OpenCV  4.2.0
Open Source Computer Vision
Extract horizontal and vertical lines by using morphological operations

Prev Tutorial: Hit-or-Miss
Next Tutorial: Image Pyramids

Goal

In this tutorial you will learn how to:

  • Apply two very common morphology operators (i.e. Dilation and Erosion), with the creation of custom kernels, in order to extract straight lines on the horizontal and vertical axes. For this purpose, you will use the following OpenCV functions:

    in an example where your goal will be to extract the music notes from a music sheet.

Theory

Morphology Operations

Morphology is a set of image processing operations that process images based on predefined structuring elements known also as kernels. The value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbors. By choosing the size and shape of the kernel, you can construct a morphological operation that is sensitive to specific shapes regarding the input image.

Two of the most basic morphological operations are dilation and erosion. Dilation adds pixels to the boundaries of the object in an image, while erosion does exactly the opposite. The amount of pixels added or removed, respectively depends on the size and shape of the structuring element used to process the image. In general the rules followed from these two operations have as follows:

  • Dilation: The value of the output pixel is the maximum value of all the pixels that fall within the structuring element's size and shape. For example in a binary image, if any of the pixels of the input image falling within the range of the kernel is set to the value 1, the corresponding pixel of the output image will be set to 1 as well. The latter applies to any type of image (e.g. grayscale, bgr, etc).

  • Erosion: The vice versa applies for the erosion operation. The value of the output pixel is the minimum value of all the pixels that fall within the structuring element's size and shape. Look the at the example figures below:

Structuring Elements

As it can be seen above and in general in any morphological operation the structuring element used to probe the input image, is the most important part.

A structuring element is a matrix consisting of only 0's and 1's that can have any arbitrary shape and size. Typically are much smaller than the image being processed, while the pixels with values of 1 define the neighborhood. The center pixel of the structuring element, called the origin, identifies the pixel of interest – the pixel being processed.

For example, the following illustrates a diamond-shaped structuring element of 7x7 size.

A structuring element can have many common shapes, such as lines, diamonds, disks, periodic lines, and circles and sizes. You typically choose a structuring element the same size and shape as the objects you want to process/extract in the input image. For example, to find lines in an image, create a linear structuring element as you will see later.

Code

This tutorial code's is shown lines below.

Explanation / Result

Get image from here .

Load Image

Grayscale

Grayscale to Binary image

Output images

Now we are ready to apply morphological operations in order to extract the horizontal and vertical lines and as a consequence to separate the the music notes from the music sheet, but first let's initialize the output images that we will use for that reason:

Structure elements

As we specified in the theory in order to extract the object that we desire, we need to create the corresponding structure element. Since we want to extract the horizontal lines, a corresponding structure element for that purpose will have the following shape:

and in the source code this is represented by the following code snippet:

The same applies for the vertical lines, with the corresponding structure element:

and again this is represented as follows:

Refine edges / Result

As you can see we are almost there. However, at that point you will notice that the edges of the notes are a bit rough. For that reason we need to refine the edges in order to obtain a smoother result:

cv::Mat::rows
int rows
the number of rows and columns or (-1, -1) when the matrix has more than 2 dimensions
Definition: mat.hpp:2086
cv::String
std::string String
Definition: cvstd.hpp:150
cv::Mat::clone
Mat clone() const CV_NODISCARD
Creates a full copy of the array and the underlying data.
cv::IMREAD_COLOR
@ IMREAD_COLOR
If set, always convert image to the 3 channel BGR color image.
Definition: imgcodecs.hpp:67
cv::moveWindow
void moveWindow(const String &winname, int x, int y)
Moves window to the specified position.
cv::cvtColor
void cvtColor(InputArray src, OutputArray dst, int code, int dstCn=0)
Converts an image from one color space to another.
cv::THRESH_BINARY
@ THRESH_BINARY
Definition: imgproc.hpp:317
cv::erode
void erode(InputArray src, OutputArray dst, InputArray kernel, Point anchor=Point(-1,-1), int iterations=1, int borderType=BORDER_CONSTANT, const Scalar &borderValue=morphologyDefaultBorderValue())
Erodes an image by using a specific structuring element.
cv::samples::findFile
cv::String findFile(const cv::String &relative_path, bool required=true, bool silentMode=false)
Try to find requested data file.
cv::MORPH_RECT
@ MORPH_RECT
a rectangular structuring element:
Definition: imgproc.hpp:231
cv::waitKey
int waitKey(int delay=0)
Waits for a pressed key.
highgui.hpp
core.hpp
cv::Size
Size2i Size
Definition: types.hpp:347
cv::destroyWindow
void destroyWindow(const String &winname)
Destroys the specified window.
cv::dilate
void dilate(InputArray src, OutputArray dst, InputArray kernel, Point anchor=Point(-1,-1), int iterations=1, int borderType=BORDER_CONSTANT, const Scalar &borderValue=morphologyDefaultBorderValue())
Dilates an image by using a specific structuring element.
cv::imread
Mat imread(const String &filename, int flags=IMREAD_COLOR)
Loads an image from a file.
cv::Mat::empty
bool empty() const
Returns true if the array has no elements.
cv::Mat::cols
int cols
Definition: mat.hpp:2086
cv::ADAPTIVE_THRESH_MEAN_C
@ ADAPTIVE_THRESH_MEAN_C
Definition: imgproc.hpp:332
cv::imshow
void imshow(const String &winname, InputArray mat)
Displays an image in the specified window.
cv::blur
void blur(InputArray src, OutputArray dst, Size ksize, Point anchor=Point(-1,-1), int borderType=BORDER_DEFAULT)
Blurs an image using the normalized box filter.
cv::Point
Point2i Point
Definition: types.hpp:194
cv::Mat
n-dimensional dense array class
Definition: mat.hpp:792
cv::imshow
void imshow(const String &winname, const ogl::Texture2D &tex)
Displays OpenGL 2D texture in the specified window.
cv::Mat::copyTo
void copyTo(OutputArray m) const
Copies the matrix to another one.
cv::CommandLineParser
Designed for command line parsing.
Definition: utility.hpp:797
CV_8UC1
#define CV_8UC1
Definition: interface.h:88
cv::COLOR_BGR2GRAY
@ COLOR_BGR2GRAY
convert between RGB/BGR and grayscale, color conversions
Definition: imgproc.hpp:542
cv
"black box" representation of the file storage associated with a file on disk.
Definition: affine.hpp:52
cv::getStructuringElement
Mat getStructuringElement(int shape, Size ksize, Point anchor=Point(-1,-1))
Returns a structuring element of the specified size and shape for morphological operations.
imgproc.hpp
cv::Mat::ones
static MatExpr ones(int rows, int cols, int type)
Returns an array of all 1's of the specified size and type.
cv::bitwise_not
void bitwise_not(InputArray src, OutputArray dst, InputArray mask=noArray())
Inverts every bit of an array.
cv::adaptiveThreshold
void adaptiveThreshold(InputArray src, OutputArray dst, double maxValue, int adaptiveMethod, int thresholdType, int blockSize, double C)
Applies an adaptive threshold to an array.
cv::Mat::channels
int channels() const
Returns the number of matrix channels.