OpenCV is our recommended package for image processing. In robotics, OpenCV can be deployed in Single Board computers as Raspberry Pi to give the power of vision to the robot.

You can make robots as SID2 using the power of OpenCV in your robots.

The Jupyter notebook for the code is at the Github link here.

The aim of this blog:

This is a beginner's blog which will look at below aspects using OpenCV2 -



a) Read an image from the folder path

b) Grayscale the image

c) Perform Canny edge on that image

d) Identify lines in an image using the Hough Line detection algorithm

e) Learn about Affine transformation

The codes are shared via Jupyter Notebook in our GitHub directory. The link is here for GitHub.

Pre-requisites for the code:

You should have OpenCV2 installed along with matplotlib. You may consider using the Anaconda environment as we have used or may use a standalone environment as well.

You must also have a webcam in your machine which acts like the camera.Please make sure your local antivirus does not block Python from using the webcam.

You can have a look at our old blogs on Anaconda here if you are new to Anaconda.

You must also have Python2 installed along with Numpy. For more information of Numpy installation you can refer our old blog here.

Ok, so let's get started.

Open a new Jupyter Notebook session over the Python2 kernel and get started with below code blocks.

Reading an image with OpenCV2 -

To read an image import OpenCV and use the below two lines of code.

import cv2

img = cv2.imread('images/TajMahal.jpg')

cv2.imshow('read image',img)

Grayscale an image with OpenCV2 -

Grayscale refers to black and white and it is often done as many image processing algorithm in OpenCV have a pre-requisite to convert the image into grayscale. There are two ways to do it. The most common way is to use the OpenCV inbuilt function cv2.COLOR_BGR2GRAY.

The other way is to amend a '0' (numeric zero not alphabet O) when you read the image using the imread function. We will use the easy second option as -

img = cv2.imread('images/TajMahal.jpg',0)

cv2.imshow('Grayscale', img)

cv2.waitKey()

cv2.destroyAllWindows()

As you can see in the image above of the Taj Mahal. OpenCV can perfectly grayscale the source image from the left.

Rotate an image :

As you know images in OpenCV are represented by Matrix. To rotate an image in OpenCV we would need to know the following -

a) The center with respect to which the image will rotate

b)The angle to be rotated. In OpenCV a positive angle is counter-clockwise

c)Optional: A scale factor

Affine transformation is expressed in the form of a matrix multiplication (linear transformation) followed by a vector addition (translation).

This is conducted using a concept known as Affine transformation. According to OpenCV documentation Affine transformation is 'any transformation that can be expressed in the form of a matrix multiplication (linear transformation) followed by a vector addition (translation)'.

In simple terms, Affine transformation retains the parallelism of the source image whereas Non Affine transformation does not do that.

To implement it in code we use the two steps to -

a) use function getRotationMatrix2D to rotate the image around it's center

b) warpAffine function to to get the 2X3 rotation matrix

image = cv2.imread('images/TajMahal.jpg')

height, width = image.shape[:2]

# Divide by two to rotate the image around its centre

rotation_matrix = cv2.getRotationMatrix2D((width/2, height/2), 90, .5)

rotated_image = cv2.warpAffine(image, rotation_matrix, (width, height))

cv2.imshow('Rotated Image of Taj', rotated_image)

cv2.waitKey()

cv2.destroyAllWindows()

The end result looks as above which is a rotation of the image we provided of the Taj Mahal.

Canny edge detection:

Opencv2 provides us with a standard function called as canny which can be used to perform canny edge detection. For the mathematics please refer the section at the end of this blog. In Opencv2 the function parameters are : cv.Canny(image, edges, threshold1, threshold2, aperture_size=3)

Parameters -

image – single-channel 8-bit input image. edges – output edge map; it has the same size and type as image .threshold1 – first threshold for the hysteresis procedure. threshold2 – second threshold for the hysteresis procedure.

Now let is implement this in Python as below:Opencv2 provides us with a standard function called as canny which can be used to perform canny edge detection. For the mathematics please refer the section at the end of this blog.

In Opencv2 the function parameters are : cv.Canny(image, edges, threshold1, threshold2, aperture_size=3)

Now let is implement this in Python as below:

img = cv2.imread('images/TajMahal.jpg')

cv2.imshow('read image',img)

edges = cv2.Canny(img,100,200)

plt.subplot(121),plt.imshow(img,cmap = 'gray')

plt.title('Original Image'), plt.xticks([]), plt.yticks([])

plt.subplot(122),plt.imshow(edges,cmap = 'gray')

plt.title('Edge Image'), plt.xticks([]), plt.yticks([])

plt.show()

The output for couple of input images is shown below and you can see it does a good job to detect the edges.

Detect a line in an image by Hough line:

Line detection is very interesting and can form the base for a line follower robot where you can your webcam to enable the robot to detect lines.

To complete this part you would need to know how OpenCV2 detects lines. Please note that OpenCV2 uses Polar coordinates as it needs to take note of the pixels from top left of the image.

If you need to brush up Polar coordinate equations we have provided some links at the source section.

Line detection is very interesting and can form the base for a line follower robot where you can your webcam to enable the robot to detect lines.

The line in Polar coordinate is represented by r= xcos (theta) + ysin (theta) meaning that each pair represents each line that passes by (x,y).

There are two ways in OpenCV2 for Hough line transforms i.e

The Standard Hough Transform:

It consists in pretty much what we just explained in the previous section. It gives you as result a vector of couples

In OpenCV it is implemented with the function HoughLines

The Probabilistic Hough Line Transform:

A more efficient implementation of the Hough Line Transform. It gives as output the extremes of the detected lines

In OpenCV it is implemented with the function HoughLinesP

We will use the simple HoughLines in Python as below approach.

We import the needed numpy and opencv library and then call the image and grayscale/canny edge the image using the methods described above.

The code is implemented as below.

image = cv2.imread('images/buildingBlog.jpg')

# Grayscale and Canny Edges extracted

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

edges = cv2.Canny(gray, 100, 170, apertureSize = 3)

The image we are using is that of a building as shown.

It is now time to call the standard Hough Transform.

lines = cv2.HoughLines(edges, 1, np.pi / 180, 190)

Here we declare lines using the defined HoughLines function and have using a rho accuracy of 1 pixel with theta as accuracy of np.pi / 180 which is 1 degree. We set the threshold as 190 but you can experiment with this value.

Affine transformation retains the parallelism of the source image whereas Non Affine transformation does not do that.

Then iterate through each line and convert it to the format required by cv.lines (i.e. requiring end points) as below code:

for rho, theta in lines[0]:

a = np.cos(theta)

b = np.sin(theta)

x0 = a * rho

y0 = b * rho

x1 = int(x0 + 1000 * (-b))

y1 = int(y0 + 1000 * (a))

x2 = int(x0 - 1000 * (-b))

y2 = int(y0 - 1000 * (a))

cv2.line(image, (x1, y1), (x2, y2), (255, 0, 0), 2)

When we run this we see the lines detected as below.

As you can that the standard Hough line does a decent job and it was able to detect the lines on all floors of the image but had some errors on the bottom left side of the image. You may play around with the parameter value and see if you get better results.

We now try to change the image to a Soduku image which is a better input image for standard Hough line transform. We would just need to place the image in the same folder and change the file name.

The new input file is shown below.

We change the code as to change the image file name and also the threshold value (changed to 270 but you can play with this value) in two lines of code as shown below.

image = cv2.imread('images/soduku2.jpg')

lines = cv2.HoughLines(edges, 1, np.pi / 180, 270)

The output is more impressive in this attempt as shown below.

We hope this blog was helpful for you. Please comment and share.

Sources and references:

a) http://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/warp_affine/warp_affine.html

b) Source code as Jupyter notebook for this blog - https://github.com/MieRobot/Blogs/blob/master/Blog_OpenCV_CannyEdgeWithHoughLineDetection.ipynb

c) OpenCV2 link and some background on mathematics for Polar coordinates and Hough line

d) https://en.wikipedia.org/wiki/Canny_edge_detector

e) http://docs.opencv.org/2.4/modules/imgproc/doc/feature_detection.html?highlight=houghlinesp#canny

f) Attribution for Taj Mahal image : # Image source and attribution : By This image was produced by David Castor Own work, Public Domain, # https://commons.wikimedia.org/w/index.php?curid=3996907

g) Adffine transformation at wiki - https://en.wikipedia.org/wiki/Affine_transformation



