In this article a few popular image processing problems along with their solutions are going to be discussed. Python image processing libraries are going to be used to solve these problems.

Image Transformations and Warping

0. Display RGB image color channels in 3D

A gray-scale image can be thought of a 2-D function f(x,y) of the pixel locations (x,y), that maps each pixel into its corresponding gray level (an integer in [0,255], e.g.,). For an RGB image there are 3 such functions, f_R(x,y), f_G(x.y), f_B(x.y). matplotlib’s 3-D plot functions can be used to plot each function.

The following python code shows how to plot the RGB channels separately in 3D:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 def plot_3d(X, Y, Z, title, cmap): import matplotlib.pylab as plt from mpl_toolkits.mplot3d import Axes3D plt.show() im = imread( '../new images/parrot.jpg' ) Y = np.arange(im.shape[ 0 ]) X = np.arange(im.shape[ 1 ]) Z1 = im[..., 0 ] Z2 = im[..., 1 ] Z3 = im[..., 2 ] plot_3d(X, Y, Z1, cmap = 'Reds' , title = '3D plot for the Red Channel' ) plot_3d(X, Y, Z2, cmap = 'Greens' , title = '3D plot for the Green Channel' ) plot_3d(X, Y, Z3, cmap = 'Blues' , title = '3D plot for the Blue Channel' )

The RGB image

https://sandipanweb.files.wordpress.com/2018/07/parrot.jpg?w=150&am... 150w, https://sandipanweb.files.wordpress.com/2018/07/parrot.jpg?w=300&am... 300w" sizes="(max-width: 570px) 100vw, 570px" />

https://sandipanweb.files.wordpress.com/2018/07/parrot_red.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/parrot_red.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/parrot_red.png 691w" sizes="(max-width: 676px) 100vw, 676px" /> https://sandipanweb.files.wordpress.com/2018/07/parrot_green.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/parrot_green.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/parrot_green.png 691w" sizes="(max-width: 676px) 100vw, 676px" /> https://sandipanweb.files.wordpress.com/2018/07/parrot_blue.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/parrot_blue.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/parrot_blue.png 691w" sizes="(max-width: 676px) 100vw, 676px" />

1. Wave Transform

Use scikit-image’s warp() function to implement the wave transform. Note that wave transform can be expressed with the following equations:





We shall use the madrill image to implement the wave transform. The next python code fragment shows how to do it:

1 2 3 4 5 6 7 8 9 10 11 def wave(xy): xy[:, 1 ] + = 20 * np.sin( 2 * np.pi * xy[:, 0 ] / 64 ) return xy from skimage.io import imread from skimage.transform import warp import matplotlib.pylab as plt im = imread( 'images/mandrill.jpg' ) im = warp(im, wave) plt.imshow(im) plt.show()

The next figure shows the original mandrill input image and the output image obtained after applying the wave transform.

https://sandipanweb.files.wordpress.com/2018/07/mandrill.jpg?w=150&... 150w" sizes="(max-width: 320px) 100vw, 320px" /> https://sandipanweb.files.wordpress.com/2018/07/mandrill_w1.png?w=1... 150w" sizes="(max-width: 317px) 100vw, 317px" />

2. Swirl Transform

Use scikit-image’s warp() function to implement the swirl transform. Note that swirl transform can be expressed with the following equations







We shall use the madrill image to implement the wave transform. The next python code fragment shows how to do it:

1 2 3 4 5 6 7 8 9 10 11 12 def swirl(xy, x0, y0, R): r = np.sqrt((xy[:, 1 ] - x0) * * 2 + (xy[:, 0 ] - y0) * * 2 ) a = np.pi * r / R xy[:, 1 ] = (xy[:, 1 ] - x0) * np.cos(a) + (xy[:, 0 ] - y0) * np.sin(a) + x0 xy[:, 0 ] = - (xy[:, 1 ] - x0) * np.sin(a) + (xy[:, 0 ] - y0) * np.cos(a) + y0 return xy im = imread( '../images/mandrill.jpg' ) im = warp(im, swirl, map_args = { 'x0' : 112 , 'y0' : 112 , 'R' : 512 }) plt.imshow(im) plt.axis( 'off' ) plt.show()

The next figure shows the original mandrill input image and the output image obtained after applying the swirl transform.

https://sandipanweb.files.wordpress.com/2018/07/mandrill.jpg?w=150&... 150w" sizes="(max-width: 320px) 100vw, 320px" />

Compare this with the output of the scikit-image swirl() function.

3. Very simple Face morphing with α-blending

Start from one face image (e.g., let image 1 be the face of Messi) and end into another image (let image 2 be the face of Ronaldo) iteratively, creating some intermediate images in between. At each iteration create an image by using a linear combination of the two image numpy ndarrays given by

3. Iteratively increase α from 0 to 1.

The following code block shows how to implement it using matplotlib’s image and pylab modules.

1 2 3 4 5 6 7 8 9 10 11 im1 = mpimg.imread( "../images/messi.jpg" ) / 255 im2 = mpimg.imread( "../images/ronaldo.jpg" ) / 255 i = 1 plt.figure(figsize = ( 18 , 15 )) for alpha in np.linspace( 0 , 1 , 20 ): plt.subplot( 4 , 5 ,i) plt.imshow(( 1 - alpha) * im1 + alpha * im2) plt.axis( 'off' ) i + = 1 plt.subplots_adjust(wspace = 0.05 , hspace = 0.05 ) plt.show()

The next animation shows the simple face morphing:



There are more sophisticated techniques to improve the quality of morphing, but this is the simplest one.

4. Creating Instagram-like Gotham Filter

The Gotham filter

The Gotham filter is computed as follows (the steps taken from here), applying the following operations on an image, the corresponding python code, input and output images are shown along with the operations (with the following input image):

https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

A mid-tone red contrast boost 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 from PIL import Image import numpy as np import matplotlib.pylab as plt im = Image. open ( '../images/city.jpg' ) r, g, b = im.split() red_levels = [ 0. , 12.75 , 25.5 , 51. , 76.5 , 127.5 , 178.5 , 204. , 229.5 , 242.25 , 255. ] r1 = Image.fromarray((np.reshape(np.interp(np.array(r).ravel(), np.linspace( 0 , 255 , len (red_levels)), red_levels), (im.height, im.width))).astype(np.uint8), mode = 'L' ) plt.figure(figsize = ( 20 , 15 )) plt.subplot( 221 ) plt.imshow(im) plt.title( 'original' , size = 20 ) plt.axis( 'off' ) plt.subplot( 222 ) im1 = Image.merge( 'RGB' , (r1, g, b)) plt.imshow(im1) plt.axis( 'off' ) plt.title( 'with red channel interpolation' , size = 20 ) plt.subplot( 223 ) plt.hist(np.array(r).ravel(), normed = True ) plt.subplot( 224 ) plt.hist(np.array(r1).ravel(), normed = True ) plt.show() https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png 1172w" sizes="(max-width: 676px) 100vw, 676px" /> Make the blacks a little bluer 1 2 3 4 5 6 7 8 9 10 11 12 13 plt.figure(figsize = ( 20 , 10 )) plt.subplot( 121 ) plt.imshow(im1) plt.title( 'last image' , size = 20 ) plt.axis( 'off' ) b1 = Image.fromarray(np.clip(np.array(b) + 7.65 , 0 , 255 ).astype(np.uint8)) im1 = Image.merge( 'RGB' , (r1, g, b1)) plt.subplot( 122 ) plt.imshow(im1) plt.axis( 'off' ) plt.title( 'with transformation' , size = 20 ) plt.tight_layout() plt.show() https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" /> A small sharpening 1 2 3 4 5 6 7 8 9 10 11 12 13 from PIL.ImageEnhance import Sharpness plt.figure(figsize = ( 20 , 10 )) plt.subplot( 121 ) plt.imshow(im1) plt.title( 'last image' , size = 20 ) plt.axis( 'off' ) im2 = Sharpness(im1).enhance( 3.0 ) plt.subplot( 122 ) plt.imshow(im2) plt.axis( 'off' ) plt.title( 'with transformation' , size = 20 ) plt.tight_layout() plt.show() https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" /> A boost in blue channel for lower mid-tones A decrease in blue channel for upper mid-tones 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 blue_levels = [ 0. , 11.985 , 30.09 , 64.005 , 81.09 , 99.96 , 107.1 , 111.945 , 121.125 , 143.055 , 147.9 , 159.885 , 171.105 , 186.915 , 215.985 , 235.875 , 255. ] b2 = Image.fromarray((np.reshape(np.interp(np.array(b1).ravel(), np.linspace( 0 , 255 , len (blue_levels)), blue_levels), (im.height, im.width))).astype(np.uint8), mode = 'L' ) plt.figure(figsize = ( 20 , 15 )) plt.subplot( 221 ) plt.imshow(im2) plt.title( 'last image' , size = 20 ) plt.axis( 'off' ) plt.subplot( 222 ) im3 = Image.merge( 'RGB' , (r1, g, b2)) plt.imshow(im3) plt.axis( 'off' ) plt.title( 'with blue channel interpolation' , size = 20 ) plt.subplot( 223 ) plt.hist(np.array(b1).ravel(), normed = True ) plt.subplot( 224 ) plt.hist(np.array(b2).ravel(), normed = True ) plt.show()

The output image obtained after applying the Gotham filter is shown below:

https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

Down-sampling with anti-aliasing using Gaussian Filter

Start with a large gray-scale image and reduce the image size 16 times, by reducing both height and width by 4 times. Select every 4th pixel in the x and the y direction from the original image to compute the values of the pixels in the smaller image. Before down-sampling apply a Gaussian filter (to smooth the image) for anti-aliasing. Compare the quality of the output image obtained by down-sampling without a Gaussian filter (with aliasing).

The next code block performs the above steps. Since the Gaussian blur is a low-pass filter, it removes the high frequencies from the original input image, hence it’s possible to achieve sampling rate above the Nyquist rate (by sampling theorem) to avoid aliasing.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 from scipy.ndimage import gaussian_filter im = rgb2gray(imread( 'images/umbc.png' )) print (im.shape) plt.figure(figsize = ( 20 , 20 )) plt.imshow(im) plt.show() plt.figure(figsize = ( 20 , 20 )) im_blurred = gaussian_filter(im, sigma = 2.5 ) plt.imshow(im_blurred) plt.show() n = 4 w, h = im.shape[ 0 ] / / n, im.shape[ 1 ] / / n im_small = np.zeros((w,h)) for i in range (w): for j in range (h): im_small[i,j] = im[n * i, n * j] plt.figure(figsize = ( 20 , 20 )) plt.imshow(im_small) plt.show() im_small = np.zeros((w,h)) for i in range (w): for j in range (h): im_small[i,j] = im_blurred[n * i, n * j] plt.figure(figsize = ( 20 , 20 )) plt.imshow(im_small) plt.show()

Original Image

https://sandipanweb.files.wordpress.com/2018/07/orig_umbc.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/orig_umbc.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/orig_umbc.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/orig_umbc.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/orig_umbc.png 1159w" sizes="(max-width: 676px) 100vw, 676px" />

Image blurred with Gaussian Filter LPF https://sandipanweb.files.wordpress.com/2018/07/blur_umbc.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/blur_umbc.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/blur_umbc.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/blur_umbc.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/blur_umbc.png 1159w" sizes="(max-width: 676px) 100vw, 676px" />

Down-sampled Image from the original image (with aliasing)

https://sandipanweb.files.wordpress.com/2018/07/alias_umbc.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/alias_umbc.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/alias_umbc.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/alias_umbc.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/alias_umbc.png 1159w" sizes="(max-width: 676px) 100vw, 676px" />

Down-sampled Image from the blurred image (with anti-aliasing) https://sandipanweb.files.wordpress.com/2018/07/anti_alias_umbc.png... 150w, https://sandipanweb.files.wordpress.com/2018/07/anti_alias_umbc.png... 300w, https://sandipanweb.files.wordpress.com/2018/07/anti_alias_umbc.png... 768w, https://sandipanweb.files.wordpress.com/2018/07/anti_alias_umbc.png... 1024w, https://sandipanweb.files.wordpress.com/2018/07/anti_alias_umbc.png 1159w" sizes="(max-width: 676px) 100vw, 676px" />