Storing an Image: Other Color Spaces

To make image manipulation easier, we commonly convert images from the RGB color space to HSV or Hue, Saturation, Value.

figure 10 — from slides

In RGB, the color is tied with its luminescence, whereas in HSV they’re separated, which allows us to manipulate an image in a much more straightforward manner. We’ll cover some examples of this later.

This conversion is very difficult to explain in text, but the diagram below illustrates it well:

figure 11 — from slides

This RGB cube (in figure 10) is tilted and squashed so that all the colors are on the same plane. The black point is then expanded to create the base of the HSV cylinder shown in figure 10.

Every image in the HSV color space is still represented by a 3D tensor with three channels. The difference is that the channels are no longer red, green, and blue (RGB) but instead hue, saturation, and value (HSV).

As we’ve expanded the black point into the base of the cylinder, it’s completely black, and the color is then most vibrant at the top of the cylinder. This is the value in HSV. Hue represents which color the pixel is, and the central column of this is white. This means that the closer a point is to the central core of the cylinder, the whiter it is (or less saturated).

We can see this in the following example:

figure 12 — from slides

figure 13 — from slides

You can see that the hue still shows the yellow wall and red bike from the original image, but the floor looks blue, even though it’s grey in the original image.

This is because even though the hue is blue, the saturation is very low. In the saturation image, you can see that the wall and bike are very bright, which tells us that the yellow and red in the hue image look that color. The floor, however, has a low saturation and is therefore grey.

The value image looks grayscale but isn’t actually, as grayscale images are a lot more complicated than you’d initially think. As we know (from part 1), green matters a lot more than red and blue in how we see the world, but this information is lost in value images and is therefore less accurate at representing our perception. The value image is just in black and white and represents how luminescent a pixel is.

In this color space, we can perform image manipulations that are much more difficult to perform if the image was represented in the RGB color space. For example, we could double all saturation values to over saturate the image. We also covered this in part 1 of this series and pointed out that this is a very common trick used to sell houses.

figure 14 — source

To reiterate, this is quite difficult to do in the RGB color space but very easy in HSV.

Similarly, we can very easily manipulate the exposure of an image by modifying its value (again, exposure is covered in part 1).

figure 15 — source

Changing the hue values changes the colors in the picture. This is often used in Photoshop to tweak colors of objects in photos; but is has another very interesting use.

Machine learning, especially deep learning, requires a lot of data to train. To complicate matters, this data needs to be annotated, which is difficult to collect at scale. By manipulating the hue of an already annotated image, you can generate more labeled data to train models. These images look very similar to us but don’t to computers, so this can be used to significantly increase the size of a dataset.

figure 16 — from slides

These examples are fairly simple, but you can also perform more complicated manipulations easily. For example, you could replace the hue matrix with a pattern to get psychedelic results or add threshholding to an image’s saturation matrix to desaturate all colors apart from one to get some cool results like this:

figure 17 — source

These manipulations are all used very practically every day for things like Instagram filters and branding. Another image manipulation we use all the time is image resizing. Whether viewing images on different screens or pinching your phone screen to zoom, this is done in different ways depending on the use case.