Note: The author of the demo, Stéphane Roucheray, is a member of the PIMS team. The demo was first posted on the Pims World Labs weblog.



View the demo in Firefox 3.5.

Content Aware Image Resizing is a way to re-target an image size without modifying its content ratio, in other words : non-linear image resizing. The algorithm was first explained by Shai Avidan and Ariel Shamir and published in 2007 (“Seam Carving for Content-Aware Image Resizing.”)

Since then several excellent Open Source implementations have been released. For example, there is a plugin for The Gimp and CAIR a standalone application in C++.

But thanks to Canvas and JavaScript it’s now possible to do this in the browser without a plugin.

Since version 1.5, Firefox offers bitmap manipulation through the Canvas API. Version 3.5 introduced not only the fastest Firefox JavaScript engine ever, but also a new Canvas method – createImageData – providing a much more powerful environment.

For this demo, a sub part of the Content Aware Image Resizing algorithm has been implemented. The width of the image can be reduced interactively without modifying its height. This implementation uses seam carving to re-size the image, subtracting the less visible vertical lines. It is a four step iterative algorithm. One iteration is one pixel width re-size. First an image is loaded into the Canvas context and then the iteration starts :

A grayscale version of the image has to be calculated The edges of the image (Sobel convolution is used in our case) and its energy matrix is computed The seam of least energy (1 pixel vertical line from the bottom to the top of the energy matrix) is detected Then the pixel of the detected seam is removed from the original image and the result is re-injected as a source image to step 1

Each of the previous steps stores a whole matrix of data at the source image size. While these matrices are not all images but actually artifacts of the algorithm, storing them in an ImageData object is more convenient than using simple Arrays. This is why the createImageData method of the Canvas context is used. One of the benefits of this process is to allow showing the intermediate computations made under the hood.

This demo shows that’s possible to do more intelligent image resizing than just flattening an image’s pixels with CSS. Having computational and image manipulation capabilities directly in the browser opens up a new range of possibilities of how image data can be displayed to users. This is only one small demonstration of that.