The New York Times has a fascinating piece on some new software that automatically tweaks pictures of human faces to make them more attractive by reducing the concept of facial beauty to simple vector-based algorithms.

The image on the right is a ‘before and after’ picture of the software at work, and the researchers have a page for the project with many more examples and the full-text of the academic paper.

The researchers asked participants to rate the attractiveness of a series of faces and they then used software to calculate distances and directions between key facial landmarks.

By combining the attractiveness ratings and the landmark vectors they created a statistical model of which general facial attributes are most attractive. Their software allows new faces to be subtly altered to more closely approximate the general model of attractiveness.

I’m fascinated by the fact that software advances are increasingly taking advantage of the quirks of our mind and brain.

The MP3 format is perhaps the most well known, which allows audio files to be compressed because it takes advantage of a psychological effect called auditory masking where, when two sounds of certain frequencies are present, we can only perceive one.

The MP3 encoding algorithm simply scans sound files for times when auditory masking would eliminate the perception of one sound, and then actually eliminates the data from the file, thereby making it smaller.

Another wonderful idea is chroma subsampling used in jpg and digital video compression. It’s based on the finding that our visual system is less accurate at pinpointing colour differences than brightness differences.

Chroma subsampling takes advantage of this by storing colour information at a lower resolution than brightness information. For example, rather than storing separate colour information for every pixel, it will store it for every four. When we see the image, we often can’t tell the difference.

This is particularly true for moving images, and you’ll notice sometimes when you stop YouTube videos the colours seem to be fuzzy and bleed from where they’re supposed to be (have a look at this YouTube still I used on a recent post ) even though you hardly notice this when the video is playing.

These software advances wouldn’t have happened without the psychology research to find the bugs / features in human perception and it’s curious to think that these new developments build on both the digital and neural platforms.

What will be most interesting is if software starts to take advantage of cognitive features found only in certain members of the population (for example, some women have four types of colour receptor in the retina, rather than the usual three).

In other words, we might find that some important software advance will only work on some people (or rather, will be developed with only some people in mind), and so these people might be preferentially hired to work with certain applications.

If these applications become particularly high value (usually due to their use in the military or intelligence services), people might starting attempting to engineer themselves or others to have the uncommon attribute.

Sci-fi writers, start your engines.

Link to NYT piece ‘The Sum of Your Facial Parts’.

Link to researcher’s page with photos and full-text.