Analysing images to find what aspects evoke which feelings could allow computers to create art that packs an emotional punch

Art and emotion: what joyful squiggles (Image: Michael Kappeler/AFP/Getty)

ECSTASY. Joy. Sadness. Despair. The sweeping lines and blocks of colour in abstract art prompt us to respond emotionally in ways that we do not really understand. Now computers are getting in on the act, and the results could add a new dimension to the weird world of artificial creativity.

The pioneering abstract painter Wassily Kandinsky (whose work is pictured) suggested that the emotional effects of abstract art are “objective, determined by the characteristics of the colours and their interactions”. If that is true, machines should be able to get a handle on those emotions, too.

It turns out that they can. A team led by Nicu Sebe at the University of Trento in Italy used machine vision to analyse 500 abstract paintings at the Museum of Modern and Contemporary Art of Trento and Rovereto. The system measured how colour is distributed across each work, as well as the occurrence of different shapes or outlines. Using data on how 100 people responded to the paintings, the system then worked out what emotional impact these elements had. For example, black, spiky features tended to correspond to the bleaker end of the emotional spectrum, whereas bright, smooth features were more feel-good.


To test the system, the team gave it other artworks to scan and asked it to predict the typical viewer’s emotional response on a sliding scale, from extremely negative to extremely positive. Nearly 80 per cent of the time the system came up with a score that matched the average response of 100 volunteer viewers. The study was funded by a CARITRO grant and was presented to the ACM Multimedia conference in Nara, Japan, at the end of October.

James Wang of Penn State University in University Park, Pennsylvania, says that the work opens the door to using emotional data in the creation of more advanced machine art. At the conference, he presented a similar system which predicts the emotions that certain images are likely to arouse.

Wang trained his system using a set of photos that had already been tested on viewers to determine the typical emotional response – for example, disgust, excitement, anger or joy. The system analysed the photos for shapes and complexity, and so learned how such features might be associated with those emotions.

Simon Colton of Imperial College London, who studies the intersection of creativity and computation, says such projects could help the AI artist that he has developed, called The Painting Fool, add an emotional dimension to its work.

Given the ability to recognise what aspects of an image elicit emotion, The Painting Fool could choose a basic theme for a piece, scan the web to find strongly emotional images, then use the results to inspire a swathe of pieces, he says. The system could even analyse its own creations to choose the most poignant piece.

The system could scan the web to find emotional images that then inspire a swathe of its own pieces

Colton says that computer-generated art “can do things that no one can do”, like sampling every tweet on Twitter for inspiration. Artificial creativity means that anyone can own original, unique art, he says.

Wang has potential applications in mind, too. The work could help us search for images using emotional keywords, and blogs might be supercharged with pictures designed to arouse particular feelings, for example.

The work could also protect children using the internet. “You might not want children to look at pictures that contain a lot of anger, fear, disgust or violence,” he says.

The linguistic meets the artistic The Mona Lisa not cheerful enough for you? Just say the word. Xiaohui Wang at Tsinghua University in Beijing, China, and colleagues have developed software that adjusts an image to reflect the emotions evoked by words such as “twilight”, “deserted”, “gentle” or “romantic”. The system picks a colour scheme to match, then repaints the image. The researchers asked 15 volunteers whether they preferred images touched up by the system or by an artist using Photoshop. Just under 70 per cent said they preferred the results of the automatic system (The Visual Computer, doi.org/jp4). The software has now been released as an iPad app called Emotion Modifier. Douglas Heaven