IT MAY have occurred to you, during the course of a dismal trawl round a supermarket indistinguishable from every other supermarket you have ever been into, to wonder why they are all the same. The answer is more sinister than depressing. It is not because the companies that operate them lack imagination. It is because they are all versed in the science of persuading people to buy things—a science that, thanks to technological advances, is beginning to unlock the innermost secrets of the consumer's mind.

In the Sainsbury's in Hatch Warren, Basingstoke, south-west of London, it takes a while for the mind to get into a shopping mode. This is why the area immediately inside the entrance of a supermarket is known as the “decompression zone”. People need to slow down and take stock of the surroundings, even if they are regulars. In sales terms this area is a bit of a loss, so it tends to be used more for promotion. Even the multi-packs of beer piled up here are designed more to hint at bargains within than to be lugged round the aisles. Wal-Mart, the world's biggest retailer, famously employs “greeters” at the entrance to its stores. Whether or not they boost sales, a friendly welcome is said to cut shoplifting. It is harder to steal from nice people.

Immediately to the left in Sainsbury's is another familiar sight: a “chill zone” for browsing magazines, books and DVDs, tempting impromptu purchases and slowing customers down. But those on a serious mission will keep walking ahead—and the first thing they come to is the fresh fruit and vegetables section.

For shoppers, this makes no sense. Fruit and vegetables can be easily damaged, so they should be bought at the end, not the beginning, of a shopping trip. But psychology is at work here: selecting good wholesome fresh food is an uplifting way to start shopping, and it makes people feel less guilty about reaching for the stodgy stuff later on.

Shoppers already know that everyday items, like milk, are invariably placed towards the back of a store to provide more opportunity to tempt customers. This is why pharmacies are generally at the rear, even in “convenience” stores. But supermarkets know shoppers know this, so they use other tricks, like placing popular items halfway along a section so that people have to walk all along the aisle looking for them. The idea is to boost “dwell time”: the length of time people spend in a store.

Traditionally retailers measure “footfall”, as the number of people entering a store is known, but those numbers say nothing about where people go and how long they spend there. But nowadays, a ubiquitous piece of technology can fill the gap: the mobile phone. Path Intelligence, a British company working with the Massachusetts Institute of Technology, tracked people's phones at Gunwharf Quays, a large retail and leisure centre in Portsmouth—not by monitoring calls, but by plotting the positions of handsets as they transmit automatically to cellular networks. It found that when dwell time rose 1% sales rose 1.3%.

Having walked to the end of the fruit and vegetable aisle, Basingstoke's hard-core shoppers arrive at counters of prepared food, the fishmonger, the butcher and the deli. Then there is the in-store bakery, which can be smelt before it is seen. Even small supermarkets now use in-store bakeries. Mostly these bake pre-prepared items and frozen dough, and they have boomed even though central bakeries that deliver to a number of stores are much more efficient. They do it for the smell of freshly baked bread, which makes people hungry and thus encourages people to buy not just bread but also other food, including frozen stuff.

Most of the information that shoppers are bombarded with is visual: labels, price stickers and advertising. But the wafting bread aroma shows smell can usefully be stimulated too, says Simon Harrop, chief executive of BRAND sense agency, a British specialist in multi-sensory marketing. In the aisle by the laundry section he suggests introducing the smell of freshly laundered sheets. Even the sound of sheets being folded could be reproduced here and contained within the area using the latest audio technology. The Aroma Company, which Mr Harrop founded, has put the smell of coconut into the shops of Thompson, a British travel agent. Some suntan oils smell of coconut, so the scent is supposed to remind people of past holidays. The company even infuses the fresh smell of citrus into a range of clothing made by Odeur, a Swedish company. It can waft for up to 13 washes.

Such techniques are increasingly popular because of a deepening understanding about how shoppers make choices. People tell market researchers and “focus groups” that they make rational decisions about what to buy, considering things like price, selection or convenience. But subconscious forces, involving emotion and memories, are clearly also at work.

Scientists used to assume that emotion and rationality were opposed to each other, but Antonio Damasio, now professor of neuroscience at the University of Southern California, has found that people who lose the ability to perceive or experience emotions as the result of a brain injury find it hard or impossible to make any decisions at all. They can't shop.

Oh, that's what I want

Researchers are now exploring these mechanisms by observing the brain at work. One of the most promising techniques is functional magnetic resonance imaging (fMRI), which uses a large scanner to detect changes in the blood flow in parts of the brain that correspond to increases or decreases in mental activity. People lying inside the scanners are shown different products or brands and then asked questions about them. What they say is compared with what they are thinking by looking at cognitive or emotional activity. The idea is that if, say, a part of the brain that is associated with pleasure lights up, then the product could be a winner. This is immensely valuable information because eight out of ten new consumer products usually fail, despite test marketing on people who say they would buy the item—but whose subconscious may have been thinking something different.

“We are just at the frontier of the subconscious,” says Eric Spangenberg, dean of the College of Business at Washington State University and an expert on the subtleties of marketing. “We know it's there, we know there are responses and we know it is significant.” But companies commissioning such studies keep the results secret for commercial reasons. This makes Dr Spangenberg sure of one thing: “What I think I know, they probably know way more.”

We are just at the frontier of the subconscious

Retailers and producers talk a lot about the “moment of truth”. This is not a philosophical notion, but the point when people standing in the aisle decide what to buy and reach to get it. The Basingstoke store illustrates some of the ways used to get shoppers' hands to wobble in the direction of a particular product. At the instant coffee selection, for example, branded products from the big producers are arranged at eye-level while cheaper ones are lower down, along with the supermarket's own-label products.

Often head offices will send out elaborate plans of where everything has to be placed; Albertsons, a big American supermarket chain, calls these a “plan-a-gram”. Spot-checks are carried out to make sure instructions are followed to the letter. The reason for this strictness is that big retailers demand “slotting fees” to put suppliers' goods on their shelves, and these vary according to which positions are considered to be prime space.

But shelf-positioning is fiercely fought over, not just by those trying to sell goods, but also by those arguing over how best to manipulate shoppers. Never mind all the academic papers written on how best to stack shelves, retailers have their own views. While many stores reckon eye-level is the top spot, some think a little higher is better. Others charge more for goods placed on “end caps”—displays at the end of the aisles which they reckon to have the greatest visibility (although some experts say it all depends on the direction in which people gyrate around a store—and opinion on that is also divided). To be on the right-hand-side of an eye-level selection is often considered the very best place, because most people are right-handed and most people's eyes drift rightwards. Some supermarkets reserve that for their own-label “premium” goods. And supermarkets may categorise things in different ways, so chapatis may not be with breads, but with ready-meals of the Indian variety. So, even though some suppliers could be paying around $50,000 per store a year for a few feet of shelf space, many customers still can't find what they are looking for.

Technology is making the process of monitoring shopper behaviour easier—which is why the security cameras in a store may be doing a lot more than simply watching out for theft. Rajeev Sharma, of Pennsylvania State University, founded a company called VideoMining to automate the process. It uses image-recognition software to scan the pictures from security cameras of shoppers while they are making their selections. It is capable of looking at the actions of hundreds of thousands of people. It can measure how many went straight to one brand, the number that dithered and those that compared several, at the same time as sorting shoppers by age, gender and ethnicity.

VideoMining analysed people in convenience stores buying beer. Typically it would take them two minutes, with the majority going straight to one brand. “This shows their mind was already made up; they were on autopilot,” says Dr Sharma. So brewers should spend their marketing money outside, not inside, the store. The analysis can also help establish the return on investment to a new advertising campaign by showing what proportion of beer-buyers can be persuaded to consider rival brands. Another study in a supermarket some 12% of people spent 90 seconds looking at juices, studying the labels but not selecting any. In supermarket decision-making time, that is forever. This implies that shoppers are very interested in juices as a healthy alternative to carbonated drinks, but are not sure which to buy. So there is a lot of scope for persuasion.

Reducing the selection on offer might help too. Cassie Mogilner of Stanford University and her colleagues found in a study that consumers like unfamiliar products to be categorised—even if the categories are meaningless. In a study of different coffees they found people were more satisfied with their choice if it came from a categorised selection, although it did not matter if the categories were marked simply A, B and C, or “mild”, “dark roast” and “nutty”.

Despite all the new technology, simply talking to consumers remains one of the most effective ways to improve the “customer experience”. Scott Bearse, a retail expert with Deloitte Consulting in Boston, Massachusetts, has led projects observing and quizzing tens of thousands of customers about how they feel about shopping. It began when a client complained that he had mountains of data on the one in four people that entered his store and bought something, but knew hardly anything about the vast majority who left without making a purchase. The “customer conversion” rate varies between types of store: it could be around 20% in some department stores but reach almost 100% in a grocery. And within the same store the conversion rate will vary in different sections.

People say they leave shops empty-handed more often because they are “unable to decide” than because prices are too high, says Mr Bearse. Working out what turns customers off is not difficult, yet stores still struggle with these issues: goods out of stock, long queues at the checkouts and poor levels of service. Getting customers to try something is one of the best ways of getting them to buy, adds Mr Bearse. Deloitte found that customers using fitting rooms convert at a rate of 85% compared with 58% for those that do not do so.

Often a customer struggling to decide which of two items is best ends up not buying either. A third “decoy” item, which is not quite as good as the other two, can make the choice easier and more pleasurable, according to a new study using fMRI carried out by Akshay Rao, a professor of marketing at the University of Minnesota. Happier customers are more likely to buy. Dr Rao believes the deliberate use of irrelevant alternatives should work in selling all sorts of goods and services, from cable TV to holidays.

The notion of shoppers wearing brain-scanning hats would be ridiculous

A lack of price tags is another turn-off, although getting that right will become crucial with the increasing use of Radio Frequency Identification (RFID) tags. These contain far more information than bar codes and can be scanned remotely. People have been predicting for years that they would shortly become ubiquitous; but, with costs continuing to fall, they eventually will. Tills will then become redundant, because everything shoppers put in their trolleys will be automatically detected and charged to their credit cards.

The basic mechanisms to do this are already in place. A store or loyalty card can be fitted with an RFID tag to identify customers on arrival. A device on the trolley could monitor everything placed in it, check with past spending patterns and nudge customers: “You have just passed the Oriels, which you usually buy here.”

Mind over matter

Technology will also begin to identify customers' emotions. Dr Sharma's software has the potential to analyse expressions, like smiles and grimaces, which are hard to fake. And although fMRI scanners presently need a crane to move, something that provides a similar result might one day be worn on your head. Researchers believe it is possible to correlate brain patterns with changes in electrical activity in the brain, which can be measured with electroencephalography (EEG) using electrodes placed on the scalp. Small EEG machines are already available, especially for computer gamers, which fit on the head.

The notion of shoppers wearing brain-scanning hats would be ridiculous if it were not so alarming. Privacy groups are already concerned about the rise of electronic surveillance that records what people do, let alone what they might be thinking. The San Francisco-based Electronic Frontier Foundation is concerned that because RFID tags can be read at a distance by anyone with the necessary equipment they could create “privacy pollution”; being used to discover what is in not only someone's shopping trolley, but also their cupboards.

To some degree shoppers would have to “buy in” to the process: a bit like having an account with an online retailer which comes with the explicit knowledge that your past purchases and browsing history will be monitored and used to pitch purchase suggestions. And if that makes shopping easier—especially if sweetened with discounts—then consumers might sign up to it. When Dr Sharma asks shoppers what they think about his video-monitoring he says most people do not mind.

But what if psychological selling is done stealthily? That way lies grave perils. It is the anger not of privacy groups that retailers should fear, but of customers at being manipulated from behind the scenes.

There have been backlashes before: “The Hidden Persuaders” by Vance Packard, an American journalist, caused a sensation when it was first published in 1957 by revealing physiological techniques used by advertisers, including subliminal messages. It is what got Dr Spangenberg interested in the subject. He thinks shopping science has limits. “I don't think you are going to be able to make someone buy a car or a computer that they don't need,” he says. “But you might persuade them to choose one model instead of another. And importantly, they wouldn't know it.” But if they did realise psychological methods were being used to influence their choice, “the counteraction can be so huge it can put someone off buying anything at all,” he adds.

Which is probably why at the end of this shopping trip there is not much in the trolley. At least the temptations at the checkout are easy to avoid: a few celebrity magazines and bags of sweets at the eye-level of children. But that will change too.

Barry Salzman, the chief executive of YCD Multimedia in New York, has big plans for the area around a cash till. He is using digital video screens displaying ads that relate to what someone is buying and which can also be linked with facial-recognition software to refine the displays according to the customer's age or sex. His system is already being used in Aroma Espresso Bars in America to present, say, an advert for a chocolate croissant to someone buying only a cappuccino.

But the checkout in this Sainsbury's comes to a halt because the teenager at the till is not old enough to sell alcohol and can't attract the attention of a supervisor for permission to ring up a multi-pack of beer, which is therefore left behind on the counter. The science of shopping is a marvellously sophisticated business; the practice is still a little more primitive.