Online retailers are increasingly using your personal data to decide how much to charge you. And high-street shops are about to get in on the act

In 1861 a shopkeeper in Philadelphia revolutionised the retail industry. John Wanamaker, who opened his department store in a Quaker district of the city, introduced price tags for his goods, along with the high-minded slogan: “If everyone was equal before God, then everyone would be equal before price.” The practice caught on. Up until then high-street retailers had generally operated a market-stall system of haggling on most products. Their best prices might be reserved for their best customers. Or they would weigh up each shopper and make a guess at what they could afford to pay and eventually come to an agreement.

Wanamaker’s idea was not all about transparency, however. Fixed pricing changed the relationship between customer and store in fundamental ways. It created the possibilities of price wars, loss leaders, promotional prices and sales. For the first time people were invited to enter stores without the implied obligation to buy anything (until then shops had been more like restaurants; you went in on the understanding that you wouldn’t leave without making a purchase). Now customers could come in and look and wander and perhaps be seduced. Shopping had been invented.

If you have enough data you can get closer to the ideal of giving your customers what they want at the time they want it Roy Horgan, Market Hub CEO

For the last 150 years or so, Wanamaker’s fixed-price principle has been a norm on the high street. Shoppers might expect the price of bread or fish or vegetables to go down at the end of a day, or when they neared a sell-by date, but they would not expect prices to fluctuate very often on durable goods, and they would never expect the person behind them in the queue to be offered a different price to the one they were paying. That idea is no longer secure. Technology, for better and worse, through the appliance of big data and machine intelligence, can now transport us back to the shopping days of before 1861.

The notion of “dynamic pricing” has long been familiar to anyone booking a train ticket, a hotel room or holiday (Expedia might offer thousands of price changes for an overnight stay in a particular location in a single day). We are used to prices fluctuating hour by hour, apparently according to availability. Uber, meanwhile, has introduced – and been criticised for – “surge pricing”, making rapid adjustments to the fares on its platform in response to changes in demand. During the recent tube strikes in London, prices for cab journeys ‘automatically” leapt 400%. (The company argued that by raising fares it was able to encourage more taxi drivers to take to the streets during busy times, helping the consumer.)

What we are less aware of is the way that both principles have also invaded all aspects of online retailing – and that pricing policies are not only dependent on availability or stock, but also, increasingly, on the data that has been stored and kept about your shopping history. If you are an impulse buyer, or a full-price shopper or a bargain hunter, online retailers are increasingly likely to see you coming. Not only that: there is evidence to suggest that calculations about what you will be prepared to pay for a given product are made from knowledge of your postcode, who your friends are, what your credit rating looks like and any of the thousands of other data points you have left behind as cookie crumbs in your browsing history.

Facebook has about 100 data points on each of its 2 billion users, generally including the value of your home, your regular outgoings and disposable income – the kind of information that bazaar owners the world over might have once tried to intuit. Some brokerage firms offering data to retailers can provide more than 1,500 such points on an individual. Even your technology can brand you as a soft touch. The travel site Orbitz made headlines when it was revealed to have calculated that Apple Mac users were prepared to pay 20-30% more for hotel rooms than users of other brands of computer, and to have adjusted its pricing accordingly.

The algorithms employed by Amazon, with its ever-growing user database, and second-by-second sensitivity to demand, are ever more attuned to our habits and wishes. Websites such as camelcamelcamel.com allow to you monitor the way that best-buy prices on the site fluctuate markedly hour by hour. I watched the price of a new vacuum cleaner I had my eye on – the excitement! – waver like the graph of a dodgy penny stock last week. What is so far less certain is whether those price changes are ever being made just for you. (Amazon insists its price changes are never attempts to gather data on customers’ spending habits, but rather to give shoppers the lowest price available.)

Until quite recently this facility to both monitor the market and give consumers best price offers has looked like another advantage of the digital retailer over its bricks and mortar counterpart. Recently there have been efforts to address that inequality and replicate the possibilities of dynamic pricing on the high street.

Ever since data has been collected on customer purchases it has been possible to place shoppers into what analysts call “different consumer buckets”: impulse shoppers who were likely to buy sweets at the checkout counter; Fitbit obsessives willing to pay over the odds for organic kale. In her cheerily titled book Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy Cathy O’Neil notes how by 2013, as part of a research project by the consultant Accenture using data from a major retailer, “it was possible to estimate how much it would cost to turn each shopper from one brand of ketchup or coffee to another more profitable brand. The supermarket could then pick out, say, the 15% most likely to switch and provide them with coupons. Smart targeting was essential, [as] they didn’t want to give coupons to shoppers happy to pay full price.”

Facebook Twitter Pinterest Dynamic pricing is familiar to users of online travel websites such as Expedia. Photograph: Alamy

The obstacle to creating such “smart” pricing strategies in store has been the stubbornness of the paper price tag. A price change in most British retailers still involves a laconic employee manually updating them. In that, the UK currently lags quite far behind its neighbours in Europe (a fact noted last year by Nick Boles, then minister for skills, who praised French retailers for having systems that could change prices “90,000 times a day” while we still had minimum-wage employees traipsing along the aisles). Electronic price tags, which allow those 90,000 dynamic price changes, are a fact of life in most larger stores not only in France, but also in Germany and Scandinavia.

Within a couple of years it is likely they will become the norm here too – not least because cheap “price gun” labour might be harder to come by for supermarkets post-Brexit. That is certainly the view of Roy Horgan, chief executive of a company called Market Hub, which not only offers electronic shelf labels but also data analysis to keep prices competitive. Market Hub was created in 2010 by Horgan in part as a response to what he saw as a “race to the bottom” by retailers in his native Ireland in response to the financial crash. “We just thought that this can’t be the way to compete,” he says. “One of the things we are sure of is if you are copying your competitors’ strategy and you are losing, then they are losing too…” There had to be a smarter way.

Earlier this year the French market leader SES took a majority share in Horgan’s firm, giving it access to 15,000 stores. Only two or three of those at the moment are in Britain – Spar stores in Walthamstow and Hackney in London, where they are experimenting with dynamic pricing in the food hall, particularly with bread. The retailers show not only an uplift in revenue and profit (of 2.5%), but also a drop in wasted food of around 30%, according to Market Hub. They are selling their products in part as an eco-efficient system that prevents waste.

“When we set out,” Horgan says, “there were literally hundreds of startups analysing where customers were going in the store, or whatever. But there was also a ‘so what?’ about that. It didn’t make any difference without the ability to execute price [changes] and to make that change at the shelf. We developed a piece of software called Pulse, which analyses sales, weight, stock, and competitors’ prices that allows you to basically decide or not decide to take a trade. A city centre store will want to catch customers at the end of the day before they head home, so what level do you set your price at?”

Horgan suggests that British retailers are still a bit terrified that customers will be put off by changing prices – they notice one shift in price of a loaf of bread, but don’t see 50 changes of price in the vacuum cleaner they are browsing on Amazon. He believes that the system can benefit both consumer and retailer though, because it is about getting the right deal. “If you have enough data you can get closer and closer to the ideal, which is giving your customers what they want and at the time they want it, rather than overwhelming them with deals.”

Algorithms: AI’s creepy control must be open to inspection Read more

It also perhaps has the potential to offer a glimmer of hope for the beleaguered high street. Shops are all too aware of the habit of “showrooming”, by which customers look at products in stores before going home and browsing the best deals for them online. Electronic price-tag systems can not only track online prices, they can – and sometimes do – also display at point of sale the hidden cost of shipping if the same product was bought online – a cost that most online customers don’t factor in. “There is a way for [high street] retailers to become profitable again,” Horgan insists.

So far, such systems have not entered the murkier waters of using the data to offer different customers different prices for the same product at the same moment. A couple of years ago B&Q tested electronic price tags that display an item’s price based on who was looking at it, using data gathered from the customer’s mobile phone, in the hope, the store insisted, “of rewarding regular customers with discounts and special offers” – rather than identifying who might pay top price for a product based on their purchasing history.

That trial hasn’t become a widespread practice, although with the advent of electronic systems and the greater possibilities of using your phone apps as a means of payment, it is probably only a matter of time. Should such pricing policies alarm us? The problem, as with all data-based solutions, is that we don’t know – no one knows – exactly which “consumer bucket” we have been put in and precisely why. In 2012, a Wall Street Journal investigation discovered that online companies including the office-supply store Staples and the furniture retailer Home Depot showed customers different prices based on “a range of characteristics that could be discovered about the user”. How far, for example, a customer was from a bricks-and-mortar store was factored in for weighty items; customers in locations with a higher average income – and perhaps more buying choice – were generally shown lower prices. Another study, in Spain, showed that the price of the headphones Google recommends to you in its ads correlated with how budget-conscious your web history showed you to be.

Increasingly, there is no such thing as a fixed price from which sale items deviate. Following a series of court judgments against other retailers advertising bogus sale prices, Amazon has tended to drop most mentions of “list price” or recommended retail price, and use instead the reference point of its own past prices.

This looks a lot like the beginning of the end of John Wanamaker’s mission to establish “new, fair and most agreeable relations between the buyer and the seller” and to establish something closer to a comparison site that works both ways – we will be looking for the low-selling retailer, while the retailer will equally be scanning for the high-value customer. The old criticism that consumer societies know the price of everything and the value of nothing is under threat: even the former certainty is up for debate.

Store wars: the future of shopping

Vending machines 2.0



The Luxe X2 Touch features facial recognition software to identify users and suggest purchases based on spending history or context, such as iced drinks on a hot day. It can also prevent children from buying cigarettes or alcohol, or keep hospital patients away from sugary or salty foods.

The Amazon Go store



The Seattle store is the first to eliminate checkout lines by using a mobile app. Customers simply scan their smartphone on entry and pick up what they want. Computer vision technology keeps track of their purchases and their Amazon account is debited when they are finished.

Automated assistants



US home improvement store Lowe’s is introducing a new employee into its workforce: a robot that finds products for you. The robots, which will start roaming the aisles in San Jose, California, during the course of the year, speak several languages and can answer customers’ questions.

Beacons of hope



Beacons are small, battery-operated, wireless devices that transmit a bluetooth signal to an appropriate smartphone app. This technology can be used to nudge customers into the store, suggest offers and purchases, and also stores information to monitor customer behaviour.

The Starship delivery bot



Conceived by the founders of Skype, this is designed to deliver anything from groceries to books. The autonomous six-wheeled robot is speedy and saves you from lugging shopping bags, although it is questionable how safe it will be roaming the streets of Britain.

Pop-up shops

Facebook Twitter Pinterest Photograph: Katy Dillon



Got an idea for a shop, but worried about the commitment of opening one? Appear Hear is a website that helps you find short-term retail space and is designed to connect retailers, entrepreneurs, brands and designers alike. It has so far been used by top brands including Nike and Moleskine.