Stealing from a cashier-less store (without you, or the cameras, knowing it)

Yoshimasa Takahashi visits Standard Market, San Francisco's first cashierless and checkoutless grocery store, Sept. 7, 2018. Shoppers who have downloaded the store's app can grab items and simply leave.; cameras identify the shopper and the items, and determine when said items leave with said shopper. At least, that's the idea. less Yoshimasa Takahashi visits Standard Market, San Francisco's first cashierless and checkoutless grocery store, Sept. 7, 2018. Shoppers who have downloaded the store's app can grab items and simply leave.; ... more Photo: CAYCE CLIFFORD, NYT Photo: CAYCE CLIFFORD, NYT Image 1 of / 43 Caption Close Stealing from a cashier-less store (without you, or the cameras, knowing it) 1 / 43 Back to Gallery

One recent afternoon, San Francisco’s newest grocery market was trying to figure out whether I would buy, steal or leave behind a bag of white cheddar popcorn — and so was I.

On its side: 27 cameras along the ceiling and a wealth of behavioral data.

On my side: crippling indecision.

This month, the city got its first cashier-less store, Standard Market. Shoppers who download the store’s app can go into the 1,900-square-foot space, grab items and simply leave. There is no check-in gate or checkout swipe. Ceiling cameras identify the shopper and the items, and determine when said items leave with said shopper. Or, at least, that’s the idea.

The startup behind this is Standard Cognition, which has raised $11.2 million in venture capital and formed partnerships with four retails chains around the world. This first market is a prototype to showcase the technology and work on the bugs. The ambitious goal is to add the tech in 100 stores a day (each day!) by 2020.

Five of the seven founders came from the Securities and Exchange Commission, where they built artificial intelligence software to detect fraud and trade violations, before starting Standard Cognition in 2017. Now these fraud experts are working to discern something equally complicated: whether I am stealing a snack.

Standard Market is the latest entry in the emerging fray of retail automation, where companies are throwing cameras, sensors and machine learning into grocery stores to replace the checkout line. In January, Amazon opened its first cashier-less Go market in Seattle to the public; it has since opened more of the stores. In China, experiments in cashier-less stores abound, using radio frequency identification tags and a self-checkout process that involves scanning a Quick Response code or your face.

Standard Cognition’s approach is different. It relies exclusively on the ceiling cameras and artificial intelligence software to figure out what you are buying. The cameras document shoppers’ movements, speed, stride length and gaze. The store knows when I glance at a poster and for how long. It knows if I slowed down, grabbed a chocolate bar and put it back. It knows if my body is facing the dried mangoes but my face is set on the popcorn.

And it knows (or is trying to know) when I am planning to steal.

The goal is to predict, and prevent, shoplifting, because unlike Amazon’s Go stores, which have a subway turnstile-like gate for entry and exit, Standard Market has an open door, and the path is clear.

“We learn behaviors of what it looks like to leave,” said Michael Suswal, Standard Cognition’s co-founder and chief operating officer. Trajectory, gaze and speed are especially useful for detecting theft, he said, adding, “If they’re going to steal, their gait is larger, and they’re looking at the door.”

Once the system decides it has detected potential theft behavior, a store attendant will get a text and walk over for “a polite conversation,” Suswal said.

Predicting theft requires a lot of data about shoppers, much of which does not exist yet — “or at least no one is willing to give it to us,” he said.

So a few days before Standard Market opened, Standard Cognition hired 100 actors to shop there for four hours. In Japan, the team has worked with a convenience store chain, whose name it has not disclosed, in a very useful data collection effort.

Standard Cognition said that unlike facial recognition, it did not collect biometric information, a possibility that has troubled privacy experts watching the technology evolve.

The growth of cashier-less technology could hurt the American labor force; there are nearly 5 million retail sales workers in America. But as Suswal has pitched Standard Cognition’s technology, he said, he has found that most shop owners are not looking to replace workers. Instead, they want their workers wandering the stores more, in hopes of luring shoppers back into brick-and-mortar retail.

“They all talk about new services, making shopping more fun, making it worthwhile to shop offline,” Suswal said.

And they talk about data. While a store owner can look at receipts to see who bought a generic ketchup, cashierless technology can help tell if the shopper first picked up a Heinz bottle and how long he or she looked at it. Basically, now an owner can see what a customer did not buy.

On a recent Friday, a line stretched down the street from Standard Market as a bouncer at the door took in one shopper at a time for the automated experience. The store is in the Mid-Market neighborhood, between Chanvi Eatery, a Pakistani restaurant, and Huckleberry Bicycles, a high-end bike shop. People outside were downloading the app and typing in their credit card numbers.

Walking out was Yoshimasa Takahashi, 32, who works nearby in finance. A receipt popped up on his phone. It said he had bought noodles and Kraft Macaroni & Cheese — except, he had not bought the Kraft Macaroni & Cheese.

“I was playing with it but didn’t take it out,” Takahashi said, smiling at his win over the tech.

The bouncer gave him a refund.

Store hours are short for the next few weeks — the store will be open only half days on Wednesdays and Fridays while the tech is tweaked. For now, the selection of food is extremely limited. The store has only 25 square feet devoted to food because, the founders said, they have not yet gotten the permits required for more. So there is an odd assortment of items — Fritos, Apple Jacks and Starbucks Frappuccinos — that leans heavily toward dorm-room-style snacks.

To shop, I opened my phone, which flashed blue, letting the store know I had entered. I wandered, throwing items into my tote. Then I left.

Outside I found Suswal. A minute went by, and a notification popped up on my phone with my receipt: one white cheddar popcorn and one roll of toilet paper for a total of $1.19.

In fact, I had left with two bags of popcorn. I had toyed with the second bag, debated buying it, considered my dinner plans, put it back and finally took it with a quick impulsive grab. The system missed it.

“That shouldn’t happen,” Suswal said. And yet it did. He shrugged and said I had won it.

MBA BY THE BAY: See how an MBA could change your life with SFGATE's interactive directory of Bay Area programs.

So I left with the extra 99-cent bag of popcorn, and I did not feel bad, really. Soon, Standard Cognition and others will probably get better, will perfectly detect where that snack went, and my movement will be subsumed and predicted by artificial intelligence’s endless data maw.

But for now it’s not quite good enough. And I’m covered in crumbs.

Nellie Bowles is a New York Times writer.