Bruno De Hogues | Photolibrary | Getty Images

When a Google executive found a high-end Bluetooth headset selling at a steep discount on the company's shopping site earlier this year, he didn't consider that the deal may have been too good to be true. He ordered the product and waited. And waited. The expected delivery date passed. He tried calling the website's customer service number. It was disconnected. The headset never arrived. The money was lost. In reality, the merchant wasn't based in the U.S., as its website indicated. Google Shopping had sent the buyer about 8,000 miles away, to a bogus seller in Vietnam who took the Google employee's credit card information with no intention of ever sending out a headset. The prospective buyer kicked the case over to his co-workers to start an investigation. But instead of simply banning the bad actor from listing new products, Google Shopping's trust and safety team initiated a global probe that ultimately tracked down 5,000 merchant accounts wrapped up in a sophisticated scheme to defraud users. "I think we caught them right at the tip of when they were trying to scale up," Saikat Mitra, Google Shopping's director of trust and safety, told CNBC.

The story, which Mitra is sharing publicly for the first time, reflects Google's never-ending battle against scams, a fight that requires engineers and their increasingly sophisticated machine learning tools. It also illustrates the risks that consumers face as Google aggressively tries to win back product searches from Amazon and stay relevant in the future of e-commerce. Although Google Shopping may look like a marketplace, it really isn't. Amazon and eBay operate shopping platforms that connect sellers with buyers and offer protections like money-back guarantees. Google, by contrast, sends shoppers off its site after they click on an item, and thus has no visibility into what happens after the transaction. Nor does Google take responsibility for scams. If you order something from a sketchy website you found through Google Shopping and don't pay through a service like PayPal, which has its own robust fraud checks, you're likely out of luck. Like Google's dominant search engine, Google Shopping is an advertising site. If you want a new camera, a pair of Comme des Garçons sneakers or a sequin backpack, and you start your search on Google, you'll be greeted with a big, scrollable list of ads. If you then click on Google's "Shopping" tab, you'll be directed to a more robust page where you can search products by price, color, or size. If you didn't notice the little "Sponsored" messages tag in the corner, you might not recognize all these listings as advertisements.

That model keeps Google in the high-margin business of online ads and away from the costly operations of holding inventory and managing a massive logistics and shipping network. (Google also has a separate shopping product, called Express, that only partners with specific retailers and does guarantee purchases.) E-commerce is a market Google can't afford to lose. Digital agency Merkel said its clients increased spending on Google Shopping by 40 percent in the first quarter from a year earlier, and ad impressions on the shopping site surged 47 percent. Shopping is becoming even more important as the battle for users turns to voice-controlled speakers in the home, a market where the Amazon Echo has an early lead over Google Home. Without owning and operating the platform, Google has less control over who sets up shop. For a business to create a Google Shopping ad, it has to follow a variety of policies meant to keep consumers safe. But, as the Bluetooth headset example illustrates, there's still room for fraudsters to game the system.

What Google's doing to keep its platform safe

That's where Mitra has work to do. Once his team was made aware of the Google employee's experience, it got to work using advanced data algorithms and looking for subtle connections between merchant accounts, Mitra said. Through machine learning, Google discovered a cluster of merchants with similar data and online habits. More than 5,000 well-designed websites that looked like they belonged to U.S. based companies were actually being run by a ring of scammers in Vietnam. Google booted those merchants from its ads product and was able to recognize when members of the same group tried to rejoin the platform. Mitra's team determined that the merchants had likely been preparing sleeper accounts for at least six months to make them look legitimate. When one shop got rejected, there were many more that would be on standby to fill the ranks. "The coordination among them, and their persistence and desperation to get back on the system was stark," Mitra says. "That was really unique." Scamming is nothing new to the internet, particularly when it comes to commerce. Counterfeit goods are a common problem across all platforms and something that Amazon has struggled with for years. In the past it was common for people to order goods through Google Shopping only to receive shoddily made products that looked nothing like the pictures. Those practices were hard to police, Mitra says, but Google addressed the issue by integrating seller reviews and requiring that all merchants meet a minimum review score.