What if the advertisers who pay Google most of the money it earns are getting a negative return on their investment, to the tune of -63%?

That’s the question raised by a new paper that uses internal eBay data to assess whether the company’s investment in search-engine marketing is actually paying off. When major brands spend to make sure their results are at the top of a Google search page for certain keywords, they like to think they’re effectively targeting the customers looking for their products. But what if they are paying for something that they could be getting for free?

That’s what the following chart implies. When eBay experimented by shutting off paid search traffic connected to the company’s name, people found their way to the site anyway: “Since users intend to ﬁnd eBay, it is not surprising that shutting down the paid search path to their desired destination simply diverts traffic to the next easiest path, natural search, which is free to the advertiser.”

no-caption

That’s not what Google (or the people paying Google for targeted advertising) want to hear. Next, eBay wanted to look at keywords that don’t mention the company’s name; since it’s possible that those do direct more traffic to their site. EBay purchases ads on an astonishing 100 million or more keywords (for example: “used gibson les paul”) and was able to shut down advertising in randomly selected geographic areas to test their efficacy. This experiment showed some value in Google’s product: Traffic to eBay declined by 2%. But clicks aren’t sales, and the researchers think that most of those potential customers still would have found their way to eBay.

But there’s a hope for search-engine marketing: eBay is a mature, well-known company, and much of its potential customer universe is aware of it already. The researchers found that Google’s marketing was most effective at reaching customers who hadn’t heard of the company or its products. That might suggest that search-engine marketing is in fact useful for younger companies seeking to make their names. It also plays into one theory of why eBay pays so much for search engine marketing: Its a prisoner’s dilemma, where all competitors would be be better off not bidding on keyword ads—but if one does, they all must seek an advantage.

But even if this advertising is devalued only for the biggest companies, that’s still an issue for Google: The top-ten spenders on online advertising spent $2.36 billion out of a total of $31.6 billion in 2011. If they start cutting back, it will make a dent.

Of course, this is one piece of research, based on one company’s data, and there may be better ways to value the kind of targeted marketing that Google provides. A Google spokesman told Quartz that Google’s internal research shows that 89% of visits (pdf) to an advertiser’s site would not have occurred absent their campaign, and even when an advertiser’s website was featured as a top natural search result, 50% of traffic to their site from Google comes from their ad. “Since outcomes differ so much among advertisers and are influenced by many different factors, we encourage advertisers to experiment with their own campaigns,” the company said in a statement.

Still, this paper could offer some insight into why Google is spending so much money on satellite internet and driverless cars: The need for a back-up plan to its advertising business (which provides 95% of its revenue) is perhaps more pressing than the company might like to admit.