In early 2014, hackers gained control of a small number of eBay employee log-ins and breached a company database containing customer information. Inside it were phone numbers, home addresses and dates of birth, among other details that users entrusted the online auctioneer to protect. The breach would prove so vast that eBay was forced to ask more than 100 million people to reset their passwords.

PayPal, then the digital-commerce arm of eBay, sought to ensure it wouldn’t have to ask its users to do the same.

So, PayPal did something it acknowledges is a “regular course of business.” It tasked a middleman to buy data from criminals: a small sample of 32 accounts offered online for about $100.

Such sales are widely known within cybersecurity circles, but purchases of data advertised as stolen are seldom discussed publicly, in part because of the messy ethical questions they raise.

A dozen people — including several current and former senior executives at major Silicon Valley mainstays and cybersecurity vendors — detailed to The Chronicle the process and its importance to counterintelligence investigations.

Companies that engage in the practice include top technology firms and banks, which reportedly bought back stolen credit and debit card numbers in the wake of the breach at Target in 2013.

Criminal ecosystem

According to insiders, the tactic requires companies and intelligence vendors to infiltrate a complex criminal ecosystem of chat rooms and forums where stolen data are bought and sold, and participants are often vetted for their underworld bona fides.

The payments are often so small — or so well hidden — that there’s little evidence they ever took place.

At PayPal, a company spokeswoman said, the practice helps “identify larger sets of compromised accounts that can be used to support law enforcement investigations and to protect customer accounts.”

The now-separate San Jose company’s payments to criminals total from $50 to $75 a year, she said, yielding less than a dozen accounts.

It is just “one of many (actions) that helps keep our customers secure,” she said. PayPal’s data were not said to be compromised in the eBay breach.

Some companies use their own employees to go undercover and broker these deals. Others, like PayPal, rely on intermediaries.

In those cases, such purchases are often part of a larger contract for countersurveillance work, ensuring corporate accounting departments don’t need to green-light individual black market buys.

The practice allows companies to sort out whether the data being hawked on dark corners of the Internet are actually as advertised — or whether a hoaxer is simply trying to profit by tricking other criminals into buying useless data from past breaches.

If the information is legitimate, such buys can determine whether the data were stolen directly from users or third-party partners, or if the leak was the result of a much more serious company gaffe.

Small purchases to find leak

In the rare case of a breach, information taken from inside a company can sometimes include subtle clues about where it came from, leading to other records that may have been compromised or the people involved.

Such hints can be incredibly useful to a company facing a potential crisis, despite the fact it means funneling money to criminals.

“From a counterintelligence point of view, I don’t have an issue with it,” said Rodney Joffe, senior vice president and technology fellow at information services firm Neustar, who has advised the White House on cybersecurity. “It’s a legally gray area, but for me it’s not really morally gray.”

Joffe said he sees it as no more problematic than someone buying back a stolen wallet or car.

Others repudiate the practice.

“There is a line, and you don’t want to cross that line,” said Kevin Haley, a director of product management for Security Response at Symantec. “Buying things from criminals is getting across that line.”

Though many companies are wary of being in receipt of stolen goods, it’s the intent behind the purchases that matters, attorneys who specialize in information security said.

“Ultimately, there is nothing wrong with it other than the fact that it would require you to use some level of subterfuge,” said Jon L. Praed, head of the Internet Law Group of Arlington, Va. He said his firm has represented clients who have engaged in such buybacks in the past.

‘A common practice’

Paying criminals for access to a company’s own user data seems no different, said CipherLaw Group information security attorney James Denaro, than paying a hacker to unlock a computer infected with ransomware — a practice that in some circumstances even the FBI has recommended.

The FBI and the Department of Justice declined to discuss companies buying data from the criminal underground.

For companies that turn to outside vendors to conduct such buybacks, there are only a few boutique cybersecurity firms that offer the service, insiders told The Chronicle.

None publicly advertises it.

Buying stolen data twice

Alex Holden, the chief information security officer of Hold Security in Milwaukee, called it “a common practice” that “unfortunately happens too often.”

Holden, whose firm specializes in surveilling criminals on the Dark Web, said he’s been asked by clients multiple times to perform such buys.

“I’ve said over a hundred times no to these customers,” he said. “Because it may not be consistent with our practice, it may not be consistent with the law or maybe simply not of any use.”

Despite Holden’s personal objections, he said he’s purchased stolen data twice, spending roughly $30 and $2,000 on criminally obtained information for “brand names that you and I and everyone else has heard of.”

One was in financial services, the other a tech firm, he said.

In both of these situations, Holden said, he was “100 percent” sure the information he was buying had been stolen directly from his clients.

Companies turn to people like Holden “because it requires a relationship,” he said.

“You still can’t go on the Internet, on the Deep Web, and contact a hacker and say: ‘I want to buy this,’” he said. “The companies themselves, they (sometimes) don’t have the time, or resources, or knowledge.”

Online markets where stolen data are bought and sold are tough to crack even for professionals.

Navigating vetting process

Some are heavily policed by the criminals themselves, requiring buyers to provide references proving they are criminals, too. Some go so far as to monitor IP addresses previously linked to law enforcement or other outsiders.

Those with enough street cred to make it through the criminals’ vetting process risk losing access the second they play their hand. If thieves notice that data that they’ve shown as a sample or sold to a specific customer become unusable, that person may be blacklisted.

A law enforcement agent conducting an undercover buy of stolen credit cards in 2015 was locked out by a black market’s internal security system, which identified him as a “pig,” according to the authoritative security blog Krebs on Security.

Despite the challenge, security experts see further upsides in such cloak-and-dagger operations.

By infiltrating these markets and fooling bad actors into giving up their data, they undermine two things: the potential monetary value of the data being sold by criminals and the trust that those criminals place in the platforms they use.

“The goal here is to ruin the economic advantage for stealing user name, password pairs,” said a senior security executive at a Silicon Valley giant who did not have permission to speak on the record.

“But you can’t fight these guys by hiding in the corner and pretending it doesn’t exist. You have to engage.”

Buying black market data — while cheap and effective — is only one tactic employed by companies.

It’s the equivalent of doing a biopsy. It may be necessary to discover the scope of the cancer in the body. But it’s not the primary treatment used to fight the disease.

Using canary credentials

In some cases, there are ways for companies to obtain stolen information without having to pay for it.

On some online forums, for instance, criminals brag about what they’ve done and even offer samples for free to prove their validity, multiple industry experts said.

Some companies place canary credentials inside databases — say, user names and passwords for people who don’t exist.

If anyone tries to log in with those credentials, it’s clear something is wrong.

Techniques like these may be more palatable to companies concerned about the ethical implications of paying criminals.

It’s not a practice recommended for everyone, said Katie Moussouris, a former security strategist at Microsoft and the chief policy officer at HackerOne.

“Doing business with people who traffic in stolen goods is not the best way to build a sustainable ecosystem,” Moussouris said.

Companies must ask if it’s worth it to safeguard customers — and bottom lines.

“When you funnel any money of any kind into a market that has to do with cybercrime, you are giving it to some very bad people,” said Ryan Kalember, a senior vice president of cybersecurity strategy at Proofpoint.

But, he added, it’s not as if there is a “pope for cybersecurity” who passes down edicts on what’s ethical and what’s not.

Sean Sposito is a San Francisco Chronicle staff writer. E-mail: ssposito@sfchronicle.com Twitter: @seansposito