“Alternative data” is going mainstream as fund managers are projected to spend more than $1 billion this year to beat the market averages and stave off the rise of low-cost passive investing.

The explosive growth in the amount of alternative data sets — an array of information gleaned from the web, satellites and even consumers’ wallets — hasn’t proved to be a panacea. But what was once considered the domain of quantitatively oriented hedge funds and other well-heeled investors has become a must-have for traditional asset managers struggling to deliver market-beating returns.

Traditional investment managers face three options, said Octavio Marenzi, chief executive officer of Opimas, a capital-markets-focused management consulting firm. The first option is to embrace alternative data and effectively adopt a more quantitative approach. The second is to go into passive investing, tracking an index and abandoning research altogether. “And the third option is to go home and give up,” he told MarketWatch.

Also see:How a mix of psychotherapy and financial advice could solve your money issues once and for all

Investors have been embracing the first option, if spending is anything to go by. While sources offer varying figures on the size of the alternative-data industry, there’s widespread agreement that business is growing rapidly with no sign of an imminent slowdown. Schroders, Fidelity Investments, Capital Group, Neuberger Berman, T. Rowe Price and Invesco are among the companies that have built data teams or brought on a number of alternative-data-focused full-time employees, a 2018 study by AlternativeData.org showed.

Explosive growth in spending

Total spending on alternative data by buy-side firms — mutual funds, hedge funds, pension funds, private-equity firms and the like who buy securities or other assets for their own or their clients’ accounts — will jump from $232 million in 2016 to a projected $1.1 billion in 2019 and $1.7 billion next year, according to AlternativeData.org, an industry trade group supported by data provider YipitData. The group reckons there are now 447 alternative-data providers.

More broadly, Opimas in 2017 projected total spending, including outlays for data sources, data science, IT infrastructure, data management and systems development, would rise from more than $4 billion to north of $7 billion by 2020 — a prediction that appears on track, according to Marenzi.

It’s an industry described as being very much in its infancy. Many investment firms are still “getting their feet wet” when it comes to using data from alternative sources, said Richard Johnson, principal for market structure and technology at research firm Greenwich Associates.

The firm estimates the total amount budgeted for alternative data by investment managers stands at around just $400 million to $500 million, but is set to grow rapidly as users follow through on plans to ramp up usage. A recent study by the firm found alternative-data budgets swelled by 76% in 2017 and 52% last year.

The growth in alternative data has a number of drivers, most of which offer few surprises. After all, the advent of machine-learning technology and the drop in the cost of computing power have made it much less expensive to crunch ever-larger sets of data — a phenomenon that’s led a number of traditional, active asset managers to increasingly incorporate quantitative investing techniques into fundamental-oriented models.

Wave of new information

So what makes data “alternative”? Investors, vendors and research firms use the term to refer to data that’s outside the realm of the usual government- or company-provided data on the economy, earnings and other traditional metrics. The growth of the internet and the digital economy, along with technological advances in computing and data crunching, have created a tidal wave of information and, crucially, the ability to process it quickly.

Video:How ‘alternative data’ gives an investing edge — if you can afford it

There’s also a gee-whiz factor. News reports often focus on cloak-and-dagger-style data sources, such as the aircraft-tracking service that spotted a corporate jet owned by Occidental Petroleum Corp. OXY, -3.94% at an Omaha airport in April, triggering hunches that executives of the oil giant were in talks with Warren Buffett’s Berkshire Hathaway Inc. BRK.B, +0.82% BRK.A, +0.43% in an effort to find an ally in their bid to acquire Anadarko Petroleum Corp. Buffett indeed stepped up to the plate two days later with a $10 billion investment in Occidental.

Satellites that can count the number of cars in retailers’ parking lots or detect when manufacturers are adding shifts or reducing their workforces have also garnered plenty of attention. In the commodity world, Genscape Inc., a unit of Daily Mail & General Trust PLC DMGT, -0.15% , uses helicopters to beam infrared signals at oil-storage tanks to gauge inventory levels ahead of government data. Data-analytics provider Verisk Analytics VRSK, +2.31% last month signed an agreement to acquire Genscape from DMGT for $364 million in cash.

Other popular products include data sets based on web scraping. By tracking prices and inventory on public retail sites, for example, investors can glean insights into the performance of brands and companies, Greenwich Associates said in a recent report.

Crowd-sourced data and social-media sentiment are also popular alternative-data forms. And investors have been willing to pay up for data from credit-card providers and point-of-sale systems. Such data sets collect figures straight from the source, Greenwich Associates noted, with some companies forming a panel of consumers who agree to share credit-card statements, while others work directly with technology providers that handle retail payments.

Some of the industry’s best-known players include alternative-data aggregator Quandl, acquired in December by Nasdaq Inc. NDAQ, +2.25% ; Orbital Insight, which applies artificial-intelligence technology to satellite and other geospatial data sources; Genscape, the outfit being acquired by Verisk from Daily Mail & General Trust, which provides an array of data based on its network of monitors across the commodity and energy sectors; financial-research platform Sentieo; and Dataminr, which gleans social-media feeds for insights into breaking news events and market sentiment.

No silver bullet

Does it pay? Users think so. The Greenwich Associates study said 72% of investment firms reported that alternative data enhanced their signal, with over a fifth of respondents saying they got over 20% of their alpha — industry lingo for the ability to beat the market — from alternative data.

But analysts and data users caution that the newfound sources of insight aren’t by themselves a silver bullet capable of curing active managers’ performance ills.

“Many investors still have unrealistic expectations,” said Yin Luo, managing director for quantitative analysis, strategy and economics at Wolfe Research, in an interview.

The problem is that the outside-the-box nature of the information can lead to the impression that merely shelling out for the right data set will point the way to profitable stock picks and market-beating performance. Veteran users say it’s more realistic to view alternative data as an additional tool in the toolbox, not a one-shot replacement of the investment process.

“I think of alternative data as very much an additional input into our process,” said Katherine Glass-Hardenbergh, associate portfolio manager at Acadian Asset Management, a quantitative investment-management firm with around $96 billion in assets under management that’s been using alternative data sets for years.

To put the role, and potential, of alternative data in perspective, it helps to look back at an event that helped drive its adaptation.

It was a rough patch for quantitative traders that marked an important turning point, according to Luo. The firms, which rely almost exclusively on computer models to generate trading ideas, suffered sharp losses in 2007 after crowding into similar trades, an episode known as the “quant quake.”

That inspired quantitative managers to look for data outside their traditional toolboxes toward sources that weren’t being widely used, he said.

Alternative data sets should generally be expected to deliver performance similar to traditional data sets, he said. So while the data aren’t necessarily “better” when it comes to providing performance-enhancing insight, they are “different,” Luo explained.

It echoes the investment-industry adage that describes asset diversification as the only “free lunch” in finance, he said. The same principle applies to data, in that the usage of alternative sources can provide data inputs that are also uncorrelated.

Separating signal from noise

So there are plenty of inputs. But how do investors go about filtering the signal from the noise?

For big asset managers, it’s often done in-house. Data scientists remain in high demand in the investment-management industry. Their job is often to vet the myriad sources of data available, and then figure out what works and how to deploy it.

Vendors, too, are fueling demand for data scientists as they build off-the-shelf data sets. There’s also a role for sell-side firms — brokerages and companies that sell investment services to asset managers — with investors looking to them to provide research based on alternative data much in the same vein they provide other traditional market research.

But those choices can lead to other dilemmas. Consider a vendor touting an off-the-shelf data set as a back-tested, highly reliable signal. Even if taken at face value, there’s the concern the vendor will be selling that data set to a wide range of investors, which means its advantage could soon be arbitraged away, Glass-Hardenbergh said. Or there may be little “visibility” into what makes up the data.

“What we’re generally looking for is something a little bit more raw, a bit more unprocessed, where we can really understand what the data is,” she said. “We can dive into it, we can do our analysis, we can do cleaning, we can apply it and back-test it” with the goal of figuring out whether the data will provide a useful signal.

At the same time, the firm does consider on a case-by-case basis whether it makes sense to hire a vendor to provide data in a particular area, such as web scraping, or to do it in-house. “It’s looking at where we can add our edge and figure out what’s really going on,” she said.

Overall, there are probably only a few dozen shops capable of taking in substantial amounts of raw data, which means many investors will continue to turn to established market vendors, including traditional data providers like Bloomberg, IHS Markit and Refinitiv, said Johnson at Greenwich Associates.

Meanwhile, active asset managers’ attempts to beat their benchmarks and justify their fees amid an onslaught from low-cost passive investors will keep demand for alternative data sets strong, according to Johnson.

“There’s also a branding thing; people want to tell their investors that they’re using alternative data because the asset owners are going to start asking these questions,” he said.

Four popular types of alternative data and how they’re used

Web scraping: The most widely used form of alternative data, according to research firm Greenwich Associates, web scraping collects data from targeted websites in a bid to gain information on brands, companies and corporate activity. Types of web-scraped data in high demand include job listings and employee-satisfaction rankings, which can offer clues to a company’s growth prospects, according to the Greenwich report. Data providers also pay attention to product rankings and sales promotions, looking for clues to the performance of individual brands and companies.

Satellites and aerial surveillance: Satellite images can be used to count cars in parking lots, a potential source of insight into sales activity for retailers or output at factories. Industry professionals caution, however, that satellite and other types of aerial surveillance data are best supplemented with other types of data able to provide more detailed estimates of actual foot traffic when it comes to gauging retail sales. Satellites are also used to track ships, monitor crops and detect activity in ports and oil fields.

Credit-card data: Investors are looking for insights straight from the cash register. Some data providers have put together large panels of consumers who agree to share their credit- and debit-card activity. Panels made up of more than 3 million consumers are considered big enough to be useful, according to industry trade group AlternativeData.org, which noted that the resulting data are among the most expensive on offer and are often used to track retail revenue.

Sentiment: Social-media feeds, news flow, corporate announcements and other items are monitored and analyzed for clues to sentiment on stocks, products and the economy. Data sleuths are also analyzing language used by executives on earnings calls and elsewhere for clues to corporate prospects.