Companies are waging an invisible data war online. And your phone might be an unwitting soldier.

Retailers from Amazon and Walmart to tiny startups want to know what their competitors charge. Brick and mortar retailers can send people, sometimes called "mystery shoppers," to their competitors' stores to make notes on prices.

Online, there's no need to send people anywhere. But big retailers can sell millions of products, so it's not feasible to have workers browse each item and manually adjust prices. Instead, the companies employ software to scan rival websites and collect prices, a process called “scraping.” From there, the companies can adjust their own prices.

Companies like Amazon and Walmart have internal teams dedicated to scraping, says Alexandr Galkin, CEO of the retail price optimization company Competera. Others turn to companies like his. Competera scrapes pricing data from across the web, for companies ranging from footwear retailer Nine West to industrial outfitter Deelat, and uses machine-learning algorithms to help its customers decide how much to charge for different products.

Walmart didn’t respond to a request for comment. Amazon didn’t answer questions about whether it scrapes other sites. But the founders of Diapers.com, which Amazon acquired in 2010, accused Amazon of using such bots to automatically adjust its prices, according to Brad Stone's book The Everything Store.

Scraping might sound sinister, but it’s part of how the web works. Google and Bing scrape web pages to index them for their search engines. Academics and journalists use scraping software to gather data. Some of Competera’s customers, including Acer Europe and Panasonic, use the company’s “brand intelligence” service to see what retailers are charging for their products, to ensure that they are complying with pricing agreements.

For retailers, scraping can be a two-way street, and that’s where things get interesting. Retailers want to see what their rivals are doing, but they want to prevent rivals from snooping on them; retailers also want to protect intellectual property like product photos and descriptions, which can be scraped and reused without permission by others. So many deploy defenses to subvert scraping, says Josh Shaul, vice president of web security at Akamai Technologies. One technique: showing different prices to real people than to bots. A site may show the price as astronomically high or zero to throw off bots collecting data.

Such defenses create opportunities for new offenses. A company called Luminati helps customers, including Competera, mask bots to avoid detection. One service makes the bots appear to be coming from smartphones.

Luminati’s service can resemble a botnet, a network of computers running malware that hackers use to launch attacks. Rather than covertly take over a device, however, Luminati entices device owners to accept its software alongside another app. Users who download MP3 Cutter from Beka for Android, for example, are given a choice: View ads or allow the app to use "some of your device's resources (WiFi and very limited cellular data).” If you agree to let the app use your resources, Luminati will use your phone for a few seconds a day when it’s idle to route requests from its customers’ bots, and pay the app maker a fee. Beka didn’t respond to a request for comment.

The ongoing battle of bot and mouse raises a question: How do you detect a bot? That’s tricky. Sometimes bots actually tell the sites they’re visiting that they’re bots. When a piece of software accesses a web server, it sends a little information along with its request for the page. Conventional browsers announce themselves as Google Chrome, Microsoft Edge, or another browser. Bots can use this process to tell the server that they’re bots. But they can also lie. One technique for detecting bots is the frequency with which a visitor hits a site. If a visitor makes hundreds of requests per minute, there’s a good chance it’s a bot. Another common practice is to look at a visitor’s internet protocol address. If it comes from a cloud computing service, for example, that’s a hint that it might be a bot and not a regular internet user.