James Martin/CNET

When news broke this week that Google was letting Gmail app developers scan and even read your email, we heard what's become Silicon Valley's usual excuse: This is what you signed up for.

Don't like how Facebook shares your data with third-party developers? Too bad, it's right there in the privacy policy. How about how Twitter tracks your activity across websites? The company spelled that out in its data policy. (What? You didn't bother reading it?) And maybe you were upset when you learned last year that Unroll.Me was selling information taken from your email inbox. The company's CEO found that "heartbreaking," but that's how businesses make money from a free service.

Privacy advocates have been pushing back against the industry's way of doing business for years. Marc Rotenberg, executive director of the Electronic Privacy Information Center, said in an email that companies including Google and Facebook need to take responsibility for how software developers leverage your data.

"It is not reasonable, practical or efficient to expect users to know how third-party companies will make use of their personal data," Rotenberg said. "Just like Facebook, Google bears responsibility for the misuse of personal data by app developers."

Now playing: Watch this: Keep your Gmail from prying eyes

You can opt out of data sharing in some cases -- or you can stop using the service. Still, it's hard not to feel as if our privacy is being violated. Consumers' reaction is becoming a major challenge for tech companies as they face lawmakers, lawsuits and the threat of regulation over data policies they say they've been telling us about all along.

Fatemeh Khatibloo, an analyst at Forrester, said tech companies need to make clear to users what the tradeoff is for receiving services for free. "It should very clearly say, 'We provide this service for free because we monetize your data in other ways,'" Khatibloo said.

The issue resurfaced this week after an examination by The Wall Street Journal found that "hundreds" of outside software makers could scan your inbox through third-party Gmail apps. (Gmail has more than 1 billion monthly active users.) In some cases, developers' employees had access to thousands of Gmail users' emails.

One developer, Return Path, a marketing company that offers free email organization tools, let its workers read about 8,000 user emails two years ago to help develop the company's software. Another free app, called Edison Software, which helps users manage their email, let its employees read thousands of Gmail messages to train the "Smart Reply" feature in its app, the Journal reported.

Both Return Path and Edison Software said they've now stopped the practice. But both also defended giving employees access to Gmail data, saying that humans need to see such data to build their software. "As anyone who knows anything about software knows, humans program software – artificial intelligence comes directly from human intelligence," Return Path said in a blog post on its website.

Giving developers access to your data might part of the terms of service -- whether that's for tech giants like Google or Facebook, or 20-person startups -- but people often don't realize just exactly what they're agreeing to.

Google vets all apps it allows to request user data through Gmail, according to a blog post published Tuesday by Suzanne Frey, Google's director of security, trust and privacy for Google Cloud. "We strongly encourage you to review the permissions screen before granting access to any non-Google application," Frey said. To review which apps already have access to your account, you can go through the security check-up on the Google Account page linked to your Gmail account.

Google said last year it would stop scanning user emails for data that helps marketers target ads. Since then, data privacy from third-party app developers has been a global hot-button topic. In March, Facebook acknowledged that Cambridge Analytica, a digital consultancy that had ties to the Trump presidential campaign, improperly accessed personal information on up to 87 million of the social network's users.

Khatibloo pointed out that the Gmail controversy affected more than just Gmail users. She noted that if she'd emailed someone who was using Return Path or Edison, its employees could have read her emails, too.

"I didn't agree to have my data evaluated by Return Path, but by signing up for the service, somebody I sent an email to has opted me into it," Khatibloo said. "And I think that is the big privacy violation."

Google's Gmail controversy: Third-party app developers can read your mail.

Cambridge Analyitica: Everything you need to know about Facebook's data privacy scandal.