New data privacy rules are pushing marketers to unload millions of dollars in liability on the agencies that help them buy their media, forcing the shops to take on new levels of financial risks and adding a layer of tension in client-agency relationships.

At one large global brand, a marketing executive limited a review for its account to agencies that agreed to pay more than $15 million in fines if they were involved in a data breach or violation. The mandate was less about avoiding potential fines than making sure the brand’s agency took data privacy seriously, the executive said.

The focus on data privacy has heated up following the arrival of the European Union’s General Data Protection Regulation, which in many cases requires publishers and advertisers to obtain consumers’ explicit consent before using their information to tailor advertisements. A violation of the regulation, which applies globally to companies that deal with the personal data of people located in the EU, could lead to a fine of up to 4% of a company’s revenue for the prior year.

While risk-averse advertisers want to avoid such fines, a number of brands are more concerned about the impact of a privacy violation on their brand reputation. They hope imposing more liability on their agencies will push them to take every precaution when collecting and using consumer data.

A data breach could be “massively damaging” to a company’s reputation, said the marketing executive, who spoke on condition of anonymity to discuss sensitive business matters. In the aftermath, consumers might not “register for an ad and give us their private information,” the executive said.

Clients are seeking differing terms based on their size and the value they place on their data and brand reputation, said Simon Francis, chief executive at Flock Associates, which helps advertisers run agency reviews and advises on marketing operations. The amount of liability companies are asking agencies to accept can range from $5 million to $100 million or even unlimited liability, he said.

Five years ago, only a handful of large clients were making detailed demands around data privacy liability, he added. Now, it’s nine out of 10 clients, and it comes up early in agency pitches.

“There’s no consistency in the price in the marketplace yet,” Mr. Francis said.

“It’s an unmitigated mess,” said Douglas Wood, a partner at law firm Reed Smith, which works with advertisers. “It’s not resolved yet. There’s no standard solution to this.”

In recent years, digital platforms and technologies have made it easier for advertisers to collect reams of data to send personalized ad messages to consumers. The ad industry’s migration to digital also has led to increasing concern around data breaches.

The changes have prompted discussions of increased regulation around consumer privacy in the U.S. California has already passed a strict new privacy law that takes effect Jan. 1, 2020.

Hard Lines

In the past, agencies’ liability to their clients for privacy violations was sometimes capped at a certain amount of fees agencies would need to return to a client. In recent years, “clients said that’s not enough, it doesn’t give us enough indemnification for damage you might deliberately do to my brand,” Mr. Francis said.

“This is something that clients push for, which is fully understandable,” said Florian Adamski, chief executive of Omnicom media agency OMD Worldwide. “Agencies push back, and usually they reach an agreement.”

But there isn’t always much room for negotiation, with some clients giving agencies little time to decide whether they’re willing to agree to the terms and refusing to consider any that don’t, he said. Such an approach is adding stress to agency-client relationships that are already strained over concerns about how agencies are using brands’ data and making money from their digital buying efforts.

“I struggle with the concept of pushing a potential partner, a service provider, into a corner where you say, ‘Take over this multi-multi-million-[dollar] risk within the next 24 hours or you might be out of that process,’” said Mr. Adamski.

The parties are also struggling over whether agencies should be responsible for any violations by the publishers that they ultimately use to reach consumers.

“I’ve seen numerous contracts by now, where clients have specifically asked for a commitment or guarantee that we’d ensure they’d have access and transparency into media vendors’ books,” Mr. Adamski said. “That’s something we can simply not provide.”

Agencies have taken steps to ensure they are working with compliant publishers, but they can only assume so much risk on behalf of a third party, he said.

The clause in an agency-client contract about who is responsible for a data violation involving the publisher is a gray area, Mr. Francis said.

The largest digital publishers and technology companies years ago added indemnification clauses, unloading data privacy liability to their business customers, said Tanya Forsheit, chair of the privacy and data security group at law firm Frankfurt Kurnit Klein & Selz PC.

Advertisers and agencies are “new to this dance,” she said.

Write to Alexandra Bruell at alexandra.bruell@wsj.com