In the letter, the groups say they are concerned that the government will pressure tech companies into taking drastic steps without first considering the human-rights or free-speech implications.

“When the government sits down secretly with those companies that have practical control over a broad swath of public speech and private communication, and especially if and when those conversations lead to voluntary surveillance or censorship measures that would be illegal or unconstitutional for the government to undertake itself, the consequences are truly global,” the letter says.

The message is signed by prominent human-rights organizations like the American Civil Liberties Union, Amnesty International, and Human Rights Watch; they were joined by technology-focused groups like Access Now, the Center for Democracy and Technology, and the Open Technology Institute at the New America Foundation.

“They definitely deserve a seat at the table,” said a spokesperson for one of the main companies involved in the Madison Valleywood Project. The spokesperson, who spoke on condition of anonymity, said most of the participants already respect privacy principles, but that the presence of civil-rights groups would “keep them honest.”

(After the initial publication of this story, a Justice Department spokesman said he would reach out to a representative for the civil-rights groups that signed the letter.)*

The groups’ worry is that the U.S. government will follow the pattern of other governments worldwide, which have tried to use American tech companies to censor content or silence dissidents. Platforms like Twitter and Facebook field thousands of takedown requests every year from officials in nearly every country they operate in—and reject many that aren’t legal.

The U.S. has a far greater respect for free speech than the countries where the companies operate, but even here, national-security fears can can prod the officials into making moves that encroach on free-speech and privacy rights.

Ongoing court cases that pit Apple against the FBI, like the high-profile conflict over an iPhone that belonged to one of the San Bernardino shooters, offer an example of how far the government is willing to push tech companies in the name of fighting terrorism.

Yesterday’s letter also makes note of the targeted nature of anti-extremism programs that focus on Americans. “The potential threat to human rights is especially acute because so-called U.S. counter-extremism programs, while framed as not addressing a particular ideology or religion, currently overwhelmingly target Muslim and other marginalized communities and individuals,” they wrote.

The groups asked the White House to make two changes. First, they said, the conversations should be expanded to include civil-society organizations that can act as a check to harmful policies and initiatives. And second, the groups asked the White House and its tech-company partners to be more forthcoming about the measures they’re taking on.