Most citizens would object to their government searching their homes without a warrant. If you were told that while you were at work, your government was coming into your home and rifling through without cause, you might be unsettled. You might even consider this a violation of your rights specifically and the Bill of Rights generally.

But what if your government, in its defence, said: "First of all, we're searching everyone's home, so you're not being singled out. Second, we don't connect your address to your name, so don't worry about it. All we're doing is searching every home in the United States, every day, without exception, and if we find something noteworthy, we'll let you know. In the meantime, proceed as usual."

Yes, it's been strange to live in the USA in this, the era of the NSA. Not just because of the National Security Agency's seemingly boundless and ever-more-invasive collection methods, but because, for the most part, Americans have been proceeding as usual. In the wake of the Snowden revelations, there's been some outrage, and a flurry of lawsuits filed by organisations such as the ACLU, but most polls show about 50% of the population – including a shockingly high percentage of Democrats – find the NSA's domestic spying programme more or less acceptable.

No doubt many moderate Democrats have been caught in a paralysis of cognitive dissonance. That is, on a gut level, this level of spying seems horrific and unconstitutional, but, then again, would President Obama, himself a constitutional scholar, actually endorse – much less expand – a domestic spying programme unless it were morally acceptable and constitutional? And thus moderates twist themselves into pretzels trying to defend, or at least allow, the NSA's collections.

And so it has been up to an unlikely coalition to bang the drum. It surely has to be the first time the ACLU, the Electronic Frontier Foundation and the Tea Party have found themselves (somewhat) politically aligned. And in one of the more significant, if not unexpected, developments, Richard J Leon, a federal judge appointed by George W Bush, on Monday issued a 68-page decision denouncing the NSA's surveillance as "Orwellian" and saying: "I cannot imagine a more 'indiscriminate' and 'arbitrary invasion' than this systematic and high-tech collection and retention of personal data on virtually every single citizen for purposes of querying and analysing it without prior judicial approval." He added: "Surely, such a programme infringes on 'that degree of privacy' that the founders enshrined in the fourth amendment." Will this, finally, turn the tide of US opinion? Don't count on it. Judge Leon's ruling has no binding effect at the moment, and could be reversed on appeal.

In an effort to illuminate the NSA's effect on free expression, PEN America Centre recently surveyed its US members on their feelings about the NSA's unbounded reach. The resulting report, "Chilling Effects: NSA Surveillance Drives US Writers to Self-Censor," reveals that 88% of the writers polled are troubled by the NSA's surveillance programme, and that 24% have avoided certain topics in email and phone conversations. Most disturbingly, 16% of those answering the survey said they had abandoned a project given its sensitivity.

The survey is troubling on many levels. First, it's deeply dismaying that any writer would give up so easily – that any writer would be so readily cowed into submission. After all, to date, the NSA's surveillance hasn't landed any writers in jail, and, though there's no doubt a watchlist, so far no one on PEN's membership has been hauled in for questioning based on their phone calls, searches or internet activity. But living under the cloud of suspicion, or wondering not if, but when this collected data will be misused, runs, shall we say, counter to the idea of freedom of expression in a democracy.

The recent petition by Writers Against Mass Surveillance, issued last week and signed by 562 writers around the world, is an essential step toward an international digital bill of rights. But until there is such a thing, there will be hundreds of millions of people, writers among them, living under the assumption that every inquiry or communication they make could later be used against them.

Wajahat Ali is an American lawyer, essayist and playwright with Pakistani roots who has written extensively about Muslim-American issues. "When I read that," he said of the "Chilling Effects" report, "my first reaction was, welcome to our world. Muslim-Americans have been living with the presumption of being potential suspects for 12 years now. We've had to assume that all our phone calls, emails, social media and text messages are being monitored in some way."

Ali says he doesn't self-censor, but he is exceedingly aware at all times that his messages might be scrutinised. What if the government doesn't like an essay he writes, and then, searching through their metadata haystack, sees that he's donated to some Islamic relief organisation deemed questionable? In minutes they have enough on him to make a mess of his life. Ali and legions of other Muslim-Americans have had to adopt a gallows sense of humour about it, he says, citing the times when he's written 'Hello NSA!' in text messages and emails. "But I can't structure my life around fear," Ali says. He's so un-cowed that he recently took a job with Al Jazeera America.

"A writer's job is to look for trouble." That line was uttered by a character in The Front, the 1976 film about the McCarthy blacklist era. In the film, Woody Allen plays Howard Prince, a small-time bookie who is asked by an old friend, a blacklisted screenwriter, to be a front, signing his name to scripts penned by writers suspected of communist sympathies. Prince agrees, and soon attracts the notice of the House Committee on Un-American Activities himself.

Watching the movie now, parallels between that era and our own are many – the generalised air of suspicion, the sinister feeling of being watched but not knowing when. The movie's screenwriter, Walter Bernstein, himself was blacklisted and deprived of work. His phone was tapped, he was followed by FBI agents, his friends were harassed, and he was denied a passport.

I spoke to Bernstein, now 94 and still writing, by phone at his home in New York. When the Snowden revelations became public, he was surprised not that there was such ongoing surveillance, but by the stunning extent of it. "Then again," he said, "if they're able to do it, they will do it." I asked him if he thought our current era of domestic spying was as dangerous as the one he lived through. "In some ways it's worse now," he said."Now the surveillance extends to everyone. And it's going to get worse. The crimes committed in the name of national security are very great, and there's no answer to it."

Bernstein got his say in The Front, though. At the end of the film, Howard Prince decides not to co-operate with McCarthy's minions. In a private meeting in which he's supposed to sign a loyalty oath and provide names of communist sympathisers, Prince, until then decidedly apolitical, finally has had enough. He turns on his interrogators and roars: "I don't recognise the right of the committee to ask me these kinds of questions. And furthermore, you can all go fuck yourselves." Then he goes to jail.

Bernstein notes that at least back then there were faces attached to one's accusers. There was McCarthy himself. There were the FBI questioners. There were senators and courts and hearings. Now it's a shadowy agency that seemingly answers to no one. Not long ago, novelist William T Vollmann successfully used the FOIA, and found that the FBI had an extensive file on him; in the 1990s they had suspected that he was the Unabomber. But now, anyone trying to find out what the NSA has collected on them would be out of luck. "I would say you have no chance," says Rachel Levinson-Waldman, an attorney with the Brennan Centre for Justice at the NYU School of Law. In "What the Government Does with Americans' Data," Levinson-Waldman wrote what is perhaps the most comprehensive and lucid study about the NSA's spying and its potential for misuse. She pointed out something interesting about the PEN study. If writers assume they're being watched, "it could eliminate whole areas of inquiry. They could say, 'It's not worth it to me to come to the attention of the government.' And difficult subject matter might only be pursued by those who think themselves invulnerable to scrutiny."

Think back to all the messages you have ever sent. All the phone calls and searches you've made. Could any of them be misinterpreted? Could any of them be used to damage you by someone like the next McCarthy, the next Nixon, the next Ashcroft? This is the most pernicious and soul-shattering aspect of where we are right now. No one knows for sure what is being collected, recorded, analysed and stored – or how all this will be used in the future.

A few years ago, John Villasenor at the Brookings Institution wrote a terrifying study called "Recording Everything: Digital Storage as an Enabler of Authoritarian Governments", which explains how easy it is for a government to record and store the entire contents of all calls made within any country. The technology is readily available now, and the storage costs are so low that Syria, for example, could record and store all phone calls made by its citizens in a given year for under $1m (£610,000), or about nine cents a person. Taking the leap from any government, including our own, collecting metadata, to collecting all audio, period, is no leap at all.

Unchecked, the NSA will surely avail itself of these economies of scale. If and when it begins to record all phone calls, or when a whistleblower reveals that it is already doing this, again the NSA will say that there is no harm done, given that no one is being targeted specifically, that computers are simply scanning all of this audio for certain keywords. It will say that the vast majority of us – those who are presumably doing nothing wrong – will have nothing to worry about. No doubt the NSA looks at the poll numbers, that enabling 50%, and sees it as a mandate to continue and expand.

And so it will be up to this strange coalition to continue the fight. The PEN survey is not the most important barometer of the minds of the citizenry, but if writers are altering their behaviour, then millions more are, too. "Citizens of a democracy need a zone of privacy, and to have control over it," Levinson-Waldman says. "Without some assurance of privacy, it's impossible to participate robustly in the dissent and debate that are critical to our society." The effect of an entire nation of individuals choosing to abstain from certain phone calls, email messages, internet searches, for fear of what could be done with that information in the future, threatens not just a chill, but a permanent intellectual ice age.

Bernstein, who survived McCarthy, whose former friends used to cross the street rather than be seen talking to him, is as scared as he's ever been. I asked him to convey advice to writers and to us all. "Well," he said, "All I can say is that you need to resist. Resist. Resist. Resist. Resist."