How many​ living novelists does the FBI keep files on? Is there a filing cabinet in Washington that contains a rundown of Jonathan Franzen’s feud with Oprah Winfrey? Do the Feds keep track of how many words Joyce Carol Oates writes in a day? Do they monitor Karl Ove Knausgaard’s border crossings? Did they know who Elena Ferrante was before the editors of the New York Review of Books did? Or how much Martin Amis drinks these days, and where? These scenarios seem unlikely. But perhaps they keep a file on Toni Morrison, who – as a photo that recently went viral made clear – hung out with Angela Davis in the early 1970s, and surely Don DeLillo’s speculations on Lee Harvey Oswald in Libra merited attention. There is at least one known case. In 2013 William Vollmann wrote about getting hold of his own FBI file and discovering that during the 1990s, following an anonymous tip, he was suspected of being the Unabomber. ‘UNABOMBER, not unlike VOLLMANN, has pride of authorship and insists his book be published without editing,’ the agents wrote. They interviewed an acquaintance who told them he had a ‘death wish’; noted that his appearance – an unshaven man wearing sunglasses and a hood – fitted the composite sketches of the suspect; reported that he owned firearms and, erroneously, a flamethrower; commented on the violence and torture depicted in his novels (which they didn’t seem to finish); suspected that he might have learned about explosives from reporting on the mujahedin in Afghanistan in the 1980s, and remarked: ‘By all accounts, VOLLMANN is exceedingly intelligent and possessed with an enormous ego.’ The squad investigating the Unabomber had 2406 suspects, 111 of whom were female. Vollmann was never high on the list. But the FBI kept its file on him open for another decade, he was subjected to lengthy interrogations at border crossings and came under suspicion for the anthrax attacks that followed 9/11.

Nearer to the present, many prominent writers were associated with, or at least publicly supportive of, Occupy Wall Street and Black Lives Matter. I could name names, but then I’d be naming names. What about signatories to an open letter requesting that Obama pardon Edward Snowden? I spoke on the phone this afternoon to the guy who wrote it. Was the FBI listening?

Probably not, but at least the metadata are within reach. Things weren’t always so convenient for the bureau. Writers under Surveillance: The FBI Files (MIT, £20) collects facsimiles of ‘Bufiles’ on 16 famous writers, most of whom were in their prime during the directorship of J. Edgar Hoover. It was a Cold War habit in the days of domestic radicalism to collect such files, and the documents are at once fascinating and underwhelming. Most of the information so dutifully gathered would now be available through a call to the author’s publicist, and the rest would probably come up if an agent googled, say, ‘James Baldwin communist?’

Like lazy students of literature everywhere, the agents filing reports didn’t usually bother to do their reading. In the documents collected here only the report on Norman Mailer’s Miami and the Siege of Chicago delivers the scrutiny a literary hack might apply, with attention paid to ‘his usual obscene and bitter style’. Elsewhere, agents crib from newspaper reviews and interviews and watch their subjects on TV. They do maintain some scruples. In response to a query from Hoover – ‘Isn’t Baldwin a known pervert?’ – an Agent Jones responds:

It is not a matter of official record that he is a pervert; however, the theme of homosexuality has figured prominently in two of his three published novels. Baldwin has stated that it is also ‘implicit’ in his first novel Go Tell It on the Mountain. In the past, he has not disputed the description of ‘autobiographical’ being attached to the first book.

The same quest for proof on the ‘official record’ informs their attempts to smoke out commies. Baldwin drew the bureau’s attention because he was in the habit of telling the press that he was working on a book about the FBI and the South that would drop ‘like an atomic bomb’. Baldwin never wrote the book. He also told interviewers that getting Hoover out of his job or stripping him of power would bring ‘a great deal more hope’ to the civil rights movement. In contemporary terms, Hoover might be forgiven for thinking Baldwin was trolling him. Then again, Baldwin was absolutely right.

Then there are snitches. What did the local liquor store owner in Woody Creek, Colorado, know about Hunter Thompson? The make of his car and that he was going to appear on the Today show. The main snitch in Ray Bradbury’s file was the actor and screenwriter Martin Berkeley, a former Communist Party member who gave around 160 names to the House Committee on Un-American Activities. He told the FBI that Bradbury and other sci-fi writers ‘reached a large audience through their writings which are generally published in paperbacked volumes in large quantities’. ‘The general aim of these science fiction writers,’ he went on, ‘is to frighten the people into a state of paralysis or psychological incompetence bordering on hysteria which would make it very possible to conduct a Third World War which the American people would seriously believe could not be won since their morale had been seriously destroyed.’ Berkeley couldn’t peg Bradbury for party membership, but had heard him speak out against ‘McCarthyites and cowards’ at a meeting of the Writers’ Guild of America, so perhaps it was personal shame that drove him to hyperbole. Not a bad sci-fi plot: a nation lulled into acquiescence to the pinkos by reading The Martian Chronicles in book club.

The FBI was made aware of Hannah Arendt by a concerned parent of a Berkeley undergraduate whom Arendt had encouraged to move to France to study with Paul Ricoeur. Daddy couldn’t confirm that Arendt was a communist but believed she was ‘advocating a totalitarian philosophy in her political courses’ and offered a description of his target: ‘She is a small, rotund, stoop-shouldered woman with a crew-like haircut, masculine voice and a marvellous mind.’ An investigation into Susan Sontag conducted at Hoover’s request went on for four years and yielded little beyond Sontag’s résumé, her travels (most of which resulted in published articles), and a record of her anti-war activities, which were public and meant to bring attention to the cause anyway. As with Allen Ginsberg, the bureau stopped short of interviewing Sontag because they knew that an interview would only result in bad publicity when she inevitably wrote about it.

Some of the files were opened as a result of the efforts of the writers themselves or their publishers. Bennett Cerf sought the bureau’s help on behalf of Truman Capote when he went to Kansas to research In Cold Blood. Because the agents hadn’t read his stuff in the New Yorker they declined to provide a reference. Ayn Rand, a veteran HUAC snitch, offered her own collection of clippings of literary attempts to ‘inject communist philosophy’ into the culture. Later, she was excited to see Hoover quoted in an interview saying he was an ‘objectivist’, and asked to meet him. He brushed her off – it turned out that he’d meant to say he was ‘objective’ (as if).

The biggest file on display belongs to Ernest Hemingway, who worked for the FBI in Cuba over the winter of 1942-43. It was a marriage arranged by the US ambassador, Spruille Braden, and not a happy one. In exchange for uncovering Falangists and Germans Hemingway procured an allocation of gasoline, firearms, depth charges and a budget of $500 to $1000 a month to pay his team of informants, mostly bartenders and waiters, Republican refugees from Spain. The FBI was suspicious of Hemingway because he’d signed a letter against its treatment of veterans of the Spanish Civil War and at a jai alai match he’d introduced an agent to a friend as ‘gestapo’. Within a couple of months the local FBI attaché was warning that if Hemingway’s intelligence activities continued ‘they are undoubtedly going to be very embarrassing.’ His information was unreliable, the FBI wasted a lot of time checking it out, and he and his informants, who favoured ‘European-style denunciations’, were meddling in internal Cuban affairs that were outside American jurisdiction (an admirable note of restraint) and would only anger the government. Hemingway was cut off, but with funding from the navy he continued his investigations into the oxygen-fuelled German submarines he’d read about in the New York Times, taking pains to locate the island’s network of oxygen tanks.

Writers under Surveillance is a quaint collection, charmingly reproducing the documents in facsimile, typewritten, stamped and annotated. Most of the material and much more besides was covered by Herbert Mitgang in his 1988 book Dangerous Dossiers, which makes up for what it lacks in gritty reproduction with more detailed contextualising narratives and a wider cast of tragic characters (Dorothy Parker, who was blacklisted in Hollywood; H.L. Mencken, who came under suspicion for his German background during both wars; John O’Hara, whose many failures included a job application to the fledgling CIA). The book’s editors, JPat Brown, B.C.D. Lipton and Michael Morisy, founded and run the website MuckRock, where anyone can file a Freedom of Information Act request with the US government, and which is the source of the files in Writers under Surveillance. Files on the living can only be obtained by their subjects (Vollmann’s was unavailable to the editors), but anyone can obtain files on the dead. When I checked MuckRock, recent requests for files on Philip Roth and V.S. Naipaul were pending.

These are paranoid days in America. A couple of weeks ago, a guy who used to work in magazines asked me with a straight face if a journalist friend of mine was a Russian plant. Two of the most celebrated figures of recent months are one of Hoover’s successors, James Comey, and presumed national saviour Robert Mueller. In the liberal imagination now, the FBI and its erstwhile directors are your friends in the #resistance. If FBI agents are as lazy as they used to be when it comes to investigating writers, at least their job is much easier than it once was: these days they are just a click away from news of literary shamings, humiliations, denunciations and sundry invasions of writers’ privacy, most of it provided by fellow writers. But I have a hunch they’re not looking too hard at novelists, poets or critics. You can say one thing for Hoover: like Stalin, at least he cared.