Previously unpublished documents have revealed the scale of a plot by the media mogul William Randolph Hearst to discredit Orson Welles and destroy Citizen Kane, the 1941 film about the rise and fall of the fictional newspaper proprietor Charles Foster Kane.

Welles and RKO Pictures faced extortion, media manipulation and other underhand tactics in a plot that was much darker and began earlier than was previously known, according to research by Harlan Lebo for a forthcoming book.

The book cites a memo of 1941, in which Welles’s lawyer-manager, Arnold Weissberger, warned his client that the Hearst empire would stop at nothing: “This is not a tempest in a teapot, it will not calm down, and the forces opposed to us are constantly at work.”

Elsewhere, Weissberger said: “[Hearst] may decide to use all his legal machinery to harass RKO.”

Records show that Welles felt Hearst was not linked to the attacks on him and Citizen Kane, but that his minions wanted “to show the boss that they were on the ball”. On a lecture tour before Citizen Kane’s release, Welles was warned by a police investigator: “Don’t go back to your hotel. They’ve got a 14-year-old girl in the closet and two photographers waiting for you to come in.” But the director at the time blamed a “hatchet man” from a local Hearst paper.

Lebo, who acted as historical consultant to Paramount Pictures for Citizen Kane’s 50th anniversary, told the Guardian: “It’s typically been assumed that Hearst probably didn’t know about it and it was probably just his lackeys trying to protect the boss. But it’s clear he knew about it the entire time.”

Facebook Twitter Pinterest William Randolph Hearst, pictured around the time Citizen Kane was released. Photograph: Hulton Archive/Getty Images

He described the plot as “much more complicated and dark than has been recognised before”, with evidence showing Hearst’s executives conspiring to undermine Welles’s personal credibility and stop the film’s release.

In one memo, Richard Berlin, the ambitious head of Hearst’s magazine division, wrote to his boss about his “preliminary, and rather hasty” investigation of Welles, telling Hearst that the film-maker “acted as a front for the Communist party”.

Lebo said Welles was no Communist, yet “it was a Communist witch-hunt that was planned and managed at the top level of the Hearst organisation”.

A letter from Berlin to Hearst’s assistant shows the media baron was colluding with Congressional investigators hunting Communists in Hollywood: “We have the complete assurance from our friends in Washington that the result of the investigation made by them … of the motion picture industry is available to us … This should be extremely valuable.”

From the beginning, Welles never denied that his picture was about a newspaper publisher. But, as his film progressed, word spread of an exposé of Hearst, one of the most powerful men in the US, whose empire included 26 newspapers, 16 magazines and 11 radio stations.

Why Orson Welles lived a life like no other Read more

Lebo said: “There are numerous parallels between Kane and Hearst, including their respective fortunes built on the discovery of precious metals, their age, their private pleasure palaces. Both were overbearing, manipulated the media and used sensationalist journalism to sell papers.”

But the film was not solely based on Hearst, and Weissberger advised his client: “It is essential that we maintain consistently, emphatically and unequivocally that Hearst has nothing to do with this picture.”

Unpublished letters are at Indiana University Bloomington, among other archives. “Thousands of pages about Citizen Kane have never really been written about in any length, if at all”, Lebo said.

The conspiracy against Citizen Kane is generally assumed to have started after footage was seen by Hollywood columnists Hedda Hopper and Louella Parsons. Lebo said: “As the story goes, Hopper, after seeing a rough cut … alerted the Hearst organisation to the ‘dangers’ of Citizen Kane, and Parsons, who wrote for Hearst, then saw the film and took up the Hearst cause.”

The documentary evidence includes a note from Hearst’s assistant to Parsons confirming that “the Chief” had received her letter. It also shows that the Hearst organisation had been investigating the film for weeks earlier. Berlin wrote of going after Welles and stopping “this vicious picture”, adding: “It looks to me as if Citizen Kane will not have much of a showing.”

The Hearst organisation banned mentions of the film from its publications and dangled the threat of a lawsuit over the studio and any exhibitor until RKO eventually just released it.

Lebo’s book, Citizen Kane: A Filmmaker’s Journey, will be published by St Martin’s Press on 26 April.

In it, he writes: “One can only imagine the flood of national news coverage and highly charged commentary that would pour forth today if a 21st-century media mogul such as Rupert Murdoch, for purely personal reasons, banned all mention of a motion picture by his organisation’s outlets, and then actively tried to suppress or destroy the film.”