Labelled and filed away in secret documents . . . (clockwise from top left) Rock Hudson, Guy Burgess, Donald MacClean and Anthony Blunt. The American Psychiatric Association finally rebuked "reparative" or "conversion" therapy for homosexuals in 1998. In Britain similar inhumane persecution occurred. In 1952, Alan Turing, 41, a mathematical genius, master decoder of Nazi ciphers and arguably an inventor of the modern computer, was arrested on homosexuality charges and sentenced to a year of "conversion" hormone treatment. Also accused of subversion, he committed suicide in 1954, apparently by eating a cyanide-laced apple. On the other hand, Soviet intelligence – analysed in an unclassified CIA study by defector Alexander Orlov on Theory and Practice of Soviet Intelligence – was "amazed at the mutual consideration and loyalty which prevailed among homosexuals". Prejudice in the West proved dangerous to its own survival. The Soviet use of homosexual spies and recruiters in the West was to result in possibly the espionage coup of the 20th century with the penetration and devastation of Western intelligence by the Cambridge University spy ring, initiated in the 1930s. The FBI kept copious newspaper and magazine files. US magazines of the 1950s – such as Confidential, at one point outselling Time and a precursor to today's tabloid culture – were an explosive, terrorising mix of hard right-wing politics, sex scandals and outing gays in Hollywood and even Washington, as in the case of former US under secretary of state Sumner Welles in March 1956. Hysterical articles claimed the government was infiltrated with "sex deviates" and in 1950 the Senate announced it would investigate Washington police department estimates that 3500 homosexuals had federal jobs. Per FBI files Burgess – in Washington at the time of the hearings with Philby, to help and liaise with the CIA – was disturbed by the hearings but obviously had the last laugh.

Major events in 20th-century history – including World War II, the atomic bomb, the Korean War – were all affected by the activities of probably the most effective spy ring on record. Historians are still trying to sort out the mess, since they were in place for decades. For example, an FBI file notes Maclean had access to "practically all high level US and UK projects" and secret coded communications. When you consider Kim Philby was not only head of the (anti) Soviet Section of British Intelligence but helped establish the CIA, you get an idea of the catastrophe. The huge FBI investigation of the watershed English spy scandal was an indication of the intelligence disaster. Censored files show much is still not public, even now. Burgess and Maclean had also contributed to, if not triggered, attacks on the Cold War's so-called Lavender Menace. A crude, broad brush now painted communists and traitors as gay. Hoover's secret files grew as politician Adlai Stevenson, prominent lawyer Alger Hiss and others were rumoured in memos as "queer". In the ensuing accusatory melee – immortalised in Arthur Miller's The Crucible, albeit a heterosexual version – even principal red baiter senator Joe McCarthy was "gay baited". McCarthy and his assistant Roy Cohn outed Communists and sometimes homosexuals in the notorious 1950s hearings. Trampling contemporary shades of political correctness, Cohn himself was homosexual and a nasty piece of work, destroying fellow homosexuals and smearing others. McCarthy himself was targeted by writer Hank Greenspun. "It is common talk among homosexuals in Milwaukee who rendezvous in the White Horse Inn that Senator Joe McCarthy has often engaged in homosexual activities." McCarthy shortly thereafter married his secretary. Alleged and real Communists and homosexuals were purged from government jobs. Confidential magazine revved it all up: "A conspiracy of silence among Washington bigwigs and news correspondents has kept the public ignorant of how high homosexuals were able to penetrate the State Department, or how decisive was their influence." Secret FBI files show that Hoover closely tracked Blunt, the "Fourth Man", from 1956 on, whenever he came to lecture on art in the US. Blunt's FBI files have many deletions but it is clear Hoover was on to him in secret memos as a friend of Burgess and Maclean, and in 1963 as a "confirmed homosexual". A "Secret and Personal" memo to Hoover in 1967 notes an informant is "positive Blunt was a Soviet agent". This was, of course, true but did not become public until 1979, embarrassing British prime minister Margaret Thatcher and becoming prominent again in the Spycatcher trial in Australia in 1986.

The Cambridge spies feature in 3219 unclassified forensic and other FBI documents. They reflect a justified panic, leavened by some inadvertent bureaucratic humour. A top-secret 1955 memo to Hoover notes "peculiarities" of super-spy Burgess include "excessive drinking, slovenly [appearance], chain-smoker, homosexuality". In fact, more than likely one of Burgess's covers was acting like a mad, drunken homosexual to the point that counter-intelligence never suspected he could be a serious threat, until he defected. The myth that most homosexuals were "femme" or not tough also helped the spies. In fact, these very hard men had minds like steel traps and the ability to betray friends, family and country was essential to promulgate their core political beliefs. Some US presidents, like Nixon, were openly homophobic. Nixon is caught on released tapes of May 13, 1971, saying northern California has "gotten so faggy . . . that I won't shake hands with anybody from San Francisco . . . I don't want to see this country go that way. You know what happened to the Greeks. Homosexuality destroyed them." Unclassified FBI files reveal that Nixon asked Hoover to find out who was homosexual in the Washington press corps. In 1964 Johnson was shocked when close friend and presidential assistant Walter Jenkins was arrested for oral sex in a YMCA toilet in Washington. Formerly secret tapes reveal Johnson feigning naivety to FBI chief Hoover, saying: "I guess you are going to have to teach me something about this stuff . . . I swear I can't recognise them. I don't know anything about them." Hoover replies: "It's a thing you can't tell sometimes . . . just like poor Jenkins. There are some that walk kind of funny. That you might think a little off or queer." Demurring to the master's superior knowledge, Johnson asked Hoover to find out who else was gay in his White House. The exquisite irony of Hoover ferreting out homosexuals for presidents and national security reasons – and advising LBJ on how to spot homosexuals for dummies – can't be lost on anyone who knows Hoover himself was the centre of rumour and gay urban legend for decades. He lived with his mother until he was 43. On his death, he bequeathed his estate to his FBI deputy Clyde Tolson, now buried a few metres from him. Acutely aware of his image and this issue, Hoover's uber files or "DO NOT FILE" files transferred to the US National Archives in 2005, reveal he had FBI agents investigate rumours that he was "queer" as early as 1944, long before any "Lavender" panic in the US government. Despite sensational allegations by scandal connoisseurs that Hoover was a cross-dresser and the mafia had blackmail photos of Hoover being intimate with Tolson, there is no conventional hard evidence, only circumstantial notes. However, these "Official and Confidential Files" contain numerous references to homosexual allegations and "derogatory information" about famous people covering decades. FBI files show derogatory remarks about Hoover himself were noted when heard in "beauty salons" in 1951 and a memo was filed when it was rumoured the Los Angeles Times's Jack Nelson planned to write that Hoover was homosexual in 1970. The government's hounding and persecution of homosexuals had become a major civil rights issue, so obviously the matter was extremely sensitive, if, in fact, Hoover was gay. The FBI was tracking and bugging the Gay Activists Alliance in the 1970s and memos about the activists to Hoover marked "secret" are alarmingly headed, "Revolutionary Activities". If Hoover, McCarthy and Cohn were gay, then politics trumped sexual preference in their energetic pursuit of alleged or real "enemies of the state". But if gay men who were pillars of the establishment got into a jam, then files reveal attitudes could somersault, as exemplified by the extraordinary tale of Joseph Alsop. On April 1, 1957, CIA director Allen Dulles wrote a "SECRET EYES ONLY" memo to Hoover, enclosing a nine-page confession by Alsop, Washington socialite and conservative journalist, detailing that he was homosexual and that, consequently, eight weeks earlier he had been framed by the KGB in Moscow when men burst into a hotel room, catching him in "the act" with a young Russian "pleasant-mannered fellow". This was not an April Fool's day joke, this was the hard-core side of the Cold War. So sensitive was the matter it took 42 years for the confession to become unclassified. Alsop initially became suicidal when caught by the Russians, who tried, but failed according to Alsop, to get him to change sides. He was helped by US ambassador Charles Bohlen in Moscow (himself subject to rumours of homosexuality and an appointment opposed by McCarthy). Initially, Hoover and Dulles accepted Alsop's abject mea culpa and self-described "folly". A memo to Hoover marked "top secret" indicates Dulles was still suspicious: "I am not at all satisfied with the subject's evaluation of his own situation or his refusal to co-operate more fully in the interview."

Nevertheless only six years later, Alsop is recorded in tapes three days after John F. Kennedy's assassination strongly urging, in a subtle blend of insistence and grovelling, that Johnson set up a blue ribbon commission that would "carry unchallengeable convictions". Alsop added, "I'm giving you public relations advice and not legal advice" to handle the assassination aftermath. Johnson had wanted the matter settled in Texas as a police matter, as he coolly said to Hoover on the same day, "We can't be checking up on every shooting scrape in the country." Earlier LBJ had asked Alsop: "Why can't the FBI transmit [details of the investigation]?" Alsop replied: "[The commission] will carry absolute conviction . . . no one on the left will believe the FBI . . . and the FBI does not write very well . . . I hate to interfere, sir . . . I only dare to do so because I care so much about you." Johnson replied: "I know that, Joe." Alsop prevailed and the Warren Commission was born. Dulles would be a member. From being framed by the KGB in a gay assignation in Moscow, to giving advice to the brand new president on a key event of the century, is quite a turn of events. Graham Greene, eat your heart out. tagSpacemaker2 er1