When Winston Churchill became prime minister in May 1940, he realised immediately - if he had not realised before - that he had to achieve one thing in order to ensure that Britain was not defeated by Hitler's Germany: he had to enlist the US as Britain's ally. With the US alongside Britain, Hitler would be defeated - eventually. Without the US (Russia was neutral at the time), the future looked unbearably bleak. Roosevelt, as president, was predisposed to help - after a fashion and for cash on delivery - but the situation in America was overwhelmingly isolationist. One easily forgets this, in the era of our much-vaunted, so-called "special relationship", but at the nadir of Britain's fortunes, polls in the US still showed that 80% of Americans were against joining the war in Europe. Anglophobia was widespread and the US Congress was violently opposed to any form of intervention.

After the fall of France in June 1940, Britain's position became even weaker - it was assumed that British capitulation was simply a matter of time; why join the side of a doomed loser, ran the argument in the US. Roosevelt's hands were therefore firmly tied. Much as he might have liked to help Britain (and this, I feel, is a moot point: just how enthusiastic was FDR himself?) he dared not risk alienating Congress - and he had a presidential election looming that he did not want to lose. To go to the country on a "Join the war in Europe" ticket would have been electoral suicide. He had to be very pragmatic indeed - and there was no greater pragmatist than FDR.

All the same, Churchill's task, as he himself saw it, was clear: somehow, in some way, the great mass of the population of the US had to be persuaded that it was in their interests to join the war in Europe, that to sit on the sidelines was in some way un-American. And so British Security Coordination came into being.

BSC was set up by a Canadian entrepreneur called William Stephenson, working on behalf of the British Secret Intelligence Services (SIS). An office was opened in the Rockefeller Centre in Manhattan with the discreet compliance of Roosevelt and J Edgar Hoover of the FBI. But nobody on the American side of the fence knew what BSC's full agenda was nor, indeed, what would be the massive scale of its operations. What eventually occurred as 1940 became 1941 was that BSC became a huge secret agency of nationwide news manipulation and black propaganda. Pro-British and anti-German stories were planted in American newspapers and broadcast on American radio stations, and simultaneously a campaign of harassment and denigration was set in motion against those organisations perceived to be pro-Nazi or virulently isolationist (such as the notoriously anti-British America First Committee - it had more than a million paid-up members).

Stephenson called his methods "political warfare", but the remarkable fact about BSC was that no one had ever tried to achieve such a level of "spin", as we would call it today, on such a vast and pervasive scale in another country. The aim was to change the minds of an entire population: to make the people of America think that joining the war in Europe was a "good thing" and thereby free Roosevelt to act without fear of censure from Congress or at the polls in an election.

BSC's media reach was extensive: it included such eminent American columnists as Walter Winchell and Drew Pearson, and influenced coverage in newspapers such as the Herald Tribune, the New York Post and the Baltimore Sun. BSC effectively ran its own radio station, WRUL, and a press agency, the Overseas News Agency (ONA), feeding stories to the media as they required from foreign datelines to disguise their provenance. WRUL would broadcast a story from ONA and it thus became a US "source" suitable for further dissemination, even though it had arrived there via BSC agents. It would then be legitimately picked up by other radio stations and newspapers, and relayed to listeners and readers as fact. The story would spread exponentially and nobody suspected this was all emanating from three floors of the Rockefeller Centre. BSC took enormous pains to ensure its propaganda was circulated and consumed as bona fide news reporting. To this degree its operations were 100% successful: they were never rumbled.

Nobody really knows how many people ended up working for BSC - as agents or sub-agents or sub-sub-agents - although I have seen the figure mentioned of up to 3,000. Certainly at the height of its operations in late 1941 there were many hundreds of agents and many hundreds of fellow travellers (enough finally to stir the suspicions of Hoover, for one). Three thousand British agents spreading propaganda and mayhem in a staunchly anti-war America. It almost defies belief. Try to imagine a CIA office in Oxford Street with 3,000 US operatives working in a similar way. The idea would be incredible - but it was happening in America in 1940 and 1941, and the organisation grew and grew.

From a novelist's point of view, to discover such a forgotten corner of 20th-century history is a wonderful and unique gift. I had long wanted to write a novel about a spy, a woman spy in fact, but to have her spying in America - rather than in Russia or Germany or Occupied France - seemed an irresistible bonus. The more I investigated BSC's activities, the more intrigued I became. Some of BSC's schemes verged on the absurd; some were highly sophisticated media manipulation.

BSC invented a game called "Vik", described as "a fascinating new pastime for lovers of democracy". Printed booklets described up to 500 ways of harassing and annoying Nazi sympathisers. Players of Vik were encouraged to ring up their targets at all hours of the night and hang up. Dead rats could be put in water tanks, air could be let out of the subject's car tyres, anonymous deliveries could be made to his house and so on. In the summer of 1941, BSC sent a sham Hungarian astrologer to the US called Louis de Wohl. At a press conference De Wohl said he had been studying Hitler's astrological chart and could see nothing but disaster ahead for the German dictator. De Wohl became a minor celebrity and went on tour through the US, issuing similar dire prognostications about Hitler and his allies. De Wohl's wholly bogus predictions were widely published.

However, one of BSC's most successful operations originated in South America and illustrates the clandestine ability it had to influence even the most powerful. The aim was to suggest that Hitler's ambitions extended across the Atlantic. In October 1941, a map was stolen from a German courier's bag in Buenos Aires. The map purported to show a South America divided into five new states - Gaus, each with their own Gauleiter - one of which, Neuspanien, included Panama and "America's lifeline" the Panama Canal. In addition, the map detailed Lufthansa routes from Europe to and across South America, extending into Panama and Mexico. The inference was obvious: watch out, America, Hitler will be at your southern border soon. The map was taken as entirely credible and Roosevelt even cited it in a powerful pro-war, anti-Nazi speech on October 27 1941: "This map makes clear the Nazi design," Roosevelt declaimed, "not only against South America but against the United States as well."

The news of the map caused a tremendous stir: as a piece of anti-Nazi propaganda it could not be bettered. But was the South America map genuine? My own hunch is that it was a British forgery (BSC had a superb document forging facility across the border in Canada). The story of its provenance is just too pat to be wholly believable. Allegedly, only two of these maps were made; one was in Hitler's keeping, the other with the German ambassador in Buenos Aires. So how come a German courier, who was involved in a car crash in Buenos Aires, happened to have a copy on him? Conveniently, this courier was being followed by a British agent who in the confusion of the incident somehow managed to snaffle the map from his bag and it duly made its way to Washington.

The story of the South America map and the other BSC schemes was written up (in an extensive document of some hundreds of pages) after the war for private circulation by three former members of BSC (one of them Roald Dahl, interestingly enough). This secret history was a form of present for William Stephenson and a selected few others; it was available only in typescript and only 10 typescripts ever existed. Churchill had one, Stephenson had one and others were given to a few high officials in the SIS but they were regarded as top secret.

When Stephenson's highly colourful and vividly inaccurate biography was written (A Man Called Intrepid, 1976), the BSC typescript was drawn on by its author, but very selectively - in order to spare American blushes. The story of BSC seemed to be one of those wartime secrets that was never to be wholly revealed, like Bletchley Park and the Enigma machine decryptions. But the Enigma story was eventually made public and has been written about endlessly since the mid-1970s, fostering films, TV plays and novels in the wake of the revelations. But somehow BSC and the role of British agents in the US before Pearl Harbor has remained almost wholly undisclosed - one wonders why.

In 1998 the BSC typescript (one of only two remaining) was eventually published. To say it fell stillborn from the press would be an understatement. Yet here is a book of some 500 pages, written just after the war by former BSC agents, telling the whole story of Britain's US infiltration in great detail, recounting all the dirty tricks and the copious and widespread news manipulation that went on. I think it's fair to say that historians of the British Secret Services know about BSC and its operations, yet in the wider world it still remains virtually unheard of.

The reason is the story of BSC and its operations before Pearl Harbor is deeply embarrassing and remains so to this day. The document is explicit and condescending about American gullibility: "The simple truth is the United States is inhabited by people of many conflicting races, interests and creeds. These people, though fully conscious of their wealth and power in the aggregate, are still unsure of themselves individually, still basically on the defensive." BSC set out to manipulate "these people" and was very successful at so doing - hardly the kind of attitude countries involved in a "special relationship" should display. But that relationship is a Churchillian myth, invented and fostered by him after the war, and has been bought into wholesale by every subsequent British prime minister (with the possible exception of Harold Wilson).

As the secret history of the BSC unequivocally shows, sovereign states act exclusively to serve their own interests. A commentator in the Washington Post who read the BSC history remarked, "Like many intelligence operations, this one involved exquisite moral ambiguity. The British used ruthless methods to achieve their goals; by today's peacetime standards, some of the activities may seem outrageous. Yet they were done in the cause of Britain's war against the Nazis - and by pushing America towards intervention, the British spies helped win the war." Would BSC's activities eventually have encouraged the US to join the war in Europe? It remains one of the great "what ifs" of historical speculation. The tide of US public opinion seemed to be turning towards the end of 1941 - though isolationist sentiments remained very strong - and BSC's propaganda and relentless news manipulation deserved much of the credit for that change but, in the event, matters were taken out of BSC's hands. On the morning of Sunday, December 7 1941 the Japanese bombed Pearl Harbor - the "day of infamy" had dawned and the question of American neutrality was gone for ever.

· William Boyd's novel, Restless, is published by Bloomsbury on September 4, priced £17.99.