He played some of the most sublime chess ever seen. Then, as a new book and film illustrate, he disappeared from view. What made such a brilliant mind go into freefall?

In 1999, I spent three days sitting in a variety of thermal baths dotted around Budapest. As grand and attractive as the Hungarian capital's spas are, I wasn't stewing myself for therapeutic or leisure purposes. Instead, I was waiting for someone I'd been told frequented the baths, someone who was said to be a genius and a paranoid obsessive, the greatest chess player who ever lived and an obnoxious crackpot. I was looking for Bobby Fischer.

For the last four decades of his life, that's what people did with Fischer – they looked for him. Fans, journalists, biographers, friends, they all tried to find this mythical creature, either in person or in that fabulous abstract realm that he continued to haunt: chess. He had ventured deep into the alternate world of this most intellectually demanding of games, a daunting contest of infinite possibilities, and succeeded in becoming world champion. Like some chequerboard version of Conrad's Kurtz, the experience seemed to leave him in a state of dread. Then he vanished.

As with those other great disappearing acts, JD Salinger, Greta Garbo and Howard Hughes, Fischer was almost as well known for his withdrawal from public life as he was for the achievements that brought him fame in the first place. There was even a feature film made called Searching for Bobby Fischer. It wasn't actually about Fischer, but based on the life of another chess prodigy, Joshua Waitzkin. Fischer's name was employed as a metaphor for his total commitment, what Garry Kasparov, Fischer's only rival for the title of best-ever player, has described as "pathological determination". Fischer was apoplectic when he heard about the film, which he called a "monumental swindle" and even angrier when he discovered that he had no legal grounds on which to sue the film-makers.

Had I run into him, I wasn't expecting him to be any happier. My intentions weren't metaphorical. I'd been prompted to seek him out after he'd made one of his rare public statements. In a live interview on Hungarian radio, he said: "As Adolf Hitler wrote in Mein Kampf, the Jews are not the victims, they are the victimisers!", before launching into a Holocaust-denying rant. Over half-a-million Hungarian Jews are estimated to have been killed during the Holocaust. Fischer was born a Jew, and if Hitler had had anything to do with the matter, he would have died a Jew, too. I wanted to discover how or why Fischer's obsessive character had taken such a self-destructive turn. The word was that he remained a peerless analyst of chess games. Would it not be possible to appeal to his rational side?

In the event, the bath-house stake-out was a failure. None of the bearded strangers I spent my time staring at through the saturated air turned out to be Fischer. He didn't show up at any of the baths. I left Budapest with Fischer seeming even more elusive than before I arrived. With the exception of a former girlfriend, most of the people who knew him refused to speak to me. He was fiercely protective of his privacy, which was the reason the story of his progress from prodigy to pariah remained the subject of so much speculation and rumour.

Among the many perceived betrayals for which his friends and intimates were permanently expunged from his life, the gravest was speaking to the press or biographers. Only with Fischer's death in 2008 did the atmosphere of omerta that surrounded the legend begin to dissipate and a more accurate testimony emerge. The fruits of this candour are a new biography, Endgame: Bobby Fischer's Remarkable Rise and Fall, by Frank Brady, and the soon-to-be-released HBO film, Bobby Fischer Against the World. Brady knew Fischer in the 60s and is the author of Bobby Fischer: Profile of a Prodigy, perhaps the only other worthwhile biography on the subject. He is also credited as consultant for the documentary. Together, the film and the book give shape to a more complete picture of Fischer: brash, complex, troubled, bold, vulnerable, lonely, occasionally loving, but fundamentally enigmatic.

The tortured genius and the celebrity recluse are two archetypes by which the popular imagination appears incurably enthralled. They occupy extreme but ambiguous positions in the social firmament, simultaneously familiar and unknowable, often winning our sympathy even as they fail our understanding. Working as Mephistophelean morality tales, they reassuringly remind us that exceptional talent can be an affliction as well as a gift and that sometimes the price of success is one that we – the average, the normal, the unchosen – would not wish to pay. No one in recent times has combined these two roles with more tragedy or pathos than Fischer.

His descent into wild and irrational behaviour is far from a unique narrative, particularly in chess. The history of the game contains many similar trajectories. As GK Chesterton noted in arguing that reason bred insanity: "Poets do not go mad, but chess players do." Akiba Rubinstein, the early 20th-century Polish grandmaster, would hide in the corner of the competition hall between moves, owing to his anthropophobia (fear of people), retiring from the game when schizophrenia got the better of him. William Steinitz, the Austrian who was the world's first undisputed chess champion, died in an asylum. Then there was Paul Morphy, the American who was said to be the 19th-century's finest player and to whom Fischer has frequently been compared: he quit the game, having beaten all his rivals, and began a decline into paranoid delusion. Aged 47, he was found dead in his bath, surrounded by women's shoes.

In his book Outliers, Malcolm Gladwell contends that genius is not an accident of birth but the combination of innate ability and intense application – typically 10,000 hours of practice. Nigel Short, the British grandmaster who was once ranked third in the world, agrees with Gladwell. "You will always find that the most naturally talented players have put in an incredible number of hours," he says. "There are no exceptions."

Fischer, who registered an IQ of 180, once said that he did not consider himself to be a genius at chess. "I consider myself to be a genius who happens to play chess." He was not only furnishing his own myth when he made that statement, but also playing to our romantic notions of genius as a kind of destiny. The truth is that not even the exceptional Fischer was an exception.

Born in Chicago in 1943, Fischer moved around America during the war years with his mother, Regina, and his older sister, Joan. His mother worked as a welder, a riveter, farm worker, schoolteacher, stenographer and toxicologist's assistant. The one-parent family eventually moved to New York, settling in a rundown area of Brooklyn. Having shown a precocious talent for kids' puzzles, Fischer began playing chess at six when his sister bought him a $1 plastic set. By the age of nine, he was practising and studying the game to the exclusion of all else.

The 13-year-old Bobby Fischer practises in New York, 1956 - less than two years later he was US champion. Photograph: Bettmann/Corbis

Nothing interrupted his obsession. He played while eating. He played in the bath. He played when he should have been at school. He cultivated an extraordinary facility for reading chess games, absorbing pages of dense notation in seconds, and learned Russian just so that he could study Soviet chess literature. He could play blindfold and recite games by heart. "Chess and me," Fischer later said, "it's hard to take them apart. It's like my alter ego." Such was Fischer's all-consuming preoccupation with chess that his mother took him to see two psychiatrists, both of whom told her not to worry.

Perhaps it was not worry but guilt that caused Regina to seek professional advice about her son. She didn't just work at a multiplicity of jobs, often leaving him alone in their small apartment, she spent most of her spare time studying for a medical degree or organising political protests. A communist sympathiser, she was anxious about FBI attention, telling Fischer never to speak to the authorities should they approach him. His earliest life lessons were in paranoia and loneliness.

If chess offered an escape from his humdrum life in Brooklyn, it also provided a much-needed structure. He mastered its rules and valued its traditions in a way that he never quite grasped social norms or conventions. The more he learned about chess, the less he cared about school, friendship, girls, job prospects or any other teenage concerns. Throughout his adolescence, Fischer showed little or no interest in sex. Legend has it that Henry Stockhold, a chess-playing journalist, took the 19-year-old Fischer to a brothel to lose his virginity. When Stockhold asked him how it went, Fischer is said to have replied: "Chess is better."

Nearly everyone who showed an interest in him when he was a child was primarily drawn to his chess ability. They ignored his less charming attributes – his lack of curiosity about others, his sense of entitlement, his tendency to cry and sulk when he lost – because he displayed such promise at moving pieces around the board. His chess coach as a teenager was a man named Jack Collins, who gave Fischer free tuition. Yet when Collins became popular with other young players, as a result of Fischer's successes, his star student resented Collins's financial gain. For the rest of his life, he remained ever vigilant to exploitation. As Brady writes: "He hated the idea of people making money off his name."

And all the time, he just got better and better. He became a chess master at 13, and the same year he defeated Donald Byrne, one of America's strongest players, with such magnificent precision that it became known as the "game of the century". Two months before his 15th birthday, he became the youngest-ever US champion. He played in and won a further seven US championships – once achieving the remarkable feat of a perfect score – 11 wins and no losses. Yet partly owing to his withdrawal from international competition in the mid-60s in protest at what he claimed (probably correctly) was the collusion of Soviet players, it would not be until 1972 that he challenged for the world title.

The showdown between Fischer and Boris Spassky, the reigning world champion, in Reykjavik in 1972 was promoted as if it were an intellectual third world war, with chess pieces standing in for tactical nuclear weapons. If that sounds ridiculous, it's worth noting that when Fischer looked like pulling out of the match on the eve of the tournament, the phone call that persuaded him to go to Iceland came from Henry Kissinger, the US national security adviser.

This was the era of the cold war and, in popular terms, Spassky represented the Soviet subsidised system that had ruled international chess for decades, while Fischer embodied the spirit of American free market individualism standing up to communist collective might. In fact, each man would in the years to come renounce his nation's citizenship and ideology, but at the time the symbolism of their designated roles fired the world's imagination.

And there was no shortage of drama. First of all, Fischer, spooked by the media and his impending moment of truth, went to ground in New York, forcing the organisers to postpone the starting date of the competition in Iceland. He hid at his friend and fellow chess player Anthony Saidy's parents' house, unconcerned by the fact that Saidy's father was dying.

When he finally got to Reykjavik, he arrived late for the first game and lost, making an inexplicable blunder during the endgame. He forfeited the second game by not turning up and refused to continue unless the match was moved from a public hall to a private room. Amazingly, the organisers gave into his demands and he started to win. It was then Spassky's turn to complain of a mysterious contraption hidden in the room that was sapping his energy. The furniture and light fittings were disassembled to mollify Spassky. All that was found were two dead flies.

Were these all psychological ploys? Fischer once said: "I don't believe in psychology. I believe in good moves." No one prepared harder or longer in contemplation of those moves than Fischer. But chess is not just a game of the mind. It's also the ultimate mind game. The confidence with which chess players speak of themselves and the condescension they display to their opponents illustrate how much of the action takes place away from the board. "The greatest pleasure," Fischer told the talkshow host Dick Cavett when asked what he relished most about winning, "is when you break his ego – that's where it's at."

Spassky's ego was broken in Iceland. Fischer accelerated away to the title, playing some of the most sublime chess ever witnessed in championship competition. In Bobby Fischer Against the World, Saidy refers to game six, the most celebrated of the match, as "a symphony of placid beauty". It's reminiscent of the scene in Amadeus when Salieri describes the aching majesty of Mozart's "Gran Partita" adagio. Salieri's torment was that he was good enough to recognise genius, but not good enough to be one. Yet even those who are not musically minded can hear the splendour of Mozart's score. For those who don't know the difference between a King's Gambit and a Sicilian Defence, the brilliance of Fischer's moves must be taken on trust, although Brady's succinct description of Fischer's play captures his signature style: "crystalline – transparent but ingenious". It's an appreciation that Short echoes.

"The thing that strikes me about Fischer's chess," he says, "is that it's very clear. There are no mysterious rook moves or obscure manoeuvrings. He's very direct. There's a great deal of logic to the chess. It's not as though it's not incredibly difficult – it is incredibly difficult. It's just that when you look at it you can understand it – afterwards. He just makes chess look very easy, which it isn't."

Making the supremely difficult look easy may be one definition of genius. One category of social dysfunction is making the easy look supremely difficult. Fischer possessed both traits. Short, who has played 12 world champions, never met Fischer, but he's read widely about him, including Brady's new book. He thinks that before Reykjavik Fischer showed some eccentricities but that "no fair person would say that he was crazy". That is true, yet there were visible signs of growing unease. There is footage online of a crew-cut 15-year-old Fischer taking part in the US quiz show I've Got a Secret. He looks alert, engaged, open to the world. At one point, as the panel tries to guess his achievement (becoming the youngest US chess champion), he's asked if it made people happy, to which he quips: "It made me happy!" and breaks into a big smile. Fourteen years later, in the interview with Cavett, he gives nervous, laconic answers that put me in mind of Christopher Walken in The Deer Hunter. His leg moves in anxious spasms. He looks edgy rather than alienated. Wearing a sharp blue suit, he cuts a handsome, athletic figure, uncomfortable but not unappealing.

His triumph in Iceland was the culmination of 23 years' relentless work on chess. When questioned about the effect of this monomania, he replied, looking disturbed at the thought: "Well, it would have been better if it had been a little more balanced, a little more rounded. But what can you do?" Yet on finally winning the crown, Fischer announced: "My goal now is to play a lot more chess. I feel I haven't played enough chess."

He never played another competitive game again, except for one strange match in 1992 against a fading Spassky. After Reykjavik, he was offered and turned down millions of dollars of sponsorship. There was a $5m deal on the table to defend his title against the Soviet challenger, Anatoly Karpov. Fischer insisted that Fide, the world chess federation, change the format of the final. Fide went nearly all the way to meeting his demands, but not far enough for Fischer. He resigned his title in June 1974. Saidy, like Kasparov, believes that a fear of losing lay behind the decision, even though Fischer would almost certainly have triumphed. Karpov, who became world champion by default, put it another way: "I think he couldn't cope with his own invincibility."

And so began what have become known as the wilderness years and what Kasparov has called "one of the greatest known bouts of psychoanalysis in absentia the world has ever seen". Fischer donated a large percentage of his Reykjavik winnings to the Worldwide Church of God. The church set him up with women – "vivacious girls with big breasts" were his type – but, frustratingly for Fischer, the dates were chaste. There is a wealth of Freudian literature about chess and sex, in which, for example, a queen sacrifice is associated with a sublimation of homosexuality. Brady doesn't indulge in such remote or fanciful diagnoses, but it seems apparent that Fischer was not at ease with his sexual desires.

In 1975, after the no-show of the second Christ the church had been promising, he abandoned the sect, believing it to be part of "a satanical secret world government". By this time, he was living in a small basement apartment in Pasadena, near Los Angeles. His elegant wardrobe and fitness regime were things of the past. He cut off his phone, played chess on his own and spurned the friends who visited him. He started quoting the infamous antisemitic fabrication "The Protocols of the Elders of Zion", even at dinner with his sister, her Jewish husband and children. "It was as if he was at war with himself," says Saidy, who told him that if he didn't play chess, he'd be forgotten. Fischer was so upset at the idea that he never spoke to him again.

Brady, who has written a largely sympathetic book, maintains that the widely circulated story that Fischer had his fillings removed to stop them being used by the Soviets for radio signals is an "urban legend". He did get them taken out, but Brady argues that he was only interested in his periodontal health. It's a toss-up as to which explanation is the more lunatic, because either way Fischer lost his teeth as a result of the procedure.

Well aware of chess's baleful reputation in matters of psychiatric health, Short notes that it was when Morphy and Fischer stopped playing competitive chess that their mental conditions deteriorated. "So," he says, "there is a counterargument that chess keeps people, or certain obsessive people, sane."

None the less, he acknowledges that the game can inflict mental damage. He draws an analogy with sport, where players pick up physical injuries as a result of constant strain. High-level competitive chess can also take its toll, only not on the physique.

"I don't think outsiders understand the levels of stress involved," says Short. "If you're in time trouble, your heart rate can easily double during a game. Normally when your heart rate is doubling you should be physically moving – so in chess there is no outlet for this stress. I think what happens is that people are getting some sort of mental injuries that are not necessarily detected. Whereas a sportsman with a hamstring problem will receive immediate treatment, you can develop neurotic ideas in chess and they're just never treated."

Fischer didn't believe in doctors, much less psychiatrists. His neuroses were left to the media to diagnose and Fischer had even less time for the media than he did for shrinks. Brady records Fischer's slide down the Los Angeles housing ladder, from the Pasadena basement to a MacArthur Park flophouse. He was reduced to living off his mother's social security cheques. By 1990, he was a half-forgotten, penniless paranoiac.

That's when he received a letter, via the United States Chess Federation, from a 17-year-old female chess player from Hungary called Zita Rajcsányi. She told him that he was "the Mozart of chess". He entered into a correspondence with the teenager – hoping to develop a sexual relationship – and she persuaded him to play chess once more. In 1992, a $5m match against Spassky was set up by Jezdimir Vasiljevic, the Balkans' own Bernie Madoff, in war-torn Yugoslavia. There was a UN embargo in place and Fischer was warned in writing by the American government that his participation would contravene US law. He responded by spitting on the official letter at a press conference.

After a sticky start, in which Spassky seemed to take pity on his plight, Fischer easily won the series. But the match exposed his decline as a player and, more conspicuously, as a person. He was also now an outlaw, facing a potential prison sentence should he return to America. With two Serbian bodyguards in tow, he moved to Budapest, where he drew heavily on the hospitality of Hungary's leading chess families, the Polgars and Lilienthals, both of whom were Jewish. They put up with his antisemitic tirades because his company – or legend – was still prized within the Hungarian and international chess communities. It was Fischer who eventually turned his back on them, citing, as ever, real or imagined betrayals.

Fischer lived a leisurely but dislocated life in Budapest. His money problems were behind him, but he didn't speak the language and he alienated anyone who came close. Even Rajcsányi fell out with him after he refused to accept that she wanted to marry someone else. "I've won from worse positions than this," he told her. He spent his time reading Holocaust-denial literature and watching American films. Having seen The Truman Show, he said he identified with Jim Carrey's character, seeing himself as the lone honest person surrounded by dissembling actors.

While I was in Budapest, I met up with Rajcsányi, a warm and sharp-witted young woman who was training to be a psychologist. I asked her about the origins of Fischer's persecution complex. "If a person is not able to confess his own mistakes," she said, "he tries to project them on to other people."

Had he ever explained his hatred of Jews? "He told me that, in his childhood, his mother had lots of Jewish friends who spent lots of time at the apartment in endless conversation," she told me. "At 12 years old, he decided this wasn't normal."

It's often been said that Fischer grew to hate his mother and that he broke off all contact with her. Brady makes a persuasive case that Regina and her son were never estranged, remaining close until her death in 1997. He even suggests that Fischer may have visited Regina's funeral incognito to avoid the American authorities she had always warned him about.

In 2000, Fischer relocated to the far east, splitting his time between Japan, where he formed a romantic relationship with Miyoko Watai, the head of the Japanese Chess Association, and the Philippines. In Japan, by all accounts, he lived an almost normal, settled married life. But in the Philippines, he revelled in an ageing playboy lifestyle common to many wealthy westerners in the poorer areas of east Asia. There were several girlfriends, including one, Marilyn Young, who claimed to have given birth to Fischer's daughter.

The beginning of the end of this double life came on 12 September 2001, when Fischer gave an interview to a Philippines radio station praising the previous day's terrorist attacks on America. In so doing he realerted the US government to his fugitive status. It took a while, but in 2004 the consequences caught up with him when he was imprisoned in Japan over alleged visa irregularities, pending extradition to America. The case dragged on for eight months and Fischer was only saved from American justice by the decision of Iceland, scene of his greatest triumph, to grant him citizenship. He moved to the volcanic island in the north Atlantic and promptly took against most of the people who had secured his liberty. In Bobby Fischer Against the World, there's film of him ranting at one of his last remaining friends, Dr Kári Stefánsson, a neurologist. "His genius and his illness," says Stefánsson, "were joined at the hip."

Fischer succumbed on 17 January 2008 to renal failure in Reykjavik hospital, having declined medical treatment. "Nothing," he is reported to have said in the final days of his life, "is so healing as the human touch."

He was 64 – one year, as was duly observed, for each square of the chess board.

What is Fischer's legacy? Rather as John McEnroe's antics helped raise the profile of tennis, so did Fischer's uncompromising outlook revitalise chess. There are now around 600 million chess players in the world and Bobby Fischer Teaches Chess remains one of the bestselling chess books. He also demanded and got a huge increase in prize money, which attracted more competitors to the game, and improved conditions for the players who followed him. Spassky called him "the honorary chairman of our trade union".

"I doubt I would have been a professional had it not been for Bobby Fischer," says Short. "It's as simple as that."

Fischer made chess seem dangerous, a board game fit for armchair warriors. Much of this was to do with the backdrop of the cold war, but there was also Fischer's personality – driven, arrogant, ultra-competitive. He introduced physical fitness to a game that involved sitting down for five hours. There was also the dazzling quality of his chess.

"For me," says Short, "Kasparov was the greatest player of all time. But I think Fischer burned brighter than anyone else. The difference is he burned brighter for a shorter period."

Ultimately, this may prove the lasting image of Fischer, the brilliant but unstable star who blazed with such intensity that he imploded. There's something cosmically cleansing about the idea that helps vitiate the unpleasant memories of his vile outbursts. That said, it simply wasn't in Fischer's character to go out with a final, conclusive bang. His spirit was endlessly disputatious in life and, as it would turn out, in death.

He left around $2m but no will. Four different parties made contesting claims on the estate: Miyoko Watai, who said she was his wife; Jinky Young, the Filipino girl to whose mother Fischer paid child support; his American nephews; and the American government. Last July, Fischer's body was exhumed for a DNA test that found he was not related to Young. In March this year, an Icelandic district court ruled that Watai was Fischer's legal widow and should therefore inherit his estate. An appeal is likely to be the next stage in the saga.

But eventually all that will live on is the chess. "Just his games," as former US chess champion Larry Evans says in Bobby Fischer Against the World. "That's his monument – his games."

Endgame, by Frank Brady is published by Constable on 26 May, £20. To order a copy for £15.19 inc free UK p&p go to guardianbookshop.co.uk or call 0330 3336847.

The film Bobby Fischer Against the World is released on 15 July