On the morning of September 11, Francis Fukuyama was working in his seventh-floor office at the School of Advanced International Studies in Washington. When American Airlines flight 77 crashed into the Pentagon he crossed to the other side of the building and watched the smoke rising over the city. While his first thoughts were about the safety of himself, his family and friends who worked for the defence department, it soon became clear that Fukuyama was in for a period of professional as well as personal anxiety. The following week the Guardian diary deadpanned that, "efforts to contact thinking-man's thinker Francis Fukuyama to ask whether he feels moved to publish a sequel to his 1989 essay The End of History prove fruitless". It seemed that every other media outlet had had the same idea, and the man who had so spectacularly anticipated the collapse of communism, declared that the alternatives to liberal democracy had exhausted themselves and had been described as the "court philosopher of globally triumphant capitalism" was called to account.

Fukuyama was no stranger to his ideas being scrutinised, challenged and occasionally publicly ridiculed. When the Berlin Wall fell a few months after his essay first appeared it seemed to cement his status as both prophet and sage. But within a year the Gulf war had been and gone. Then, in 1993, the Harvard political scientist Samuel Huntington published a competing theory that said, rather than arriving at the end of history, we were about to be launched into a clash of civilisations. Following the subsequent war in the Balkans and genocide in Rwanda, the Nation magazine summed things up with the front-page headline "The End of Fukuyama". Following the New York and Washington attacks, Huntington's stock inevitably rose further as Fukuyama's declined. But as the communitarian thinker Amitai Etzioni points out, Fukuyama "is one of the few enduring public intellectuals. They are often media stars who are eaten up and spat out after their 15 minutes. But he has lasted."

Fukuyama spent the autumn thinking about the implications of September 11, and the spring teaching a postgraduate class on the subject. "It was obviously a huge analytical challenge," he says, speaking in the same Washington office. "The question was 'what the hell were we confronting here?' I didn't want to be one of those people who stake out a position then stick to it, even though it has become untenable. This was a whole new set of data coming at us from the real world." His hopeful conclusion was that the terrorism was in essence a last-gasp, rearguard action "by a culture that will over time be modernised". "Even within the Islamic world," he explains, "the hijackers do not represent a dominant trend, and over time they will have to confront modernisation, and modernisation will win."

It is alleged that when Margaret Thatcher first heard about Fukuyama's theory her response was "end of history? the beginning of nonsense!" But his theory is, of course, more subtle and intellectually complex than the apparent simplicity of the title implies. History in this sense is the direction of history, not the history of events or people. Liberal democracy is the culmination of ideological evolution, with no non-barbarous serious systemic alternative, and so the debate over fundamentals has been concluded.

The theory draws heavily on Hegel and, more specifically, the way the 20th-century French-Russian philosopher, Alexandre Kojeve, interpreted Hegel. It was Kojeve who asserted that Hegel had been essentially right in claiming that history, in this sense, really came to an end when Napoleon, representing the ideals of the French revolution, defeated the Prussians at Jena in 1806. And the motor for the global progress towards liberal democracy is the human need for recognition which, for instance, ensures that people eventually reject even prosperous dictatorships. "Hegel saw rights as ends in themselves," wrote Fukuyama," because what truly satisfies human beings is not so much material prosperity as recognition of their status and dignity."

Even those who reject his conclusions acknowledge that they are consistent and intellectually robust. Professor Alex Callinicos of the University of York has written about Fukuyama from a Marxist perspective. While he complains, for instance, that Fukuyama too readily equated socialism only with what happened in the Soviet Union, he acknowledges that Fukuyama's theory is difficult to refute. "He has a sufficiently sophisticated and supple theoretical framework for him to look at something like the Gulf war and say it is the wrong sort of history. He says the advanced capitalist societies of the west have gone beyond history and ideological conflict, but the more backward bits of the world are still trapped in history. It gives him a philosophical framework that allows him to explain away things like Islamic fundamentalism and September 11."

That is not to say his ideas have been universally accepted. Initial objections from the right claimed he was too optimistic about the death of communism, and the left said he was too complacent about the contradictions in western societies. Some of the most barbed criticisms came from Fukuyama' s former colleagues at Harvard. He had written that communism was finished, although "there may be some isolated true believers left in places like Managua, Pyongyang or Cambridge, Massachusetts". Daniel Bell, the Harvard social scientist whose book The End Of Ideology was published 30 years before Fukuyama, responded by saying Fukuyama's work was "Hegel at third remove", and also "wrong". More recently, a critique has come from the movement against corporate globalisation, which accepts the western values of freedom but argues that the system is failing to deliver.

Even some of those closest to the theory's genesis have their reservations. Owen Harries was editor of the conservative journal The National Interest that published the essay in 1989. "I never bought the thesis in its entirety," says Harries, "but I thought it was brilliantly interesting and provocative. Frank thought that what was happening spelt the end of the realpolitik world. I think that is a way of talking about the [kind of] world that has stood up pretty well from Thucydides to Kissinger, and it was a bit premature to say it was all going for a burton on the basis of three or four years of Gorbachev." But Harries is nevertheless anxious to protect Fukuyama from the "vulgarisers who have never read him properly and think he is saying that history is going to end tomorrow. Even in his boldness he was much more careful than that. He said there would be huge remnants of the old that would persist and he was really talking about the main trend. He allowed for all sorts of hold-ups."

Perhaps Fukuyama's greatest influence is that everyone else has subsequently had to define their position in relation to the ground he staked out. Robert Cooper was until recently Tony Blair's personal adviser on foreign affairs and is about to take up a job in Brussels with the EU's security and foreign policy chief, Javier Solana. Speaking in a personal capacity, he confirms Fukuyama's impact at the highest levels. "I'm a fan," he declares. "I think the point he was making that there is only one viable system and only one superpower is undeniable. The world Fukuyama described is an accurate description of the world we operate in."

Amitai Etzioni reinforces the point. "I've just come back from Iran and I was stunned as to how much western values are winning. It looks from the outside that it is all Islamic fundamentalism, but if you talk to anyone they don't want to wear the scarves, treat women like second-class citizens, and they want a drink. The western trend is unmistakeable."

Although September 11 forced Fukuyama to go public with his latest thinking, he had in fact already been reassessing his theory for some time. "The one unanswerable criticism of the whole End of History idea was 'how can you talk about an end of history if you don't have an end of science'," he explains. "Further technological progress establishes your politics and what is possible in the economy, and so forth. And with the recent huge explosion in the life sciences, it is obvious that we are not at the end of science."

He has recently joined President Bush's bioethics committee and his new book, Our Posthuman Future [serialised next week in the Guardian], examines developments in the life sciences and assesses the implications for the social sciences. It is his third book since The End of History - the essay later appeared as a book - and follows his 1998 work, The Great Disruption, which looked at 40 years of rising crime and disintegrating families in the west and optimistically proposed that the trend was coming to an end. His 1995 book, Trust, posited the concept of social capital and looked at how the way people relate to each other socially relates to the way they perform economically and politically.

Harries says if there is an identifiable trend in Fukuyama's work it is an underlying interest in human nature, a view shared by JK Galbraith who, in reviewing Trust, said "his work is important not least because it provides a necessary antidote to a restricted view of human nature shared by extremists on both left and right. Marxists on the left, and rational utility maximisers on the right, forget that neither man nor woman lives by bread alone. We hairless monkeys have cultivated higher aspirations."

Of course, Fukuyama is aware that any discussion of human nature, genetics, race and sex is an incendiary combination. "I went into this territory a little in my last book and felt the heat," he smiles. "But it seems to me that a lot of things about the world are crying out to be said, and people don't say them for various reasons." While he says he doesn't think of himself as being constitutionally suited to public controversy, his wife has pointed out how often he gets involved, "so that obviously can't be true". He says culturally the Japanese don't like to argue but that he is probably one step on from his father, who was born in America, and who, while having "a certain public reticence, didn't stop arguing a lot with me at home".

Fukuyama was born in Chicago in 1952 into a family of academics. His maternal grandfather had founded the economics department at Kyoto University and was part of the Japanese generation that went to Germany for their education before the first world war. A by-product of this saw Fukuyama inherit a first edition of Marx's Das Kapital. He says because his mother came from this westernised elite - "she was brought up listening to Beethoven" - and his father was a sociologist and an ordained protestant minister, Fukuyama did not learn Japanese or even see many Japanese people when he was growing up.

His paternal grandfather was forced to sell his hardware business "for a pittance" and move from Los Angeles to a detention camp in Colorado after Pearl Harbour in 1941. Fukuyama's father avoided detention by winning a scholarship to go to university in Nebraska. He then moved to the University of Chicago, where he met Fukuyama's mother. Francis is their only child, and soon after he was born the family moved to Manhattan where he was brought up.

Fukuyama says his father's work for the Congregational Church - "an old Protestant line, very left-wing" - was a source of friction between them. "That kind of Protestantism is barely religion, and while he was obviously religious in some sense, he spent most of his life looking down his nose at fundamentalists and people with any more direct form of spirituality. For him, religion was mostly social activism and politics." A few years ago Fukuyama and his wife, Laura, joined a Presbyterian church, but he says he is not active in the church and is "probably an agnostic. I've found it quite a struggle to think of myself as a believer."

By the time he went to Cornell University in 1970 to read classics - he learned Attic Greek to do it, and also has French and a reading knowledge of Russian and Latin - he was already politically a conservative. At Cornell he came into the orbit of Professor Allan Bloom, who went on to write the best-selling 1980s conservative assault on moral relativism, The Closing Of The American Mind, and was, posthumously, the eponymous model for Saul Bellow's novel Ravelstein.

Fukuyama arrived at Cornell just after student protests had virtually shut down the university. "They were on the cover of Time Magazine, wearing bandoliers of ammunition. It was a horrible spectacle because basically the whole university administration capitulated to them. They admitted it was a racist institution, and that academic freedom didn't exist. Bloom was part of a group of professors who were outraged by the whole thing and left Cornell, but he owed one more semester, which I took." Fukuyama says "the first half of Bellow's novel is very good in showing what a charismatic teacher he was. My interest in human nature really dates from then." In a series of neat links it was Bloom who first translated Kojeve's work into English, and then in 1989 it was Bloom who invited Fukuyama to deliver his End of History lecture in Chicago.

Fukuyama went on to postgraduate study in comparative literature at Yale and then had six months in Paris studying under the high priests of deconstruction, Roland Barthes and Jacques Derrida. He says now "perhaps when you're young you think that something must be profound just because it is difficult and you don't have the self-confidence to say 'this is just nonsense'". He did write a novel while in Paris that remains in a drawer, but by the time he returned to Harvard to complete his comparative literature course he was so disillusioned that he changed courses to political science. "It was like this huge burden being taken off my shoulders when I changed over," he says. "It was a huge relief to move from these very academic and abstract ideas to very concrete and real policy problems of Middle East politics, arms control and that sort of thing."

Professor Nathan Tarcov of the University of Chicago first met Fukuyama at Harvard. In 1989 he was the joint sponsor with Bloom of Fukuyama's End of History lecture. "Frank was very bright and had this interesting and unusual intellectual trajectory that had taken him to Paris," recalls Tarcov. "As a social scientist and policy person he was interested and knowledgeable about literary and aesthetic matters, which was a rare combination." Etzioni says Fukuyama is unusual among public intellectuals in being "a very fine human being and a joyous guy to be with. You want to read a lot of these other people but you don't want to be near them."

He completed his thesis on the Soviet threats to intervene in the Middle East and in 1979 joined the RAND Corporation, the huge public policy organisation based in Santa Monica. He has been associated with them ever since. It was also in California that he met his wife, Laura Holmgren, who was a graduate student at UCLA. They live near Washington and have three children, Julia 14, David 12 and John 10.

James Thomson is now president and chief executive officer at RAND. He remembers Fukuyama being someone who "got into sub jects that other people hadn't thought about. I remember he did a great job on a project for the air-force on Pacific strategy. He told them things they didn't want to hear and had a real skill in that; getting people to listen and then see the rationale. If he had chosen he could have taken on increasingly senior leadership roles but he didn't want to give up the freedom to roam intellectually."

This freedom has been part of why he has never sought elected office. "Although I have strong ideas about policy, particularly foreign policy," Fukuyama says, "there is too much hand-shaking and baby-kissing for me. And you have to simplify things so much. I would never feel happy to say the things you do to get elected." While Ronald Reagan is one of the politicians he admires, Fukuyama says during the 80s he was uncomfortable with what he saw as his oversimplifications, "but in retrospect I appreciate him more. I think the way he could cut through things is what made him a great president and I think the consensus on Reagan had shifted. It is hard now not to recognise that he represented a set of coherent ideas that changed the landscape for a generation."

Having worked in the state department during both the Reagan and first Bush administrations, Fukuyama is close to many at the heart of George W's presidency. The hard-liner Paul Wolfowitz - now deputy secretary of defence - as Reagan's director of policy planning in 1981 invited Fukuyama to join his staff . He has also known national security advisor Condoleezza Rice since they were in college. Fukuyama says, "Every day I am really glad I am not in their position making those sorts of decisions," he says.

In those days his spheres of operation couldn't have been more sensitive and nearly all have an eerie resonance with the key geopolitical questions of today. His early reports at RAND were on security issues relating to Iraq, Afghanistan and later Iran. He also wrote an influential report on Pakistan in the immediate aftermath of the Soviet invasion of Afghanistan. He recalls being a raw 28-year-old just out of graduate school and making contact with the ISI, the controversial Pakistani intelligence service. "No one knew anything abut the mojahedin and so I spent two weeks there getting briefed. I concluded that the mojahedin should be supported and in order to do that the Pakistani military needed to be bolstered. I then took up my job at the state department and the next thing the Reagan administration did was give Pakistan some F16s. I had nothing to do with that decision, although I supported it, but it made me one of the most unpopular people in the Indian sub-continent and for the next six months I was routinely denounced in the Indian press as the man who had arranged this."

During his first two year spell in government he was a member of the US delegation to the Egyptian-Israeli talks on Palestinian autonomy. He then went back to RAND, but following George Bush's 1988 election win Fukuyama was re-appointed to the state department as deputy director of policy planning working for secretary of state James Baker. This was the period in which he made his name. It was his policy advice that was best attuned to the rapid overturn of the old world order. As early as May 1989 he wrote a memo to Baker urging consideration of German reunification, while up until late October, a month before the Wall fell, the German experts in the department were saying reunification would never happen within their lifetimes. Next he was the first to propose planning for the dissolution for the Warsaw Pact. It was a line again first treated with incredulity by the career Kremlinologists.

"That was a fascinating year," he smiles. "The advantage I had was that I was about six months ahead in my thinking. It was increasingly obvious to me that the Soviet ice flow was melting very quickly. Usually the problem in government is that things move so slowly, but that year the problem was that people were stuck in their grooves. They said the future would be the reformed communists, and then they were swept away. Then they said it's happened in Hungary but it certainly won't happen in East Germany, and that turned out to be wrong. It was very intellectually bracing that the absolute boldest thinker always won that year."

He thinks the Bush administration got it just about right by not freezing the Soviet position and settling for what six months earlier would have been a spectacularly successful result but would, for instance, have left a divided Germany. Robert Cooper describes that year as "the best period of US foreign policy ever", and the triumph of the west fed in to the cult of Fukuyama. Nathan Tarcov had asked him to give the lecture before he was appointed to government, but by the time it was delivered he was part of the administration. For the first few weeks after the subsequent National Interest article - although the magazine sold out - there was little initial feedback. "And then it took off vertically," recalls Owen Harries. "The Washington Post, Time magazine, Newsweek, the Wall Street Journal all ran long articles. There was this huge-felt need to define this post-cold-war world that was coming into being."

The move into the mainstream precipitated some gross distortions of his message - James Thomson established a Fukuyama scale to measure the degree of denunciation by people who haven't read your book - but it provoked some unexpected reactions amongst those who did read it. While in America it attracted more intellectual interest on the right, in Britain it was the left who took Fukuyama most seriously. Alex Callinicos says in part this was a consequence of the theoretical language he used. "Philosophy, history and Hegel represent familiar terrain for lots of people on the left and he got quite a positive response," he says. "To my mind it was too positive, although the theory has survived longer than I first thought. I think what he has written since seems much less interesting and he is really a one-idea man, but it has proved a big idea."

When Fukuyama won a $600,000 deal to write the book he resigned from government. "My wife says that I always wanted to be a former government official," he laughs. As a government official, "you have no family life and most of what you do is mind-numbingly dull and stupid." While he says it was, of course, exciting to have access to political decision makers, the reality of government is that "you get very familiar with the details of policy but six months later the issues are all different and you are not left with very much for your work. Kissinger used to say that in government you spend your intellectual capital, you never build it up, and that is fundamentally true. I like what I am doing now. I like being an academic, writing books and thinking about interesting things. I like the freedom to learn and write about other things and it's also satisfying to shape public debate about issues that I think are important."

Although he is seen as a product of the American right, and much of his work has been sponsored one way or another by conservative think tanks, not all of his thinking fits with orthodox rightwing theory. In his new book he argues for more regulation of biotechnology, and he thinks it is a good thing that the downsizing of government begun with Reagan and Thatcher is now being reversed. "You saw a very arrogant techno-libertarianism among the elite of Silicon Valley," he says. "They said they were now the real value creators and the government should just get out of their way. But it turned out that they were creating sham value. The one basic truth that came out is that everyone in society is inter-dependent and you need things like government."

He is also not as isolationist as some of his political soul-mates. "I was upset that Bush did not mention Britain in the State of the Union address. Blair went out of his way to support American policy after September 11. I know it makes sense for Britain to do that as it maximises their influence by being the one European country that is cooperative, but it should be a two-way street." He sees a possible American split with Europe as the most troubling political cloud on the horizon. "I think it is probably going to get worse before it gets better. A lot of that rift I blame the Europeans for. But I do think the United Sates could have done a better job of trying to explain itself. I think that we should take more care of that relationship. Europe has gotten to a state where they don' t like conflict and they don't like pushing themselves in front on issues generally. One thing that has bothered a lot of Americans is that when confronting terrorism a lot of Europeans seem to think 'if we duck they won't hit us, and we will continue to live in peace'. I don't think that is a very responsible attitude in the long run. It breeds a resentment on the part of Americans, which just isn't healthy."

Despite the success of Trust And The Great Disruption, his career as an academic, "and things going well with my family and so on", Fukuyama was out of sorts with the 1990s. But although he has a particular dislike of President Clinton he does have a surprising soft spot for the radical baby-boomer generation. He has noticed that in recent college generations "there has been a basic lack of curiosity about anything other than that which would help get them a good job. Bloom felt this too. In a way he loved arguing with all those student radicals because they did care passionately about ideas and justice and they worried about things larger than themselves."

And for someone who has been accused of a naively Candidian level of optimism about the ultimate and just victory of western values, it is perhaps unsurprising that he even sees silver linings emerging from the clouds of smoke over Washington and New York. "In the 90s there was a really unpleasant self-satisfaction with a simple economic prosperity. You think about the issues people cared about - like the dot.com bubble and Monica Lewinsky - and they are utterly trivial. One thing that has happened since September 11 is that people have been reminded that serious matters happen in the world. That puts a certain perspective on things."

Life at a glance Francis Fukuyama

Born: October 271952, Chicago.

Education: BA Classics, Cornell 1974; PhD Soviet Foreign Policy, Middle Eastern Politics, Harvard 1981.

Married: Laura Holmgren, one daughter and two sons; Julia 14, David 12 and John 10.

Career: RAND Corporation 1979-'81, '83-89, '90-94; State Department policy planning staff 1981-82, deputy director policy planning, '89-90; professor of public policy, George Mason University 1996-2001; professor of international political economy, Johns Hopkins School for Advanced International Studies 2001-.

Books: The Soviet Union And The Third World (edited with Andrzej Korbonski) 1987; The End Of History And The Last Man '92; Trust: The Social Virtues And The Creation Of Prosperity '95; The Great Disruption '99; Our Posthuman Future: Consequences Of The Biotechnology Revolution 2002.

· Our Posthuman Future is serialised in G2 next Monday and Tuesday. The book is published by Profile Books on May 20 at £17.99. To order a copy for £15.99 plus p&p call the Guardian book service on 0870 066 7979. Francis Fukuyama leads a debate about the future of human genetics on May 30 at 7pm, Institute of Education, 20 Bedford Way, London, WC1H. For tickets call 0207 269 9229 or 0207 269 9230.