My first thought, on meeting Niall Ferguson, is that he looks too smart to be an academic. It's a Wednesday afternoon, and the Philippe Roman chair in history and international affairs is sitting in his shoebox-shaped office in the Ideas centre at the London School of Economics. Though the setting is hardly glamorous, Ferguson is dressed in the informal-but-smart get-up of a movie executive or hedge-fund manager: suave blue suit, pressed white shirt, gleaming Chelsea boots. His skin is ruddy and his hair is coiffed. Somehow it seems improbable that he has spent the day supervising seminars or reading dissertations. He begins by asking me to wait a few moments. "I'm afraid I have to write a cheque," he says, reaching for his fountain pen. "One of life's more tedious burdens." I stifle an urge to lean over his shoulder and try to catch a glimpse of the number he is etching. Ferguson, one suspects, is used to writing big cheques.

To describe Ferguson as an academic is, of course, to fail to do justice to his lofty position within the intellectual firmament. For he really is, as the LSE website puts it, "one of the world's most eminent scholars". Though perhaps less instantly recognisable than his two main TV historian rivals, David Starkey and Simon Schama, he eclipses both when it comes to scholarly heft and sheer productivity. At 46, he is the author of an astounding number of highly acclaimed, and mostly very fat, books, works such as The World's Banker, The War of the World and The Ascent of Money. (He can't be accused of choosing low-key titles.) His last book, High Financier, was a biography of the banker Siegmund Warburg. Apart from his current one-year posting at the LSE, he is the Laurence A Tisch professor of history at Harvard, the William Ziegler professor of business administration at Harvard Business School, and a senior fellow of the Hoover Institution at Stanford. He has presented numerous television series, served as an adviser to John McCain and written reams of journalism (currently he is a columnist for Newsweek). He gets up at six every morning and says that he doesn't have hobbies: he just works. Whatever you make of the man and his views it is hard not to be impressed by his dedication.

Ferguson's latest book, published next month, is called Civilization: The West and the Rest (the accompanying six-part Channel 4 series starts on 6 March). Coming just eight months after the Warburg biography, it's a book that belongs at the more populist end of the Ferguson oeuvre. In fact, he says, he wrote it largely with his children in mind. (He has three, two sons and a daughter, ranging from 11 to 17.) "The book is partly designed so a 17-year-old boy or girl will get a lot of history in a very digestible way, and be able to relate to it," says Ferguson, who, along with the many other irons he has in the fire, is advising his friend Michael Gove, Britain's education secretary, on how to redraft the history curriculum. "I have a sense that my son and daughter's generation is not well served by the way they are taught history. They don't have the big picture. They get given these chunks, usually about Adolf Hitler, so I wanted to write a book that would be really accessible to them."

Civilization sets out to answer a question that Ferguson identifies as the "most interesting" facing historians of the modern era: "Why, beginning around 1500, did a few small polities on the western end of the Eurasian landmass come to dominate the rest of the world?" In other words, the book attempts to explain the roots of something – western power – that has long fascinated its author. Although Ferguson's background is as a financial historian – his research at Oxford and then Cambridge in the late 80s and early 90s was into German hyperinflation and the history of bond markets – he has, over the past decade or so, drifted increasingly into writing about empire. In two consecutive books, Empire and Colossus – published, not by accident, around the time of the Iraq invasion – he charted the respective imperial histories of Britain and America, concluding not only that Britain should be prouder of its colonial past, but that the world would be a better place if America imitated Victorian Britain and became a fully fledged liberal empire. Though both books were bestsellers and won Ferguson scores of new admirers, especially in the US, they also, not surprisingly, drew heavy criticism from the left.

Civilization, too, starts from the premise that western dominance has been a good thing. In order to explain how it came about, Ferguson deploys an unexpectedly cutting-edge metaphor. The west's ascendancy, he argues, is based on six attributes that he labels its "killer apps": competition, science, democracy, medicine, consumerism and the work ethic. Each chapter of the book (and each episode of the TV series) sets out to explore how it was that western nations possessed one of these "apps", while other nations failed to acquire it. So, in the chapter on competition, he shows how the political structure of western Europe in the early modern era encouraged rivalry both between and within states, while the monolithic rule of the Ming dynasty led China to rest on its laurels. Likewise, in the medicine chapter, he argues that the civilising goals of western European empires produced pioneering medical advances that ultimately benefited the whole world.

Ferguson is clearly more than a little in love with his "killer apps" conceit, as well as his "west versus the rest" dichotomy, which he slips into conversation at every available opportunity. (In the TV series, he even starts talking at one point about "westerners" and "resterners".) Doesn't he worry that this kind of thing detracts from his standing as a serious historian? "No," he says. "Apart from anything else, this terminology is absolutely ubiquitous. And I think it captures something quite important. We actually had a good argument when I first came up with the killer apps concept. Not everyone at Channel 4 liked it. But I just thought it was an absolutely great idea. You explain this book to any group of people and what usually happens is there's a competition to see if I've missed something out. People love it. It's like a game: play Civilization Killer App! It's designed to be slightly annoying, so that you talk about it."

Ferguson is not, it seems, a man given to self-doubt. When I suggest that his views have changed somewhat in the past decade – one moment he was calling on America to establish an empire, now he talks in terms of the west's "civilisational software" being "downloaded" by other countries – he replies: "I'm not sure my position has changed so much as the circumstances." In what comes across as a well-rehearsed spiel, he proceeds to explain why his thought has developed logically across his last six books, and why, on every occasion, his arguments have been prescient. (In the case of the financial crisis, this self-congratulatory impulse is fair enough: he noted that America was seriously over-extended as early as 2004.)

Ferguson's self-confidence – which, if it wasn't accompanied by considerable charm, might be downright insufferable – is no doubt partly a matter of temperament. But it also has something to do with the kind of historian he is. His approach to the past is overwhelmingly materialistic. Questions of right and wrong, or indeed of personality and psychology, don't appear to preoccupy him greatly. What gets him going is hard data, facts and figures – the stuff, in other words, that is most measurable (and, by extension, provable). No doubt this outlook has a lot to do with his grounding in economic history. Yet his materialism goes beyond this, almost to the point, oddly, of seeming Marxian. "Something that's seldom appreciated about me," he declares, "is that I am in sympathy with a great deal of what Marx wrote, except that I'm on the side of the bourgeoisie."

When it comes to thinking about empire, Ferguson's preoccupation with material forces allows him to undertake what amounts to a cost-benefit analysis, weighing the good that imperial regimes have done against the bad without being unduly bothered by the kind of moral questions that traditionally concern the left. (Does one country have a right to invade another? Does colonialism leave a psychological scar that makes it hard for previously occupied countries to progress?) He is able to remain relatively sanguine about the less than glorious aspects of, say, Britain's occupation of India, or French rule in west Africa, because he always seeks to ask what the alternatives might have been. (As a rule, he thinks they would have been far worse.) "The moral simplification urge is an extraordinarily powerful one, especially in this country, where imperial guilt can lead to self-flagellation," he explains. "And it leads to very simplistic judgments. The rulers of western Africa prior to the European empires were not running some kind of scout camp. They were engaged in the slave trade. They showed zero sign of developing the country's economic resources. Did Senegal ultimately benefit from French rule? Yes, it's clear. And the counterfactual idea that somehow the indigenous rulers would have been more successful in economic development doesn't have any credibility at all."

As many of his critics have noted, Ferguson's emergence as an advocate of empire coincided with the rise of neoconservatism in the US and the drive to displace Saddam Hussein from power in Iraq. (In the run-up to the war, Ferguson was a vocal supporter of invasion.) Although he became critical of US policy once the occupation took place, and now distances himself from the neocons, he remains unrepentant about his pro-war stance, arguing that the real problem was that the invasion was "botched" because of the Bush administration's failure to commit sufficient manpower and resources to it. ("The problem I constantly wrote about then was that if you invade and overthrow the bad guy, hold elections and then piss off, it doesn't work.") Nor does he rule out supporting similar campaigns in future. "It's all very well for us to sit here in the west with our high incomes and cushy lives, and say it's immoral to violate the sovereignty of another state. But if the effect of that is to bring people in that country economic and political freedom, to raise their standard of living, to increase their life expectancy, then don't rule it out."

Ferguson, who describes himself as a "classic Scottish enlightenment liberal", clearly enjoys provoking the left, which he does with a relish that at times borders on callousness. At one point he remarks: "I think it's hard to make the case, which implicitly the left makes, that somehow the world would have been better off if the Europeans had stayed home. It certainly doesn't work for north America, that's for sure. I mean, I'm sure the Apache and the Navajo had all sorts of admirable traits. In the absence of literacy we don't know what they were because they didn't write them down. We do know they killed a hell of a lot of bison. But had they been left to their own devices, I don't think we'd have anything remotely resembling the civilisation we've had in north America."

Yet when our conversation moves to more personal matters, all traces of lofty detachment disappear. In the past couple of years, Ferguson's professional interest in civilisations and the relations between them has intersected intriguingly with his private life, thanks to his relationship with the dedicatee of Civilization, who is identified in the book only as "Ayaan". In his preface, Ferguson writes that she "understands better than anyone I know what Western civilisation really means – and what it still has to offer the world".

"Ayaan" is Ayaan Hirsi Ali, the Somali-born writer, activist and former Dutch MP who has emerged as one of the west's most strident critics of Islam, especially its treatment of women. Ferguson met her in May 2009, at Time magazine's annual "100 most influential people in the world" party. (Both Ferguson and Hirsi Ali have previously appeared on this list, in 2004 and 2005 respectively.) They embarked on their relationship a few months later, shortly after his separation from his wife of 16 years, Sue Douglas, a former editor of the Sunday Express. Their affair soon prompted a storm of gossip and the publication of several muck-raking articles.

Ferguson's relationship with Hirsi Ali is further complicated by the fact that she lives under constant police protection. In 2004, she wrote the script for the short film Submission, which attacked Islam's subjugation of women and contained shots of a woman's naked body inscribed with verses from the Qur'an. The film's director, Theo van Gogh, was assassinated by an Islamic extremist in Amsterdam soon after its release; pinned to his body was a letter calling for a jihad against Hirsi Ali. Being forced into hiding certainly hasn't made her any less outspoken. In 2006, she told a German magazine that Islam is "not compatible with the liberal society that has resulted from the Enlightenment."

As soon as he starts talking about Hirsi Ali, Ferguson's demeanour changes. His voice becomes softer, infused with feeling. Suddenly, he is no longer the super-confident scholar; he seems almost humble. "Ayaan comes from a completely different civilisation," he says, explaining what he meant by saying she knows what western civilisation "really means". "She grew up in the Muslim world, was born in Somalia, spent time in Saudi Arabia, was a fundamentalist as a teenager. Her journey from the world of her childhood and family to where she is today is an odyssey that's extremely hard for you or I to imagine. To see and hear how she understands western philosophy, how she understands the great thinkers of the Enlightenment, of the 19th-century liberal era, is a great privilege, because she sees it with a clarity and freshness of perspective that's really hard for us to match. So much of liberalism in its classical sense is taken for granted in the west today and even disrespected. We take freedom for granted, and because of this we don't understand how incredibly vulnerable it is."

In talking of liberalism in its "classic" sense, Ferguson is perhaps pointedly drawing a distinction between the liberalism espoused by Hirsi Ali (and himself) and that of certain left-leaning liberals – notably Timothy Garton Ash and Ian Buruma – who have been critical of her anti-Islamic stance. The French philosopher Pascal Bruckner depicted their attitude as the "racism of the anti-racists". Does Ferguson agree? "I think Ayaan's critics – Ian Buruma in particular – were more guilty of sexism than racism," he says. "But certainly they underestimate her intellectual rigour at their peril. She's just smarter than they are, as well as having a great deal more courage. I mean, there aren't many people who really put their life on the line for human freedom. And I think when you come across someone like that you've got to be a little bit respectful. It just sticks in my throat a bit to have middle-aged men who've had cushy lives turning up their noses at someone who has gone through what she's gone through. There's a particular role you're supposed to play as an oppressed woman... you're supposed to smile and look pretty and not say too much."

I ask whether Ferguson has been surprised by the reaction their relationship provoked, the gossipy articles and so forth. His tone changes again and he suddenly sounds angry. "I was nauseated. Just nauseated. It makes me quite ashamed to be part of a culture that regards the private life of a professor as something that should be in the paper. It's just so tawdry. The British press has an insatiable appetite for making public things that should be private. It's a prurience that I've never understood. I don't give a monkey's about the so-called celebrities that they write about. But the idea that my private life should be the subject of articles I find deeply, deeply infuriating. Because there's absolutely no way to control or resist that process unless you're very rich, which I'm not. They of course claim I am by massively magnifying my income." (Various articles put Ferguson's annual earnings at $5m, a figure he labels "ridiculous".)

Ferguson has long been somewhat ambivalent about Britain – he quit his Oxford professorship in 2002 to teach in America. He seems invigorated by the prospect of returning there, which he plans to do as soon as the current academic year ends. Near the end of our conversation, he talks of how, growing up in Glasgow, where there "wasn't a lot to do except football and drinking", he immersed himself in American culture. "I read Steinbeck, Fitzgerald, Kerouac. And I listened to American music. I remember once after school going to see Woody Allen's Manhattan and thinking: I want to be there. And as soon as I arrived in New York, I just felt at home."

In Britain, by contrast, he says that "the abuse of the freedom of the press has now reached the point... where it's no longer tolerable". He decreasingly feels at home here and says he "really only took this job so I could see more of my kids". It's a damning verdict from a man who clearly has a huge love of British history, and who acknowledges that he owes much to the country, not least his education. But there's a final twist to the tale. One thing he "hasn't missed at all about England", he says, is the experience of being "condescended to" by public school boys. "I have a very negative relationship to the aristocracy. And having them out my life is on balance a benefit." So it's something of an oddity (and yet typical of a man who clearly enjoys being contradictory) that the one thing Ferguson says gives him hope for Britain's future is David Cameron's government – made up overwhelmingly of public school boys.

Killer 'apps': the ideas that propelled the west to world domination

1.Competition: In the 15th century, China was the most advanced civilisation in the world, while Europe was a backwater. But then things changed and by the late 18th century Adam Smith could observe that China had been "long stationary". What happened? Ferguson argues that Europe's fragmented political structure led to competition and encouraged Europeans to seek opportunities in distant lands. The increasingly insular China, by contrast, stagnated.

2. Science: The 16th and 17th centuries were the age of science, with an extraordinary number of breakthroughs occurring. This revolution was, Ferguson writes, "by any scientific measure, wholly European". In the Muslim world, clericism curtailed the spread of knowledge, while in Europe, aided by the printing press, the scope of scholarship dramatically widened. Ultimately, breakthroughs in science led to improvements in weaponry, further cementing the west's advantage.

3. Property: Why did the empire established by the English in north America in the 17th century ultimately prove so much more successful than that established by the Spanish in south America a century earlier? It was, Ferguson contends, because the English settlers brought with them a particular conception of widely distributed property rights and democracy, inherited from John Locke. This proved a far better recipe for success than the Spanish model of concentrated wealth and authoritarianism.

4. Modern science: According to Ferguson, modern medicine was the west's "most remarkable killer application". Western medical advances in the 19th and 20th centuries increased life expectancies across the world, including in the colonies. The French in particular, largely thanks to a lofty conception of their imperial mission, brought significant improvements to public health in western Africa, developing effective vaccinations for diseases such as smallpox and yellow fever.

5. Consumption: The west's dominance of the rest of the world was not only achieved by force; it was also, as Ferguson shows, achieved through the market. The industrial revolution in 18th and 19th century Britain created a model of consumerist society that has proved irresistible, as shown, for example, by the way that the western style of dressing has swept the globe. Yet there's a paradox: how was it that an economic system designed to offer infinite choice has ended up homogenising humanity?

6. Work ethic: As Max Weber noted a century ago, Protestantism was a form of Christianity that encouraged hard work (and just as importantly, Ferguson adds, reading and saving). It isn't a coincidence, he says, that the decline of religion in Europe has led to Europeans becoming the "idlers of the world" (while the more religious US has remained hard-working). Interestingly, Ferguson also argues that China's embrace of hard work is partly because of the spread there of Protestantism.

Niall Ferguson's Civilization begins on 6 March on Channel 4