Big thinkers make for big targets and they don't come much bigger, physically and intellectually, than Daniel Dennett. The tall, 71-year-old philosopher looks every inch the enthusiastic sailor he is, with his white beard and broad torso. With his mild-mannered, avuncular confidence, he comes across a man who could calmly fend off an assault or two. That's just as well, since as a leading cheerleader for Darwin and atheism, he is as much a bete noire for their opponents as a hero for their advocates.

Dennett is in London from his native Massachusetts talking to me about his latest book, Intuition Pumps and Other Tools for Thinking. It's a kind of greatest hits collection, pieced together from mainly previously published work to present both a summation of his central ideas about meaning, consciousness, evolution and free will, and to share some of the philosophical tools he has used to craft them. "After half a century in the field I've got some tricks of the trade which I'd like to talk about," he says.

His critics agree, to the extent that they see him as a clever philosophical trickster. Yet time and again it seems the best way to understand Dennett is to understand why so many criticisms miss their targets.

Most assaults are variations on the theme that Dennett takes a too narrowly scientific worldview. The late Stephen Jay Gould, for instance, coined the "shocking" terms "ultra-Darwinist" and "Darwinian fundamentalist" to hurl at him. Yet Dennett has consistently resisted the crude extension of scientific thinking into areas where it does not belong. Nowhere is this more evident than in the issue of free will, which he describes in the book as "the most difficult and the most important philosophical problem confronting us today".

"It's important because of the longstanding tradition that free will is a prerequisite for moral responsibility," he says. "Our system of law and order, of punishment, and praise and blame, promise keeping, promise making, the law of contracts, criminal law – all of this depends on one notion or another of free will. And then you have neuroscientists, physicists and philosophers saying that 'science has shown us that free will is an illusion' and then not shrinking from the implication that our systems of law are built on foundations of sand."

Dennett argues: "There is nothing we have learned from neuroscience that undercuts the foundation for both the law of contract and criminal law." It is true that we do not have "ultimate responsibility" because our choices are always in some ways the result of things we didn't choose, such as our core personalities and the values we have absorbed from our society and families. But we have enough self-control to make sense of the difference between the psychopath and the criminal murder, the person who murders unwittingly in a sleepwalk and the cold-blooded killer.

That is the kind of free will we need and is worth wanting. Dennett worries that there is good evidence that promulgating the idea that free will is an illusion undermines just that sense of responsibility many scientists and philosophers are worried about losing. Critics maintain that Dennett's kind of free will, with its modest idea of "enough" responsibility, autonomy and control, is not really enough after all.

There's a pattern here, "the story of my life", as Dennett puts it. People assume unrealistic ideals of what free will, selfhood or rationality are and then get upset when Dennett says: "It's not the overwhelming supercalifragilisticexpialidocious phenomenon that you thought it was." But it's still real enough. The problem is simply: "Both free will and consciousness have been, by my lights, inflated in the popular imagination and in the philosophical imagination," and so "anybody who has a view of either one that is chopped down to size" is accused of "a wretched subterfuge", as Kant memorably put it.

This can't be explained wholly in cold, rational terms. There is also an "emotional stake that philosophers often have and betray in their argumentation", which "it's dangerous and also even verges on offensive to draw attention" to. "I see what I think is white-knuckled fear driving people to defend views that are not really well-motivated, but they want to dig the moat a little further out than is defensible because they're afraid of the thin end of the wedge."

Some, however, accuse the "new atheists" of being motivated by fears and less than generous prejudices of their own. The most serious charge is that they attack the most simplistic forms of religious belief and leave their most intellectually subtle versions untouched. Dennett acknowledges "there's some smidgen of truth" in that, but insists: "They bring it on themselves by changing the rules and shifting the goalposts. You ask them: 'What do you actually mean, what do you actually believe?'" But "the real claim won't sit down and be examined and so after a while you get impatient with that. I have made a concerted effort over the years to understand sophisticated philosophical theology and have never come up with anything that I thought could sit in the light of day and be defended."

He also points out: "The spokespeople for religion who chastise the new atheists for this philistine dismissal of their sophisticated work are not speaking for your average churchgoer, certainly not in the United States."

He may not be crudely scientistic, but it is true that these days Dennett spends more time around scientists than other philosophers. "I find the discoveries in those fields mind candy, just delicious," he says. "If I go to a scientific conference I come away with a bunch of new things to think about. If I go to a philosophy conference I may come away just having learned four more wrinkles in the debate about something philosophers have been thinking about for all my life."

But Dennett also maintains that we need philosophy to protect us from scientific overreach. "The history of philosophy is the history of very tempting mistakes made by very smart people, and if you don't learn that history you'll make those mistakes again and again and again. One of the ignoble joys of my life is watching very smart scientists just reinvent all the second-rate philosophical ideas because they're very tempting until you pause, take a deep breath and take them apart."

Ridicule and misrepresentation are in some sense an occupational hazard for the philosopher. "The best philosophers are always walking a tightrope where one misstep either side is just nonsense," he says. "That's why caricatures are too easy to be worth doing. You can make any philosopher – any, Aristotle, Kant, you name it – look like a complete flaming idiot with just a slightest little tweak."

His courting of a popular audience also unfairly undermines his credibility in a profession that has little concern or sometimes even respect for the wider public conversation. His often breezy tone only makes it worse. "They want you to take it seriously!" he mocks in a suitably pompous voice. "I do take it very seriously, but not solemnly."

Dennett is also unashamedly expansive in a way that goes against the academic grain, trying to paint the bigger picture of the biggest issues of the day. In contrast, he believes: "One of the real problems in philosophy is that people can be a little bit myopic." The nadir of this narrow insularity came while Dennett was a graduate student in the 60s. "It was comically cautious and unambitious. The very idea of trying for a large view of anything! This was miniature little piecework stuff. I thought it was dreadful."

Public intellectuals can come in the role of the helpful clarifier or the challenging provocateur. Dennett sees himself as the former, and is "sometimes quite taken aback that people are so provoked". His best explanation for this is that a lot of what he does is about dissolving the mysteries that surround ideas such as selfhood and consciousness. "I think the people who don't like magic tricks explained to them are also the people who don't like free will explained to them, or consciousness. 'How rude! How philistine, to explain, to even try! How dare you!'"

But Dan dares, and, like what he says or not, few do so with such intelligence, insight and flair.