My recent collision with Ben Affleck on Bill Maher’s show, Real Time, has provoked an extraordinary amount of controversy. It seems a postmortem is in order.

For those who haven’t seen the show, most of what I write here won’t make sense unless you watch my segment:

So what happened there?

I admit that I was a little thrown by Affleck’s animosity. I don’t know where it came from, because we hadn’t met before I joined the panel. And it was clear from our conversation after the show that he is totally unfamiliar with my work. I suspect that among his handlers there is a fan of Glenn Greenwald who prepared him for his appearance by simply telling him that I am a racist and a warmonger.

Whatever the reason, if you watch the full video of our exchange (which actually begins before the above clip), you will see that Affleck was gunning for me from the start. What many viewers probably don’t realize is that the mid-show interview is supposed be a protected five-to-seven-minute conversation between Maher and the new guest—and all the panelists know this. To ignore this structure and encroach on this space is a little rude; to jump in with criticism, as Affleck did, is pretty hostile. He tried to land his first blow a mere 90 seconds after I took my seat, before the topic of Islam even came up.

Although I was aware that I wasn’t getting much love from Affleck, I didn’t realize how unfriendly he had been on the show until I watched it on television the next day. This was by no means a normal encounter between strangers. For instance: I said that liberalism was failing us on the topic of Islamic theocracy, and Affleck snidely remarked, “Thank God you’re here!” (This was his second interruption of my interview.) I then said, “We have been sold this meme of Islamophobia, where every criticism of the doctrine of Islam gets conflated with bigotry toward Muslims as people,” and Affleck jumped in for the third time, more or less declaring the mid-show interview over: “Now hold on—are you the person who understands the officially codified doctrine of Islam? You’re the interpreter of that?”

As many have since pointed out, Affleck and Nicholas Kristof then promptly demonstrated my thesis by mistaking everything Maher and I said about Islam for bigotry toward Muslims. Our statements were “gross,” “racist,” “ugly,” “like saying you’re a shifty Jew” (Affleck), and a “caricature” that has “the tinge (a little bit) of how white racists talk about African Americans” (Kristof).

The most controversial thing I said was: “We have to be able to criticize bad ideas, and Islam is the Mother lode of bad ideas.” This statement has been met with countless charges of “bigotry” and “racism” online and in the media. But imagine that the year is 1970, and I said: “Communism is the Mother lode of bad ideas.” How reasonable would it be to attack me as a “racist” or as someone who harbors an irrational hatred of Russians, Ukrainians, Chinese, etc. This is precisely the situation I am in. My criticism of Islam is a criticism of beliefs and their consequences—but my fellow liberals reflexively view it as an expression of intolerance toward people.

And the tension on the panel only grew. At one point Affleck sought to cut me off by saying, “Okay, let him [Kristof] talk for a second.” As I finished my sentence, he made a gesture of impatience with his hand, suggesting that I had been droning on for ages. Watching this exchange on television (his body language and tone are less clear online), I find Affleck’s contempt for me fairly amazing.

I want to make one thing clear, however. I did not take Affleck’s hostility personally. This is the kind of thing I now regularly encounter from people who believe the lies about my work that have been sedulously manufactured by Reza Aslan, Glenn Greenwald, Chris Hedges, and many others. If I were seated across the table from someone I “knew” to be a racist and a warmonger, how would I behave? I don’t honestly know.

Kristof made the point that there are brave Muslims who are risking their lives to condemn “extremism” in the Muslim community. Of course there are, and I celebrate these people too. But he seemed completely unaware that he was making my point for me—the point being, of course, that these people are now risking their lives by advocating for basic human rights in the Muslim world.

When I told Affleck that he didn’t understand my argument, he said, “I don’t understand it? Your argument is ‘You know, black people, we know they shoot each other, they’re blacks!” What did he expect me to say to this—“I stand corrected”?

Although I clearly stated that I wasn’t claiming that all Muslims adhere to the dogmas I was criticizing; distinguished between jihadists, Islamists, conservatives, and the rest of the Muslim community; and explicitly exempted hundreds of millions of Muslims who don’t take the doctrines about blasphemy, apostasy, jihad, and martyrdom seriously, Affleck and Kristof both insisted that I was disparaging all Muslims as a group. Unfortunately, I misspoke slightly at this point, saying that hundreds of millions of Muslims don’t take their “faith” seriously. This led many people to think that I was referring to Muslim atheists (who surely don’t exist in those numbers) and suggesting that the only people who could reform the faith are those who have lost it. I don’t know how many times one must deny that one is referring to an entire group, or cite specific poll results to justify the percentages one is talking about, but no amount of clarification appears sufficient to forestall charges of bigotry and lack of “nuance.”

One of the most depressing things in the aftermath of this exchange is the way Affleck is now being lauded for having exposed my and Maher’s “racism,” “bigotry,” and “hatred of Muslims.” This is yet another sign that simply accusing someone of these sins, however illogically, is sufficient to establish them as facts in the minds of many viewers. It certainly does not help that unscrupulous people like Reza Aslan and Glenn Greenwald have been spinning the conversation this way.

Of course, Affleck is also being widely reviled as an imbecile. But much of this criticism, too, is unfair. Those who describe him as a mere “actor” who was out of his depth are no better than those who dismiss me as a “neuroscientist” who cannot, therefore, know anything about religion. And Affleck isn’t merely an actor: He’s a director, a producer, a screenwriter, a philanthropist, and may one day be a politician. Even if he were nothing more than an actor, there would be no reason to assume that he’s not smart. In fact, I think he probably is quite smart, and that makes our encounter all the more disheartening.

The important point is that a person’s CV is immaterial as long as he or she is making sense. Unfortunately, Affleck wasn’t—but neither was Kristof, who really is an expert in this area, particularly where the plight of women in the developing world is concerned. His failure to recognize and celebrate the heroism of my friend Ayaan Hirsi Ali remains a journalistic embarrassment and a moral scandal (and I told him so backstage).

After the show, a few things became clear about Affleck’s and Kristof’s views. Rather than trust poll results and the testimony of jihadists and Islamists, they trust the feeling that they get from the dozens of Muslims they have known personally. As a method of gauging Muslim opinion worldwide, this preference is obviously crazy. It is nevertheless understandable. On the basis of their life experiences, they believe that the success of a group like ISIS, despite its ability to recruit people by the thousands from free societies, says nothing about the role that Islamic doctrines play in inspiring global jihad. Rather, they imagine that ISIS is functioning like a bug light for psychopaths—attracting “disaffected young men” who would do terrible things to someone, somewhere, in any case. For some strange reason these disturbed individuals can’t resist an invitation to travel to a foreign desert for the privilege of decapitating journalists and aid workers. I await an entry in the DSM-VI that describes this troubling condition.

Contrary to what many liberals believe, those bad boys who are getting off the bus in Syria at this moment to join ISIS are not all psychopaths, nor are they simply depressed people who have gone to the desert to die. Most of them are profoundly motivated by their beliefs. Many surely feel like spiritual James Bonds, fighting a cosmic war against evil. After all, they are spreading the one true faith to the ends of the earth—or they will die trying, and be martyred, and then spend eternity in Paradise. Secular liberals seem unable to grasp how psychologically rewarding this worldview must be.

As I try to make clear in Waking Up, many positive states of mind, such as ecstasy, are ethically neutral. Which is to say that it really matters what you think the feeling of ecstasy means. If you think it means that the Creator of the Universe is rewarding you for having purged your village of Christians, you are ISIS material. Other bearded young men go to Burning Man, find themselves surrounded by naked women in Day-Glo body paint, and experience a similar state of mind.

After the show, Kristof, Affleck, Maher, and I continued our discussion. At one point, Kristof reiterated the claim that Maher and I had failed to acknowledge the existence of all the good Muslims who condemn ISIS, citing the popular hashtag #NotInOurName. In response, I said: “Yes, I agree that all condemnation of ISIS is good. But what do you think would happen if we had burned a copy of the Koran on tonight’s show? There would be riots in scores of countries. Embassies would fall. In response to our mistreating a book, millions of Muslims would take to the streets, and we would spend the rest of our lives fending off credible threats of murder. But when ISIS crucifies people, buries children alive, and rapes and tortures women by the thousands—all in the name of Islam—the response is a few small demonstrations in Europe and a hashtag.” I don’t think I’m being uncharitable when I say that neither Affleck nor Kristof had an intelligent response to this. Nor did they pretend to doubt the truth of what I said.

I genuinely believe that both Affleck and Kristof mean well. They are very worried about American xenophobia and the prospects of future military adventures. But they are confused about Islam. Like many secular liberals, they refuse to accept the abundant evidence that vast numbers of Muslims believe dangerous things about infidels, apostasy, blasphemy, jihad, and martyrdom. And they do not realize that these doctrines are about as controversial under Islam as the resurrection of Jesus is under Christianity.

However, others in this debate are not so innocent. Our conversation on Real Time was provoked by an interview that Reza Aslan gave on CNN, in which he castigated Maher for the remarks he had made about Islam on the previous show. I have always considered Aslan a comical figure. His thoughts about religion in general are a jumble of pretentious nonsense—yet he often speaks with an air of self-importance that would have been embarrassing in Genghis Khan at the height of his power. On the topic of Islam, however, Aslan has begun to seem more sinister. He cannot possibly believe what he says, because nearly everything he says is a lie or a half-truth calibrated to mislead a liberal audience. If he claims something isn’t in the Koran, it probably is. I don’t know what his agenda is, beyond riding a jet stream of white guilt from interview to interview, but he is manipulating liberal biases for the purpose of shutting down conversation on important topics. Given what he surely knows about the contents of the Koran and the hadith, the state of public opinion in the Muslim world, the suffering of women and other disempowered groups, and the real-world effects of deeply held religious beliefs, I find his deception on these issues unconscionable.

As I tried to make clear on Maher’s show, what we need is honest talk about the link between belief and behavior. And no one is suffering the consequences of what Muslim “extremists” believe more than other Muslims are. The civil war between Sunni and Shia, the murder of apostates, the oppression of women—these evils have nothing to do with U.S. bombs or Israeli settlements. Yes, the war in Iraq was a catastrophe—just as Affleck and Kristof suggest. But take a moment to appreciate how bleak it is to admit that the world would be better off if we had left Saddam Hussein in power. Here was one of the most evil men who ever lived, holding an entire country hostage. And yet his tyranny was also preventing a religious war between Shia and Sunni, the massacre of Christians, and other sectarian horrors. To say that we should have left Saddam Hussein alone says some very depressing things about the Muslim world.

Whatever the prospects are for moving Islam out of the Middle Ages, hope lies not with obscurantists like Reza Aslan but with reformers like Maajid Nawaz. The litmus test for intellectual honesty on this point—which so many liberals fail—is to admit that one can draw a straight line from specific doctrines in Islam to the intolerance and violence we see in the Muslim world. Nawaz admits this. I don’t want to give the impression that he and I view Islam exactly the same. In fact, we are now having a written exchange that we will publish as an ebook in the coming months—and I am learning a lot from it. But Nawaz admits that the extent of radicalization in the Muslim community is an enormous problem. Unlike Aslan, he insists that his fellow Muslims must find some way to reinterpret and reform the faith. He believes that Islam has the intellectual resources to do this. I certainly hope he’s right. One thing is clear, however: Muslims must be obliged to do the work of reinterpretation—and for this we need honest conversation.