Claiming that you were “just joking” is a common way to deflect criticism, or anger. You might, for example, mask disapproval of your partner’s driving in a clever remark, preferably with a lilt in your voice and a wink. If this is met with anger, you might then say “but I was only joking” or “fooling around.” It’s a get out of jail free card that is usually worth trying, even if it doesn’t always work.

But exactly why does it ever work? Roughly, jokes have two parts: a set-up and a punchline: “Why did the moron stare at the carton of orange juice? It said ‘concentrate.’” Here the question is the set-up; the answer is the punchline.

Ideally, the punchline gets a laugh. It should be absurd, silly, nonsensical, or as philosophers like to say, “incongruous” – not the way things are or should be. And, because it is patently incongruous, it can’t be taken seriously. It is not a proper assertion, since the teller of the joke cannot be thought to actually believe in the statement. This is why criticism that is veiled as humour can get its teller out of trouble.

But there is no guarantee that this gambit will work. Jokes may be employing figures of speech, like hyperbole, that can be readily translated into assertions. Many racist and ethnic jokes, for example, employ absurd exaggerations in order to disparage the groups they despise.

The refrain “just joking” is not fool-proof. It will not automatically protect racist, ethnic (or sexist, or homophobic) humour from moral criticism. However, a second line of defence may be raised on its behalf. Call it “aesthetic.” Even if such jokes are morally reprehensible, they may be permitted if they are funny. In these cases, wit seems to have a redeeming social value.

The underlying conviction here is that comic amusement and morality are distinct. A joke can be evil at the same time as it is funny. It is never the case that the immorality of a joke can cancel out its humour. A moral defect in joke is not the same as a mirth defect.

This is supposedly true of disreputable jokes. Evidence for this claim is that the stereotypes presupposed by many racist and ethnic jokes are shared both by those who despise the group in question and by members of the targeted group. That is, insiders and outsiders can, in some sense, tell the same jokes. Irish jokers may tell stories about excessive drinking, Jewish jokers may weave fictions of financial cunning, African Americans may revel in tales of hyper-sexuality, and so on.

The “Redneck Jokes” about working-class and hillbilly southerners in the USA sometimes take the form “If ___, you might be a redneck”. For example: “If you’ve been married three times and still have the same in-laws, you might be a redneck.” Jokes like these can be enjoyed by redneck-hating Yankees and by good-old boys alike. They appear funny whether or not they are vehicles of racial or ethnic hatred.

The trouble with this view is that it takes the relevant joke structure to be something like a piece of literature, akin to a short story. But joking is a performing art and the joke text as it might appear in a joke book or on the computer screen, is merely a script, readily and flexibly adaptable in the moment of telling.

Think of how flat a joke is when read silently. It sits there dead on the page, unless you imagine yourself telling it to your mind’s ear or hear inwardly some master joker embellishing it with his or her practiced voice. It’s the joke as delivered rather the joke as written that’s the thing. Whether the instance of the joke tonight in the pub is funny or not depends on the performance. Some joke tellers can provoke roaring laughter with the most unpromising material; others will make a pig’s ear out of a potential silk purse, with timing, accents and delivery all off-key.

But the context in which a joke is delivered relates to its moral valence – and the intentions of the teller. Does someone telling an Irish drinking joke intend it as a malicious put-down, or is it offered in a celebratory spirit as one Irish person to another? Is the attempt to harm, or to cheer?

It is this possibility – that the same joke script may be told with such different intentions – that explains why those who are the butt of a joke may still find it funny. But when the intention of the teller is to demean the group in question, the effect will be very different – predictably indignation rather than laughter.

Why? Comic amusement is not simply a matter of the perception of incongruity. Many incongruities are not amusing. Some are frightening. An infant may laugh when her caregiver makes a “funny face.” But when a stranger adopts an identical expression, the child is apt to cry. Movie zombies are incongruous, but they are not intended to invite laughter.

To make people laugh, a punchline must be free from anxiety or malice. This is a condition of comic amusement. When the joke-teller uses a racial or ethnic stereotype maliciously and the members of the disrespected group recognise this, they will justifiably recoil from what they fear is an attempt to reinforce discrimination and denigrate its target. This sense of anxiety or fear will undermine the attempt at comedy.

If a teller violates these conditions of comic amusement, the joke will fall on its face. Its moral defect will also reflect a defect in the comic address. In cases like this, morality unseats humour. They’re not so distinct after all – especially when the teller of the joke is not really just “fooling around”.

Noël Carroll is distinguished professor of philosophy at the CUNY Graduate Center. He is the author of Beyond Aesthetics and A Philosophy of Mass Art.

This article is part of the Agora series, a collaboration between the New Statesman and Aaron James Wendland, assistant professor of philosophy at the Higher School of Economics.