In an essay from 2003 – The Problem of Thinking Too Much – the statistician Persi Diaconis recounts being unable to make his mind up about a move from Stanford University to Harvard. After much discussion, a colleague says: “You’re one of our leading decision theorists – maybe you should make a list of the costs and benefits and calculate your expected utility.” To which Diaconis replies: “Come on, Sandy, this is serious.” Even statistical decision theorists do not make serious choices by consulting cold, textbook models. Like the rest of us, they resort to a knottier combination of deliberation, gut feel and blind hope. For choices, so too for beliefs, which, when met with evidence, are pushed and pulled by processes that are equally mysterious.

The Australian philosopher Eleanor Gordon-Smith’s first book, Stop Being Reasonable, explores how we really go about changing our minds. Her approach is to probe six real-life stories of people who have had to radically rethink in high-stakes situations. Like Dylan, who, in a flash of insight, realises the apocalyptic cult that he was born into is bogus. Or upper-crust Alex, who appears on a reality TV show in which he has to fake it as a London bouncer and ends up with an unrecognisably new identity. Each story functions as “a miniature of a much larger complex sprawl”, and collectively they give a sense of the richness and strangeness of human reason.

The book came out of an item that Gordon-Smith produced for the radio programme This American Life. Her idea was to wander around a nightlife district of Sydney, wait for the predictable catcalls to start and then try to dissuade her catcallers from engaging in this type of behaviour again. But despite her orderly arguments, the catcallers left the conversations still convinced that it was “OK to grab, yell at, or follow women”. Gordon-Smith’s takeaway from the experience was that people are often unmoved by dispassionate logic, peer-reviewed research and statistics, but in fact are swayed by ego, emotion, self-interest and identity. If we want our public discourse to succeed in changing attitudes, Gordon-Smith insists, we have to ditch our idealised, sterile picture of persuasion and be more sensitive to how people behave in real life.

She has a nose for identifying the highlights of this difficult academic literature, and a talent for making it relevant

Orthodox theories of rationality adopt the sterile picture – the kind of thing a statistician like Diaconis works on – as a model of correct reasoning. Gordon-Smith takes a heterodox approach, suggesting that we can discover the secrets of rationality in passionate, messy real-world reasoning – her idea is that the way Diaconis actually makes up his mind may have a lot to recommend it. Her six stories – of Dylan, Alex and others – function both as portraits of fringe psychology and as corner cases for a theory of rationality.

Contemporary epistemology, the branch of philosophy that deals with these areas, addresses fascinating and fundamental topics, such as the relationship between reason and emotion, or the role of ethics in the pursuit of truth. It can also be somewhat divorced from real life: one of its subfields analyses the concept of knowledge by devising thought experiments about, for example, mules disguised as zebras. Gordon-Smith has a nose for identifying the highlights of this difficult academic literature, and a talent for making it relevant and accessible (a phenomenon termed “testimonial injustice” is glossed as “the bullshit when people dismiss your words for no good reason”).

Her book also raises difficult questions. For instance, sometimes all of us just reason sloppily, and any theory of rationality should take this into account. A potential pitfall for an approach to rationality like Gordon-Smith’s is that this basic observation is in danger of being overlooked: by tightly tethering the normative to the descriptive her proposal risks leading to the absurd consequence that all human reasoning is deemed rational. For example, are we comfortable accepting that climate change denial, or for that matter believing that women enjoy being catcalled, are appropriate responses to the evidence? No doubt Gordon-Smith would shrink in horror at the suggestion. But her book flirts with this odd claim and we are never told how her view distinguishes psychology from rationality, or bad reasoning from good.

Gordon-Smith’s silence on this question – of what rationality requires of us – might seem like a shortcoming, but in the end it is strategic. She debunks easy answers while shrewdly adopting a position of intellectual humility. She thinks we are ignorant of rationality’s demands, and “very close to the edge of what we know how to talk about at all sensibly”.

It is sometimes held that rationality defines us as human, a claim written into our species name, Homo sapiens. If this is right, it follows from Gordon-Smith’s witty, intelligent book that, like the people she profiles, we do not really know who, or even what, we are.

• Stop Being Reasonable: Six Stories of How We Really Change Our Minds by Eleanor Gordon-Smith is published by Scribe (£14.99) in the UK and by NewSouth ($27.99) in Australia. To order a copy go to guardianbookshop.com or call 0330 333 6846. Free UK p&p over £15, online orders only. Phone orders min p&p of £1.99