JOAN DIDION WROTE 'we tell ourselves stories in order to live', and so often has the idea that humanity runs on stories been asserted of late that it has come to resemble a self-evident truth until, in next to no time it seems, we have started talking in excited voices about humans being hardwired for stories: except if we keep reading Didion, she goes on to note 'we look for the sermon in the suicide, for the social or moral lesson in the murder of five'. And then 'we interpret what we see, select the most workable of the multiple choices.' Which is to say, the telling of stories is a human compulsion that can make our view of the world less despairing and defensive, can make life itself more bearable. No mean feat – a feat fit to save our sanities – yet still storytelling, the kind of storytelling Didion writes about, does not in itself or by itself take us closer to the truths of our lives with anything like the inevitability that gets ascribed to it these days. Saying we need to tell stories because we are human, is not quite the same as saying we are human because we need to tell stories.

Okay, so the hyperbole, so the ten kinds of hoo-hah… Big deal. I am not sure if it is a big deal or if it is not. And of course philosopher Alasdaire MacIntyre called us storytelling animals already in 1981, and before MacIntyre there was Lakoff and Johnson's Metaphors We Live By and before them Joseph Campbell's The Hero with a Thousand Faces and before him (and after him) came the folklorists, the formalists, the anthropologists and philosophers, some famous, others forgotten, peeling back for us exactly how and how much stories, myths, fables, fairytales make the world go round. Didion's 'we tell ourselves stories' was written in 1979. And we can backtrack further, forever, indefinitely – to, say, 'we spend our years as a tale that is told' (Psalm 90:9) or to oral storytelling traditions, many of them enduring to this day, and the congress-sized libraries and public squares they carry within them.

What is happening now, if something is in fact happening, has to do with the new, particular idea of what stories can do for us. It has to do with the way stories are being pressed into the service of some yet-to-be-fully-glimpsed zeitgeistian thing. And so we see live storytelling events springing up everywhere, like nobody's business, and millions of people, myself included, savouring first-person narrative by way of radio on This American Life ('The power of story is pretty animal,' says Ira Glass, the show's host and executive producer.) This may have started as an American thing but it's no longer that. In Australia, Radio National broadcasts storytelling slams, and storytelling nights draw crowds in every city. Right across the Anglophone world, professional storytellers preside over the openings of corporate gatherings and arts festivals, and business gurus proclaim the ability to tell a story about your aspirations or your brand to be the most indispensable business skill of our times. 'An unnarrated life is not worth living,' is how the philosopher Richard Kearney puts it, and it starts to sound more and more natural. And another expression has been cropping up lately: 'the golden age of storytelling'. That's us apparently. We are living it.

Just the other day I took my son and his friend to our local cinema. The cinema – the Classic, in the Melbourne suburb of Elsternwick – is one of the things that makes me want to stay where we are. It has been in the same spot doing the same thing for more than a hundred years (no trivial claim for a cultural institution). Screening at the start of every session, the cinema's promo shows archival footage of injured World War I soldiers being wheeled through the building's seemingly unchanged front doors. I'd seen the promo many times and always found it smart and affecting. Only this latest time did I notice the tagline: '100 Years of Storytelling'. I noticed it because my son and his friend read the line out loud to each other and scrunched up their faces – they, months out from their seventh birthdays, thought it rather ridiculous. I kind of agreed. Were there no other words to describe what cinema has been doing for the past century?

Last year's Sydney Writers' Festival, the first under artistic director Jemma Birrell, announced storytelling as its overarching theme. Instantly the festival smelled not of books but of scoops, excitement, danger even. The much-complimented advertising campaign, with its slogan 'Have We Got a Story for You', signalled that this was to be no ordinary talkfest. Delivering the opening address before an audience of eight hundred and fifty festival-goers was an intense Welshman with syllables that hang and hammer. Daniel Morden is a professional storyteller, twenty-odd years on the beat, who can go from Homer to Haitian fables in a breath, such is his range. 'We live our lives as a narrative,' was how Morden began. 'We remember and anticipate in narrative – hate, love, hope, despair, doubt and learn from it. We even create narratives while we sleep.'

On Morden went: we make sense of ourselves and the world through stories. We process our experiences through stories. We acquire morality and develop compassion through stories. There are no new stories, only old stories in new clothes. Speaking of clothes, truth walks our world 'dressed in the clothes of story', because people cannot bear the naked truth.

Morden was great (judging by the podcast). I particularly liked him saying that the process of telling stories is, for him, a process of chasing them as they change shape and become stories about something else. 'If a story stops changing, if I catch it, I have to stop telling it.' Lovely. What bothered me was a kind of implied awe in the air (even on the podcast). There, after all, stood Morden, practising his ancient, sublime art – and his audience, you just knew, was expected to undergo some near-ecstatic experience as they rediscovered, in the face of stories, their sense of childlike wonder and, along with wonder, a sort of fundamental affinity with the people of all times and cultures. George Dawes Green started the mega-popular The Moth – champions of live, public storytelling – in his bedroom. He has described the act of telling real-life, first-person stories in front of other people as 'so moving, so simple, so perfect'. And undoubtedly – why not? – it can be just that: moving, simple, perfect. But the pressure on stories to act as conduits for the universal, for the transcendental, can also produce a curious effect. It can make friction-and-silence-laden spaces created by the telling and the listening feel smooth, elementary, back to how-it-once-was, like a woman after a Brazilian.

There was a moment in Rai Gaita's 2011 public lecture 'To Civilise the City?' – a lecture ostensibly about the future and past of universities – when Gaita said our culture's emphasis on crafting good stories exists, at least partly, at the expense of good thinking. 'Some people say that where there is good writing there is also good thinking,' observed Gaita, 'but that is not true, except perhaps at the highest level.' I don't know if I fully agree or, in fact, if I believe the two processes are manifestly distinct at any level, but surely Gaita is right in suggesting that the near-automatic equating of good writing with good thinking is misguided. Perhaps it is very simple: much as we like to think otherwise, we cannot do all our thinking through stories. Neither can we generate meaning – macro, mini, whatever – through stories alone; nor take stock. No doubt a story is a singularly powerful thing, what Ira Glass calls 'the back door into the deepest parts of us not accessible in other ways. Its power is not explicable to rational analysis, it is far more animal, far deeper, far more pre-rational. Narrative "gets to us" in ways that other things don't.'

I heard Glass say this about narrative at Melbourne's Athenaeum Theatre during his 2012 'Reinventing the Radio' tour of Australia. And it's true, I thought, especially the 'getting to us' bit. Glass spoke of narrative acting prophylactically – 'when a story gets inside of us, it makes us less crazy' – and as a shield, à la Scheherazade in 1001 Arabian Nights. 'You can save your life a thousand times by suspense.' I came out of the theatre that night feeling warm, fuzzy, and nodding all the way back to Elsternwick.

But, also, something about This American Life was making me uneasy. 'A story is like a train going to a station,' Glass likes to say – except sometimes, and increasingly, it can feel like a tank crushing all sorts of things under its tracks. Something in the way the form pushes itself onto the experience; something about how the obligatory reflection framing the story often feels subtly untrue.

I couldn't put my finger on what was troubling me. Then I read a piece by Eugenia Williamson in a magazine called The Baffler in which she argued – bravely, I thought – that the story told by Mike Daisey about appalling working conditions in Apple factories in China, which quickly became This American Life's most popular episode, and which turned out to be largely fabricated, was the logical outcome of the show's predilection for tackling complex subjects by means of a 'dramatic non-fiction narrative in the form of a personal journey'.

Glass and his team dedicated a whole follow-up episode to dissecting what went wrong and how Daisey fooled them. But there was, I am afraid, a structural issue at play. It may be that there is something Trojan horse-like about a certain kind of narrative: it can sneak, unnoticed, past the usual well-oiled protocols of authentification, the usual ethical questioning. Sometimes stories can lead you down foxholes you cannot fact-check your way out of. Or, worse, perhaps falsehoods are inevitable when 'the method of "exploring" ideas through stories is now the preferred mode of, or replacement for, serious thought and argument' as Steven Poole writes in his review of Malcolm Gladwell's latest book David and Goliath: Underdogs, Misfits and the Art of Battling Giants (Little, Brown, 2013). Actually Gladwell, he of the better-than-best-selling Blink (Little, Brown, 2005), The Tipping Point (Little, Brown, 2000) and Outliers (Little, Brown, 2008), is an interesting case. He has perfected the formula of combining affecting stories with prestigious-sounding and freely abbreviated academic research, particularly in the fields of psychology and sociology. His schtick is what philosopher John Gray describes, a little uncharitably, as a 'mix of moralism and scientism'. Surely, as well, Gladwellism is a product of this moment, in which people insist on using stories as the primary means of working out what they think and of saying what they mean.

BENJAMIN BRATTON, IN his recent TED talk about the failure of TED talks, mentioned 'middle-brow megachurch infotainment' – and it is hardly accidental if many of today's more successful live storytelling events have a whiff of this. Plenty of them also wrap around themselves a cult of faux simplicity (oh, the timeless artisan feel of stories), which obscures the artifice involved in packing the entire world into a series of tellable tales. Invariably, out of such a process, an exacting aesthetic emerges – in TED talks, it's a combination of epiphany and personal testimony that Bratton calls 'epiphimony'. At various The Moth events, organisers ask those wishing to tell their story to, as part of their two-minute pitch, 'give us a sense of your story arc and how your story changed you'.

I am sure you can see the problem we're running into here: if turning an experience or idea into a story for public consumption requires this kind of preliminary work-over, then we are back in the middle of the Coen brothers' movie Barton Fink (1991)and the conversation between Jack Lipnick, president of a major Hollywood studio, and New York playwright Fink, played by John Turturro. Fink is dispatched by his agent to Hollywood in the wake of 'Bare Ruined Choirs', his play about 'the common man', becoming the toast of Broadway. It's 1941. Both Fink and Lipnik are, I should say, as grotesque as each other (Fink arguably more so, towards the end). Fink, anxious, is attempting to write, as instructed, a Hollywood screenplay about a wrestler. Lipnick tells Fink not to worry one bit about his total lack of knowledge and cinematic experience. 'We're only interested in one thing: can you tell a story, Bart? Can you make us laugh, can you make us cry, can you make us wanna break out in joyous song?'

What happens when that question – 'Can you tell a story, Bart?' – is asked of our scientists, our thinkers, our educators, our artists, our journalists, our human rights advocates? 'If the public is not with us yet, it means we haven't told the story of global warming well enough,' said a visiting environmental journalist (if I understood her correctly, she prefers to be called an environmental storyteller) at a recent ideas festival in Sydney. Telling the story better, hmm – as if it were a web that must be made wider, thicker, with more sticky threads in it, to increase the general public's chances of getting caught up in it. And how do you tell this kind of story better? Oh, yes, you work on your story arc, you describe a moving personal journey, you take your audience on that journey with you, you land the story well, with a bang, with a message (but don't make it heavy-handed) too, and all the while you skilfully steer your audience away from despair and towards a sense of empowerment.

I think you know what I want to say. To me, one of the great artistic achievements of Seinfeld was not that here was, purportedly, a show about nothing (it was no such thing) but that not one character in the show's main quartet was changed, let alone empowered, in any take-home way by their experience. And as far as emotional identification afforded by going 'on a journey' is concerned, well, in Nabokov's lectures on Russian literature he says the good reader 'identifies himself not with the boy or the girl in the book, but with the mind that conceived and composed that book'. Surely we cannot let Robert McKee's screenwriting bibles and commandments set the template for the way ideas, experiences, bodies of knowledge and arguments are articulated in the public sphere. And how can we fall for this stuff anyway, when we have right in front of our noses the Indigenous storytelling traditions, answerable to the highest forms of moral seriousness? 'The everyday contemporary Indigenous story world,' writes Alexis Wright, is 'epic' – it reaches back, it reaches forward, it materialises and transmits the invisible. Stories are given as gifts vibrating with power and urgency. Telling a story to your people is telling it to your ancestors is telling it to your land. How can we read these words and not feel at least slightly nauseous next time someone mentions 'story arc' or going 'on a personal journey'?

When public conversations become distorted by particular storytelling tics, you have to question what is being missed and what is not allowed to happen. Also, what does happen to those scientists, thinkers, artists and advocates who do not make the public laugh, cry or wanna break out in song? Do they simply get left behind? And let's for a moment take the best-case scenario – let us say you put into the public domain a story that is lucidly and persuasively told, one that elicits big thoughts and heavy emotions. This kind of story is still no substitute for a genuine public debate with its bumping of heads, its pushes and pulls, its peculiar and all-important labours – defending your position, nailing your opponents, issuing rejoinders, synthesising thought, changing your mind – that are so different to the exertions of storytelling, even storytelling at its best. And all the while the stakes are high and the public is breathing down your neck.

I am not against stories. I am, in fact, very much for stories – a big fan, that's what I am – but these days when I hear someone talk about the universal power of storytelling I do feel like reaching for my gun. No one needs to convince me of storytelling's power, but it's not a no-brainer, okay, not a ready-made thought. It has to be, or rather it should be, what journalist Katherine Boo calls an 'earned fact'.

I WAS THINKING of starting this essay with a story but then I thought again. Starting an essay or any substantial piece of journalism with a story is pretty much the law these days. The reader will swallow your story and then here she is with your shiny hook in her innards and you can start pulling on it this way or that. I am sick of reading these opening stories, sick of their intent to seduce, they remind me of the candles, wine and music put out by a man who wants to sleep with you. Here he is waiting for you to be two-thirds through the first glass and then, mark my words, he will make his move.

This is the story I was going to start with.

Primo Levi, free and back home in Italy after the liberation of Auschwitz, talked to whoever would listen. During that winter of 1945 he cornered strangers on the Milan–Turin Express. None shushed him. Levi made no apologies. His need to speak of what he'd witnessed and survived was 'an immediate and violent impulse'. Later he would write about the feverishness of the immediate post-Lager period. 'I felt,' he wrote, 'like Coleridge's Ancient Mariner, who waylays on the street the wedding guests going to the feast, inflicting on them the story of his misfortune.'

Levi was talking, and he was writing. If This is a Man is his account of a year in Auschwitz that took him less than a year to write, and in it he describes a recurring dream. In the dream he is back home, telling people of his experiences, but they are 'completely indifferent…speak confusedly of other things among themselves, as if I was not there'. In the dream he is free, he will live, but all he feels is 'a desolating grief'. The dream doesn't let up and Levi discovers that others in the camp, including his best friend Alberto, have the dream too – it comes to them again and again, just like the other dream, that one accompanied by men licking lips and moving jaws in their sleep, of food being close but not close enough. Both dreams are dreams of things needed beyond all others. Of things needed and denied. 'Why is the pain of every day,' asks Levi, 'translated so constantly into our dreams, in the ever-repeated scene of the unlistened-to story?'

TO BRING PRIMO Levi into a conversation about telling and listening is a well-worn trick. You turn to Levi when you want to say that certain stories must be told even when they cannot really be told, and must be listened to even if they can never be sufficiently heard. But to me the most important thing that Levi is saying, above all else, the thing that's at stake, is we must see clearly the devastation – personal, cultural – wreaked by 'the ever-repeated scene of the unlistened-to story'.

Eric Santner, a professor in modern Germanic studies at the University of Chicago, has an illuminating concept he calls 'narrative fetishism'. 'By narrative fetishism,' writes Santner, 'I mean the construction and deployment of a narrative consciously or unconsciously designed to expunge the traces of the trauma or loss that called that narrative into being in the first place.' Narrative, when fetishised, can become an evolved and brilliantly disguised way of shutting our ears to what hurts and scares us the most, a way not of sharing our experiences but of multiplying the vast archives of unlistened-to stories.

The final nail Eugenia Williamson put in This American Life's coffin was to call the show on its habit of 'massaging painful realities into puddles of personal experience, its preference for pathos over tragedy'. What a devastating diagnosis that is – a preference for pathos over tragedy. Is it true? I think it just might be. Actually, I think it might just be true of this cultural moment more broadly.

At the very last conference I went to before leaving academia – it was held at Massey University in Wellington, and an unusually beautiful, non-turgid conference it was – one of the speakers was British neuropsychologist Paul Broks. A middle-aged man, solid, something of a bird in his face; I seem to remember he was wearing a baseball cap, because he made a joke about it. (My memory, typically, has retained the act of his making a joke but not the joke itself.)

On a big screen behind him Broks put up a baby photo. 'Is it me?' he asked. It was him but the contrast between the shiny baby and the man looking, like most of us in that room, a little scratched and chipped around the edges, and perhaps even more to the point the intrusion of this image of a learned man as a baby into our sober scholarly discourse…well, it was a nice touch. And nice bait. 'What exactly,' Broks continued, 'is common between me then and me now? What has persisted? Knowledge? No. And on a molecular level, the baby and this ageing man in front of you are completely different.' That, I must say, surprised me. I expected the solution to Broks's conundrum to be pleasantly sciencey, like that question that delighted me as a child, in which you are asked to name the body's largest organ and everyone goes brain, lungs, spleen, kidneys, while the answer is skin.

'The connection between me then and me now is at the narrative level only.' And he said: 'We string ourselves through the stories.' Broks, who does not believe in ego or in immaterial soul, believes instead that we are built from language and memory, and that our self, as something unified and continuous with the past, present and future, has 'a narrative structure'. The thing holding our self together is, in other words, stories. That was in 2010, and unbeknownst to me, the conversation about narrative's central role in human biology, evolution, adaptation, cognition and socialisation had been going on for decades. (Philosopher and cognitive scientist Daniel Dennett wrote, in 1992, that self is best understood as an abstract 'centre of narrative gravity'.) Broks's talk made a big impression on me – the primacy he gave to stories as the pipes of the self, not just the water in the pipes. I am still impressed, still re-read his stuff. How come 2010 feels so far away?

In his TED talk that I like and keep coming back to, Benjamin Bratton says 'innovation' is not a strong enough ideal, it cannot be the horizon, in anticipating the future. 'I am just telling a story' – the platitude du jour of our times – is, similarly, not a strong enough construction to speak of the essence of what happens, of what gets passed on between humans, in the act of communication. And it is certainly not strong enough, not even close, to fully convey what it is that writers, filmmakers, artists, performers do, that whole doomed Flaubertian enterprise of 'longing to move the stars to pity'.