This is a story about a book that just kept selling, catching publishers, booksellers and even its author off guard. In seeking to understand the reasons for the book’s unusually protracted shelf life, we uncover important messages about our moment in history, about the still-vital place of reading in our culture, and about the changing face of publishing.

The book is Sapiens, by the Israeli academic Yuval Noah Harari, published in the UK in September 2014. It’s a recondite work of evolutionary history charting the development of humankind through a scholarly examination of our ability to cooperate as a species. Sapiens sold well on publication, particularly when it came out in paperback in the summer of 2015. What’s remarkable about it, though, is that it’s still selling in vast numbers. In its first two and a half years of life, Sapiens sold just over 200,000 copies in the UK. Since 2017, when Harari published Homo Deus, his follow-up, Sapiens has sold a further half million copies, establishing itself firmly at the top of the bestseller lists (and convincingly outselling its sequel). Sapiens has become a publishing phenomenon and its wild success is symptomatic of a broader trend in our book-buying habits: a surge in the popularity of intelligent, challenging nonfiction, often books that are several years old.

It was trade publication, the Bookseller, that first noted the rise of what it called the “brainy backlist”. It also highlighted a concomitant fall in the sales of the books that had been such a staple of publishers’ catalogues – celebrity biographies. We are turning away from glitzy but disposable stories of fame and excess and towards more serious, thoughtful, quiet books that help us understand our place in the world. Analysts at the Bookseller parsed data from Nielsen BookScan, and saw over the past five years a dramatic rise in the sales of “long-tail” nonfiction titles, often works on politics, economics, history or medicine that attempted to synthesise or challenge received thinking on the subject. Kiera O’Brien, charts and data editor at the Bookseller, and one of the authors of the initial study, is convinced the publishing landscape has changed over the past few years. “It’s a rare thing for nonfiction to carry on selling like this,” she says. “Often fiction will, when there’s a film adaptation or something like that, but nonfiction tends to be very much of its time. Now it feels like we’ve broken that mould.”

Facebook Twitter Pinterest Yuval Noah Harari’s Sapiens continues to sell well years after publication. Photograph: Daniel Thomas Smith

These are febrile, unpredictable times, with society facing new challenges and quandaries each day, from the rise of populist politics to the migrant crisis to climate change. Mark Richards, publisher at John Murray Press, sees the return to serious works of nonfiction as a response to the spirit of the age. “We’re living in a world that suddenly seems less certain than it did even two years ago, and the natural reaction is for people to try and find out as much about it as possible,” he says. “People have a hunger both for information and facts, and for nuanced exploration of issues, of a sort that books are in a prime position to provide.”

The idea that Richards sets out here – that books retain a special place in our culture, an aura that means we look to them first when searching for deep truths about the world – rings particularly true when you scan through the list of those works of nonfiction that we’ve been buying consistently over recent years. From Siddhartha Mukherjee on cancer to Peter Frankopan on the history of trade, from Elizabeth Kolbert on extinction to Atul Gawande on end-of-life care, great nonfiction offers us levels of detail, breadth of scope and depth of engagement that we simply don’t get from other media.

It may be, though, that the picture is more nuanced than merely a desperate yearning for voices of authority at a time of crisis. Karolina Sutton, an agent at Curtis Brown who represents writers from Malala Yousafzai to Oliver Bullough to Thomas Friedman, sees a number of different forces driving this latest publishing trend. “There are two things going on in serious contemporary nonfiction,” she says. “We’re seeing these big sweeping narratives that people are reaching for as a way of making sense of this unstable world, and on the other hand you have a new generation of activist-writers who are telling stories of gender, politics or race and doing it on their own terms, very much going against what went before. These are bold books, quite radical, really exciting and tend to be younger writers writing for younger readers.”

Whether it’s the likes of Owen Jones and Douglas Murray on politics, or Roxanne Gay and Ta-Nehisi Coates on race, or Samantha Irby and Laura Bates on feminism, a host of younger polemical authors are writing for their peers about issues that seemed to have taken a back seat during the frolicsome years in which we’d rather read about Jordan the person than Jordan the place. Now the geopolitical backdrop demands that we take positions: it’s no longer acceptable not to have an opinion on the big subjects. Again, it appears that we are looking to books to deliver us these opinions, nonfiction written from a position of authentic authority.

Facebook Twitter Pinterest Waterstones’s decision to stop accepting payments from publishers to promote books in its stores has been very influential. Photograph: Alamy

One of the most striking recent successes in this vein is Reni Eddo-Lodge’s Why I’m No Longer Talking to White People About Race. Eddo-Lodge’s editor at Bloomsbury is Alexa von Hirschberg, who knew that there was public appetite for a book on the subject. “I had witnessed the mainstreaming of feminist writing with Caitlin Moran and Laura Bates,” she says, “and it became clear to me that a book about race was the next logical step. There’s lots of great writing in the States about race but this gaping hole in the UK, and it’s a subject that people here want to understand. I just knew that Reni could lead the charge, that she’d written a handbook for understanding race in this country.” It was important also that Eddo-Lodge’s book spoke to a whole new market of readers. “It’s a younger generation. Reni’s readership is teens and younger students, people who feel a real urgency for change.”

Sapiens has been near the top of Amazon’s charts both as a physical book and an ebook. The latter is rare for nonfiction – we tend to use our Kindles for novels. It’s clear that one of the things driving the resurgent popularity of nonfiction is Amazon’s recommendation system, which tends to direct readers towards books of a similar type. So the breakout success of Sapiens has brought with it other, like-minded books identified by Amazon’s computers – by Jared Diamond, Daniel Kahneman and Steven Pinker, for instance. It was, though, the work of another bookseller to champion nonfiction that was most often mentioned by the publishing insiders we spoke to as the major force behind this latest trend – Waterstones. “They know their customers,” Mark Richards says, “and they know they can make books happen. Prisoners of Geography [by Tim Marshall, published in 2015] was made by them – a book from quite a small publisher, not particularly noticed on initial publication – but they realised what a brilliant synthesis it was, and to what extent it helped explain a lot of what’s happening in the world now. And once they get behind a book, other booksellers will often follow.” O’Brien agrees. “There’s a lot of recent historical nonfiction that has been given a bump by Waterstones, particularly their book of the month promotions. And it’s often stuff from tiny publishers which, given that initial boost, can then establish itself in the charts and stay there for a very long time.”

Waterstones has seen a stunning turnaround since coming under the stewardship of James Daunt in 2011, moving from a series of loss-making, down-at-heel stores with three-for-two offers piled high on tables to 280 highly profitable bookshops carefully curated to offer readers the books they want. Importantly, they not only work with publishers to promote new releases, but also pore over backlists to seek out books that might have been overlooked and deserve a wider audience.

Sapiens would have had an expectation of selling a few thousand. It's not one you'd expect bestseller status for James Daunt

Clement Knox is Waterstones’s nonfiction buyer for history and politics and explains the power of the chain’s promotions. “We made Prisoners of Geography book of the month in the summer of 2016. For the first few months we had 90% of the market for that book, and then for a long time after that we had over 50%. Now it’s much less, because everyone has got hold of it, but we’re fine with that. It’s the book I see most around and about, when I’m on the bus or whatever and it makes me very happy to know where it started.” The path of Sapiens was similar: Waterstones plucked it from relative obscurity, making it book of the month in May 2015 and helping to propel its initial success. Then, as Knox put it, “Everyone else joined the party. Sapiens is still near the top of our bestseller lists. Three years after publication, that’s incredible. And it’s still selling. It just went from a rate of around 8,000 books a month to 11,000 on the back of him appearing over here promoting his next book.”

Daunt agrees that the book of the month choices have had a profound impact on the publishing landscape. One of the key changes he has made to the chain is that, unlike peers such as WH Smith, publishers can no longer buy their books a place in its promotions. This was why celebrity biographies had such a good run for so many years – they were bankable hits and both publishers and bookshops were locked into promoting them. Now, freed from these shackles, Waterstones is able to take risks with its promotions. “Sapiens, for instance,” Daunt says, “would have had an expectation of selling a few thousand in hardback, maybe 5,000 copies in paperback. It’s a worthy book and a necessary book, but not one you’d expect bestseller status for and a publisher would therefore never pay to have that book piled up in front of people. When Waterstones stopped taking payment for putting books in front of people it liberated us to follow our bookselling instincts.”

How real books have trumped ebooks Read more

In the end, the story of Sapiens is about a book becoming part of a national conversation. “Readers like to read the same book, particularly if it’s a good and interesting one,” Daunt says, “because they like to discuss it. But with something like Sapiens, which is a challenging book, and perhaps one you wouldn’t expect to be at the top of the bestseller lists, let alone for years on end, you need to get enough people to read it to ignite that interest.” At a time when politics is more furious and fragmented than ever, when technology is colonising our everyday existence, when medicine is reshaping our lives, we still look to books to make sense of things, to feel ourselves part of a great communal effort to understand our age. These are serious times and they demand serious, intelligent and challenging books.