This book isn’t about what it seems to be about: the setting up of a survivalist community in the Scottish highlands. Believing that the collapse of civilisation was inevitable and possibly imminent, Dylan Evans decided to become an off-grid crusader, a pioneer for a future without fossil fuels, technology, communications and general comfort. But the real story of the book is about delusion and depression, about how the appeal of primitivism can so unhinge an academic that he sells a cottage in the Cotswolds to live in a damp yurt and bathe in a barrel.

Structurally, the book is smart: instead of beginning at the beginning, full of optimism and hope, it begins with Evans in a psychiatric unit, having been broken by the stresses of running his post-apocalyptic project. The psychiatrist reads back to Evans parts of his founding document – an imagined future of power cuts, martial law and cultural darkness: “Global supply chains snapped. Panic buying ensued, and within a day there was nothing left on the supermarket shelves. Looters took to the streets, and the army was deployed in all the major cities.”

From then on, the book is on parallel tracks. One track talks about “the Utopia experiment”, the establishing and running of a microcommunity of catastrophists. The other track describes Evans’s retrospective attempt to understand why he gave up a job in academia, got married and divorced again in the midst of the experiment, and then spiralled downhill into a state of near catatonia.

The actual community-building sounds extraordinarily unthought through. Even allowing for Evans’s witty self-deprecation, it’s clear that the project was hampered by a lack of leadership and clarity. He had a vague daydream that, at the centre of the community, “unassuming but indisputably wise, would be me, the founder, loved and revered”. But it doesn’t happen like that, and his utopia is chaotic and ideologically incoherent. Everyone still goes to the supermarket, to hospital or to a friend for a hot bath. In a way, that adds to the charm of the book: the attempts to grow vegetables, to cook, wash and defecate in primitive conditions, are haphazard but amusing, and the eccentric characters who join him, from Adam the chancer to hard-core Agric (“a Hobbit on speed”), come alive. Despite the ironic narrative tone, the bonfires, fresh bread and friendships sound enchanting.

Retracing the reasons he started the experiment, Evans deconstructs the seductive appeal of communalism and ludditism. It wasn’t just that western civilisation seemed, to him, destined for extinction like the Maya or Easter Islanders; Evans also thought that machines were becoming as dangerous as humans. He was working in artificial intelligence and was converted to a scared scepticism by reading Bill Joy, a software pioneer turned doubter, who in 2000 wrote in Wired that “our most powerful 21st-century technologies are threatening to make humans an endangered species”. Evans also quotes Theodore Kaczynski AKA the Unabomber: “The human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines’ decisions.” Evans gets so into the idea of the apocalypse that he urges his sister to have a horse on hand so that she could “escape the nightmarish last days of London”.

Evans makes a convincing case that his depression was the cause of the madcap adventure, not the other way round; that he fantasised about civilisation’s collapse as a projection of his own internal and personal disintegration. “The experiment”, he writes much later, “was simply a misguided attempt to create some kind of ‘therapeutic community’ in which I could seek refuge.” As one character quietly suggests, Evans was perhaps only obsessing about the end of the world as a way of wondering about his own mortality.

Things don’t get too dark in the communal experiment, but there are shadows. There’s talk among the survivalists of pipe bombs, booby traps and chopping off hands. There’s paranoia that they’re being watched by a helicopter. Ever the maverick, Evans begins to doubt the whole premise of the exercise, retreating to his friends’ houses to watch from a distance.

One imagines this must have been an incredibly painful book to write: the experiment happened nine years ago, and it’s obviously taken this long for Evans to process and understand it. There seems to be sincere regret, possibly even shame, about the way he toyed with people’s lives. He writes of the volunteers, “They were my lab rats”, and wonders whether he harboured “some secret desire to be a kind of guru or cult leader”, and had “some unacknowledged form of megalomania”.

He also offers some good analyses of why communities fail: the fact that idealists are seldom very practical people, and that communities tend to attract cantankerous misfits, who then find themselves in a pressure cooker. Evans doesn’t, though, pose the opposite, and much more interesting, question of why some communities succeed. Perhaps because he’s an avowed atheist, he recognises but never inquires into the intriguing fact that almost all the enduring communities are religious in foundation.

My other reservation about the book is simply that he bounces from one extreme to the other – disdaining civilisation, then being appalled by primitivism, and so returning in the end to civilisation and another job in AI. The subtleties of the middle ground are missed: might it not be possible that stability and balance are actually found in the overlap between the noble savage and the noble sophisticate? By the closing pages, Evans is as disdainful of communitarians and back-to-the-landers as he once was of the industrial world, lumping Thoreau in with the Unabomber as “on a deluded quest”. His experiment didn’t work out, but there are plenty that do, and plenty that have something eloquent to say to the world.

• Tobias Jones’s book about his woodland community in Somerset, A Place of Refuge, will be published by Quercus in July. To order The Utopia Experiment (RRP £14.99) for £11.99, go to bookshop.theguardian.com or call 0330 333 6846.