Rockwell Shah speaks with almost evangelical zeal about sleep. He is the CEO of Pzizz, an app that “designs dynamic audio” to get you to “sleep at the push of a button”; for him, bedtime is a “sleep experience”. Does he use his own app? “Oh my God! All the time.” As a sleep entrepreneur, what is his bedtime routine like – does he swear by camomile tea or special pyjamas? “I have a Purple mattress. I love the darn thing; it’s not like anything you’ve ever experienced with a mattress before, you basically float on top of it.” He does not, he clarifies, have any affiliation to the company. He is just truly that excited about shuteye.

Who can blame him? A good night’s sleep helps our memory, learning and mood. So it is no wonder that an industry of bright-eyed sleep entrepreneurs has awoken around our quest for better, deeper, longer sleep. They are offering everything from sleep trackers to white noise machines and hi-tech pyjamas that claim to create “an advanced sleep system for better rest and recovery”, made from bioceramic material that “absorbs the body’s natural heat and reflects that energy back into the skin”. Then there is a new robot, versed in “thousands of years of Buddhist breathing techniques”, that promises to soothe you to sleep, if only you spoon it. Yours to order for €539 (£466).

In the world of sleep, business is booming: according to a 2017 McKinsey report, the sleep-health industry – anything from bedding and sound control to sleep consultants and prescription sleep aids – “is collectively estimated to be worth between $30bn and $40bn and has historically grown by more than 8% a year, with few signs of slowing down”.

At a time when our innate ability to sleep is being kiboshed by work, life and disruptive partners – one recent study found that 30% of Americans wanted a “sleep divorce” – capitalism is, for better or worse, finding a way to sell it back to us.

Just look at the mattress market. In recent years, mattresses have become a highly desirable commodity, sold by companies that increasingly behave like tech startups, putting growth at their core and accessing venture capital markets more usually associated with Silicon Valley. The New York-based online mattress retailer Casper reached $100m in sales in 2015, the year after it launched; British company Simba expects sales of £100m by next year, having launched in 2016.

The Pzizz app launched in October 2016 and now has more than half a million downloads across 160 countries. The Duke of York declared himself a fan, and JK Rowling said it was the “best I’ve used by a mile”.

Shah spent 10 years working in a medical software company before starting the app, fuelled by his own past struggles to nod off as well as a “recognition that sleep has been declared a public health crisis”. He describes in more detail how Pzizz works: “dreamscapes engineered to lull your body into sleep” are paired with voiceovers “based on clinical sleep interventions, things like progressive muscle relaxation, clinical sleep hypnosis, breathing exercises and autogenic training”, a technique that teaches your body to respond to verbal commands. The scripts are modular, meaning “literally billions of variations”, and the voice actors are chosen for possessing “that special quality” – they know how to “speak in a certain way that just … gets you …” – he slows his voice right down – “to … relax”.

It certainly sounds relaxing. But what does the meteoric rise of this industry say about our lives – are we in a sleep crisis? “The simple answer is ‘yes’,” says Dr Guy Meadows, the co-founder and clinical director of the Sleep School, which runs insomnia clinics in central London, “we are in a sleeplessness epidemic.” A perfect storm has settled over our bedrooms, and it is stopping us from drifting off. “Tiredness,” he says, “is the new norm.”

Whatever happened to a nice cup of camomile tea before bed? Photograph: ALAMY

The internet is awash with concern about sleep, its quality, length and regularity. Recent articles warn us that “One bad night’s sleep may increase levels of Alzheimer’s protein”; that “Late risers [are] at increased risk of early death” and explain “Why going to bed in the wrong pyjamas could be affecting your sleep”.

Children around the world are sleeping less – in the UK, for instance, hospital attendances for children under 14 with sleep disorders have tripled over the past 10 years. The Centers for Disease Control and Prevention reports that a third of US adults say they usually get less than the recommended amount of sleep. The World Health Organization recommends between seven and nine hours a night, but a 2013 study by the National Sleep Foundation showed the average adult in the UK is getting just 6hr 49min each weeknight. People are falling asleep on New York’s subway so frequently that Mayor Bill de Blasio backed a scheme to start waking them up.

From academics to entrepreneurs, everyone agrees that a large portion of blame lies with digital technology. Watching The Good Place on Netflix with one eye on Instagram and another on the news is not, it turns out, a recipe for good sleep. And it is not just about the blue light of screens that we have all come to dread, the wavelength of which affects levels of the sleep-inducing hormone melatonin. “We are more connected, and more stimulated – in a cognitive sense,” says Meadows. “Our brain is not switching off, which is affecting its ability to gradually downshift its gears into sleep.”

Going back to a much earlier tech revolution, the sheer fact of electricity means we can choose to stay up until all hours. “We’ve invaded the night,” says Dr Russell Foster, the director of the Sleep and Circadian Neuroscience Institute at the University of Oxford, “and we’ve fitted more and more into the working day. Sleep has been the first victim.”

Work was once unlikely to be allowed into the bedroom, but can now commonly be found there, and not just in the form of midnight email sessions propped up on pillows. Shah points to the insecurity of the gig economy: “It has ramifications. Everybody is worried all the time about where the next paycheck is coming from,” he says.

Therein lies what a lot of the chatter around sleep seems to miss – that many people can’t afford to get enough; a good night’s sleep has become a luxury. Those in richer countries tend to get more. And the richer people in those countries tend to get more than the poor. According to a University of Chicago study from 2006, US adults are more likely to get more sleep, and sleep better, if they are white, wealthy and – perhaps surprisingly – women.

Our obsession with sleep has coincided with, and in some ways been consolidated into, the wellness industry. Sleep has been given that most modern of makeovers – it has been Goop-ified, given the clean-sleeping treatment, with Gwyneth Paltrow evangelising about making sleep a priority and her 10-hour-a-night ideal. People are being encouraged to douse vetiver-scented wellness oil between their toes or do a five-minute foam-rolling session right before heading to bed.

But that doesn’t negate the fact that all of us, even those with expensive pillows and oily toes, could probably use more sleep. Even if the numbers on sleeplessness make for grim reading, it is still good news that our attitudes are shifting. Foster says: “We’re right to take sleep seriously. It is 36% of our biology, and it has been largely marginalised and ignored.”

When Trump declared, at a campaign event in Illinois in 2015, “I have a great temperament for success. You know, I’m not a big sleeper, I like three hours, four hours, I toss, I turn, I beep-de-beep,” his sleep bravado sounded retrograde, a vestige of 1980s Wall Street, where money never slept because sleeping was for wimps.

The cult of “manly wakefulness” (as opposed to female “beauty sleep”), as Prof Alan Derickson coined it in his 2013 book Dangerously Sleepy, has been superseded – now we see a bit more shuteye and a bit less beep-de-beep as no bad thing. It perhaps helps that Bill Clinton, who was said to get only four to six hours while in office, has since admitted: “Most of the mistakes I made, I made when I was too tired.” Anyone can sleep for a meagre few hours a night, but only a “minute percentage” can do that and function, according to Meadows.

The corporate workshop side of Meadows’ sleep school has expanded massively since its launch in 2008. Now one of the largest parts of the business is going into banks, law firms, management consultancies and ad agencies to provide a programme of sleep education for employees. Where once businesses were paying their staff not to sleep, now they are paying to teach them how to get better sleep. Little wonder when 200,000 working days a year are lost to absenteeism caused by lack of sleep in the UK, and sleep-deprived workers cost the UK economy £40bn a year.

It has been just over a decade since Arianna Huffington collapsed from sleep deprivation and arose to stage her call to arms – and to the bedroom – putting paid to the idea that CEOs or indeed anyone can function on what many of us would count as a long nap each night. And many people have tried to heed her message, if they can afford to (it is worth noting that Huffington has “nine or so” assistants).

In recent years, Amazon’s bestseller list has been topped by a children’s book called The Rabbit Who Wants to Fall Asleep. It is written by a Swedish psychologist and its focus and structure are designed to lull children into slumber. And adults are queueing up in pyjama’d throngs to be put to sleep by classical music – from Berlin to Sydney, Max Richter’s Sleep, an eight-hour “personal lullaby for a frenetic world”, has turned a sleepover into a good night out.

So what led to this change in mindset? One answer lies in the lab. “What has fundamentally changed,” says Foster, is that “serious neuroscientists have started to take sleep seriously – it was a bit of a graveyard of the neuroscience world.” But not any more – and the “data emerging are quite spectacular”.

He rattles some off in a torrent so convincing it will make you want to head straight to bed: the “beautiful experiment” published by Jan Born in 2004 that showed the massive impact sleep can have on problem-solving. The “nice data” from Eve Van Cauter’s University of Chicago lab that found that sleep loss in healthy young adults increased their risk of type 2 diabetes. And the experiment that showed you are less likely to remember words with a positive value (think: love or joy) if you are sleep-deprived – “Our level of sleep will very much reflect the way we remember positive and negative experiences.” Anecdotally, it stacks up – who isn’t moody when they are functioning on too little sleep?

“If you’re not fully rested,” Foster continues, “then you tend to be overly impulsive – jumping that red light; unreflective of things that you do; you lack in empathy, so your ability to pick up the social signals of others is not good. Tired people not only fail to come up with innovative solutions to complex problems – to use this extraordinary brain – but their ability to function generally, sense of humour, social interactions fall apart really quickly.” And that, he says, is just relatively short-term sleep loss.

“For a long time,” says Meadows, “insomnia has been thought of as a symptom of poor mental health. Now we know that actually it is also a trigger – sleep is considered an early warning sign, a canary down the coalmine for anxiety, depression, bipolar.” With the science of sleep proving Virginia Woolf was playing with fire when she dismissed shuteye as a “deplorable curtailment of the joy of life”, it is no wonder we’re anxious to get enough.

Many of us turn to sleep trackers for help. Trackers claiming to measure how long we are sleeping, and what kind of sleep we are getting – light, deep or REM – are now common bed companions. Foster is wary: “They have been half validated and they sort of work for people who have very stable sleep/wake patterns, but if you have any irregularity or are falling outside of the normal range, and frankly that’s most of us, they fall apart very quickly.”

While “they are great for empowering you to say: ‘Yeah, there’s a bunch of stuff I can do to improve my sleep,’” he says, “I don’t think we’re there yet with these devices … [But] everybody’s jumped on the bandwagon … there are a lot of people who take these things very seriously.”

Cut to sleep’s very modern, meta disorder: orthosomnia. Dr Sabra Abbott, a professor in neurology and sleep medicine at Northwestern Memorial hospital, co-coined the term with her colleague Dr Kelly Baron in a 2017 paper, Are Some Patients Taking the Quantified Self Too Far?. She tells me how they started seeing patients who “didn’t necessarily initially have sleep complaints – their primary concern was that their tracker was telling them they weren’t getting the right amount or right type of sleep. It seemed,” she says, “that the device was creating a sleep problem that may not have otherwise been there.”

Foster likens this newfound interest in sleep trackers to “when domestic electrification first came in. A whole bunch of people started to wire up houses and a number of them burned down, because they didn’t know how to use the equipment.”

Orthosomnia seems to be one symptom of an industry that grew rapidly and has left consumers with more data than we know what to do with (albeit not always accurate). It is tempting to draw a parallel with the world of social media – we are using a ton of it and we are not yet sure what impact it is having. “Every step change encounters the same thing,” reflects Foster. “It is a massive interest – things have pushed forward so fast that there is a great vacuum behind it.”

Like a hug in bed … the Gravity Blanket

But the sleep industry is not all gadgetry. At the pleasingly lo-tech end you will find the weighted Gravity Blanket, available in three gradations of heaviness: seven, nine or 11kg. According to the company’s CEO Mike Grillo, it “mimics the feeling of being hugged or embraced”. This “releases serotonin”, he says, “stimulates melatonin, helps decrease cortisol levels, which is linked to stress and anxiety, and that is what induces the calming and grounded effect”. Not bad for a heavy blanket.

When it comes to the evidence behind it, “there is still a lot of science to be done”, Grillo concedes – according to the New Yorker, Gravity early on deleted a section from its Kickstarter page claiming it “can be used to treat a variety of ailments”, including insomnia, post-traumatic stress disorder, obessive-compulsive disorder and attention deficit hyperactivity disorder.

But it obviously struck a chord: it has raised just shy of $5m on Kickstarter to date. Its original fundraising goal had been $21,500. It is tempting to slot it into the tumultuous present – what had been “used in more niche patient populations for some time”, says Grillo, began to have wider appeal after the 2016 election of Donald Trump “and the Brexit vote in your part of the world”.

In a recent article about Gravity in the New Yorker, the writer Jia Tolentino describes how it “enacts a fantasy of immobilisation that is especially seductive in a world of ever-expanding obligations – to work, to monetise, to take action, to perform”.

An industry for an anxious age, then, where screens and work have invaded our bedrooms and world leaders sit up into the small hours beep-de-beep-ing. Now, where’s that breathing robot? I might need to spoon it.