Matt Berninger, lead singer of the five-piece rock band the National, prowls the main stage of the Haven festival in Copenhagen, sloshing a pint of white wine on the rocks, what he calls “Cincinnati sangria”. He’s smartly dressed in varied shades of dark grey, charcoal and faded black, and has the air of the charismatic lecturer whose female students all have a crush on him. Looking on from the side of the stage, watching the rapt expressions of the fans, it calls to mind the scene in one of the Indiana Jones films where he’s trying to talk to a class about an obscure archeological find only for a girl in the front row to blink and reveal she has “Love You” written on her eyelids.

Berninger, who is 46, is a compelling frontman. He is known for his rambling, drunken forays into the audience; so much so that he began to wear shin guards under his suit to protect his legs as he clambered over railings. But he also, counterintuitively, suffers from panic attacks and extreme stage fright. At the Eaux Claires festival in Wisconsin in 2015, Berninger forgot the lyrics to a handful of songs. After that, he began to use a teleprompter on stage, and his anxiety has since been much reduced. It does, however, lead to the incongruous situation, such as at Haven, where Berninger will look out and see 20,000 people mouthing words he has written, and he’s the only one who doesn’t know what comes next.

“Having the lyric monitor has made me enjoy the shows, I would say, almost for the first time,” Berninger tells me, before he goes on to perform. “You know, it feels humiliating to be on stage singing love songs with lights on you in front of people. I mean, it just is. But I’ve learned to own that and to enjoy the nakedness of it. I just feel if you’re going to be that naked, then just run around with your arms out.”

If you need an introduction, the National are America’s Radiohead. They are unabashedly cerebral: brilliant composers and classically trained musicians slumming it in a guitar band. Though you will find them on the Pyramid stage at Glastonbury, it’s also no surprise when they turn up at Moma’s PS1 gallery in New York playing one of their songs, the three-minute “Sorrow”, for six straight hours, only stopping for a pork-rib sandwich, as part of a piece conceived by the Icelandic artist Ragnar Kjartansson. (The recording is now available in a nine-LP box set.) Their regular albums – the seventh of which, Sleep Well Beast, is about to be released – sell by the shed-load. They have performed a track for Game of Thrones, but they devote similar rigour to scoring ballets or chamber music.

I don’t know what notes are, I literally can’t play a G chord

Like Radiohead, too, the National are made up of brothers and school friends. This gives the outfit a permanence and a solidity that allows it to survive and thrive despite its members’ frequent infidelities with other artists. All five are from Cincinnati, Ohio, in the midwest, but the National were formed in New York in the late 1990s. Berninger, a graphic design graduate, wanted to record songs on an eight-track with his college roommate Scott Devendorf, a bassist. Devendorf enlisted his brother Bryan, a drummer, who in turn nominated a pair of identical twins, Aaron and Bryce Dessner, with whom he’d been playing in bands since he was 15. Bryce had just finished at the Yale School of Music, where he studied guitar, and Aaron, a Columbia graduate, was working in Yale’s Holocaust library.

The National did not hurtle out of the traps. They spent years playing to empty rooms, working day jobs, watching contemporaries become successful. One time, according to band legend, they were paid not to perform. Berninger had yet to find a way to channel his nerves into a riveting, unpredictable live spectacle.

But two things were happening: they were becoming a unit, tight as a family, and they were improving. Specifically, a curiously productive, occasionally antagonistic tension was becoming established between Berninger and the Dessner twins. Berninger has no musical training – “I don’t know what notes are, I literally can’t play a G chord” – and was always pushing to make the National sound more simple and sparse. The Dessners, meanwhile, whose father was a jazz drummer, and who play more instruments than you can shake a maraca at, wanted to push it ever weirder and wilder.

Titanic rows ensued, but out of the compromises came songs like no other band were making: both grand and bombastic, but also intimate and melancholic. A strong hint was given on Alligator, their 2005 album, and it became unmistakable a decade ago with the release of 2007’s Boxer. That record took its name from a recurring theme in the lyrics about people who love each other often having the most vicious fights. And this neatly sums up the dynamic of the members of the National themselves. (Thank the lord for the Devendorfs, who are also virtuosic musicians, but much more laid-back.) The stand-out track from Boxer, Fake Empire, was adopted by Barack Obama’s 2008 campaign to unseat George W Bush and entrenched an idea that the National made music for people somehow discombobulated by the modern world.

Each album since – High Violet in 2010, 2013’s Trouble Will Find Me – has developed this singularity and added admirers. And now, after a four-year break, comes Sleep Well Beast. Rumour has it that during the recording the National were more harmonious that they have ever been. All five are in their 40s now, all have partners and young children. Instead of living on top of one another in Brooklyn, the band have dispersed around the world, and the album was made in a bucolic spot in upstate New York, overlooking a tranquil pond where bullfrogs congregate at dusk.

This newfound harmony proves to have been overstated: “Matt will be off in a corner, writing something incredibly distilled,” says Aaron Dessner, of the studio experience. “Then he comes back and we’ll have made this Willy Wonka and the Chocolate Factory arrangement, with all these beautiful colours. And he’s like, ‘What’s all that? Just turn it off!’ And we’re like, ‘We’ve literally just spent three years on that!’”

Bryce Dessner, the less headstrong of the twins, has earned the nickname “Switzerland” because of his skill in cutting a deal between Berninger and his brother. But when Sleep Well Beast was being mixed, Bryce had a newborn at home and was often absent. “The usual balance of power went way off basically,” he sighs. “It got pretty nasty.”

Aaron shakes his head, “Yeah, I’m barely recovered.”

Haven, a two-day festival held in early August, is perhaps the purest vision of the National universe. It has been curated by the Dessners, along with two Danes: Claus Meyer, the co-founder of Noma, often called the world’s best restaurant, and Mikkel Borg Bjergsø, a cult legend in craft brewing. The idea is that, in a civilised world, you should be able to pop Japanese glazed meatballs and sip a hoppy IPA while listening to, say Bon Iver or Iggy Pop or the world-class Danish String Quartet. Elsewhere, in a shipping container, you’ll find a baroque strip club where the artist Ragnar Kjartansson, now a friend of the band, sheds his clothes down to a pipe, sailor’s cap and pair of gold, sequined underpants.

“Bryce and I are born collaborators,” explains Aaron, who helpfully tends to wear his hair down and floppy, while his brother sweeps his up and back. “We grew up literally staring at each other’s hands playing guitar. We don’t have to teach each other anything, and we finish each other’s ideas and we challenge each other. So it’s very natural for us: we’ve always been open in our process, never closed.”

The day before the festival, the National limbered up in a rehearsal room in the Rhythmic Music Conservatory in Copenhagen. It’s a picturesque campus, in the shadow of what must be the world’s most stylish power station, and the room they are playing in is just a couple of steps and a dive from an inlet in the city’s harbour. “There were girls skinny-dipping in there earlier,” says Berninger, smiling. The Observer’s photographer, Suki, wonders if they might have been groupies or at least known who the band were. “No,” replies Bryce, seemingly bemused by the idea. “When in Denmark, I guess.”

The list of places where the members of the National now reside reads like the locations for a fashionable chain restaurant: Copenhagen (Aaron), Paris (Bryce), Los Angeles (Matt), Long Island (Scott) and Cincinnati (Bryan). This is a marked change from previous records. Until 2015, Aaron owned a house in Ditmas Park, Brooklyn and rented out the upstairs duplex to Berninger; Bryce had an apartment on the same street; the Devendorfs lived not far away. When the band wanted to record an album, they just walked a few steps to the garage in Aaron’s garden.

Conquering stage fright: lead singer Matt Berninger with the National at the Sasquatch festival, Washington state, May 2017. Photograph: Mat Hayward/WireImage

“We made Alligator, Boxer, High Violet all there,” says Aaron. “And it was intense. For Matt to be creative sometimes he has to go into this dream state, maybe slightly drunk. He doesn’t hang out in a room and talk about what he’s doing. So he’d be upstairs screaming his head off in a room and we’d be in the backyard in the garage tinkering with the songs. That went on for a 10-year period and did work. Bryce would make him food and leave it outside the door.”

The National have always had an unusual way of working. Each song starts with a musical “sketch”, usually created by the Dessners and filled out by the Devendorfs. When they have something they are happy with, they send it to Berninger, who adds the lyrics, should he deem the track worthy of them. He has always scribbled odd lines and fleeting thoughts in notebooks, or he’ll sit with his laptop, using the program GarageBand, just mumbling or even singing other people’s words. More and more these days, Berninger revises his ramblings with his wife, Carin Besser, a former fiction editor at the New Yorker.

“Sometimes I use weed, sometimes alcohol – I don’t do anything else,” says Berninger. “It just lets me loosen my grip a little bit on my self-conscious editing instinct and stops me trying to refine too soon. It’s a real slow process: letting it all spill out, the ugly bad melodies, terrible lyrics, and just record it and start to piece together the stuff that fits.”

The lyrics to the National’s songs are often cryptic to say the least – “It’s a terrible love and I’m walking with spiders” – but Berninger indisputably has a gift for capturing a mood and for finding unexpected rhythmic hooks to hang his words on. “He’s like a sculptor, thinking, ‘What can I fit in here?’” says Scott Devendorf, who has good cheekbones, a shaven head and favours cheery orange aviators.

For some appetites, Berninger’s imagery can get overblown and depressing, and he understands that criticism. He laughs, “I’m a romantic, an optimist, but I also like to peel back the onion on the miserable parts of my brain.”

One song on Sleep Well Beast, Guilty Party, is about the dissolution of Berninger’s marriage. In fact, he and his wife are still together, and Besser actually helped write the lyrics. “It was a way for us to tiptoe into dark territory without actually having to talk about those things,” he explains. “Most of the songs are about things I’m afraid of or things I really want to hold on to. A lot of it is about inhabiting alternate realities and digging into it. It’s confessional but it’s not autobiographical, if that makes sense. It’s emotionally confessional, but the details aren’t always specific to my stuff. Some of it is just made up.”

For a man with a reputation for being morose, Berninger is self-aware, even self-deprecating. “To be perfectly honest, the songs are already 75% what they are before I do anything to them,” he continues. “Aaron writes most of the music and there’s a lot of sadness and desperation and melancholy in so much of what he sends me, and I’m following his lead most of the time. So I don’t take full responsibility for our band being so miserablist and dark. It’s his frickin’ minor chords!”

Then there is Berninger’s delivery: a deep, sonorous baritone that rumbles the ground like an underground train. It would be a stretch to call it a beautiful voice, but it is certainly distinctive. “To quote David Berman [the lead singer of indie band the Silver Jews], ‘All my favourite singers couldn’t sing’. And he is one of my favourite singers,” says Berninger. “You know, I never cared that much about being a great singer. Like Bob Dylan… I never thought of Dylan as being a good singer. He’s an amazing singer, but I never thought of him as having a good voice. Stephen Malkmus [Pavement] or Bob Pollard [Guided By Voices], they just sang like they meant it and that was all that mattered. Kurt Cobain was an amazing singer because his pain was genuinely in the sound of his voice.”

For Berninger, whose formal musical education amounts to some piano lessons when he was a child, there’s something punk in his lack of expertise. “I’m musical but I don’t have any skill,” he says. “Does that make sense?”

The National are often called a political band. This is 100% true. And also not. All five are Democrats and, for Obama especially, they were a go-to mood-maker; Jim Messina, Obama’s long-time “fixer”, was a fan. Fake Empire played in 2008, just before the new president took the stage for his victory speech. Ohio, where the band comes from, is a bellwether – the candidate that wins that state has gone on to take the presidency every time since 1964 – and in 2012 the National pounded the pavement in an effort to register voters. “Getting to meet the president was probably one of the proudest moments of my life, for sure,” says Scott Devendorf.

The band has also fallen into a cycle of finishing albums around elections. “We were mixing Alligator when Kerry and Bush were squaring off,” recalls Bryan Devendorf, who is neck-achingly tall, funny and hairy; a drummer imagined by Wes Anderson. “And we were mixing Trouble Will Find Me during Obama-Romney. It’s been in the background all along.”

The band pictured in 1999, the year they formed. Photograph: Alec Bemis

But, actually listening to the songs, a message is harder to discern. Fake Empire is, ironically, about not wanting to engage with politics. The same is true of Sleep Well Beast: there are certainly no excoriating anti-Trump rants, although “the beast” in the title song seems to refer to the desire to hibernate and wait for the world to come to its senses. “Matt would say, ‘Politics are in everything’,” says Bryan. “A lot of the new record is a father speaking to his daughter, and he considers anybody who makes the rules that would govern his child’s life personal. And political. So it permeates the whole world he’s creating, but it’s not like he’s a news junkie who’s glued to CNN.”

Nevertheless, coming from Cincinnati, the band all agree, left a stamp on the National in their formative years. In 1990 there was a ferocious ruckus in the city over whether a retrospective of Robert Mapplethorpe photographs was too obscene for public view. Two years later, the Ku Klux Klan, with judicial backing, walked through Cincinnati and planted a 10-foot cross. “There was a lot of awful shit there,” says Bryan. “But a lot of great stuff, too.” He’s mostly talking about music: the Afghan Whigs are from Cincinnati, and Guided By Voices from nearby. “My drum teacher was in the Afghan Whigs,” Bryan goes on. “So it seemed normal and attainable: ‘OK, why can’t I also be on stage like they are?’”

Berninger freely admits he had “a chip” on his shoulder from watching other people make it. He missed the Cincinnati heyday, and then he moved to New York and had to watch the Strokes, Interpol and Yeah Yeah Yeahs blow up. “I was always on the outside of a scene looking in,” he says. “Then the Brooklyn thing started and I felt like we were, like, part of that: the Brooklyn scene. You had TV on the Radio, Grizzly Bear, Clap Your Hands Say Yeah. That was finally where we were, like, ‘OK, this is the wave I guess we’ll catch.’ Because we missed the other ones!”

But while many of those bands have burned bright and fizzled, the National endure. Part of this is a focused, very conscious determination. About a decade ago, when Boxer came out, Berninger decided to wear collared shirts, suits and smart boots on stage. “I realised I wanted to do this for a long time,” he says. “So I started dressing like an old man, and that means when I do become an old man, people will say, ‘Oh, you haven’t aged at all!’ I remember there was this old photo where I wore a very low V-neck T-shirt, standing in a field of wheat. It was really bad. I thought it was going to be sexy, but I can’t sell that.”

We watch out for each other and pull each other away from the things that tear bands apart

There have been sticky times for the National, even moments where the band feared it might implode. Before beginning Sleep Well Beast, Bryce Dessner was explicit that he wanted some elements to change. “There was a transition in technology, which happened around Boxer, when we all suddenly had Pro Tools at home and we started working separately,” he says. “It wasn’t joyless, but it started to become a more methodical thing. There were hard times for sure. Going into this process, I know I wanted it to feel different. So we set in motion a few things, and I feel like it opened a different world for us.”

Sleep Well Beast certainly sounds like a different, looser National. The band is known for endlessly revising their demos – Lemonworld from High Violet had more than 80 incarnations – but this batch of songs feels somehow less worked on: guitars are more expansive, Bryan Devendorf’s thunderous drums verge at times on the unhinged; tracks sprawl out for an unheard-of six-plus minutes. Perfection has been sacrificed, perhaps, but the trade-off is that the kinetic danger of their live shows translates.

For Berninger, the pram in the hallway, as the English critic Cyril Connolly warned, has not been “the sombre enemy of good art”. “Children put the band in perspective, right,” he says. “It could go away and we’d all be fine, like it wouldn’t be the worst thing in the world. But because of the brothers, the family connections and the roots in Cincinnati, there is a glue. We do watch out for each other and pull each other away from the things that do tear bands apart, whether it’s drugs or just stupid shit. We fight all the time, but we watch each other’s backs, too. It’s one of those things: brothers and family more than anybody, but when the shit hits the fan you’re going to be there.”

After 18 years as a band, and in some cases 30 years of friendship, there is still an obvious fondness and fierce loyalty that binds the National. More than that, there’s an acknowledgement that together they become more than the sum of their parts. “Something happens when the five of us really put our heads down, and it’s because we’ve been doing it for so long,” says Berninger. Aaron Dessner adds: “Even in the hard times, we always still loved each other. And the other thing, I think, is that we still feel we are getting better.”

“You know,” Berninger whispers, looking over his shoulder, “we all like this thing more than we let on.”

Sleep Well Beast is out 8 September on 4AD