Over the years, Noah Baumbach, the American writer/director, has made films about all sorts of things. Kicking and Screaming (1995) is about a group of college pals who refuse to move on with their lives. While We’re Young (2014) is about a friendship between a middle-aged documentary-maker and his wife, and a couple in their 20s. The Meyerowitz Stories (2017) is about siblings attempting to live in the shadow of their egomaniacal artist father. But for some of us, he really has only one true subject: D-I-V-O-R-C-E. Consider the best movies of his career so far, made almost 15 years apart: The Squid and the Whale (2005), the bittersweet picture that first brought him to most people’s attention, is about divorce, and so, too, is Marriage Story, his hotly Oscar-tipped new film. To pinch from Philip Larkin, separation is to Baumbach what daffodils were to Wordsworth.

Not that he sees it quite like this. When we meet in a clattering New York restaurant on a cold, bright autumn morning, Baumbach is annoyingly keen to resist my passionately held theory that people whose parents split up when they were children (him, me) grow up to be somewhat preoccupied by divorce, a fascination that will probably end (at least in my case) only with death. “Yes, both those films are about divorce,” he says, in his careful, rather ponderous way. “The Squid and the Whale from the kids’ perspective, and Marriage Story from that of the adults. But what they also provide narratively is the opportunity to talk about other things.” I stare at him. What other things? What can he possibly mean?

“Well, Squid is about family,” he says, with the kind of patience a therapist might reserve for a particularly anxious patient. “It’s really about the necessary movement away from our parents, whether divorced or not, and their conscious or unconscious tugging back [of us]. And although Marriage Story is about the disintegration of a marriage, it enables you to look at marriage itself, too.” Both films, he insists, are surprisingly hopeful, in spite of what I regard as all the evidence to the contrary. “No, no,” he tells me, reassuringly. “They’re really about starting again.”

Baumbach, who is a boyish 50, has the kind of hairstyle that manages somehow to be both long and short, and the kind of clothes that pull off the rare feat of appearing both pin-neat and artfully dishevelled. He looks, in other words, like a cross between a university lecturer and an assistant in Comme des Garçons. He also looks exhausted. The shadows under his eyes match the navy of his boiled wool blazer; he clings to his coffee cup as if to a life buoy. But this is hardly surprising. His schedule appears to be quite mad. This morning, I was woken at 5am by a message informing me that our encounter might be off because he was going to have to fly at short notice to Los Angeles to appear in a Q&A at a film festival – and, in fact, when he leaves here that’s exactly what he will be doing. Even as we speak, a frazzled PR is standing at the door, stopwatch in hand.

Scarlett Johansson and Adam Driver in Baumbach’s latest, Marriage Story. Photograph: Netflix

“This has been going on since Venice in late August,” he says. Not that he’s complaining. He could hardly be more gracious: “When you finish something, you don’t come out of the gate with a fully formed idea of what you’ve done. In a way, I feel talking about the movie, and hearing what other people think about it, helps me to understand it better myself.” The beauty of a film like Marriage Story, he thinks, is that people bring their own stuff to it, and thus the conversations he has about it tend to be as much about life as about the film itself. But in any case, irrespective of the demands of producer/distributor Netflix, which clearly hopes Marriage Story is going to win it an Oscar, I think he feels a certain duty of care.

When he was growing up, lots of people used to make films like his: interior dramas (though his movies undoubtedly have their blackly funny moments, they can hardly be accurately described as comedies) starring east coast liberal intellectual types with extensive bookshelves, subscriptions to the New Yorker and complicated emotional lives. Nowadays, though, they’re increasingly a rarity. He is determined to do his best to “articulate something that is essentially inarticulate by design” to the press and to festival audiences alike.

In Marriage Story, Scarlett Johansson plays Nicole, an actor, and Adam Driver her estranged husband, Charlie, a perfectionist director of edgy off-Broadway theatre. We know that Nicole and Charlie were once madly in love – as the film opens, we hear Charlie describing all the things he adored about her when they first met – but somewhere along the way, their relationship has broken down. Having previously turned her back on Hollywood in favour of working in New York with Charlie’s theatre company, Nicole has now returned to LA with their eight-year-old son, Henry, to star in a TV series.

The film depicts not only the couple’s increasingly toxic custody battle – it comes with brilliant turns from Laura Dern, Alan Alda and Ray Liotta, all of whom play divorce lawyers of varying degrees of rapaciousness, and from Mary Hollis Inboden as a terrifyingly silent family court representative – but also its subtle effects on their sense of identity. It is in some ways a movie about power. As Nicole grows ever stronger, her world expanding, her voice becoming louder and more confident, Charlie is ever quieter and more fragile. On Halloween, when he dresses up as the invisible man, the bandages he wraps around his face speak volumes. It’s as if he is determined to erase himself before someone else does.

Naomi Watts in While We’re Young from 2014. Photograph: Scott Rudin Prod

Can Baumbach remember the tug on the line that was the beginning of this film? “It really isn’t possible to do that,” he says. “It’s so… these things are garnered all the time. I often have a lot of… I wouldn’t even call them ideas. Sometimes, they’re ideas, but it can also be an atmosphere, a line, a relationship, a location. These things are in a notebook, and in my head, and then they keep finding their way back to whatever I’m working on. Something will return, and I won’t know where to put it. It will keep bouncing around until, at a certain point, it’s like the electrical current in a dead wire suddenly goes on – and then that wire is the one, and I start looking at it in a way that I wasn’t before. When I started writing this movie – well, the actual writing was probably only six months or something, but all the stuff that went into it you can’t quantify.”

So in the beginning, he’s basically a kind of ambulant whiteboard? He smiles. “Yes. In this case, I was working on The Meyerowitz Stories and things got auditioned in that script, and then they got kicked out, and then they found their way into this one.”

Aware that a film about a custody battle might be gruelling, he knew it was vital that the audience have a sense of Nicole and Charlie’s former happiness: hence the early scene – an extended flashback – during which Charlie lists his wife’s virtues. “I wanted that to feel very immediate and intimate and active, knowing that very soon it was all going to be destroyed,” he says. “The audience need that memory for perspective. I wanted people to feel as though they had been inside this family, however embarrassing that might be – because after this, the journey is into all these sterile office environments and courtrooms. We’re going to be wrenched out of home life. In fact, the notion of home life is going to be completely redefined.”

Baumbach went through a separation himself some years ago; he and Jennifer Jason Leigh, with whom he has a son, Rohmer (named after the French film director Eric Rohmer), divorced in 2013; he now lives with Greta Gerwig, the actor and director, with whom he has a new baby (they met in 2010, when she starred in his film Greenberg, though their relationship did not begin until later). To what degree, then, is Charlie a version of himself? “He’s not me any more than Nicole is,” he says. And whose side is the film on? Some people think that Nicole fades from view; that in the end, it comes down, for whatever reason, on Charlie’s side. “I’m aware of that view. But if anything, this movie only illustrates that to take sides at all is folly. When it begins, we’re with her: we’re in LA, where she grew up, with her mum and her sister. She’s telling her story. But then he turns up, and we drift into his. I feel like the job of the last part of the movie is to say: it’s all true, and none of it’s true. These are just people trying their best. Her story is one of momentum, rebuilding and finding her voice, and his is a story of breaking down; it’s a kind of role reversal, because he’s a director and she’s an actor – not that I want to stereotype those roles. But I also think that her monologue and his song at opposite ends of the movie are mirror images of something: in a way, both of them find their voice.”

Jeff Daniels and Laura Linney in Baumbach’s 2005 film The Squid and the Whale. Photograph: Allstar

The song in question is Being Alive, from Stephen Sondheim’s Company. Charlie sings it in a bar, with exquisite sincerity and immense feeling, watched by members of his theatre company, and it’s extraordinary: an epiphany that seems to come out of nowhere, and somehow puts a bow around the whole film. Baumbach nods: “You didn’t know that you needed it, right?” This is true, but it also raises the question: how did he know that I needed it? For a few moments, he tries to explain – something about avatars and voices and perspective – but in the end, perhaps it simply comes down to this. Driver loves that song, and its lyrics speak to one of the film’s themes, which has to do with the fact that just as familiarity can breed contempt, perhaps one can only feel truly alive when one is absolutely loved and seen (“Somebody pull me up short/And put me through hell/And give me support”). Anyway, it’s perfect. That was the moment when, sitting there in the dark, something inside me gave way, and my eyes filled with tears.

Why does Baumbach think people get married at all – especially people like us, whose parents failed at it several times? “Oh, boy.” Looking agonised, he puts his forehead in his hand. “Um, um … Well, why wouldn’t people get married? It’s a very… hopeful, very… brave thing, getting married. It’s great that people want to do that. It’s a great act of hope. It’s… romantic. Perhaps it has to do with repair as well: that’s something we’re all doing, in our different ways. Whatever our family damage is, repair takes a lot of forms.” Can those who have been through a painful divorce – though not all divorces are, or need to be, painful – ever recover from the experience? “It’s like any childhood event or trauma … it’s up to us. As the Wallace Shawn character says [Shawn plays an actor in Charlie’s company who functions as an occasional Greek chorus]: it’s going to be horrible, but it’s also going to be over.” A pause. “Though as we know, it keeps going, too.”

Baumbach grew up in Brooklyn, the older son of the novelist Jonathan Baumbach (who was married four times) and Georgia Brown, a film critic at the Village Voice (she was Jonathan’s third wife). He was 14 when his parents separated. He remembers it vividly. He went out to watch Kathleen Turner and Michael Douglas in Romancing the Stone, having been warned that when he got home there was going to be a family meeting – a scene he reprises beautifully in the semi- autobiographical The Squid and the Whale, in which Laura Linney and Jeff Daniels play a version of his parents – and he knew perfectly well what that meant: “I watched the movie with some sense of dread, though I guess it took me out of my head for a while.” He began to cry even before his parents announced their complicated custody arrangements (in the interests of fairness, the two boys would visit their father every Tuesday, Wednesday, Saturday, and every other Thursday). Has he seen Romancing the Stone since, or are the glorious adventures of Turner and Douglas in the Colombian jungle forever ruined for him? He laughs. “I don’t know. I haven’t seen it for years. But I liked it at the time. Doesn’t Michael Douglas end up in a boat on Central Park South or something?”

Did he always know that he wanted to make films? “I had, at least, the dream. I didn’t know anyone in film, so I didn’t know it was possible. I wanted to do it, and I would say that I wanted to do it, but there was a second voice in my head that said: well, how would that even happen? But I did a lot of plays, and wrote stories, and when VHS happened, I got a camcorder and lugged that around. VHS was amazing. You could watch movies at home, and you didn’t have to wait till Easter to see The Wizard of Oz or whatever.” Did it help that his parents led creative lives? “It did, and they loved movies, too.” He has said that his father took him to see Truffaut’s The Wild Child when he was just five, or thereabouts. “I was lucky to have them as an example, though it also created its own burdens and challenges. I guess film, for me, was both a kind of inheritance, and a way of doing something different [from them].”

Eric Stoltz in 1995’s Kicking and Screaming. ‘His character was just slapped on,’ says Baumbach. Photograph: Everett Collection

Baumbach studied English at Vassar College, where he wrote, directed and acted in plays, but made no movies, largely on account of the fact that its film department had no decent equipment. How, then, did it happen that he came to release his first film, Kicking and Screaming, when he was only 26? He shakes his head. “There isn’t a good story. Except to say that I’d worked on a script, and I had two good friends who were my housemates in senior year, Jason Blum and Jeremy Kramer, who are both still in films, and the three of us would give it to anyone we could. Again, we had no idea how it worked, and it fell apart many times, even in pre-production. It was so random. The video company involved threatened to pull out at one point, and I had to write the part Eric Stoltz plays at the last minute to get it financed. His character was just slapped on.” He smiles. “I was in LA, and I felt so old, and yet I was so young. I just thought it was all too good to be true.”

And in a way, it was. A second film, Mr Jealousy, followed, after which there was a quite long period of silence. “Yeah. What happened? It was hard for me. I went through this time after Mr Jealousy: I’d had this early opportunity, and then I was having trouble getting anything made. I was writing pilots, things like that. I was staying busy and making a living, but not the living I wanted. I began to wonder: am I a film-maker? Will I ever make another one? I didn’t think about quitting, but it was really difficult. What I ended up doing was working on myself. I went into therapy. I grew up a bit. I started to understand myself better, and because of that I started to write and think more creatively. Then I wrote The Squid and the Whale. I had a breakthrough. It was a case of: I’m not the film-maker I thought I was.”

Baumbach with his partner, actor and director Greta Gerwig, at the New York premiere of Marriage Story last month. Photograph: Marion Curtis/StarPix for Netflix/REX/Shutterstock

The other thing that helped was his friendship with the film-maker Wes Anderson, with whom he wrote The Life Aquatic With Steve Zissou (2004) and Fantastic Mr Fox (2009). “We were both friends with [the director] Peter Bogdanovich, and we would hear about each other. We finally met at a John Waters premiere in 1998; afterwards, we ended up at a bar in Soho called Toad Hall. We started talking, and we realised we shared a lot of likes: when we exchanged numbers, we realised we both had the same kind of notebook in our pockets. He was making The Royal Tenenbaums, and just watching how he did it was so inspiring to me. His work ethic, his precision.” Anderson went on to co-produce The Squid and the Whale.

“A lot of film-makers are nice,” he says. “But it can be tough. Directing is lonely. Mike Nichols said a director always needs a buddy, and that’s wise: to have a producing partner, or a writer. Though if you write and direct, obviously you don’t have a writer.” These days, however, he has Gerwig, with whom he wrote two films, Frances Ha and Mistress America. “The press part is harder, because we’re on different schedules. But in terms of the rest of it, for us, it’s nice.” Gerwig’s adaptation of Louisa May Alcott’s Little Women, starring Meryl Streep and Saoirse Ronan, opens at Christmas. He beams. “Oh, it’s great. When you see it, you’ll realise immediately. You’ll think: of course, this is why she did it.”

For his own part, he has no idea what he’ll do next. “I was with Adam [Driver] last night, having the conversations we tend to have: how about this? How about that?” All he knows is that he and Driver, who have become close in spite of the 14-year age gap between them, will certainly work together again in the future. “He’s amazing. He’s always alive as an actor to the particular moment.” Baumbach only wants, he says, to go with what he’s doing right now. “The whole process is a conversation with my younger self, who so loved movies.” He won’t – he can’t – take it for granted. “You’re at Venice, or somewhere, and something happens, and it’s so exciting, and all you want to do is check in with your 17-year-old self who, had he known what was coming, would have thought it so cool.”

Noah’s arc: the best of Baumbach, by Guy Lodge

Kicking and Screaming (1995)

Aged just 26, Baumbach made his debut with this modestly scaled but spiky ensemble piece following a group of college graduates all struggling to start the next phase in their lives. Made with a script endorsed by Steve Martin, it earned Baumbach a pile of “names to watch” notices, though his true breakthrough would take another decade.

The Squid and the Whale (2005)

Now thematically bookended by Marriage Story, this mordantly funny divorce drama starred a peak-form Jeff Daniels and Laura Linney as warring literary spouses and was inspired by Baumbach’s parents’ split. It was the director’s fourth film, but it announced his name to a wider audience, scoring him his only Oscar nomination to date.

Margot at the Wedding (2007)

Reviews were chilly for Baumbach’s follow-up to the success of The Squid and the Whale, but this piquant study of a successful, narcissistic writer descending on her estranged sister’s Long Island wedding festivities gets too little credit. Nicole Kidman and Jennifer Jason Leigh give exquisite performances and there’s hard humanity under the sour surface.

Frances Ha (2012)

Baumbach’s collaboration with Greta Gerwig began with the droll LA story Greenberg, but it blossomed in this vibrant, New Wave-styled character study of a young would-be dancer in personal, professional and even residential limbo. Co-written with Gerwig, and shot in gorgeous, grainy black and white, it’s less jaded and more joyful than his previous work.

Marriage Story (2019)

Fourteen years after making drama from his parents’ divorce, Baumbach gives the same treatment to his own split from Jennifer Jason Leigh, following Adam Driver’s New York theatre director and Scarlett Johansson’s LA-native actor through the travails of a bicoastal separation. Brutally close to the bone and sometimes raucously comic, it’s earned the best reviews of his career.