‘New York isn’t a city for nostalgia,” Susan Seidelman says to me, matter-of-factly, as we chat over lunch in the lobby bar of Manhattan’s Ace hotel. “It’s not a city, like Paris or Rome, that rests on a glorified past. It’s a city that has no pity. It doesn’t stop for anyone or anything. It just keeps evolving.”

Seidelman, a film director best known for her hit 1985 comedy Desperately Seeking Susan, which helped cement Madonna’s stardom, is right. Newness is what New York is built on; it’s a city that, of course, never stands still. But, in recent years, as rents relentlessly rise and its residents homogenise, New York is also a city beset by a romanticised yearning. Indeed, this is largely why I’m interviewing Seidelman: later this year, two of her early movies, Smithereens and Desperately Seeing Susan, are playing at the Barbican centre in London as part of a season called the Grime and the Glamour: NYC 1976-90, featuring films “about the wild days and nights of New York’s coolest era”.

You couldn’t have a film season about New York in the 70s and 80s without including Seidelman’s work. Both Smithereens (1982) and Desperately Seeking Susan capture the zeitgeist of downtown punk-rock Manhattan through the eyes of young, independent women in ways that garnered them instant acclaim and gave them an enduring appeal. Seidelman directed Smithereens a few years after graduating from film school at NYU, and it became the first US-produced independent feature selected for the Cannes film festival. Desperately Seeking Susan was likewise released to critical commendation and has become a cult feminist classic.

The film centres on Roberta (Rosanna Arquette), married to a Jacuzzi salesman, stuck at home, and bored with life. She becomes fascinated by a series of personal ads in the papers that are “desperately seeking Susan”. One day, she heads into New York to find out who this mysterious Susan (Madonna) is. A case of mistaken identity ensues after Roberta hits her head, suffers temporary amnesia and is thought to be Susan herself. Mistaken identity then becomes reclaimed identity. Roberta’s memory loss gives her a blank slate; it frees her from being a bored suburbanite and lets her explore the woman she wants to be. At its heart, the movie is about shaking off the social conventions that come with being a woman, and the city as a place for reinvention and female freedom.

Susan Seidelman on the set of Desperately Seeking Susan. Photograph: Alamy

The eponymous Susan, by the way, isn’t Seidelman – the fact the script featured a “Susan” just happened to be a coincidence. However, a lot of Seidelman’s background is reflected in the film. She also grew up in the suburbs, always dreaming about the glamour of New York. After studying fashion at Drexel University in Philadelphia she acted on that dream, moving to the city to go to NYU Film School, one of only five or six women in a class of about 35 people, she recalls.

As a student, Seidelman lived in a tiny East Village studio. It was the mid-70s, and New York was broke, having narrowly escaped bankruptcy in 1975. Today, the East Village is lined with luxury apartments and artisanal ice-cream shops; back then, it was desperately dilapidated. There was rubble everywhere, and boarded-up buildings. The streets were full of sex workers, addicts and artists. There was a seediness to the city, but also an electric creativity and a mix of cultures and socioeconomic groups. “That was what was so great about the nightlife in the 70s and 80s,” Seidelman says. “You’d go to Max’s Kansas City, the Mudd Club or even Studio 54 and you might find Jackie Onassis sitting with Andy Warhol and then [there were] some cute Puerto Rican boys from uptown who were there to dance and have fun.” Part of Desperately Seeking Susan’s appeal is that it feels authentic; it captures the energy of its era, and a large part of this is down to Madonna’s performance. It was Seidelman’s decision to cast Madonna, who happened to be her neighbour, despite the singer’s lack of acting experience. Seidelman felt that Madonna was perfectly fitted for the role: “She wasn’t acting Susan, she was Susan. The essence of the character felt authentic to me. She wasn’t just an actress in a cool costume.”

The trailer for Desperately Seeking Susan

As well as chronicling the New York of the 70s and 80s, Seidelman also helped capture the feel of the Manhattan of the late 90s when she directed the pilot of Sex and the City. It’s fascinating to watch Seidelman’s early movies and that first episode of Sex and the City side by side and see the extent to which New York changed in just a couple of decades. In her 1996 performance piece, New York Values, Penny Arcade writes: “There is a gentrification that happens to buildings and neighbourhoods and there is a gentrification that happens to ideas.” Sex and the City doesn’t just feature a different geographical landscape from Smithereens and Desperately Seeking Susan, it shows the gentrification of the city’s imagination and the corporatisation of feminism. “Quite honestly, if someone in the late 70s said they were dating an investment banker I’d probably gag,” Seidelman tells me. “But by the late 90s, NY had changed. For better or worse, it had become more monied. In Sex and the City, the leading man was an investment banker whom you never saw working but who lived in an amazing apartment.” And, while the leading women of Desperately Seeking Susan express their independence through creativity, in Sex and the City feminism is firmly equated with consumerism, as expressed in the ability to buy $400 Manolo Blahniks.

And gentrification continues apace. New York may have no room for nostalgia, but that doesn’t stop nostalgia from creeping in. Chatting with Seidelman, it is hard not to contrast the bohemian bustle of her early years in Manhattan with the contrived cool of the lobby at the Ace hotel – mood-lit and crammed with millennials on MacBooks – and wonder, like Carrie, if it is a city desperately seeking its own soul.

Desperately Seeking Susan is playing at the Barbican, London, on 1 October as part of the Grime and Glamour season