Rochester is a place built by photography. Since 1888, this small city in upstate New York has been home to Eastman Kodak: a corporation that, at its peak, produced 90% of the film used in the US and provided jobs for more than 60,000 local people. Now, like many other company towns across the US, it has fallen on hard times. After years struggling to keep pace with the digital revolution, shedding workers and selling off its assets, Kodak was forced to file for bankruptcy protection in January 2012.

The announcement caused dismay throughout the photographic world. In April 2012, the renowned Magnum agency dispatched 11 of its members – including Alec Soth, Susan Meiselas, Martin Parr and Bruce Gilden – to Rochester for two weeks. Together, they would document life in the town at this challenging time. But, as their new book Memory City proves, for Alex Webb and Rebecca Norris Webb one trip was not enough.

In the 12 months following their first visit, the husband-and wife team returned to Rochester four more times. Each stay lasted two or three weeks, during which they steadily worked their way into the fabric of the 210,000-strong community. "I'm a street photographer at heart, so I approached Rochester the same way I often approach somewhere I'm shooting," says Webb. "Walking, meandering, allowing my camera and experiences to guide me." Norris Webb, on the other hand, took the time to forge close friendships with residents.

Webb's boldly coloured glimpses of residential neighbourhoods, markets and community gatherings show a city that has not lost its pride. But for every manicured lawn and fluttering Stars and Stripes, there are many bleaker counterpoints – a broken couch on a neglected street, a figure slumped on a public bench, exhausted workers queueing for the night bus home. "We wanted to achieve a sense of layers," Webb says. "We wanted a structure that mirrored our sense of this down-and-out yet very soulful place."

The most poetic and striking example of this can be found in a series of grainy monochrome images that appear at irregular intervals through the book. Norris Webb opted to shoot on still readily available Kodak Portra negative film, but Webb chose to work with a mix of digital cameras and expired Kodachrome. This 35mm slide film, loved by many for its rich, intensely saturated colours, was discontinued in 2009 and had not been developed commercially since 2010.

"We decided to ask around to see if some obscure technician still processed Kodachrome," says Webb. "No one did it as colour, but we did discover that Ed Praus, a photographer who runs a lab in Rochester, could develop it as black-and-white. At Rebecca's urging, I tried out a few rolls. To our surprise, it came out as negative and looked distressed, almost weathered."

14621 Neighborhood, Amanda and Her Flower Dress: Rochester, New York. Photograph: Rebecca Norris Webb/Magnum

While her husband's work evokes Rochester's past via scenes of the present day, Norris Webb took the town's history as a starting point from which to explore what may lie ahead. "Rochester was home to Susan B Anthony, probably the most noted suffragette in US history," she says. "So women were very much on my mind. My first portrait was of Amanda Webster, our assistant. Her family epitomised the changes that Rochester has gone through. Both her grandfather and father worked for Kodak. Now she's studying photography at the Rochester Institute of Technology, which was funded in part by Kodak's founder, George Eastman. Through her, I met and photographed a variety of young women whose portraits suggest, to me, something about the city's uncertain future."

Shot through closed windows and gauzy curtains, these photographs capture a sense of melancholy and introspection. But Memory City's most elegiac frame comes from the streets of downtown Rochester. In 1914, work was completed on an ornate 19-storey tower, commissioned by Eastman to house his then thriving company. For generations, it dominated the city's skyline, symbolising wealth, innovation and possibility. In an image made by Webb almost 100 years later, it stands forlornly in the distance as a lone man strides away from it, past the rubble of a crumbling building – a poignant and all-too-perfect Kodak moment.

• Memory City by Alex Webb and Rebecca Norris Webb will be published by Thames & Hudson on 30 June