(CNN) -- Editta Sherman has celebrated more than half a century's worth of new years in her palatial studio apartment above New York's Carnegie Hall. But it's unlikely the celebrated portrait photographer will be raising her glass there next year.

Editta Sherman has lived above Carnegie Hall for more than 50 years.

Known as the Duchess of Carnegie, the 96-year-old came home a few days ago to find an eviction notice on her door.

"I thought, oh, what is this? Are you kidding me that they are really going to send a woman like me down the street just like that? Have me scurry away without a fight," she said, delivering a whooping cackle, punctuated with a grandmother's tsk tsk.

"Oh, no, that's not what I am going to do. They'll have to take me out of here with their bare hands."

The city of New York wants to renovate the space above Carnegie Hall, where Marlon Brando once lived and where Sherman and five other renters, including iconic New York Times' photographer Bill Cunningham, have enjoyed rent-stabilized bliss since Frank Sinatra cut his first demo.

Sherman pays $650 a month for her studio, a drool-inducing space basked in natural light with floor-to-ceiling windows. An enormous skylight hangs over bold, black-and-white tiled floors; a cast-iron circular staircase leads to a loft stuffed with props.

Since last year when Carnegie Hall announced its facelift, 43 residents have lost their battle to stay, and one rent-controlled tenant has vacated, according to Hall spokeswoman Synneve Carlino. The push to renovate came from the Hall's chairman Sanford Weill who wants to expand the education classrooms for more than 115,000 music and art students.

Weill's son-in-law, Natan Bibliowicz, has been hired to design the studio spaces above the hall in a $150 million expansion, and taxpayers will reportedly foot part of the bill because New York state granted $5 million to cover design and planning costs, according to the New York Times.

Carnegie Hall has offered to pay for the rent-control tenants' relocation expenses and move them to apartments which are "equivalent or better" in the neighborhood. The Hall also is offering to pay the difference in rent to each of those tenants for the rest of their lives.

"We have asked Editta to come and look at spaces with us," Carlino told CNN.

But Sherman and her like-minded neighbors are not budging.

There is only one scenario that might work, the grandmother of 25 said.

"They can pay me $10 million. I'm part of history," she said. "You want to tell me they don't have enough rooms? They have a building of rooms. This place is history, and I think Carnegie, the people running it, I don't think they think about that."

Dressed in a purple zebra-cuffed shirt and black jumpsuit, Sherman ambles around her enormous studio with the sprightliness of a woman half her age. She holds up a photograph of herself with Salvador Dali, her aubergine-painted eyebrows animated as she tells stories about the famous faces who have dropped by over the years -- Andy Warhol, Henry Fonda, Eva Gabor, Tyrone Power, Carl Sandburg, Paul Newman.

"With Salvador, he had an exhibit nearby, you know, and I went there to meet him and we just hit it off. So he came back to my place and I took some pictures," she said. "He wanted to buy my (stair) railing which was pure bronze then, with some engravings from Paramount. I told him it was quite expensive and he said he'd have to think about it."

Yul Brenner brought Marlene Dietrich by once in the 1950s during a time when the two Hollywood stars were reportedly having an affair, Sherman said.

"They were just so sweet," she said. "Yul was playful, and she was quiet."

In true Warhol style, Sherman photographed the pop genius as he was photographing her.

Warhol's portrait sits next to the hundreds of other portraits piled up in rows in her studio. Sherman has hundreds of letters from Cary Grant -- a long correspondence of them trying, in vain, to get together for a portrait session.

"I never thought that taking photos would be valuable," she said. "I did it to earn a living and because I liked it."

Sherman's career took off when she got a job in New York casinos which hosted welcome home parties for World War II soldiers. Her husband, Harold Sherman, who was an inventor and proprietor of photographic technology, convinced the management of the casinos to let his wife take photos of the celebrities who entertained there, she said.

Sherman had a way of putting celebrities at ease when they posed for her, a gift she picked up from her father who was a photographer. And soon word of her work buzzed among New York's jet set. The Shermans and their five children decided they needed a place to live in New York.

While many were running for the tranquil promise of the suburbs, she wanted to be in the middle of it all. She spied a full page ad in the New York Times in 1947. It read: "Live and work in Carnegie Hall." Rent was $225 a month.

"It wasn't a big deal at the time, and when I saw the place I thought that it would be big enough for me and five children," she recalled.

But the studio was much larger and more ideal for photography than she ever imagined. Vogue magazine used to borrow it for shoots. In the 1960s, Sherman shot many images of the supermodel Veruschka, a Prussian emigre whose father, a German count, was executed for trying to assassinate Hitler. A celebrated muse of Dali, Veruschka was a pleasure to photograph, Sherman said.

"She had such a beauty," Sherman said. "I believe youth and beauty are all in how you live."

Sherman has space to live, at least for now. She occupies an entire floor. Her children are all grown and long moved out. Her husband died in his 50s from diabetes. She spends most days shooting photos and jumping rope to stay fit.

"I feel lost sometimes that I'm the only one on this floor now," she said. "But, you see, people get tired of fighting. They lived here, and they could live somewhere else, so they did.

"But I am different. I have this business here," Sherman said. "This is who I am, where I live, and I won't let someone change that."

All About Carnegie Hall