(CNN) A Manhattan developer must lop off floors from its Upper West Side construction project, a New York Supreme Court judge ruled Thursday.

The 55-story building would have towered over a busy residential and commercial neighborhood close to Lincoln Center and Central Park. The building currently under construction stands at 668 feet and 51 stories, according to the Municipal Art Society, one of the plaintiffs in the case.

"If you have to take down part of the building, that's a big deal, particularly one that's newly constructed. Not unique, but unusual," said Elizabeth Goldstein, president of the Municipal Art Society. The Committee for Environmentally Sound Development also joined the lawsuit.

Justice W. Franc Perry ruled that the permit for the development, located at 200 Amsterdam Avenue, should not have been issued. He ordered the city to revoke the building permit and remove all floors that exceed the zoning limit.

The ruling did not state how many floors should be removed, and it remains unclear if and how many would be directly affected by the ruling. The city's Department of Buildings, which approved the permit and was challenged alongside the developer, will be ruling on how many floors to remove. The department will have to analyze the city's mammoth Zoning Resolution, which is 1,600 pages long and dates back to 1916.

Read More