(CNN) -- For decades, Harlem has been a hub of African-American culture and the NAACP. But this month, the issue of charter schools has turned the New York City neighborhood into a battleground between activists and the 101-year-old civil rights organization.

The strain was as clear as the signs and chants during a rally Thursday in Harlem -- "NAACP, Don't Divide Us, Unite Us!"

About 4% of New York City's roughly 1 million students currently attend charter schools -- which are paid with public funds but typically have different mandates, approaches and staffing arrangements that traditional schools. Another 50,000 students are on waiting lists.

A proposal to expand access to charter schools and allow charters to co-locate -- or be based in the same building -- as regular schools has turned some traditional allies against each other.

The rift accelerated in May, when the NAACP and the United Federation of Teachers -- a union of 200,000 New York City teachers and school staff -- filed a lawsuit in New York State Supreme Court. Besides asking to stop 22 school closings, the lawsuit urged halting the plan to expand or co-locate 20 charter schools.

On its website, the teachers' union calls the initiative illegal and claims it would give "charter school students more access to school facilities, including libraries, auditoriums and lunchrooms, than students in the public schools that would share the buildings."

While a teachers union opposing charter schools is hardly unprecedented, leaders of the local NAACP said they decided to step in and support the lawsuit in order to best ensure a level playing field for all New York City students.

Hazel Dukes, president of the NAACP's New York chapter, said the group is fighting for equal education for all -- similar to the landmark U.S. Supreme Court decision on Brown v. Board of Education, which mandated equal education regardless of race.

"The facilities should be in good condition on both sides," said Dukes. "There should not be a 10 o'clock lunch hour and a 12 o'clock lunch hour. There should be one 12 o'clock lunch hour."

Yet Geoffrey Canada, a nationally renowned educator who is the president and CEO of Harlem Children's Zone, thinks more charter schools in New York City are the solution -- not the problem. That's why he joined parents Thursday calling for the NAACP and teachers' union to drop the lawsuit, or at least the part pertaining to charter schools.

Aisha Moore, who attended the rally, said the NAACP's stance is counterproductive if its goal is to support the rejuvenation of New York City schools.

"The NAACP has always backed minorities and African-Americans, so it's just surprising they're moving against us now," she said. "It's like moving backwards, when we should be pushing forward."

Keona Moore, a single parent whose daughter attends a charter school, said that the trickle-down effect of closing schools and limiting options could be devastating.

"I'm not going to put her in a failing public school," Moore said.

Zakiyah Ansari said that's where the efforts should be focused: on making sure no existing public schools are failing, not to create more charter schools at the expense of traditional ones. The mother of eight insisted that those rallying in Harlem on Thursday and those behind the lawsuit -- like herself -- have more in common than they might think.

"Now we're engaged with parents and having them come out in busloads to fight against other parents fighting for the same thing -- the right for a quality education," said Ansari, an organizer with the Alliance for Quality Education. "We're not asking for more."