(CNN) Voters in Kansas City, Missouri, decided to remove Martin Luther King Jr.'s name from a historic boulevard that was renamed in January to honor the civil rights leader.

In a special election Tuesday, the measure to change Dr. Martin Luther King, Jr. Blvd. back to The Paseo Boulevard received about 65% of the vote, according to figures released by the city's Board of Election Commissioners

The vote came after months of debate between groups intent on honoring King's legacy and some residents who didn't want to lose their neighborhood's identity.

The Paseo, as many locals call it, is one of the oldest boulevards in the city and runs north to south through a predominately African American section of the city.

"We don't mind doing something to honor Dr. King, but we don't want you to take Paseo away from us to do it," said former City Councilwoman Alissia Canady, who represented much of the area until her term ended in August. She helped organize the Save The Paseo group, which collected almost 3,000 signatures to get the name change on Tuesday's ballot.

Canady voted against the MLK name change in January, but it passed by an 8-4 vote.

"That really infuriated the neighborhood because they felt it was a backdoor approach that deviated from the process and took away their voice," Canady said.

That caused tensions between residents and the Southern Christian Leadership Conference of Greater Kansas City , which worked for three years to get the boulevard named after King.

"These boulevards are treasured in Kansas City and rightfully so," said Vernon Percy Howard Jr., the group's president.

He said Tuesday's vote makes Kansas City once again one of the few major cities that does not have a road honoring King.

"For 51 years since Dr. King's assassination, no one in this town moved forward to advance or develop any kind of plan or program to honor Dr. King in a major way," Howard said. "There is a park that is embarrassing in the central core of Kansas City that's not well kept and very small and insignificant with respect to any of its beauty, its architecture, its land space, its footprint and all of that."

Howard said he was surprised by the resistance to change.

"I am shocked at the vigor and hostility that has been expressed from some African Americans, who view that that generic name has more weight or carries more significance to them than Dr. Martin Luther King Jr., who shed his blood and died for our right and our ability to own homes, to obtain loans, to have equal employment opportunities, to be protected from racial discrimination in the workplace in housing, in public transportation and so forth," he said.

He also blamed gentrification of the area and an influx of money from white investors for the opposition.

"It is the epitome of white privilege and systemic and structural racism that a predominately white group would actually have the audacity to determine or dictate in a predominately African American community who, or what they should honor, and where and how," Howard said.

Canady, who is black, called that a deflection.

"The majority of the property owners on Paseo are black, the residents and property owners are black," she said. "The majority of the businesses, the majority of the residents are black. So this is black leaders oppressing the voices of black property owners."

She said they weren't respectful to the wishes of the community.

"They were trying to do a good thing, but they went about it the wrong way," she said.

Canady said she favored naming the city's new airport terminal after King, or an east-west street because it would run through black and white neighborhoods.

University of Tennessee geography professor Derek Alderman and his research team have been building a database of streets named after MLK for more than 20 years.

He says they have identified 955 streets named for King in 41 states as well as Washington and Puerto Rico as of December 2017. That includes streets that use his full name or variations such as MLK or M.L. King, but not streets that are simply named King because they may not reference the civil rights leader.

"I can confidently confirm that among the top 50 most populated cities (as of 2015 estimates), Kansas City is now one of just three cities without a street named for Dr. King. The other two cities are San Jose, California, and Omaha, Nebraska," he said.

Those cities do have public buildings named for King, he said.

Howard and his supporters said they still think naming a road after King is the best way forward.

Mayor Quinton Lucas was on the city council in January and supported the MLK name change. He told CNN affiliate WDAF on Wednesday that his job now is to bring the community together, so that residents can honor King.

"People want to make sure that we engage with enough different community stakeholders and I think it's fair to say that this did not happen, and that's why we're kind of in the position we are now," he told the station. "This wasn't so much a repudiation of the Dr. King name or any ministers involved. It's instead saying, 'Let's make sure we work together and get it done right.'"