For more on this report, see CNN affiliate WMAZ-TV in Macon, Georgia.

Atlanta, Georgia (CNN) -- A squabble at a small Georgia town's council meeting -- in which a white councilman told his black counterpart that he should be working in a cotton field -- has caught the attention of the NAACP.

The exchange took place in a recent Warner Robins City Council meeting, said Larry Holmes, president of the NAACP in central Georgia's Houston County, which includes Warner Robins.

"They need to learn to respect each other and they need to stop all the name-calling," Holmes said.

"Ever since this new council was elected, there has been different problems. For one thing, they just can't seem to function there in a more pleasant and peaceful manner," he said.

The controversial "cotton field" exchange happened earlier this month, said CNN affiliate WMAZ.

The heated argument, which was captured on tape, was between council members John Williams, who is white, and Daron Lee, who is black, the affiliate reported.

During the exchange, Lee said he was tired of being interrupted and was upset about how he had been treated at an earlier meeting.

"I was disrespected last Monday. I'm getting about tired of you all, talking to me any kind of way. I'm not working in a cotton field," he said.

"You should be," Williams interjected after a short pause.

Lee then walked out of the meeting but later returned, the affiliate reported. Afterward, Lee told WMAZ that Williams "smiles in your face and makes racial remarks. I am pretty much used to it by now [but] I had it last meeting. It is a poor representation of the council."

Williams denied there was any racial element to his remark.

I worked in the cotton field," he said. "I drove a cotton basket many miles. It is not a racial remark at all. [Lee] makes everything racial."

Both men appeared Thursday on CNN, where it appeared clear that the impasse would continue.

Williams said he had not apologized to Lee, and would not do so until Lee apologized first for bringing race into the discussion. Lee said he doesn't have anything to apologize for.

"Not only did it offend me, but more so it offended the people who make up the city of Warner Robins," he said.

Williams accused Lee of using race as a tool for division.

"This gentleman has steadily built a wall between the white and black citizens in the community," Williams said.

In the public comment period of the meeting where the interchange occurred, one resident suggested that the City Council get diversity training.

The National Association for the Advancement of Colored People sent a letter to the City Council stating that Williams should resign, Holmes said.

"We're just saying if they can't work together that maybe someone needs to resign," he said.

Williams said he has no plans to do so.

CNN's Scott Thompson contributed to this report.