MISRATA, Libya—"Traitors keep out," reads graffiti at the entrance of a housing project in an impoverished neighborhood of Misrata, the rebel-held city grappling with the physical and emotional scars of Col. Moammar Gadhafi's siege since March.

A group of men sipping tea in the courtyard on a recent afternoon say the "traitors" are those who hail from Tawergha, a small town 25 miles to the south inhabited mostly by black Libyans, a legacy of its 19th-century origins as a transit town in the slave trade.

Many Misratans are convinced that Tawerghans were responsible for some of the worst atrocities committed during their city's siege, including allegedly raping women in front of their relatives and helping Gadhafi forces identify and kidnap rebel sympathizers and their families.

The feud between Misrata and Tawergha offers a stark example of the challenges Libya will face in reconciling communities that found themselves on opposite sides of the conflict when Col. Gadhafi leaves power.

Misrata, Libya's third-largest city and its commercial hub, has been viewed with suspicion by Col. Gadhafi, who sought to promote minority groups like the Tawerghans and some Bedouin tribes in the area to counterbalance the might of the tightly knit white merchant families here.

Before the siege, nearly four-fifths of residents of Misrata's Ghoushi neighborhood were Tawergha natives. Now they are gone or in hiding, fearing revenge attacks by Misratans, amid reports of bounties for their capture.

The rebel leadership in the eastern city of Benghazi says it is working on a post-Gadhafi reconciliation plan. But details are fuzzy and rebel leaders often resort to platitudes when dismissing suggestions of discord, saying simply that "Libya is one tribe."

On Edge in Libya Track the latest events in Libya. Regional Upheaval Track events day by day in the region. More photos and interactive graphics Map: Unrest in Libya

That viewpoint could prove dangerously naive. Already the fighting has fanned historic feuds and created new fault lines across the country. In the Nafusa Mountains southwest of Tripoli, rebels from the Zintan tribe are now pitted against their old rivals the Mashashya, who are mostly pro-government.

In a bid to calm some of these tensions, Libya's former colonial ruler Italy, which is siding with the rebels, announced last week it would host almost 300 Libyan tribal leaders for a major reconciliation conference, an offer quickly ridiculed by the Gadhafi regime.

"The longer this [fighting] goes on, the more it reinforces deep mistrust across all social cleavages," said Lisa Anderson, president of the American University in Cairo who is a Libya expert.

Misrata's rebels succeeded last month in pushing Col. Gadhafi's forces out of the city, but they continue to struggle in battles on three fronts including the border with Tawergha. A teenage boy was killed Monday and six of his relatives were wounded, including his parents and siblings, said witnesses, when pro-regime forces on the city's outskirts fired rockets into Misrata. Since Friday, similar attacks in the area have killed two women and at least 26 rebels, including ten on Monday, doctors said.

Though the rebel's political leadership says it will take steps to avoid reprisals if they capture the town, others are calling for the expulsion of Tawerghans from the area.

Ibrahim al-Halbous, a rebel commander leading the fight near Tawergha, says all remaining residents should leave once if his fighters capture the town. "They should pack up," Mr. Halbous said. "Tawergha no longer exists, only Misrata."

It is unclear how many families still live in Tawergha, which has turned into staging grounds for government troops. Many are believed to be in a government-administered camp in al-Haisha farther south.

Other rebel leaders are also calling for drastic measures like banning Tawergha natives from ever working, living or sending their children to schools in Misrata.

The hatred of Tawergha stems from witnesses who say loyalist soldiers were accompanied by hundreds of volunteer fighters from Tawergha when they ransacked and burned dozens of properties in an assault against Misrata and surrounding areas on March 16 to 18.

There are also accounts of rape, with one rebel commander putting the number at more than 150, but they are harder to prove given the stigma attached to the crime in the conservative muslim nationand the lack of testimony.

Some of the hatred of Tawergha has racist overtones that were mostly latent before the current conflict. On the road between Misrata and Tawergha, rebel slogans like "the brigade for purging slaves, black skin" have supplanted pro-Gadhafi scrawl.

The racial tensions have been fueled by the regime's alleged use of African mercenaries to violently suppress demonstrators at the start of the Libyan uprising in February, and the sense that the south of the country, which is predominantly black, mainly backs Col. Gadhafi.

Bashir Amer says he was one of the victims of the assault on Misrata by loyalist soldiers and Tawerghans in March. Nothing was spared on his ranch, he said, in the farmland area of Tuminah on the road between Misrata and Tawergha.

The carcass of one of Mr. Amer's Thoroughbred horses was still baking in the sun during a recent visit. His farmhouse was set on fire after all valuables were looted, Mr. Amer said as he held up his wife's empty jewelry box. He stood in the master bedroom, which was reduced to incinerated walls and a carpet of ash.

Mr. Amer said he was having breakfast with his family when soldiers jumped over the farm's fence and started shooting indiscriminately, wounding his daughter Fatima, 16, in the leg.

Mr. Amer said they were then allowed to go to his parents' ranch farther up the road in nearby Karzaz opening the way for pro-Gadhafi volunteers from Tawergha, who eventually reached his parents' farm. There, he said all were led out before the house, like his own, was looted and set on fire. "It was terrifying when the Tawergha men came into my parents' house," Mr. Amer said.

His father and six cousins and their families were detained during the same raid on Tuminah and Karzaz. They remain missing along with more than 1,000 other Misrata residents.

The Amers, like their wealthy neighbors the Issas, have been accused by the regime of bankrolling the rebels, which they admit to doing.

Standing on the roof of his family's burned out farmhouse, Tareq Issa recalls their escape after his uncle was killed and brother gravely wounded in a shootout with Gadhafi loyalists who attacked the farm. The Issas came back to Tuminah last month to find their properties in ruins.

The incinerated body of a Lexus sedan sat in the garage of one mansion while a smashed marble urn was all that remained of the contents of another Tuscan-style villa nestled amid acres of orchards.

Mr. Issa, a lawyer who now leads a clutch of fighters in charge of security in Tuminah, blames Tawerghans for the attack and said his whole clan has scores to settle with the town.

Write to Sam Dagher at sam.dagher@wsj.com