RICHMOND, Va. -- A black teenager was called a racial slur before being attacked by two white men near his Virginia home as part of a campaign to run his interracial family out of the neighborhood, a federal lawsuit says.

The lawsuit comes more than a year after the men were convicted of assault and battery of now-18-year-old Mister Frazier. But they were cleared of hate crime charges after a judge said it appeared to be just a neighborhood brawl.

"That's like saying that the graffiti painting on LeBron James' house was just a neighborly dispute," said Frazier's attorney, Victor Glasberg, referring to the report that an unidentified person recently spray-painted the n-word on the front gate of James' home.

Get Breaking News Delivered to Your Inbox

Frazier says he was playing outside with children during an 8-year-old's birthday party at his Gloucester County home in 2015 when he heard a man in a neighboring yard of a vacant home calling the kids the n-word.

Frazier, who goes by the name Happy, went into the neighboring yard and confronted the man about the slurs, the lawsuit says. That man -- Stephen Cooke -- got in Frazier's face, called him the n-word and told him to jump before pushing his shoulder into the teen's chest, the lawsuit says.

Another man with Cooke -- Douglas Clark -- then hit Frazier in the face before putting him in a headlock, pulling him to the ground and repeatedly punching him in the head, the lawsuit says.

"The attack fractured feelings of safety and belonging that had allowed them, an interracial family, to move to Gloucester without fear of neighbor's race-based hostility or violence," the lawsuit says.

Cooke and Clark did not respond to several phone messages left at numbers listed for their homes. Their attorneys in the criminal case did not respond to phone messages left at their offices.

During their trial, attorneys for the men rejected the idea that the fight was racially motivated and argued Frazier was the aggressor.

Cooke denied using racial slurs, saying he was using the word "jigger," -- a kind of flea found in sub-Saharan Africa and central and South America. Cooke acknowledged calling the children "dirt frogs" and "monkeys," but said he yelled at them because they weren't supposed to be playing in the yard of the vacant home.

"The fact that the Mister Frazier may or may not be of a different race doesn't matter when the confrontation is just a garden variety confrontation not motivated by his race but initiated by his coming into the backyard and coming up to Mr. Cooke," Devin Henlsey, an attorney for Cooke, said during the trial.

Frazier and his sister lived with their black uncle and white aunt in Gloucester Point, which was nearly 90 percent white in 2015, the lawsuit says. Frazier's sister, who says she was pushed by one of the men during the fight, is also named in the lawsuit.

The lawsuit claims the assault was part of a "campaign of race-based hostility, harassment, and violence" aimed at forcing Frazier and his family from the home. There are no other African-Americans on the block, the lawsuit says.

Cooke's wife also spread rumors that the teen had vandalized homes on the block and slashed a neighbor's tires, Frazier's attorneys say. After the assault, she routinely blared her car horn while driving by the family's home, the lawsuit says.

The men were convicted of assault and battery and sentenced to a year in jail but were told they only had to serve a fraction of that as long as they remained on good behavior.