(CNN) The brothers accused of attacking actor Jussie Smollett, only to be cleared when prosecutors announced Smollett had orchestrated a hoax to advance his career, are suing his legal team for defamation, attorneys said Tuesday.

The federal lawsuit alleges defamation and false light by attorney Tina Glandian for saying, among other things, that brothers Olabinjo and Abimbola Osundairo attacked the "Empire" actor and dealt steroids, and "inferring" that Abimbola engaged in sexual relations with the actor.

"In short, Mr. Smollett used his clout as a wealthy actor to influence Plaintiffs, who were in a subordinate relationship to him and were aspiring to 'make it' in Hollywood," said the lawsuit filed in US District Court in Chicago. "He wanted his employer and the public to notice and appreciate him as a successful Black, openly gay actor. So, Mr. Smollett directed every aspect of the attack, including the location and the noose."

Abimbola, left, and Olabinjo Osundairo are suing Jussie Smollett's lawyers.

The lawsuit also names Mark Geragos, who heads the firm that employs Glandian.

In a joint statement, Geragos and Glandian called the lawsuit ridiculous and predicted it would be dismissed because it "lacks any legal footing."

Read More