On Wednesday, June 18, American Apparel’s Chief Executive Officer Dov Charney was fired due to allegations of sexual harassment and misconduct. The American Apparel Inc. board, of which Charney was chairman, has been contemplating Charney’s removal for a while now, but only recently decided to take legal action. For now, Chief Financial Officer John Luttrell will serve as interim CEO.

Known for their apparel basics and provocative Terry Richardson-style advertisements, American Apparel has been struggling to renew their appeal. “What they have is a tired concept,” Ken Nisch, chairman of retail branding firm JGA, told USA Today. “I imagine (the board comes) to the conclusion, saying, ‘He’s not delivering a plan to move us forward, and we’ve got all these liabilities around us. So why are we putting up with all these liabilities when we’re not seeing a clear path to the next chapter?’”

According to Fortune, the “company has reported four consecutive annual losses. The loss last year swelled sharply to $106.3 million from $37.3 million in 2012.” In total, the U.S. wholesaler has accumulated more than $250 million in debt.

Charney founded the Los Angeles-based clothing brand in 1998. The 45-year-old pioneered his vertically-integrated company, keeping all aspects of production in-house. While the Quebec-born entrepreneur made great strides in establishing a sweatshop-free company, he’s been the center of controversy for his employment malpractice.

In a 2004 profile for Jane, Charney describes the corporate culture within American Apparel. “I’m not saying I want to screw all the girls at work—I’m not a f–kin’ madman. But if I fall in love at work, it’s going to be beautiful and sexual.”

While Charney only mentioned being in a serious relationship with three of his employees in 2004, he’s been slapped with multiple lawsuits. Three former employees sued Charney in 2005 for sexual misconduct, one in 2010 for coerced sexual acts during an interview, and five in 2011 for sexual harassment.

The most recent case involved former employee Irene Morales in 2011 who sued Charney for making her into a “sex slave.” However, the lawsuit was dropped due to conflicting evidence.

As of Monday, Charney is pursuing legal action against the board. In response to a letter sent by Charney’s lawyer Patricia Glaser, co-chairman Allan Mayer said, “We’re very confident that we are on very firm legal ground.” Mayer told The Times, “It’s what we would expect from Dov’s attorney in a situation like this, but we continue to believe firmly that we did the right thing, for the right reasons, in the right way.”

Charney is expected to leave his company by July 18.