A Papa John's pizza box is seen on July 11, 2018 in Miami, Florida.

Jefferies upgraded Papa John's shares to buy from hold on Wednesday, saying the "fear that the brand is permanently impaired" has created an opportunity for investors.

The third-largest pizza chain has been embroiled in a controversy over a racist comment, claims of extortion and litigation threats — all of which are hammering pizza sales at Papa John's stores across the country. The stock has cratered over the last three months, falling over 33 percent through Tuesday's close of $41.96 a share.

Jefferies analysts said they expect Papa John's financials to worsen before improving, but they added that this expectation is already reflected in the stock.

Shares of Papa John's rose 0.7 percent in Wednesday's trading session, closing at $42.27 per share.

Some Wall Street analysts downgraded Papa John's in July as the negative publicity mounted. But Jefferies says the "ultimate upside potential outweighs" any further downside facing the stock. The firm's analysts predict that Papa John's will lower its full year 2018 earnings forecast significantly, to $1.98 a share from a range of $2.40 to $2.60.

"This is is already baked into the stock given the extent of media headlines, despite Street estimates only down modestly since latest event," Jefferies said.

"Ultimately, we recognize that the potential impact from negative sentiment on the brand is real and will make the next few quarters painful, but we believe the brand will survive," the analysts said.

Jefferies also raised its price target on Papa John's to $58 a share from $52 a share.

– CNBC's Sarah Whitten contributed to this report.