Despite its best efforts to rehabilitate its brand, Papa John's is still paying for offensive remarks made by its founder.

John Schnatter admitted last month to using the N-word on a conference call. He apologized and resigned as Papa John's chairman soon after Forbes published a story about the incident.

On Tuesday, the company said its same-store sales in North America fell 10.5% in July and 6.1% in the most recent quarter.

"Results have been challenged by recent events," Steve Ritchie, president and CEO of Papa John's (PZZA), said in a statement. "Our entire leadership team understands the importance of getting our culture and business improvements right."

Shares fell more than 10% after hours.

On a call with analysts, Ritchie said that Papa John's stakeholders agreed with the company to take Schnatter out of the public eye.

Related: Papa John's to pizza-eaters: We're listening

"I have spent a significant amount of time in recent weeks in the field, talking with our franchisees, our customers, our drivers and employees," he said. "Franchisees and partners have expressed overwhelming support for ... our decision to remove John as brand spokesperson."

"We are not dependent, nor should we be, on one person," he said.

Days after the racial slur scandal broke, Papa John's announced that it would remove Schnatter's image from its marketing materials, where he featured prominently. A special committee of the board of directors will oversee an outside auditor's investigation into Papa John's culture, including the company's existing processes, policies and systems related to diversity and inclusion.

Ritchie said that Papa John's made the decision to distance itself from Schnatter after he said in November that the NFL protests hurt sales.

"The research and analysis we conducted after comments by our founder in November of 2017 have made it clear that we needed to move away from a founder-centric marketing plan," Ritchie said. "Obviously, the recent events have further evidenced that we need to move on."

Ritchie noted that comparable sales have slumped since November. Schnatter left the chief executive seat at his namesake company in December last year.

Related: Papa John sues Papa John's

But Schnatter, who said he regrets resigning from his post as chairman, isn't walking away without a fight. He remains on the board, and owns 29% of the company's stock. He has filed a lawsuit against the company, requesting to see company documents "because of the unexplained and heavy-handed way in which the company has treated him since the publication of a story that falsely accused him of using a racial slur."

Papa John's denied Schnatter's claims in a statement. The company said it was "saddened and disappointed" by the lawsuit, which it called "needless and wasteful."

Schnatter expressed concern over the health of the company in a statement published on Tuesday.

"History shows that the Company performs better with me involved, and it declines when I step away," he said.

— CNN's Jordan Valinsky and Jackie Wattles contributed to this report.