The board of directors also said they decided to remove him from Papa John's advertising materials and revoke his office space at the company's headquarters. Schnatter was removed as chairman of the board last week after Forbes reported his comments on the call.

Schnatter told Kentucky CBS affiliate WLKY that an executive at Laundry Service threatened Papa John's that "if I don't get my F-ing money, I'm going to bury the founder." The Louisville-based pizza chain issued a press release at 11:46 p.m. Sunday, saying Schnatter was prohibited from talking to the press or making "any further statements to the media regarding the company, its business or employees."

Schnatter, in an interview Friday with a Louisville, Kentucky, CBS station, accused media agency Laundry Service of trying to blackmail Papa John's for $6 million to keep quiet about a May conference call in which the founder admitted he used the N-word. He also told the TV station that he was provoked into using that language and accused officials who removed his name from a gym of "cracking."

It's now clear why Papa John's board of directors asked former Chairman John Schnatter to stop talking to the press.

"They tried to extort us and we held firm and they took what I said and ran to Forbes and Forbes printed it and it went viral," Schnatter said.

The conference call in May came to light after Forbes magazine detailed the incident in an article Wednesday. The report, which was later confirmed by Schnatter, said he was on a call with Laundry Service when he tried to downplay comments he made about the National Football League last fall by saying, “Colonel Sanders called blacks n-----s" and never faced any public backlash at KFC.

While he again apologized for using the racially charged slur, Schnatter told the TV station he was baited into using that language.

"I was just repeating what somebody else said. I was actually kind of provoked," he said of what was supposed to be a confidential media training session. "They were promoting that kind of vocabulary, and they kept hitting it and I was like no, we're not going to do that ... that's not what we're about."

Schnatter said the other party on the call, which he didn't identify, repeatedly used the word. "By the fourth or fifth pass, I just said, 'No, we're not gonna be part of any such thing. So-and-so used the N-word, and we don't use the N-word, and we're not gonna use the N-word. And that's it,'" he said.

Neither Laundry Service nor Papa John's immediately responded to CNBC's request for comment.

"It's ironic. The very thing we were trying to avoid was the very thing that happened," he told WLKY.

Schnatter stepped down last week as chairman of Papa John's as well as from the board of trustees of the University of Louisville. The company's name was also removed from the University of Louisville's football stadium and his name was stripped from a signpost of a gymnasium in his hometown of Jeffersonville, Indiana, according to the Louisville Courier-Journal.

He also said Jeffersonville Mayor Mike Moore, who removed Schnatter's name from the gym, "overreacted."

"Everybody’s upset and they’re cracking. And they're just, they’re scared," he said. "You know Forbes is going to lie and you know the Courier’s going to lie. But you would think there would be something solid in these leaders to really acknowledge and to embrace what really went down.

In addition, Major League Baseball indefinitely suspended its Papa Slam promotion — a campaign that both sides have collaborated on since 2016 and a number of sports teams have distanced themselves from the brand.

Shares of Papa John's were down 3.9 percent in late trading Monday.

Watch the full interview from the CBS-affiliated television station in Louisville, Kentucky.

Correction: Mike Moore is mayor of Jeffersonville, Indiana. An earlier version misstated the city.