The pair never reconciled after Rivers left Tonight for her own show

Joan Rivers, who died on Thursday, was one of late-night television’s true pioneers. And inextricably tied to her legacy of trailblazing is her relationship with The Tonight Show, and the long shadow that Johnny Carson cast over her career.

In the mid-1950s, at age 23, Rivers ran away from home to become an actress. By 1958, she had decided on comedy. But she auditioned for The Tonight Show seven times – as a favor to her manager’s other client, Bill Cosby – before she was put on Carson’s show in 1965 as a gag writer.

Get push notifications with news, features and more.

“It was a mercy booking,” she told PEOPLE in 1978.

That was the beginning of a crucial and often complicated relationship.

In Carson, Rivers found a supporter, if not a mentor.

“He believed in me more than I believed in me,” Rivers told PEOPLE in 1991, building on comments she made in a 1986 op-ed for the magazine: “Johnny was the one person who said, ‘Yes, she has talent; yes, she is funny.’ He was the first person in power who respected what I was doing and realized what I could become. He handed me my career.”

But as she made clear in the op-ed, their relationship didn’t extend beyond the camera. “Our friendship existed entirely on-camera in front of America, and even then, during the commercial breaks, when the red light went off, we had nothing to say to each other,” she wrote.

Still, she filled in for Carson frequently, notching more than 80 stints as substitute by 1983, when she was named the permanent guest host of The Tonight Show.

“I always thought [Carson] was the one person in the business who understood me and really wished me well, who knew my real feelings,” Rivers said of Carson. “In return, I gave him unwavering loyalty I never wanted to do anything to hurt that man.”

But that loyalty was frequently tested. Rivers refused offers to appear on The Merv Griffin Show on CBS and turned down offers from multiple parties for her own talk show – all, by her account, for more money than she was making at NBC.

By March 1986, when Fox approached Rivers with an offer, she was worried. Since she’d taken over the guest-host position, her yearly contracts had always been renewed for the same length as Carson’s – but in 1985, he signed on for two years, while NBC only offered her a one-year contract. “It could only mean one thing,” she wrote. “The powers were uncertain about my future.”

Rivers further believed she’d never be on equal footing with Carson because of her gender. “NBC would never give [the host spot] to a woman,” she claimed. The guest-host spot, she said, was “very smart. It was no competition for him.” She described holding “in my hand an NBC interoffice memo listing suggested successors to Johnny in case he did not renew his contract. My name was not on it.”

Rivers maintained that offers of negotiations with NBC fell through for months, and “with their attitude so casual and no negotiations under way,” she accepted the offer from Fox.

Rivers said she planned to tell Carson of her decision, but news was leaked to him a day early. She called him at home to apologize and explain, but he hung up on her. They would never speak again.

The Late Show Starring Joan Rivers premiered on Oct. 9, 1986. In a PEOPLE profile on Oct. 27 of that year, Rivers recalled standing in front of her own audience and thinking, “For the first time in 53 years, I know, by God, that I truly belong. Nobody can stop me.”

But Carson’s anger over the way things had ended between them loomed large. “An NBC executive, a personal friend, told us, ‘We are going to destroy you,’ ” Rivers said. And PEOPLE noted that the show’s talent booker, Bill Sammeth, lived with “the unstated threat that a Rivers show appearance could mean a Carson blackball.”

Rivers’s tenure at The Late Show lasted less than a year. She was ousted as host in May 1987 (Arsenio Hall became her eventual replacement) after feuding with network executives and failing to find her voice as a host.

“We disagreed on the direction the show would take from the very beginning,” then-Fox Broadcasting president Jamie Kellner said at the time.

“People don’t want to see a nice Joan Rivers,” NBC programming chief Fred Silverman added bluntly of Rivers’s defanged approach to her guests.

On Aug. 14, 1987, despondent over the failure of The Late Show, Rivers’s husband Edgar committed suicide. Carson did not contact her. “It was like Stalin had sent me to Siberia,” River later said.

When Jay Leno inherited the Tonight Show desk from Carson in 1992, he upheld Carson’s blacklist for the duration of his time as host, which he chalked up to “respect for Johnny.” When the desk was passed again in 2014 – excluding the Conan O’Brien/Leno kerfuffle – new host Jimmy Fallon brought Rivers on during his first night as host for a brief cameo 49 years to the day after she made her first appearance on the show. She appeared as a regular guest within a few weeks.

Rivers brushed off speculation in 1987 that she had “peaked” with The Late Show (“I’m always peaking,” she told PEOPLE at the time), and she later hosted a series of her own shows, some with daughter Melissa. Still, her jokes about returning to the The Tonight Show betrayed a hint of the self-doubting comic whom Carson had first encouraged in 1965.

“I am crazed with nervousness,” she once said. “If you’re not a wreck in this business, you’re not around.”

On the end of her blacklisting and her return to The Tonight Show, Rivers told Variety, “It’s about time! I’ve been sitting in a taxi outside NBC with the meter running since 1987.”

Want more stories like this?

Sign up for our newsletter and other special offers:

Thank you for signing up!

Make ‘Em Laugh: Joan Rivers’s Best Lines