Fifty years later, Francois Clemmons — author of the new memoir Officer Clemmons — tells PEOPLE exclusively, “I forgive him. More than that, I understand"

From 1968 to 1993, Francois Clemmons was a member of the Neighborhood. He was, as he’s known to generations of Americans, Officer Clemmons on Mister Rogers’ Neighborhood.

And over the years, Rogers — known to Clemmons as either Fred or “Doc” because of all the honorary degrees — became Clemmons’ mentor, father figure, and ultimately “the spiritual love of my life,” he tells PEOPLE.

Get push notifications with news, features and more.

But the relationship wasn’t always easy for Clemmons. By the time the two met through Rogers’ wife Joanne at church, Clemmons had survived a tumultuous and at times violent childhood. He quickly became both an integral part of the Neighborhood and Rogers’ own life.

As he writes in his new memoir, Officer Clemmons, one day he was called into Rogers’ office at the studio.

For more from Francois Clemmons, pick up this week’s issue of PEOPLE, on newsstands Friday

Image zoom The Fred Rogers Company

“Franc, you have talents and gifts that set you apart and above the crowd,” Rogers told him, Clemmons writes in his memoir. “Someone has informed us that you were seen at the local gay bar downtown. Now, I want you to know, Franc, that if you’re gay, it doesn’t matter to me at all. Whatever you say and do is fine with me, but if you’re going to be on the show as an important member of the Neighborhood, you can’t be out as gay.”

Clemmons tells PEOPLE that he began to sob. “I could have his friendship and fatherly love and relationship forever,” Clemmons remembers today. “But I could have the job only if I stayed in the closet.”

He had given Clemmons inspiration and stardom but told him there was a cost. “ ‘You must do this Francois,’ he told me, ‘because it threatens my dream.’ ”

Image zoom Officer Clemmons

“I was destroyed,” Clemmons says. “The man who was killing me had also saved me. He was my executioner and deliverer. But, at the same time, I knew that he would know how to comfort me. I didn’t have another mother or father to comfort me. I had no one to go and be a boy with. I was just vulnerable. He got in a few slaps, some tough love, a good spanking. But I was not kicked out of the family.”

“ ‘The world doesn’t really want to know who you’re sleeping with — especially if it’s a man,’ ” Clemmons says Rogers told him. “ ‘You can have it all if you can keep that part out of the limelight.’ ”

But then, Rogers had a suggestion. “Have you ever thought of getting married?” Rogers asked Clemmons. “People do make some compromises in life.”

Image zoom Francois Clemmons Vincent Jones

“By the time I left his office,” Clemmons writes in his memoir, “I had made up my mind to marry La-Tanya Mae Sheridan. At the wedding reception, Fred and Joanne approached me and my new wife. It felt as if Fred and I were sealing some kind of secret bargain.”

He and Sheridan would amicably divorce in 1974 and he would begin living life as an openly gay man.

“Lord have mercy, yes, I forgive him,” Clemmons says today about Rogers. “More than that, I understand. I relied on the fact that this was his dream. He had worked so hard for it. I knew Mister Rogers’ Neighborhood was his whole life.”