Barry Manilow, the 73-year-old crooner who has made hearts swoon for decades decided not to share his sexuality with his fans earlier because he was afraid that he would disappoint his largely straight fan base. Now, Manilow opens up about his secret marriage to his manager, Garry Kief, and life as a married man. Speaking to PEOPLE, he discussed his 40-year romance with Kief and justified keeping his relationship with him secret by saying: “I’m so private. I always have been.”

Manilow met Kief in 1978. “I knew that this was it,” says Manilow. “I was one of the lucky ones. I was pretty lonely before that.”

“He’s the smartest person I’ve ever met in my life — and a great guy, too,” he says.

Even after persistent rumors among fans and leaked news that he married his partner, he was still hesitant to come out. “I thought I would be disappointing them if they knew I was gay. So I never did anything,” says Manilow. Turns out, “When they found out that Garry and I were together, they were so happy. The reaction was so beautiful — strangers commenting, ‘Great for you!’ I’m just so grateful for it.”