When record mogul Clive Davis came out as bisexual this week, he might have shocked those who didn’t work in the music industry, where his personal life was an open secret. But what was really surprising in this age of coming-out stories is that Davis came out as a bisexual—and not as a gay man.

Though the twice-married Davis has been in two monogamous relationships with men since 1990, he insisted, as he made the talk-show circuit in support of his new autobiography, The Soundtrack of My Life, that was he bi, not gay.

“For over 50 years, I had never had sex with a man. I wasn’t repressed; I had very good sexual relations with women. For me, this very maligned, misunderstood subject of bisexuality came up,” he told Meredith Vieira on Nightline. “I’m not lying—it exists.”

As same-sex marriage and other gay civil-rights matters become so commonplace they are almost passé, and as transgender issues have also increasingly taken center stage, bisexuality remains the most overlooked and misunderstood letter in the LGBT acronym. But despite the challenges that remain, bi advocates say they hope celebrities like Davis will help erase the stigmas that surround them.

Some bisexual advocates say they are just there—but choose to stay quiet due to the bi-phobia that runs rampant in the gay community. Many bisexuals choose not to admit that they are bi to other gays and lesbians.

“When people come out as gay, nobody says, ‘No you’re not, you must be bi or you must be something else,’” said Ellyn Ruthstrom, president and administrative director of the Bisexual Resource Center, the oldest national bi organization in the country. “Bisexuals have a really hard time. When we come out, so often somebody else is trying to say, ‘Oh no, you can’t be bi.’ Unfortunately it’s a common reaction when people come out as bi. And I think that’s what prevents some people from coming out as bi, because the reaction is often disbelief rather than supportiveness and acceptance. People should be allowed to define themselves the way they want to.”

It is ironic, then, that bisexuals have the least amount of visibility under the LGBT umbrella when they actually comprise a slight majority in that community, according to a study the Williams Institute conducted in 2011.

Tristan Taormino, a sex educator and the host of the Voice of America radio show Sex Out Loud, agreed. In the LGBT continuum, bisexuality “is the least visible,” she said.

Why? She laughed: “Well, that’s the subject of a dissertation.”

Davis joins the another recently out bisexual male in the music industry, Frank Ocean, who admitted that he had also loved men in a GQ interview and in a letter he posted on Tumblr talking about a lost love.

However, Ocean differs from Davis in that the singer didn’t proudly claim the banner “bisexual” as adamantly as Davis did.

Perhaps the singer didn’t avow himself of the label so fiercely because it is so fraught. For some, the word “bisexual” carries negative connotations, conjuring images of college girls making out to get male attention or just “experimenting.” Even Taormino prefers to call herself “queer.”

“I feel like there’s like a ton of people now who could be called bisexual—and I’m including myself in this category—but identify as queer. And I think that is generational, as well,” she said. “Queer feels much more like home, it feels much more descriptive to me than bisexual.”

In both the gay and straight communities, bisexuals get a bad rap, said bisexual porn star and feminist Nina Hartley.

“We’re seen as being unable/unwilling to ‘decide’ what they are,” she wrote in an email to The Daily Beast. “I’ve been labeled ‘wishy-washy,’ ‘afraid of commitment,’ ‘heart-breaker,’ ‘tease,’ ‘shallow,’ ‘hedonistic,’ ‘immature,’ to name just a few.”

In the gay community, said Hartley, “bisexuals who can pass for straight are thought to use that to access ‘heterosexual privilege,’ thereby abandoning” their gay sisters and brethren. And women who also sleep with men are seen by lesbians as “sleeping with the enemy.”

In the straight community, “it’s disquieting to think that a friend might be looking at you or your partner and be thinking sexual thoughts about them,” wrote Hartley. “If the bi person passes for straight (i.e., isn’t too flamboyantly queer), it’s more confusing.”

Taormino agreed: “We like things to be sort of neat and ordered. Bisexuality raises people’s ire, and anxiety, because it calls into question if in fact everyone could be bisexual and just hasn’t gone there.” She continued: “It also raises the issue of choice, and it also raises the issue of fluidity.”

While female celebrities like Anna Paquin, Ke$ha, and Evan Rachel Wood have come out as bisexual in recent years, the out-male bisexual, in particular, is a unicorn. There was even a recent study designed to prove, that yes, they really exist.

As Davis himself explained to Katie Couric: “There was a[n] ... attitude toward bisexuality that you’re either gay, straight, or lying. That’s not true.”

Many gay men, early on in their understanding of their own sexuality first come out as bisexual, before ultimately admitting that they are, in fact, gay. As a result, many gay men are incredulous when someone says they are bisexual.

“I still feel like the most elusive of all creatures, even in my world, is the bisexual man,” said Taormino. “I run with a very queer, kinky, sex-positive crowd. And I’ll tell you that the bi man, it’s hard to find them. It’s hard enough for them to find themselves.”

But as with the gay, lesbian, and transgender movement, misperceptions of bisexuality will change only as more people come out. To hear the bisexual advocates tell it, the world needs more people like Clive Davis to shout it from the rooftops.

“I think it’s really great that younger celebrities are now feeling more comfortable to be more openly bi and hopefully that really helps our community. And I hope that people will just allow Mr. Davis his self-identity and celebrate it with him,” said Ruthstrom. “He’s out and proud and that’s wonderful.”