Housing and Urban Development Secretary Ben Carson allegedly made transphobic comments at a meeting with HUD staffers earlier this week.

According to The Washington Post, during a visit to HUD’s San Francisco office Tuesday, Carson expressed concern about transgender women, whom he reportedly called “big hairy men,” entering women’s homeless shelters.

Carson also expressed concern that society could no longer differentiate between men and women, according to the Post, which cited three unnamed sources who reportedly heard Carson’s remarks firsthand and interpreted them as “an attack on transgender women.”

HUD did not immediately respond to NBC News’ request for comment, but a senior HUD official released a statement to The Post saying, “The Secretary does not use derogatory language to refer to transgendered individuals. Any reporting to the contrary is false."

Carson’s track record of anti-LGBTQ remarks and opposition to gay and trans rights began long before he joined the Trump administration in 2017. As a guest on Fox News in 2013, he compared gay people to pedophiles, likening them to members of the North American Man/Boy Love Association (NAMBLA), a group that works to abolish age-of-consent laws criminalizing adult sexual involvement with minors. In 2015, the former neurosurgeon claimed that being gay was a choice — a belief that has been disproven by science and that he has since apologized for promulgating. Later that year, Carson expressed opposition to transgender personnel serving in the military, stating that trans people should “deal with the transgender thing somewhere else.”

While running for president in 2016, Carson doubled down on his anti-transgender views, saying trans people don’t “make any sense” and comparing them to those who may wake up one day and decide to change their ethnicity.

Shortly after Carson took the helm at HUD, the department removed links that provided emergency shelters with guidance on how to best serve LGBTQ service recipients. The guidance, provided in 2009 by the Obama administration, was intended to help housing providers and shelters interpret the agency’s rules to avoid discrimination against such residents.

In May of this year, HUD proposed a rule change that would allow men’s and women’s shelters to segregate transgender people “consistent with state and local law” — an amendment many LGBTQ advocates argued would further marginalize trans people who are especially vulnerable to homelessness. As many as 1 in 5 transgender people have experienced homelessness at some point in their lives, a problem compounded by discrimination in housing and employment and family rejection, according to the National Center for Transgender Equality.

Politicians and advocates are speaking out against Carson’s alleged comments, with Zeke Stokes, vice president of programs at LGBTQ media advocacy organization GLAAD, writing in an email that Carson is “unfit to serve.”

Presidential candidate Julián Castro, who led HUD during the Obama administration, also condemned Carson’s alleged remarks, stating they “normalize violence against” black transgender women — at least 19 of whom have been murdered this year, according to the Human Rights Campaign.

“As HUD Secretary, I protected trans people,” Castro wrote on Twitter. “I didn’t denigrate them.”

Without a home, transgender individuals face increased risk of physical and sexual violence and being forced into sex work, according to the National Center for Transgender Equality.

“This man is IN CHARGE of housing. He's out there using language that makes trans folks seem disposable,” Jay Brown of the Human Rights Campaign said in a tweet. “This is quite literally killing us. We deserve better. We must DEMAND better.”

Follow NBC Out on Twitter, Facebook & Instagram