A U.S. Olympic skier said Thursday he would not visit the White House if he were invited.

Gus Kenworthy told "CBS This Morning" he would not make the visit because he has “no patience” for the administration “directly attacking the LGBT community.”

Kenworthy also opposed the decision for Vice President Mike Pence to lead the U.S. delegation at the opening ceremony of the Winter Olympics in Pyeongchang because it sent mixed messages.

“When we have people elected into office that believe in conversion therapy and are trying to strip trans rights in the military and do these things that are directly attacking the LGBT community, I have no patience,” Kenworthy said.

The skier appears to be referencing rumors that Pence supports gay conversion therapy, which the vice president’s office has denied.

Kenworthy is one of the first two openly gay men to represent the U.S. at the Winter Olympics, the other being figure skater Adam Rippon.

Rippon also said he would be skipping out on the U.S. Olympic team’s visit to the White House in order to support his community.