Dozens of members of Taylor University’s graduating class protested Vice President Mike Pence’s appearance as the school’s commencement speaker on Saturday, walking out of the ceremony minutes before he spoke.

Most of the nearly 500 graduates at the nondenominational Christian liberal arts school in Pence’s home state of Indiana remained seated for his remarks.

Pence, the former governor of Indiana, received a standing ovation during his introduction as some students walked down the aisle and out of the auditorium in the Kesler Student Activities Center at the university in Upland.

The protest was planned and discussed prior to Saturday’s ceremony, reported The Associated Press, and stemmed from Pence’s conservative religious views and role in the Trump administration.

Laura Rathburn, one of the protesters, told The Indianapolis Star she was disappointed her school chose Pence to speak.

“I think his presence makes it difficult for everyone at Taylor to feel welcomed,” she said, donning a graduation cap she decorated with rainbow colors and a message that said, “Ally Visible For Those Who Can’t Be.”

Pence, an evangelical Christian, has come under fire over his opposition to same-sex marriage and transgender rights and support for conversion therapy for LGBTQ people. In January, he defended his wife Karen’s decision to teach at a private Christian school in Virginia that bans LGBTQ students and employees.

Christine Newman-Aumiller and her sister, Marilyn Dodd, protested Pence’s appearance at their alma mater outside the commencement. Dodd held a sign saying, “VP Pence does not stand for Christian values.”

Pence “does not have a servant’s heart if he cannot defy someone who criticizes and ridicules the little ones that Jesus himself said, ‘Let come unto me,’” Newman-Aumiller told radio station WVPE, an apparent reference to President Donald Trump.

In his address, Pence warned that it’s becoming “acceptable, even fashionable, to malign traditional Christian beliefs,” and he called on his audience to stand up for their religious values. He also lauded the “growing American economy” and praised Trump for standing “so strong on national defense.”