Johns Hopkins students are campaigning to have Dr. Ben Carson pulled from this year's commencement speaker line-up, in response to comments the well-known neurosurgeon, a critic of the Obama administration, made about gay marriage.

Carson has since said he apologizes for having offended anyone and indicated he might withdraw from the commencement role, though he says his words are being misconstrued.

Carson -- who rocketed to political fame after criticizing President Obama's policies during the National Prayer Breakfast, which Obama attended -- made the gay marriage comments on Fox News Tuesday night.

Host Sean Hannity asked Carson his opinion on same-sex marriage, given the Supreme Court's consideration of two gay marriage cases this week.

"Marriage is between a man and a woman," Carson said. "It's a well-established, fundamental pillar of society and no group, be they gays, be they NAMBLA, be they people who believe in bestiality -- it doesn't matter what they are, they don't get to change the definition."

Students accused Carson of effectively comparing "gay relationships with pedophilia and bestiality."

The Health and Human Rights Student Group began circulating a petition, linked out of its Facebook page, challenging Carson's selection as the 2013 commencement speaker for the Johns Hopkins School of Medicine.

"At the time of his nomination, Dr. Carson was known to most of us as a world-class neurosurgeon and passionate advocate for education. Many of us had read his books and looked up to him as a role model in our careers," the group said. "Since then, however, several public events have cast serious doubt on the appropriateness of having Dr. Carson speak at our graduation."

The group cited his gay marriage comments, but also his comments at the National Prayer Breakfast -- Carson was hailed by conservatives, and even landed a speaking role at the Conservative Political Action Conference, for his remarks in February.

The commencement address may be in doubt.

Asked about the students' objections, Carson on Friday told MSNBC it's "their day and the last thing I would want to do is rain on their parade."

Asked whether he's told the university he would not deliver the address, Carson said he was "waiting for appropriate channels."

Carson also told the Baltimore Sun that people "have completely taken the wrong meaning out of what I was saying."

He added: "Now perhaps the examples were not the best choice of words, and I certainly apologize if I offended anyone. But the point that I was making was that no group of individuals, whoever they are, whatever their belief systems, gets to change traditional definitions."