Jay Parini, a poet and novelist, teaches at Middlebury College in Vermont. His most recent book is "The Way of Jesus: Living a Spiritual and Ethical Life." The opinions expressed in this commentary are solely those of the author.

(CNN) Even some Republican members of Congress, such as Rep. Peter King of New York, are upset about the firing of the Rev. Patrick Conroy, a Jesuit, as House chaplain. "To be the first House chaplain to be removed in the history of Congress, in the middle of a term, raises serious questions," King told reporters. "I think we deserve more of an explanation of why. Was there political pressure?"

Speaker of the House Paul Ryan took it upon himself to fire Conroy. Ryan's spokeswoman AshLee Strong denied Thursday that Conroy was pushed out for anything he said or did. What might he possibly have done, in any case? Well, there was that time in November during the tax debate when Conroy seemed to be criticizing the Republican plan from the House floor in a prayer that he offered.

"May all members be mindful that the institutions and structures of our great nation guarantee the opportunities that have allowed some to achieve great success, while others continue to struggle," Conroy prayed before the vote was taken. "May their efforts these days guarantee that there are not winners and losers under new tax laws, but benefits balanced and shared by all Americans."

A week or so after delivering these sensible words, Conroy told a reporter from The New York Times he got a message from the speaker's office. "A staffer came down and said, 'We are upset with this prayer; you are getting too political,'" he said. "It suggests to me that there are members who have talked to him about being upset with that prayer."

JUST WATCHED Is there more to the story about fired House chaplain? Replay More Videos ... MUST WATCH Is there more to the story about fired House chaplain? 07:56

Conroy told the Times that when next he saw Ryan, he was told bluntly: "Padre, you just got to stay out of politics."

Read More