The House speaker, Paul Ryan, has told fellow Republicans he fired the House chaplain after complaints from members that he was not doing a very good job – and not because of pressure over the Rev Patrick Conroy’s political leanings.

Democrats and Conroy himself have cited a prayer he offered that called for fairness as the House debated tax cuts as a reason for Republican discontent. Last November, Conroy prayed for lawmakers to make sure that “there are not winners and losers under new tax laws, but benefits balanced and shared by all Americans”.

Republicans leaving a party meeting on Friday said Ryan told them Conroy was forced out after lawmakers complained that had not been adequately tending to the pastoral needs of lawmakers.

“He had a number of complaints that the chaplain was not meeting the pastoral needs of the members in general,” said Mike Coffman, a representative from Colorado.

Paul Ryan: House speaker won't seek re-election Read more

Ryan’s office would not say why Conroy was asked to resign in what would be the first such forced exit in the history of the House.

While Ryan has the support of most of the chamber’s Republicans, many Democrats are upset and say politics may be behind Conroy’s fate. Conroy told the New York Times that Ryan warned him to “stay out of politics”.

“He was essentially dismissed for praying and for this very gray and hazy representation that he was not ministering to some of the members,” said Emanuel Cleaver, a Missouri Democrat who is himself a clergyman. “Fairness and justice is an inseparable part of the Judeo-Christian theology.”

One prominent Republican was upset too.

“I’m not aware of any discontent or any criticism and to be the first House chaplain to be removed in the history of the Congress in the middle of a term raises serious questions,” said Peter King, a New York representative.

“We deserve more of an explanation and why. Was there political pressure? The speaker said it was just because certain people felt he was not complying with their requests or was not giving them good counseling. I never heard that from anyone.”

Conroy has been the chamber’s chaplain since 2011 but offered his resignation last week at Ryan’s urging. He will leave his House service next month. Democrats sought to create a special committee to investigate “the motivations and actions” behind Ryan’s decision but the move was killed by a party-line vote.

Conroy’s resignation letter said he was offering to step down at Ryan’s request, calling his seven years of House service “one of the great privileges of my life”.

The chaplain is responsible for opening the House each day with a prayer and offering counseling to lawmakers and aides on the House side of the Capitol. Conroy is a Roman Catholic priest from the Jesuit order.