WASHINGTON (CN) – Congress has the right to hold a prayer at the start of each legislative day, and can reject an atheist’s request to give the invocation without violating the Establishment Clause, a federal judge ruled.

U.S. District Judge Rosemary Collyer, a George W. Bush appointee, said Wednesday the House chaplain’s refusal to invite an avowed atheist to deliver the morning prayer, “in the guise of a non-religious public exhortation,” also adheres to the Equal Protection Clause.

As recounted in the ruling, Dan Barker, co-president of the Wisconsin-based Freedom From Religion Foundation, sued House Chaplain Patrick Conroy and Speaker Paul Ryan in May 2016, after he was prevented from delivering an atheist invocation before Congress.

The dispute with Congress over the issue dates back to February 2015, when Rep. Mark Pocan, a Wisconsin Democrat, invited Barker to the deliver the invocation as a Congressional guest.

Conroy’s office told Barker that all guest chaplains must be “ordained by a recognized body in the faith in which he/she practices” and must present a copy of their ordination certificate as proof.

He also advised that the invocation must address a “higher power.”

Barker, a former minister, was ordained in 1975, but announced he was an atheist 10 years later. However he retained his ordination as a means to officiate at weddings.

Barker submitted his ordination certificate to Conroy’s office, and said in regard to addressing a higher power, he believes there is no higher power than “we, the people of these United States.”

The matter remained unresolved until January 2016, when Conroy informed Barker he was denying his request to give the invocation because he had publicly announced his atheism.

Barker then sued, alleging Conroy had violated his rights under the Religious Freedom Restoration Act, which prohibits government from interfering with a person’s exercise of religion unless it has a compelling reason for doing so.

But Judge Collyer concluded this argument was off the mark.

“Taking as true Mr. Barker’s allegations that atheism is his religion and assuming, but not finding, that RFRA applies to the House, the court finds Mr. Barker has failed adequately to allege a claim under RFRA because he fails to allege a substantial burden,” Collyer wrote.

She went on to explain that a substantial burden “exists when government action puts ‘substantial pressure on an adherent to modify his behavior and to violate his beliefs.’”

In this case, Collyer said, Barker failed to allege that serving as a guest chaplain was required by his religion or that Conroy or any other individual at the House forced him to engage in any conduct contrary to his religion.

She also rejected Barker’s argument that the U.S. Supreme Court’s 2014 ruling in Town of Greece v. Galloway supported his legal challenge.

In Town of Greece, the high court ruled the township could permit volunteer chaplains to open each legislative session with a prayer.

But Collyer said the ruling simply didn’t apply to Barker because nothing the justices said referenced atheists.

“To decide that Mr. Barker was discriminated against and should be permitted to address the House would be to disregard the Supreme Court precedent that permits legislative prayer,” the judge said.

Barker said he is disappointed in the ruling, saying it allowed the House chaplain’s ” personal biases against the nonreligious” to prevent him from participating in his government.

“The judge’s acquiescence in this inequity sends a crystal clear message that our government, founded upon our entirely secular Constitution, may discriminate with impunity against atheists and freethinkers,” Barker said in a statement.

Annie Laurie Gaylor, the other co-president of the Freedom From Religion Foundation, said the decision is a “wake-up call to U.S. freethinkers and nonbelievers.

“No atheist has ever been allowed to give an opening invocation in Congress, and a federal court has just blessed suppression of our views and rights by a tax-paid Roman Catholic chaplain,” Gaylor said.

“Not only are we at the bottom of the social totem pole when it comes to social acceptance, according to numerous studies and surveys, but our government can with impunity deny us as nonreligious citizens a voice in Congress that is a given to those of every religion,” she said.