Washington (CNN) Supreme Court Justice Neil Gorsuch headlined his first Washington speech on Thursday, extolling the virtues of cherishing the "din of democracy" as progressive groups gathered outside to protest the fact that the event was held at the Trump International Hotel.

U.S. Supreme Court Justice Neil Gorsuch delivers remarks at the Fund for American Studies luncheon in Washington, D.C., on Thursday, Sept. 28, 2017.

Gorsuch made no mention of the protestors at the luncheon which was sponsored by the Fund for American Studies, a conservative group dedicated to teaching students and sponsoring scholarships.

Speaking broadly, Gorsuch praised the work of the organization to teach young people civics and said that civility is an important part of such a conversation.

"To be worthy of our First Amendment freedoms we have to all adopt certain civil habits that enable others to enjoy them as well," he said.

About three dozen protesters gathered outside the hotel, many of them lamenting the optics of Gorsuch speaking at an event in a building named for the man who placed him on the bench.

Josh Orton, speaking on behalf of NARAL Pro-Choice America, said that Gorsuch's choice to appear at the hotel suggested that people might become skeptical of Supreme Court decisions if they are perceived to be impacted by politics.

"There's no question that Gorsuch knows the signal he's sending by coming here and he's not only OK with it, but he's probably happy about it," Orton said.

An organizer of the event said Gorsuch had nothing to do with the choice of the hotel.

"We chose the venue, we had no political agenda it's just a nice hotel and a new venue for us," said Steve Slattery, a spokesman for the Fund for American Studies.

As for Gorsuch, "we were looking for a speaker, Slattery said. "So we thought, 'let's invite Justice (Antonin) Scalia's replacement.'"

Gorsuch, whom Trump nominated in January and was confirmed in April, was not paid for his appearance. Television news cameras were not allowed to film the event.

Some liberal groups seized upon the venue at Trump's hotel, a property that they say is part of how the President is violating the Constitution.

Located blocks from the White House, the Trump International Hotel is the subject of multiple lawsuits . Despite a pledge to isolate himself from his business, Trump held on to his assets and placed them in a trust in his name. That arrangement means he could benefit from the success of the business, even if he doesn't reap the rewards until after he leaves office.

Elizabeth Wydra of the Constitutional Accountability Center represents more than 200 members of Congress in a lawsuit alleging in part that Trump is violating a provision of the Constitution -- known as the Foreign Emoluments Clause -- which prevents the President from receiving payments from foreign governments without the consent of Congress.

"The Trump International Hotel is ... one of the primary ways in which foreign governments are seeking to curry favor with the President, by holding events there," Wydra told CNN. "Given that these cases could very well make their way to the Supreme Court, Justice Gorsuch agreeing to speak there raises questions about his impartiality."

Nan Aron, the president of the progressive Alliance for Justice, a group that opposed Gorsuch's nomination, also criticized his judgment.

"Justice Gorsuch should have known better than to sign up as the headliner for an event that will line Donald Trump's pockets in a way that is at best ethically sketchy" Aron said in a statement.

But legal ethics expert Steven Lubet at the Northwestern Pritzker School of Law said in a recent interview that he believes the speaking event does not raise ethical questions for Gorsuch, who was appointed to the bench by Trump in 2017.

"I think the relationship between the Supreme Court cases and the hotel is too attenuated to create a problem for Justice Gorsuch -- he's not showing any favoritism from the President or benefiting from the relationship in any meaningful way," Lubet said. "The justices have no written code of conduct, but this would not violate the code of conduct for the lower courts either."

Justices have often appeared before ideologically aligned groups. Conservative justices like Samuel Alito and the late Scalia have spoken before the conservative Federalist Society, and liberals like Ruth Bader Ginsburg and Stephen Breyer have appeared before the more progressive American Constitution Society.

"This controversy is absurd, the idea that appearing at an event at this hotel would impact the emoluments lawsuit is already a stretch and the proposition that he is endorsing the hotel or the president on a broader level is even more ridiculous," said Carrie Severino of the conservative Judicial Crisis Network.

Another group, the Citizens for Responsibility and Ethics in Washington, has also filed an emoluments suit targeting the DC hotel. A spokesperson for the group declined to comment on the Gorsuch event, citing the pending litigation.

In court papers in that case, the Justice Department responded that plaintiffs' claims should be dismissed because the Emoluments Clause is meant to "prevent official corruption and foreign influence."

DOJ lawyers argue that the Foreign Emoluments clause is not meant to "reach benefits arising from a President's private business pursuits having nothing to do with his officer or personal service to a foreign policy."

JUST WATCHED How the Supreme Court is like high school Replay More Videos ... MUST WATCH How the Supreme Court is like high school 02:49

The elk in the room

Gorsuch's speech -- one of the few since taking the bench -- was peppered with humor.

He spoke of his warm return to the court so many years after his clerkship for Justice Byron White and Justice Anthony Kennedy that began in 1993.

He noted that Ginsburg was the junior-most justice when he was clerking.

"Think cafeteria committee," Gorsuch deadpanned -- referring to the fact that the junior-most justice experiences a gentle hazing that includes the assignment overseeing the court's oft-criticized cafeteria food. Gorsuch has inherited the less-than-prized assignment.

He also said that having the chance to work with Kennedy again is an "unexpected joy."

And then he turned to Scalia -- whose seat he filled -- and lamented the fact that Scalia's "booming laugh" would no longer be heard in the hallways.

But Gorsuch revealed he'd been left with something he called "a unique reminder of the man."

Apparently, some years ago, Scalia bagged an enormous elk that he hung in his office and proudly named Leroy.

Gorsuch told the audience that the elk -- because of its sheer size -- was to become homeless after Scalia's passing because it would fit on no one's living room wall.

But then, Gorsuch said, "someone got the idea that Leroy might make a sort of unusual welcome-to-the-neighborhood gift to the new guy."

"What a gift," Gorsuch said, his voice dripping with sarcasm.

"The truth is," he allowed, "I am delighted to share space with Leroy... we have a few things in common."

"We are both native Coloradans, we both received a rather shocking summons to Washington," he said. "And neither of us is ever going to forget Justice Scalia."

"It seems we've been crated... (and sent to Washington) to serve out our time together on display at the Supreme Court."

This story has been updated.