A Republican congressman from Texas told a constituent at a town hall to “shut up” after being pressed on his lack of support for legislation to help combat violence against women.

During a town hall last weekend, Rep. Joe Barton (R-Texas) was asked why he had not supported measures to protect women from violence. Barton tried to explain that he saw the issue as one that should be left to the states.

“On the first bill that I voted against, that’s a true statement, and I voted against it because I think that’s a state issue, not a federal issue,” he said, according to the Dallas Morning News. When several people started shouting, Barton, who was first elected to Congress in 1984, pointed to a man in the audience and said, “You, sir, shut up.”

The comment sparked more outrage from the audience.

“What is this? You don’t tell anyone to shut up. You work for us,” a man says.

In statement to The Huffington Post, Barton defended his actions.

“All town halls begin with ground rules, which include that you must be recognized in order to speak,” Barton said. “These are unscripted live meetings. Over the weekend in Frost, one gentlemen continued to speak over myself and many others who were seeking recognition in orderly fashion. I did, however, return to him for the last question of the meeting and allow him the opportunity to voice his concerns.”

Barton’s office also pointed to a Facebook comment written by a man named Kirk Lee, who wrote that he attended the town hall, and defended Barton even though he did not vote for him. Lee wrote that Barton had asked for no cursing, yelling or violence.

“The ‘shut up’ came when a gentleman asked a question, crowd didn’t like the answer, and he was trying to shout over a chorus of boos. I believe Rep. Barton was embarrassed by it and it was not representative of his attitude as a whole,” Lee wrote. “I know if I had been in his shoes I would have said much worse.”

The heated exchange comes as lawmakers around the country faced angry constituents last month over Republican attempts to repeal the Affordable Care Act. Despite the heated town halls, Republicans have pressed ahead with their health care plan, which would cause 24 million people to lose health insurance.