MADISON, Wis. -- It was only 6 years ago that Rep. Paul Ryan's town halls were packed and overflow crowds were the norm.

But the now-House-speaker's events these days are more like the two employee town halls he held Thursday, where the questions were easy, the general public was barred and there was no followup from reporters.

Asked on Thursday when he would take questions from reporters, Ryan said, "I gotta go."

But on Friday, he held a press conference in Madison.

CBS News asked him when he might schedule an open town hall for his constituents.

"Let me respond. Aside from the obvious security concerns, what we find is that there are people who are trying to come in from out of the district to disrupt town hall meetings," Ryan said. "I don't want to have a situation where we have a screaming fest, a shouting fest where people are being bused in from out of the district to get on TV."

Republican rallies this year have been rowdy, with members of Congress struggling to talk over protesters. That may be why we found only a handful of Republican lawmakers holding open-to-the-public town halls this week.

In Wisconsin, Ann Jamison would love to talk about health care with Ryan. She's been trying to reach his office by phone since January.

She said she has been "not successful at all" in reaching him.

Even though she lives outside his district, she says Ryan is the Speaker of the House -- a national figure -- and not merely the congressman from Janesville.

Paul Ryan took questions from reporters on July 7, 2017. CBS News

"The voicemail ends with saying that the voice box is full and you are not able to leave a message," Jamison said.

The message she says she would leave is that Paul Ryan and all the members of Congress work for the people and are supposed to listen to them.