West Chester, Pennsylvania (CNN) Thirty minutes before he'd face riled-up constituents in a Saturday afternoon town hall here, Rep. Ryan Costello looked out the window of his second-floor office.

"Give us his tax returns," he chuckled, reading a passerby protester's sign.

"A roller-blader now is telling me about it," Costello muttered five minutes later as another man skated by.

Costello is so attuned to those protesting him that in an interview, he laid out the schedules of each of the groups that protest outside his office in West Chester's town center daily and weekly. "On good days there's 50; on bad days there's 15," he said.

Costello even asked the local chapter of Indivisible, a group opposed to President Donald Trump's agenda, to delay its planned noon Saturday protest by two hours to avoid a conflict with a military academy nomination event. It obliged.

"Hopefully there will be more," he said, looking outside shortly after 2 p.m. ET. "The more people, the better. It's democracy."

That Costello is paying such close attention to progressive protesters underscores the pressure he and other more moderate Republicans in swing districts are feeling from an energized Democratic base as House Republican leadership and the White House to discuss repealing Obamacare.

Costello's district in Philadelphia's western suburbs voted in favor of Hillary Clinton over Trump by 1 percentage point -- and though Costello won his second term by 14 points, in a swing region he's sure to be a Democratic target in the 2018 midterms.

Costello didn't directly criticize the House GOP's most conservative elements -- even as he rejected key tenets in their push to undo Obamacare's protections for those with pre-existing conditions and its mandated coverage for "essential" health benefits, which include such things as maternity and mental health care.

"If you're going to be a governing majority in the House Republicans, we have to work through this stuff," Costello said. "And I get concerned ... finger-pointing can diminish our ability to marshal legislation forward."

"I tend to be very factual and very patient, particularly on something this complex," he said.

As the GOP attempts to find a path forward on health care, some Republican lawmakers have dismissed the intense, weeks-long backlash at town halls -- particularly in competitive districts like Costello's -- as ginned up by partisan activists affiliated with progressive groups such as MoveOn.org.

But on Saturday, Costello embraced the activists -- even as he dodged a question on Trump's tax returns and refused the crowd's pleas to embrace a single-payer "Medicare for all" health care system.

"How about that T-shirt?" he asked the crowd of a woman's "Nevertheless she persisted" shirt -- a slogan embraced by the left since Senate Majority Leader Mitch McConnell used the phrase to admonish firebrand Massachusetts Sen. Elizabeth Warren.

"Look at that pin!" he told the crowd when another questioner approached wearing an anti-Trump "Resist" pin on her shirt.

Costello was quick to break with Trump on some issues. He called a US-Mexico border wall impractical, said Trump's budget doesn't have "any chance of passing" the House and criticized the President for seeking to cut health research funding.

When asked how he could condone the US bombing of Syria but turn away those fleeing violence there, Costello broke sharply with Trump over his decision to stop admitting Syrian refugees.

"Good question," Costello said. "I don't."

But he was sharply jeered when he offered what many in attendance saw as a dodge to a question about whether he'd support legislation requiring Trump to disclose his tax returns.

"I think the President should disclose his taxes," Costello said. "I haven't seen even what the bill says, but I think you're going to have an issue of making a law retroactive in terms of compliance."

He ignored shouted follow-ups from attendees about whether Congress could subpoena Trump's tax returns or require them in future elections.

The toughest questions -- and most of the audience's energy -- were reserved for health care.

Costello backed Trump and House Speaker Paul Ryan's bill to repeal Obamacare in the House Energy and Commerce Committee.

But he was among the more moderate Republicans to bristle at changes sought by the hard-line conservative House Freedom Caucus to undercut Obamacare's prohibition on insurers charging higher rates for those with pre-existing conditions and the law's mandate that insurers cover some essential health benefits.

One woman said that through Obamacare's coverage, she pays $35 a month for prescription drugs to treat a disease -- which she did not name. Those drugs would cost $750 per month out of pocket, she said.

Within weeks of Obamacare's repeal, she said, "I will be dead."

"I'd like to know if you're going to come to my funeral," she asked Costello.

Costello said he has "deep concerns" about eliminating mandatory coverage without price hikes for those with pre-existing conditions -- something the House Freedom Caucus sought -- calling it a "dangerous path."

"I want to come to your birthday," he told the woman.