President Donald Trump has made no secret of the fact that he thinks lawmakers are wasting their time trying to come up with a funding solution for border security.

But the 17 members of Congress tasked with reaching that compromise are just trying to forge ahead. Staffers worked over the weekend to hammer out a deal. A subset of members are making several stops at the southern border to glean more information to aid them in writing legislation.

But while members of the committee are still expressing optimism they can work something out, there is a sense, even within the Republican Party, that the President’s comments pose a stumbling block, and that he is trying to control an appropriations process he doesn’t understand. One Republican aide, requesting anonymity to speak more candidly, described the comments “frustrating,” acknowledging that they do not help anything.

Nevertheless, Republican Rep. Chuck Fleischmann, who is on the committee, said members have started by laying the groundwork and honing in on areas where they have the biggest differences. “There are some major sticking points,” Fleischmann conceded, “but lets hope we can get it done.”

But in comments preceding and following the group’s first meeting on Jan. 30, Trump said that the members were “wasting their time,” insinuating that he would ultimately declare a national emergency if he did not receive his requisite $5.7 billion in funding for a wall. “I’ve set the stage for doing what I’m going to do,” Trump told the New York Times, when asked about declaring a national emergency in order to build the wall by executive fiat.

To make things even more difficult, Trump is set to give the State of the Union on Tuesday, a very public opportunity for him to scramble negotiations further with more threats, red lines or, as some have speculated, a public announcement that he will declare an emergency.

The conference committee has until Feb. 10 at the absolute latest to file a bill that would avert another shutdown. No meetings have been scheduled, and it is unlikely anything will happen before Tuesday.

Republicans remain adamantly opposed to both another shutdown and President Trump declaring a national emergency. “I’m for whatever works, which means avoiding a shutdown and avoiding the president feeling he should declare a national emergency,” Senate Majority Leader McConnell said last week. But as the deadline inches closer, the President continues to float this possibility. “I don’t like to take things off the table,” he told CBS News Feb. 3. “It’s that alternative. It’s national emergency, it’s other things and you know there have been plenty national emergencies called.”

The leading lawmakers on the committee have remained typically diplomatic, navigating the oft-seen tightrope of pushing for autonomy while avoiding confrontation with the White House. “It could hurt, but it could also help, because a lot of us are going to do everything we can to try to make this work, to get to yes, to avoid another shutdown” Republican Sen. Richard Shelby, who is helping to lead the negotiations, said Sunday on CNN’s “State of The Union” when asked about the President’s comments.

Shelby was quick, however, to urge the President to let the lawmakers do their jobs. “If people leave us alone … if the four of us [the top appropriations lawmakers in both chambers] could get together. Tasked by our respective caucuses, we will get this done before Wednesday night.”

Nevertheless, Congressional aides are insisting that the President’s rhetoric is not actually impacting the negotiations, which continue to hum along, despite the palpable frustration on both sides of the aisle. “We’re keeping our head down and working,” said one Democratic aide. “What the President is saying is just a chyron on a muted TV.”

Write to Alana Abramson at Alana.Abramson@time.com.