Washington (CNN) White House chief of staff John Kelly was confronted by lawmakers in a closed-door meeting about comments he made Tuesday that some immigrants who didn't apply for protections under the Deferred Action for Childhood Arrivals program were "too lazy."

Kelly reiterated his sentiment, according to people in the room, and did so with reporters after the meeting, as well, saying "some of them just should have probably gotten off the couch and signed up."

House Minority Whip Steny Hoyer brought up Kelly's comments earlier in the day that some young undocumented immigrants who came to the US as children were "too lazy to get off their asses" and apply for the DACA program under the Obama administration, according to people in the room.

The meeting in House Majority Leader Kevin McCarthy's office was a regular meeting of the party's whips or seconds in command in both chambers with Kelly, White House officials and the homeland security secretary for stalled negotiations on how to make DACA permanent after Trump decided to terminate it.

The incident reveals the continued tension between the White House and conservative Republicans and Democrats as they debate the politically toxic issue of immigration -- but also how both sides feel that they have to remain at the table despite the distrust.

According to Senate Minority Whip Dick Durbin and another source in the room at the Tuesday evening meeting, Hoyer brought up the comments and there was some back and forth. According to the source in the room, voices were not raised, but Kelly did rebut the concern.

"He took exception to that, and they had an exchange," Durbin said.

Hoyer declined to talk about it. "I'm not going to get into specific discussions because I don't think that would be helpful," he told reporters.

A source inside the meetings said that in general the sessions don't include disrespectful tones or raised voices, but the deep divisions are apparent.

"The number of times people in the room say 'with all due respect' is really funny, because it's almost to the point they joke about it now," the source said. "It's a bit of a running joke in that group, 'with all due respect,' because usually it's followed by something else. It's just a pointed policy difference, and there's nothing wrong with that."

From 'too lazy' to 'probably gotten off the couch'

Tuesday morning, Kelly told reporters about undocumented immigrants who hadn't signed up for DACA that "some would say were too afraid to sign up, others would say were too lazy to get off their asses, but they didn't sign up."

Asked to clarify later in the day, Kelly reiterated the sentiment. He also portrayed Trump, who ended DACA but has proposed a plan that would offer the whole population a pathway to citizenship in exchange for sharp immigration cuts and changes and his border wall, as the population's savior.

"Some of them didn't hear about the program. Hard to believe, but OK. Some of them were perhaps a little bit concerned about signing up when many, many people in their neighborhood signed up three times, two or three times. But OK, fine," Kelly said. "But I've got to tell you, the rest of them who are claiming, not even claiming, have been granted essentially DACA status by the President of the United States, who has become the champion, I believe, for 1.8 million people who are now considered kind of DACA.

"I've got to say that some of them just should have probably gotten off the couch and signed up, but that doesn't really matter now, because President Trump has given them the status," Kelly added.

JUST WATCHED Biden: Dreamers not bargaining chips for wall Replay More Videos ... MUST WATCH Biden: Dreamers not bargaining chips for wall 01:46

Lawmakers take offense

Kelly's comments drew condemnation from both sides of the aisle Tuesday.

Florida Republican Rep. Carlos Curbelo, a strong advocate on the right for DACA, called the comments "unfortunate" and the sentiment "insensitive."

"I don't want to attribute that sentiment to Gen. Kelly, because the way he said it was that 'others say.' I hope he doesn't personally feel that way," Curbelo said. "Whether he feels that way or not, he shouldn't echo that sentiment, because that's just totally insensitive."

Curbelo added that many DACA eligible people were afraid to come out of the shadows and provide their information to the government given the uncertainty. Others have pointed out the $500 fee could be a factor, as could being old enough for the program.

"These are young people who at the time, a lot of people told them, 'you're signing your deportation orders.' ... So I understand why many of them decided not to come forward, especially given what we see playing out now," Curbelo said.

Members of the Congressional Hispanic Caucus, which consists of all Democrats, also objected.

California Rep. Nanette Diaz Barragán referenced an earlier meeting last year when Kelly was homeland security secretary with the caucus when he said "he was the best thing that happened to DACA," in her recollection.

"John Kelly was here meeting with CHC talking about how he helped the President evolve on DACA. And what we've really seen is John Kelly evolve on DACA," Barragan said. "Just to see him come out today and talk about being lazy -- its appalling to me, it's just despicable," she added.

Hispanic Caucus Chairwoman Michelle Lujan Grisham said the comments were "not surprising" but "incredibly offensive."

"This is the Kelly I know, a guy who is incredibly consistent," Lujan Grisham said, referencing the same previous comments. "And to be again punitive by using that term 'lazy' and creating yet another discriminatory, divisive statement about minorities, I find incredibly offensive and part of the status quo with this administration."

The meeting between Kelly and top lawmakers Tuesday afternoon had members sounding positive.

"After 10 meetings, achieving nothing, anything looks like progress, and at least we had what sounded like a positive conversation, but no agreements" Durbin said.

"It was, I think, useful," Hoyer said. "I think there was some progress in that we discussed specifics, no agreements, it was a meeting and I think we went away with saying, 'Well, I think we made some progress.'"