Immigration agents catch an abysmally small percentage of the illegal immigrants who arrived on visas but overstayed their welcome, authorities admitted to Congress Tuesday, describing a loophole that those around the globe are increasingly using to gain a foothold in the U.S.

At least 480,000 people overstayed their visas last year, adding to a backlog that’s reached some 5 million total, members of Congress said. But immigration agents launched investigations into just 10,000 of them, or about 0.2 percent, and arrested fewer than 2,000, less than 0.04 percent, saying the others don’t rise to the level of being priority targets.

“We utilize our prioritization scheme along with the resources that we have,” Craig Healy, assistant director for national security investigations at U.S. Immigration and Customs Enforcement, said as he struggled to defend the administration’s meager efforts.

PHOTOS: 11 times a good guy with a gun stopped a bad guy, saving lives

He blamed a shortage of funding and a tricky environment, where authorities only have limited information, and it takes them months to decide if someone really did overstay their visa and if they are deemed serious enough offenders to make an effort to go after.

Members of Congress were stunned, saying more needs to be done to go after overstays.

Rep. Lamar Smith, Texas Republican, said the Obama administration has increasingly lost sight of the problem, deporting some 12,500 overstays in 2009, but just 6,800 in 2012 and only 2,500 last year — or less than one out of every 2,000.

PHOTOS: These celebrities are devoutly Christian — and you probably didn't realize it

“By deporting such a small percentage of the visa overstayers, the message they are sending wide and far is just get into the country, if you’re not convicted of a serious crime, [and] you’re going to be allowed to stay. You’re gonna pass go; you’re gonna get the money,” Mr. Smith said. “That is the wrong message to send because it increases more illegal immigration.”

Overstays have traditionally drawn less scrutiny than border jumpers when it comes to illegal immigration. For one, they entered legally, which means they went through at least some inspection, and they are more likely to go home of their own accord.

While border crossers draw heavily from Latin America, visa overstays come from across the globe — including from so-called “special interest” countries with ties to terrorism.

After years of delay, Homeland Security officials in January released a report detailing part of the problem. Of the nearly 50 million business and tourist visas issued, about 1 percent — or 500,000 — remained even after their permission expired in 2015.

Canadians were the largest group of offenders, accounting for nearly a fifth. Mexico, Brazil, Germany and Italy rounded out the top five, the Pew Research Center said.

And the report did not include those who came on student or work visas, who may have much different overstay rates given that many of them put down roots during their more extended time legally in the country.

Rep. Sheila Jackson Lee, Texas Democrat, defended the lower priority given to tracking down visa overstays, saying that at least, in recent years, they haven’t been implicated in major terrorist attacks.

“The question for the American people is the issue of security and the level of threat that these people represent,” she said.

However, a number of the Sept. 11, 2001, hijackers were overstays, prompting the 9/11 Commission to call for the government to complete a tracking system so all visitors to the U.S. are tracked on entry and exit. The entry system works, but the government does not have a complete working exit system.

The Department of Homeland Security was created after the 2001 attack, and John Wagner, deputy assistant commissioner of U.S. Customs and Border Protection, said he thinks the department is now in a position where it could have spotted those attackers ahead of time.

“I believe we would have identified them, yes, with the systems that are in place now and the measures in place now. I believe so,” he testified to the House Committee on Homeland Security.

Homeland Security officials said they regularly scan the visa lists for security risks and rely on information being fed from the intelligence community to figure out whom to target for extra scrutiny.

Mr. Healy said once someone is flagged, his analysts then try to figure out if they can match the name to specific derogatory information and if they can track down where that person is. If so, they become one of the active cases, and he has “well over” 100 investigators to go after them.

Of 10,000 investigations opened last year, 40 percent of the targets were either found to have belatedly left the country or to have been granted some form of new legal status by U.S. Citizenship and Immigration Services.

If his investigators can’t track someone down or deem them not a priority right now, he or she ends up on a backlog list, which investigators try to revisit every few months. That backlog list stands at 95,000 names, Mr. Healy said.