President Trump said Friday he pulled his nominee for country’s top deportation officer because the man wasn’t tough enough.

Ronald D. Vitiello, a longtime Border Patrol agent who has been acting head of U.S. Immigration and Customs Enforcement for nearly a year, had his nomination withdrawn Thursday, in what could be the beginning of a shakeup at Homeland Security.

“Ron’s a good man but we’re going in a tougher direction. We want to go in a tougher direction,” Mr. Trump said.

Mr. Vitiello’s pick had been stalled on Capitol Hill for months, but had finally shown some movement in recent weeks. He cleared the Senate Homeland Security Committee and was slated for a vote in the Judiciary Committee next week.

But Mr. Trump has grown frustrated with his Homeland Security team’s inability to make headway on denting the surge of illegal immigrants streaming across the border.

ICE has had its hands tied, by both Congress and the courts, in what it’s able to do to respond to the rising numbers on illegal immigration. The courts have imposed a catch-and-release policy that prevents ICE’s ability to hold illegal immigrant families. With the near-certainty that they will be released and have a chance to disappear into the shadows, families have headed north in record numbers.

Mr. Trump on Friday suggested it was time to do away with the immigration court system that gives some migrants myriad appeals, dragging cases out for years — and giving the migrants a chance to disappear into the shadows.

“Frankly we should get rid of judges. You can’t have a court case every time somebody steps a foot on our ground,” the president said.

He also said the country should “get rid of” the asylum system. He did not say whether he wants to do away with asylum itself, or just change the protection. Asylum is the humanitarian protection granted to people who reach U.S. soil and prove they are fleeing persecution back home.