President Donald Trump angrily said Thursday that he is considering withdrawing immigration and border control enforcement agencies from California because of what he called the state's "protection of horrible criminals."

Trump said crime would explode in California if he took such an action — and predicted that the Golden State would be "begging" for the return of federal immigration authorities within two months.

In response, California Gov. Jerry Brown told CNBC in a written statement: "In California we protect all of our people from criminals and gangs, as well as dangerous assault weapons. We do our job Mr. President, you do yours."

In his state of the state address last year, the Democratic governor issued a strong rebuke to Trump's previous threats over how California deals with immigration.

"Let me be clear," Brown said at the time. "We will defend everybody — every man, woman and child who has come here for a better life and has contributed to the well-being of our state."

In January, California became the nation's first "sanctuary state" with passage of a law that bars local authorities from asking about the immigration status of people during routine interactions. Also, the state requires employers to first obtain a warrant from immigration agents before giving them access to the workplace and restricts what kind of sensitive employee information they can voluntarily share with agents without a subpoena.

In signing the so-called sanctuary bill last fall, Brown said the measure "also bans unconstitutional detainer requests and prohibits the commandeering of local officials to do the work of immigration agents." Brown, though, has insisted that "this bill does not prevent or prohibit Immigration and Customs Enforcement or the Department of Homeland Security from doing their own work in any way."

Trump lashed out against California during remarks he made Thursday during a White House event on school safety. His tirade began as he spoke about gang members such as those in the Salvadoran group MS-13.

"These are absolutely animals, these aren't human beings, these are animals," Trump said.

He said that "while we're getting MS-13 out by the thousands," members of the gang still manage to enter the United States.

"They're smart," Trump said. "They actually have franchises going to Los Angeles."

"We're getting no help from the state of California," the president fumed, saying that the state is doing a lousy job of managing the situation, despite having "the highest taxes in the nation."

"Frankly, if I wanted to pull our people from California, you'd have a crime mess like you've never seen in California," Trump said. "All I'd have to say is ICE [Immigration and Customs Enforcement] and Border Patrol, let California alone, you'd be inundated, you would see crime like nobody's seen in this country."

"Frankly, it's a disgrace," Trump said. "The sanctuary city situation, the protection of these horrible criminals ... if we ever pulled our ICE out and said, 'Hey, let California alone and let them figure it out themselves,' in two months they'd be begging for us to come back."

"And you know what? I'm thinking about doing it!"

Last week, a five-day sweep in California led to the arrest by federal authorities of 212 undocumented immigrants and the targeting of 122 businesses in Southern California.

The action followed raids in Northern California targeting undocumented immigrant workers.

— CNBC's Jeff Daniels contributed to this report.