Washington (CNN) A former spokesman for US Immigration and Customs Enforcement said it "doesn't make sense" to get rid of ICE despite public criticism of the agency, and that agents "have a rough time with this too" in the midst of backlash over the separation of undocumented families at the border.

When CNN's Van Jones asked James Schwab, a former spokesman for ICE's San Francisco division, if he believes the agency should be abolished or shut down, Schwab said, "It just doesn't make sense to me. Immigration enforcement is important, both sides agree on that. If you get rid of it, then what? You need something there."

Calls to "abolish ICE" are growing louder as prominent Democrats, like Sen. Kirsten Gillibrand of New York, take aim at the agency.

Schwab also defended ICE agents during an interview on "The Van Jones Show," which airs at 7 p.m., ET, on Sunday, saying, "These are parents themselves. I know people aren't sympathetic to deportation officers, but they have a rough time with this too and I think more so under this administration."

President Donald Trump recently reversed course and signed an executive order aimed at keeping undocumented families together following public outcry over the separation of parents and children at the border.

Read More