(CNN) Beto O'Rourke said Thursday that if he could, he'd tear down the border wall that separates El Paso, Texas, from Mexico.

"Yes, absolutely. I'd take the wall down," he told MSNBC's Chris Hayes in an interview from a spot overlooking the border and Ciudad Juárez, Mexico.

Hailing that bollard fence during a campaign rally Monday night in El Paso, President Donald Trump called it proof that "walls work." But O'Rourke, pointing to violent crime rates, said the fence's construction in 2008 "didn't make us any safer."

The former congressman from Texas, who is weighing a bid for the 2020 Democratic presidential nomination, criticized Trump's anticipated decision to declare a national emergency in order to fund his border wall.

Read More