John Kelly, President Trump's former chief-of-staff, appeared to part ways with several of the administration's immigration policies, including Mr. Trump's promise that a "big, beautiful" wall must be built along the southern border, in a wide-ranging discussion at Duke University on Wednesday.

Kelly, who also served as Mr. Trump's Homeland Security secretary, said it's his view that "we don't need a wall from, you know, sea to shining sea." He suggested that Customs and Border Patrol agents would agree that a physical barrier in many places "seems like an awful waste of money," and appeared to question whether the crisis at the southern border needed to be deemed a national emergency at all.

"If I told them 'I can get you 2,000 miles [of wall],' they'd say 'that seems like an awful waste of money' because they don't need it everywhere is the point. So, the physical barrier in some places, many places backed up the CBP people, men and women, that have great people...and they use a great deal of technology, sensors, things like that, towers, drones. So, is it an emergency?" said Kelly.

Get Breaking News Delivered to Your Inbox

Despite the potential logistical difficulties presented by building a wall, and the obvious legal and funding challenges he currently faces as Democrats lead an effort to block his national emergency declaration, Mr. Trump has remained firm that a wall is the only way to truly secure the border.

Kelly also appeared to disagree with Mr. Trump's view that many crossing the border illegally are "rapists" or drug dealers. "They're overwhelmingly not criminals," Kelly said Wednesday. "They're people coming up here for economic purposes. I don't blame them for that."

Kelly also wouldn't say if he fully supported Mr. Trump's national emergency declaration, explaining, "So long as [the declaration is] legal and within the authority of the president, you do it." He went on to say that he believed the national emergency issue would be "wrapped up in the courts" as it continues to face mounting legal challenges.

Kelly said, however, that the larger issue of the border "gets into the alienation and lack of bipartisanship that we seem to be crippled with." Kelly noted that in the past some Democrats had signed onto the idea of expanding physical barriers along the southern border in the early 2000s.

In response to some border states pulling back their national guard troops from the southern border, Kelly said that the military does not generally enjoy being deployed for domestic operations.

"We have a long, long tradition of fighting the away game and have real sense that except for natural disasters, to interact domestically is something that most -- I would say all -- military people prefer not to do," said Kelly. He added that while National Guard troops might operate drones or provide logistical support to law enforcement, "I would look for another way to do it rather than deploy federal troops on the border."

Kelly also said that the administration's decision to implement a "zero tolerance" policy of separating children from migrant parents illegally crossing the border surprised federal agencies.

"It certainly caught [Homeland Security] and [Health and Human Services] flat footed," said Kelly. The policy has since been discontinued.

And despite criticism of how they handled the care of young migrants detained along the border, Kelly argued that the administration "did the best we could."

"They do the best they can to take care in the right way but it is still a dorm-type facility and, in many ways, better than where they came from," Kelly explained of the detention facilities along the U.S-Mexico border. "So, not the best policy and I would say it caught people in the government flat-footed and we spent a fair amount of time trying to play catch up."

Looking back on his career, Kelly suggested that if the 2016 election had gone the other way, he "would have probably" agreed to serve as secretary Clinton's chief of staff.

"Politics aside, it's all about governing the country. We are in good and bad times, there's many threats around the world. Many threats socially or domestically. And love 'em or hate 'em, whoever it happens to be, once they become president, it's in our interest as a people to assist him or her as much as we can to be successful."

Here's more highlights of Kelly's conversation:

Kelly on Trump's travel ban woes: Kelly suggested that the administration bungled the travel ban policy of banning travel from places of Muslim-majority countries due to inexperience and a lack of understanding how government works. "In my view what happened was they wrote some of these policies without going through that arduous interagency process," said Kelly He said his role "simply was to best I could react to that."

Kelly has no comment on Jared Kushner security clearance controversy: "I couldn't, and I'm not dodging, and I couldn't comment for a couple reasons. One, someone's security clearance is something that I can't talk about or reveal, number two the conversation with the president at that level would certainly be covered by privilege," said Kelly of reports that the president had overruled White House officials and ordered his son-in-law Jared Kushner be given a top-secret clearance.

Kelly calls chief of staff role the "least enjoyable job I've ever had": Kelly, who has repeatedly expressed his frustrations over the inner-workings of the Trump White House, referred to his most recent employment as "the least enjoyable job I've ever had." He did say, however, that for eighteen months he "helped the administration, the president of the United States make the very best decisions that he could based on the information that we could provided him."

Abby Kingsley, a sophomore and public policy major at Duke University, contributed to this report.