Washington (CNN) Homeland Security Secretary John Kelly sought to assure the nation's governors Sunday that immigration enforcement would be limited to criminals and the travel ban won't target Muslims.

"Gen. Kelly spelled out it's not about religion, it's not even about nation of origin," Wisconsin Gov. Scott Walker said following the security briefing at the annual National Governors Association meeting in Washington.

"It's about the previous administration, under President Obama and Congress that clearly identified there were these seven nations," Walker said, referring to restrictions Obama placed that required travelers from the nations -- Iran, Iraq, Sudan, Syria, Libya, Somalia and Yemen -- to have visas to enter the United States.

"The previous administration didn't feel confident they could get adequate information about the people coming in from those countries," Walker said.

But former CIA Director John Brennan said Sunday it was clear the travel ban was not the solution to stopping the terrorist threat.

"I don't think the travel ban is going to help in any significant way," Brennan said on CBS's "Face the Nation." "It may sound good to have a ban against individuals coming from certain countries, but you really need to take a look at what is truly going to mitigate the nature of the -- and the scope of the terrorist threat that we face."

The White House is expected to release a new version of its travel ban soon -- one that is expected to avoid some of the legal potholes the first one hit as it was rushed out the door just one week into the new administration.

Multiple federal courts across the country have granted requests to temporarily halt enforcement of the President's executive order barring foreign nationals from the seven counties from entering the US for 90 days, all refugees for 120 days and all refugees from Syria indefinitely.

Kelly has said in recent days that the White House is working on a "tighter, more streamlined version of the first executive order."

The outreach from the Trump administration Sunday comes as some Democratic governors have vowed to counter he White House's top priorities, including the travel ban and stepped-up efforts to deport illegal immigrants.

Washington Gov. Jay Inslee, a Democrat, helped stop the travel ban in court through a legal challenge and said he plans to keep fighting it.

"I was getting ready to go on a bike ride," Inslee told CNN Saturday, recounting what he was doing when the ban went into place. "I got off the bike and went to Seattle-Tacoma International Airport to help these families get into our country. And because of our lawsuit, because of our great attorney general, we got people into our country."

Virginia Gov. Terry McAuliffe, the NGA's chairman, said Kelly promised him in a one-on-one meeting that the administration would not unleash a deportation force on undocumented immigrants.

"The general made it clear to me -- I have some concerns, obviously, about the new regulations in place -- and he made it very clear that the only folks they are going after, ICE agents, were those who come into the country illegally and have been involved in a criminal enterprise," McAuliffe said.

"If you were here just illegally then they were not going after you. They're only going after individuals who came illegally and have been involved in criminal enterprises. And they are not randomly stopping people," McAuliffe said. "I took Gen. Kelly at his word."