Gary Johnson is the presidential nominee for the Libertarian Party and a former governor of New Mexico. The opinions expressed in this commentary are his.

(CNN) I'm tired of hearing about a big, beautiful wall and who should pay for it. And I'm weary of hearing politicians try to gain votes on the backs of immigrants, both legal and undocumented, by calling one another bigots.

As a white guy who served two terms as Governor of New Mexico, a border state with by far the largest percentage of Latino residents in the nation, I really wish the nativists and panderers alike would just take a break, admit what the real problems are, and let common sense prevail.

There's a reason why Donald Trump is now fumbling his way toward the "middle" on immigration policy, after having gained the Republican nomination on nativist promises to build a wall along our Mexico border and deport 11 million undocumented immigrants.

Rounding up more than 11 million people -- a population larger than all but the 7 largest states in the union -- is a ludicrous notion to begin with. Everyone knows it, including Donald Trump. It was a lie cloaked in a promise. Even if it were possible, the idea of federal authorities rounding up millions of people and loading them on buses is an image America could never stomach.

Americans know better. They know that the only realistic and, to borrow from this week's language from Mr. Trump, humane policy is to find a fair and safe way to allow non-criminal, undocumented immigrants to get right with the law and go about their lives, paying taxes, having a valid Social Security number, and earning a legal status.

No cutting the line. No "special" path to citizenship. Just a common sense way for undocumented immigrants with jobs, families, and a clean record to come forward and live by the same rules as the rest of us.

Polls show that even a majority of Republicans, the same folks who nominated Donald Trump, favor such a common sense, American approach.

Let's not forget why we came to have more than 11 million undocumented immigrants in the first place. The fear-mongers would have you believe 11 million people swam the Rio Grande, burrowed under a fence or otherwise sneaked into our communities in the dead of night. Yes, some of them did. But a significant number of undocumented immigrants actually came here legally -- and stayed.

Many didn't come—and nor do they remain--for nefarious reasons, but because they found work, established relationships or joined family members. They couldn't stay legally due to special-interest-driven restrictions on their visas. They were students who graduated or found jobs, seasonal workers who found year-round work, or children brought here by their parents.

Of those who did hike the mountains of Arizona or stow away in a container ship, how many of them would have rather come here legally if the line to enter was actually moving? Almost all of them.

Follow CNN Opinion Join us on Twitter and Facebook

But our politicians, both right and left, have created a system for legal immigration that simply doesn't work. We have artificial quotas. We have "caps" on certain categories of workers that have no real relationship to the realities of the free market. It's no coincidence that recent history shows the only successful way to reduce illegal immigration is to have a recession. Over the past 10 years, both illegal entries and the number of undocumented immigrants in the country have declined. That's not because the government did anything right.

Even for those from the right countries or with the right skills, our bureaucracy makes it ridiculously slow and cumbersome to come here legally. If it took months or years to get a driver's license, how many of us would throw up our hands, get behind the wheel, and take our chances driving without one? You know who you are.

The way to stop illegal entry is to spend our resources making legal entry efficient for people coming here for the right reasons. Instead, our politicians want to spend those resources building walls, militarizing the border and "stepping up enforcement." Did they ever stop to think we are enforcing the wrong things against the wrong people?

Try this, instead: No caps. No categories. No quotas. Just a straightforward background check, the proper paperwork to obtain a real Social Security number and work legally or prove legitimate family ties, and a reliable system to know who is coming and who is going. Border enforcement will become what it should be: Keeping out real criminals, would-be terrorists and others sneaking across the border for the wrong reasons.

Stop sending drones out to keep a mother from crossing over with her kids to join their father, and focus on stopping the actual bad guys -- or finding them if they slip by. That's what will make us safer.

We really are a nation of immigrants, and we've become the greatest nation on earth without big walls and nativism. Yes, borders must mean something. Sovereignty and civilization depend on them. But before we turn El Paso into a replica of Cold War Berlin or contemplate loading families on deportation buses, let's take a breath, recognize the real problem -- a flawed system -- and fix it.