These days, debates over immigration may seem like an inescapable and constant part of American politics. From the Supreme Court to the Presidential election, everyone has an opinion about how to fix a system that many perceive as broken.

But as immigration historian David Reimers tells TIME, that’s not actually the case. The current debate over immigration–especially when it comes to immigration from Mexico—is largely a product of the 20th century. “For so much of our history, immigrants just sort of came in,” says Reimers, a professor emeritus of history at NYU. “We had a very loose border with Mexico.”

Here, he takes TIME through some of the major moments that created the immigration debate as we understand it today.

The U.S. government only started counting immigrants in the early 19th century, and it wasn’t until the 1870s that the Supreme Court declared that immigration was a federal concern. The border patrol traces its roots to concerns over prohibition-era liquor smuggling.

Get your history fix in one place: sign up for the weekly TIME History newsletter

The system set up in the 1920s favored nationalities that were already represented in the U.S. population, because it based annual visa quotas on the 1910 census. But it only pertained to the Eastern hemisphere. Changes starting in 1965 eliminated some of the discrimination that had been baked into that system, and the Western hemisphere was given a limit for the first time. Although there have been changes since then, Reimers argues that system is essentially the one in place today.

“We have eleven million undocumented immigrants today. They’re practically all from the Western Hemisphere,” Reimers says. “If you would have not had that quota, I would argue, most of those people would be legal.”

Get our History Newsletter. Put today's news in context and see highlights from the archives. Please enter a valid email address. Sign Up Now Check the box if you do not wish to receive promotional offers via email from TIME. You can unsubscribe at any time. By signing up you are agreeing to our Terms of Use and Privacy Policy . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. Thank you! For your security, we've sent a confirmation email to the address you entered. Click the link to confirm your subscription and begin receiving our newsletters. If you don't get the confirmation within 10 minutes, please check your spam folder.

Write to Lily Rothman at lily.rothman@time.com and Arpita Aneja at arpita.aneja@time.com.