(CNN) Aaron Schlossberg, a lawyer in New York, was the subject of a viral video last week after he was caught on camera berating staff members of a restaurant for speaking Spanish with customers.

It might be news to him that nothing in the Constitution or any federal law supports his comment that "they should be speaking English" because "this is America."

A lot of multilingual countries promote an official language, but the United States has never done so with English. In fact, the US has no official language.

"The Founding Fathers didn't see a need to declare one," Dr. Wayne Wright, a professor of language and literacy at Purdue University, told CNN.

"English was pretty much the dominant language of the United States at the time so there really wasn't a need to protect it. And they didn't want to offend their fellow Americans who helped fight for independence."

People in this country have been speaking languages other than English since before the founding of the republic.

In fact, common languages spoken throughout the 13 colonies included Dutch, French and German, not to mention the many languages spoken by Native Americans.

Still, trying to force people in the US to speak English is not new.

Enslaved Africans were forbidden from using their native languages (and at the same time forbidden from learning how to read and write English) because slaveowners feared they would incite rebellions.

Native American children were forced to attend boarding schools where they were punished for speaking their own languages. And many Japanese schools started by immigrants in Hawaii were forced to shut down during World War II.

"The sad thing about debates about language is that they're rarely about language itself, but the people who happen to speak those languages," said Wright.

That's also evident in the case of the New York lawyer who threatened to call US Immigration and Customs Enforcement (ICE) on the restaurant workers.

"My guess is they're not documented," Schlossberg said of the employees. "So my next call is to ICE to have each one of them kicked out of my country."

By threatening to call ICE, he equated speaking Spanish with illegal immigration, said Wright.

'English is not under threat'

There's no question that English is the de facto language of the United States.

It's the language of government documents, court proceedings and business contracts. Immigrants feel an immense pressure to learn it. Most people in the US only speak English. But that hasn't stopped some lawmakers from trying to ensure English is used above all other languages.

In 1981, US Sen. S.I. Hayakawa of California introduced an amendment to do just that. It failed.

Since then, other lawmakers have introduced similar versions of that amendment, but to no avail.

The Senate tried again in 2006, passing an amendment to a comprehensive immigration bill that would have made English the official language. The bill never passed the House.

"There are enough representatives on both sides of the aisle that recognize that it's kind of futile," said Wright. "Number one, English is not under threat in the United States. And number two, it's divisive."

But while efforts to elevate English over other languages have failed nationally, they've seen some success at state levels.

California, Massachusetts and Arizona have, at some point in the last 20 years, implemented laws eliminating bilingual education programs and replacing them with English-only immersion programs.

The laws mandated that most public schools teach students with language barriers exclusively in English, instead of allowing them to teach students in their native languages, as many schools had done for decades.

California's law was in effect for nearly 20 years before it was repealed in 2016. In Massachusetts, it was on the books for about 15 years until a new law effectively repealed it in 2017. In Arizona, the law still applies.

Dr. Beatriz Arias, a senior research scientist at the Center for Applied Linguistics, said Arizona's law is discriminating against people based on the language they speak -- and what language people speak is often an indicator of their race.

"There is a political equation of Americanness with speaking English," said Arias. "People who don't speak English are just as American as those who do."

And although the US is increasingly becoming more multilingual, English probably isn't going anywhere any time soon. In fact, it's the languages of immigrants that are more likely to die out. As immigrants pick up English in their efforts to assimilate, the next generations are much less likely to speak the native tongues of their parents or grandparents.

While Dr. Arias says that California and Massachusetts' moves to bring back bilingual education programs indicates that the English-only movement may be becoming less prevalent, people are still telling immigrants to "speak English."

"The national temperament has gotten so isolationist that people feel empowered now to say things like that lawyer said," said Arias, alluding to recent comments about immigrants made by President Donald Trump.

"When there's permission from the highest position in the country to talk like that, then the understanding of a variety of languages and how that enriches our community is lost."