When is a button more than just a button?

That was the issue before the Supreme Court on Wednesday, in a touchy First Amendment case over a state law banning "political" garments or insignia at polling places.

It was a messy hour of oral arguments, as nearly every justice expressed frustration at lawyers from both sides who were unable to articulate a workable standard to help the court decide when personal expression -- like political buttons -- crosses the line into voter disruption or intimidation.

The case started when a Minnesota voter sued after being told to remove a "Please I.D. Me" button and Tea Party t-shirt at his polling place.

He says such "passive, non-disruptive" expression should be allowed, while the state argues the restrictions are a "reasonable, viewpoint-neutral" means to ensure polling integrity and reduce voter coercion.

"If, in fact, we are trying to have a place where a person has reflective thought for a moment after the hurly-burly of the campaign, this problem will inevitably arise," said Justice Stephen Breyer, suggesting state restrictions would be appropriate.

"It does reach quite a bit beyond what I think a reasonable observer would think is necessary," countered Chief Justice John Roberts. "Do you really think if someone has a shirt with the tiniest little logo or inscription here, that's going to have any effect on decorum?"

Seeking an acceptable constitutional standard, the justices offered a timely set of heated hypotheticals to explore the legal boundaries, including apparel that might mention the:

#MeToo movement against sexual harassment

Parkland Strong, a response to last week's Florida mass school shooting; or alternately a National Rifle Association hat, in the debate over gun rights

Competing Black Lives Matter and All Lives Matter coalitions

A rainbow t-shirt with no written message supporting gay rights

"Make America Great Again," a Trump presidential campaign slogan

All states have laws prohibiting direct campaigning, solicitation, or advocacy signs inside or near the polling station. The Supreme Court in 1992 upheld a Tennessee law prohibiting campaign materials within 100 feet of a polling place.

About 11 states and the District of Columbia go further, banning anyone wearing "political badges, political buttons, or other political insignia to be worn at or about the polling place" on election day.

Voting monitors and judges -- some government workers, others volunteers chosen from local political parties -- would ostensibly enforce the policy and settle any disputes.

Andrew Cilek, a local political activist in Minneapolis who brought the suit, was told twice in 2010 to remove his Tea Party-related apparel. When he returned a third time with his lawyer, Cilek was given a ballot but his name was recorded for possible prosecution, which can include a $300 fine.

The libertarian Cato Institute and the ACLU are among the outside groups that back him.

In the one-hour courtroom arguments, David Breemer, the attorney opposing the ban, said polling places were not "pristine retreats from the real world," and "there's not a right to vote free of being bothered at all."

That brought some bench concern.

"A t-shirt, you say, is passive," Justice Neil Gorsuch told the lawyer. "What if it were instead a sign on my head, you know, flashing lights? Is that active or is that passive [speech]? How are we supposed to police the line you're suggesting?"

"Why should there be speech inside the election booth at all?" added Justice Anthony Kennedy, his voice rising. "You're there to vote."

Roberts could be the deciding or "swing vote." He asked tough questions of both sides. On one hand, he defended the Minnesota law, saying organized voters, "a big employer, the union, teachers, whatever, say we're all going to show up and we're all going to have these buttons on." Roberts suggested "it’s subtle psychological pressure" on other voters.

"How are we supposed to police the line you're suggesting?" — Justice Neil Gorsuch

But then Roberts worried about individualized, discretionary and binding decisions made on the spot about what is acceptable to wear.

"How do you know if a mistake has been made?" he wondered. "You know, if someone makes a judgment and it's challenged, how do you know a mistake's been made?"

Daniel Rogan, the attorney for the state, received some pushback when he conceded a rainbow t-shirt would be permitted under Minnesota law, but not an NRA t-shirt.

Justice Samuel Alito bristled when told most voters in violation would be told to hide or change their offending apparel.

"The person can wear a bathrobe or some kind of coverup to go in and vote. You think that's not kind of humiliating?" he said. "Or the person can be listed as a bad Minnesotan?"

Justice Sonia Sotomayor said in the Minnesota case, the voter who sued was displaying what she said some considered a "highly charged political message," since the "Please I.D. Me" button referred to pending legislation that would require voters to show an official picture identification establishing residency.

The balancing test for the high court is a tricky one. Any limitations on speech content would have to have a "compelling government interest." But "reasonable" restrictions on speech on government property are allowed if they are viewpoint-neutral.

The case is Minnesota Voters Alliance v. Mansky (16-1435).