As the Supreme Court wraps up arguments in a big new case for LGBTQ rights, the focus is on 12 key words from swing vote Justice Anthony Kennedy: “It seems to me the state has been neither tolerant nor respectful.”

The case is Masterpiece Cakeshop Ltd. v. Colorado Civil Rights Commission. In short, it’s about whether a bakery in Colorado, Masterpiece Cakeshop, should be allowed to refuse some services to same-sex couples due to their sexual orientation. The bakery declined to make and sell a wedding cake to a same-sex couple in 2012, sparking the current legal battle after the couple filed discrimination charges and the state government sided with them.

The bakery’s owner, Jack Phillips, argues that forcing him to make and sell a wedding cake to the couple violates his First Amendment rights; he opposes same-sex marriages due to his religious beliefs, and argues that making a cake for a same-sex wedding effectively endorses the union. But the state government argues that what Phillips is trying to do is discrimination — by denying same-sex couples a service he’ll provide to different-sex couples — and that’s illegal under Colorado laws that ban discrimination based on sexual orientation from businesses that serve the public. (Much more on the legal arguments involved in Vox’s explainer.)

The potential consequences in this case are big: If the Supreme Court rules in favor of Masterpiece Cakeshop, it could open a loophole in federal and state civil rights laws that lets people like Phillips get around bans on discrimination by citing their rights to free speech and religious expression. That could lead to more — and totally legal — discrimination, not just from Phillips or bakers but from all businesses that serve the public.

During oral arguments on Tuesday, Kennedy made his key comments in the context of other remarks about how tolerance is a two-way street. It is important for, say, a baker to be tolerant of gay customers, but it’s also important for others to be tolerant of the baker’s genuine religious views, Kennedy suggested.

By arguing that the state hadn’t been “tolerant or respectful” to Phillips, Kennedy was suggesting that the state had done something wrong in enforcing its civil rights laws against Masterpiece Cakeshop. And that could mean that he thinks the state’s anti-discrimination law needs to be relaxed — or that some exemptions apply for people religiously opposed to same-sex marriages.

Kennedy’s remarks should be interpreted with caution. Justices often play devil’s advocate during oral arguments to get the best cases from both sides.

And as Supreme Court reporter Amy Howe noted on Twitter, Kennedy genuinely sent “mixed messages”: He suggested that a win for the bakery would be “an affront to the gay community” and asked if a baker would be allowed to hang a “no wedding cakes for same-sex couples sign” if the Court ruled in favor of Masterpiece Cakeshop, while also making his remarks in favor of respecting Masterpiece Cakeshop’s religious views.

Civil rights lawyer Sasha Samberg-Champion said that his “initial impression was that Kennedy is genuinely unsure what to do,” adding, “He clearly didn't like the way the couple was treated by the cakeshop. But he also didn't like the way the cakeshop was treated by Colorado.” Others, including Howe and Slate writer Mark Joseph Stern, said Kennedy seems to be leaning in favor of Masterpiece Cakeshop.

Kennedy’s views are important, because he is widely expected to be the swing vote in this case. With the four more conservative justices likely to vote on the side of Masterpiece Cakeshop and the four more liberal justices likely to vote against the bakery, Kennedy stands in the middle as the potential tiebreaker.

In the past, Kennedy has used his position to uphold LGBTQ rights — including in the decisions that struck down states’ same-sex marriage bans and anti–gay sex laws.

Now he’s sending mixed signals. That’s deeply concerning to LGBTQ and civil rights advocates, who worry that a pro–Masterpiece Cakeshop ruling could unravel anti-discrimination laws, particularly for LGBTQ people.

But we won’t find out Kennedy’s official stance until later next year, with the Court set to hand down its decision in this case by the end of June 2018.

For more on the Masterpiece Cakeshop case, read Vox’s explainer.