When Boon Sheridan decided to convert an old church into his new home, he didn’t think he would end up fending off dozens of would-be pokémon trainers. But that’s exactly what happened when Niantic, the augmented reality gaming company behind Pokémon Go, marked Sheridan’s home as a "gym" — one of the game’s hubs for training and battles, usually located at noteworthy buildings or landmarks. "Can’t wait to talk to my neighbors about it," he wrote on Twitter, noting that the church had been decommissioned decades ago. "So, all these people pulling up at all hours? We don’t know them… and we can’t stop it."

Sheridan’s case is a perfect example of how digital overlays are increasingly affecting our physical spaces. There’s something invasive-feeling about a company that was once owned by Google casually directing its millions of users to go knock on someone’s door, even if only a small set of them will ever make it there. And Pokémon Go isn’t the first location-based service to cause real-world annoyance. Traffic app Waze has irked homeowners who find their once-quiet streets crowded with drivers, for example, and an IP mapping glitch once turned a Kansas farmhouse into ground zero for angry internet users.

And unsurprisingly, nobody is sure how the law should handle it.

"This is a kind of a novel problem," says Ryan Calo, who teaches cyber and privacy law at the University of Washington’s school of law. Usually, a digital platform isn’t responsible for what its users do — whether it’s something as mild as posting inflammatory comments on a message board or as extreme as following an explosives recipe on a website. Neither is a game. But Pokémon Go isn’t just offering information, it’s actively creating a system that encourages people to visit certain locations to participate. "This is a situation where in order to play the game, individuals have to physically travel," says Calo. And if large numbers of them end up on private property, someone could allege that Niantic showed negligence in setting up the game’s locations.

"When you put a Pokémon gym in someone’s home, you have to know that that person’s going to be bothered."

"When you put a Pokémon gym in someone's home, you have to know that that person's going to be bothered, harassed," says Calo. "It's readily foreseeable that people would get hurt; it's readily foreseeable that people would be a nuisance." Similarly, Stanford Center for Internet and Society affiliate scholar Bryant Walker Smith suggests that an attorney could argue Niantic was encouraging players to trespass — which could leave the company open for liability even if it’s not engaging in outright criminal behavior.

Still, Calo says the game’s creators would have a strong First Amendment defense, and Pokémon Go’s user agreement explicitly forbids users from entering private property without permission, giving it some cover. "I think within property law and community-driven laws like noise ordinances, there are plenty of ways that law enforcement can manage day-to-day nuisance issues or other complaints by property owners," says Nathan Freitas, a fellow at Harvard’s Berkman Center for Internet and Society. "Otherwise, it will likely need to bubble up to a higher-level legislative decisions on [augmented and mixed reality] impacts on communities, and what to do about them, not unlike what has happened with mobile phones and driving."

Niantic isn’t intentionally trying to send players to anyone’s home, and as long as it responds quickly to reports, issues like Sheridan’s could be resolved almost immediately. "PokeStops and Gyms in Pokémon Go are found at publicly accessible places such as historical markers, public art installations, museums and monuments," Niantic and The Pokémon Company told The Verge in a joint statement, urging people to report inappropriate locations online. "We will take relevant steps at that point based on the nature of the inquiry." How promptly they will do so is another question — Sheridan told us that his local gym still hasn't been moved, and an older version of the support page said to only report locations that pose an immediate physical danger.

You’re not just appearing at a location, you’re (probably) taking video of it

None of this diminishes the social responsibilities that augmented reality developers have. Since Pokémon Go launched last week, we’ve seen clear cases where catching pokémon is certainly legal, but in poor taste: the Holocaust Museum in Washington, DC, for example. Freitas suggests that actively addressing complaints is in Niantic’s best interests, even if there’s no imminent legal danger. He brings up the example of Yik Yak, a geolocation-based social app that blocked access around middle and high school campuses after it was used for threats and bullying.

And while simply directing players to a location is one thing, other aspects of augmented reality gaming open up bigger privacy questions. Pokémon Go, for example, asks users to turn on their cameras to see pokémon "appear" in the real world. Effectively, this means capturing video of everything the player looks at. "If you did that in a bathroom or a boardroom — or even if it’s someone's private space, like through a window — you could implicate privacy law there," says Calo.

Niantic has been sending players out into the world for years through its previous game Ingress, and scavenger hunt-style activities like geocaching are even older. But this is the first time one has captured such a broad audience or had such a visible social impact, even if it ends up lasting only a few weeks or months. It exists at a time where someone’s physical location is turning into an increasingly potent weapon against them, through practices like doxxing and swatting — and where we’re more worried than ever about losing control over how companies collect and distribute our data. If anything is going to force us to confront the impact that augmented reality can have on the real world, it might as well be pokémon.

Update July 12th, 5:15PM ET: Added comment from Boon Sheridan on Niantic's response.

HOW TO BATTLE GYMS IN POKÉMON GO