ButtonMasher is our column about video games and gaming culture – from the offbeat fringes to the cutting-edge innovations behind the latest blockbusters

(Images: The Astronauts)

The world is our playground – especially when it is scanned and replicated exactly in a video game.

A wave of new games use a technique called photogrammetry to produce digital replicas of the real world. The technique can be used to recreate rooms, objects, streets and even whole cities, says Krzysztof Plonka of special-effects studio Better Reality in Poland.

Better Reality started out creating effects for the film industry. Now for the first time its technology will be used in a video game. Get Even is being developed by Polish studio The Farm 51. Another games studio in Poland, The Astronauts, is using a similar approach in its latest game The Vanishing of Ethan Carter. (The images in this article are all taken from this game.)


Canadian studio Pixyul wants to take the technique to its limit – scanning the whole world using drones and reconstructing it inside a game.

The first section, ReROLL, features an exact reconstruction of downtown Montreal and will be released next year. The studio plans to release new areas one by one, starting with North America, though it will have to wait for aviation authority approval for some.

Playing in an environment you recognise can give an added thrill. “The ideal world for a player is one that faithfully recreates our world but lets you do things you cannot do in reality,” says Plonka. That may be racing supercars through your home neighbourhood or surviving the apocalypse – as in ReROLL.

Photogrammetry works by gathering hundreds of photographs of an object or scene taken from multiple angles and combining them into a 3D model. Common points between photos are joined up to create a basic shell and this virtual object is then skinned with the overlapping images.

“You not only end up with the exact shape of the object but the exact look of the object as it appears in the photos,” says Andrew Poznanski of The Astronauts. The results are so accurate that photogrammetry has been used to improve aerial mapping and forensic analysis of crime scenes. Its impact could be felt most strongly in gaming, however.

To get the photographs used in The Vanishing of Ethan Carter, Poznanski takes his camera into the mountains near his studio (see picture below). It’s hard work, he says. Getting the photorealistic look he is after for the game requires a lot of photos. “It’s insane, I’ve lost count,” he says. “If you don’t count the failed experiments, it’s in the tens – if not hundreds – of thousands of photographs.”

Such detailed graphics would have brought previous consoles to a standstill. But the latest machines have a lot more muscle. The games also feature algorithms that are very good at swapping graphics in and out as and when they’re needed, freeing up space. Instead, the technical challenge now is how to squash all the extra data into a game that won’t take hours and hours to download. Faithfully recreating the world isn’t a problem, says Poznanski. “But would players be willing to download a terabyte for a few hours of entertainment?”

That depends. For the developers, it’s worthwhile if it makes a game more immersive, helping players forget they’re staring at pixels on a screen. Poznanski has worked in the games industry for 20 years and has long been frustrated by graphical glitches that make it harder for players to suspend their disbelief. Most game environments are created by teams of artists. Even when highly realistic, there are often details that make the look of the game fall short. The real-life appearance of wear and tear, for example, often fails to be captured correctly. Out of necessity, artists create sections of “texture” – a scuffed floor or brick wall – that are then repeated throughout the game.

Consciously or not, we pick up on the artifice, says Poznanski. Instead, he wants players to stop seeing graphical effects and start seeing the world. “We want players to walk through a forest and feel that it is a forest, not think ‘Wow, these are beautiful graphics’,” he says. “We want that part of the brain to switch off.”

Add in virtual reality, which The Astronauts is also experimenting with, and immersion is taken to another level. In fact, the combination of photorealism and virtual reality could make games all too real. Violence, for example, might become a bigger problem when our actions feel realistic and look increasingly gruesome.

“By removing the frame of the game screen, we begin to delve into a realm of personal responsibility that goes beyond what one feels with current game experiences,” says Mark Bolas at the University of Southern California. As a result, Bolas believes the type of games that players find enjoyable could change. “The market will sort this out,” he says.

In fact, The Astronauts found they had to tone down parts of the game, including depictions of violence and an episode in a dark tunnel, because they proved too intense for players.

“Photogrammetry gets us incredibly close to immersion,” says Poznanski. “But immersion can get too intense for a gamer.”

And ultimately, the truth may be too strange for fiction. Poznanski says he has come across several weird rock formations and strangely shaped trees that he scanned and recreated only to find that inside the game they appeared fake. “Our sense of reality is influenced by what we’ve seen previously in other games,” he says. Since he doesn’t want players to question what they find in the game he takes out anything that does not look right, whether based on the real world or not. “We have to censor reality,” he says.