Technically speaking this is the concept of the MirrorWorld, Metaverse, or

a persistent 3D digital copy of the real-world -machine-readable, 1:1 scale model of the world that is continually updated in real-time.

The AT&T Stadium, while currently existing as a “private” AR-Cloud, is now fully mapped and relocalizable. The tools we’ve written and the technology we’ve integrated could now allow anyone to design “realities” on top of the venue.

Our project, the Samsung 5G Fan Experience, was always as much as a proof of concept, as it is an experience for the Dallas Cowboys fans.

Using the Samsung S10 5G phone at the AT&T Stadium, the fan experience is transformed — here are some of the highlights…

Hype Up Chants

On game day the most iconic Dallas Cowboys players stand 100ft high integrated into the architecture of the venue, performing their signature moves to get fans excited.

Time Tackle

Our half-time AR game, Time Tackle, was designed to be played from any of the 102,000 seats in AT&T stadium. Tap to control the Dallas Cowboy, who must weave his way through the giant 80ft tall defensive bots.

Live Stats

For the first time ever, we’ve brought one of the best features of watching sports at home into the stadium. Real-time AR stats are rendered onto the field, these include passing yards, sacks, touchdowns, player stats and many more.

Hall of Heroes

In the Hall of Heroes, fans can now meet their heroes, pose for a photo or video, to share with friends. These iconic players are the highest resolution AR holograms ever created, streamed over 5G to your phone.

Background

Our journey really begins with HotStepper, our lovable dude who can walk you anywhere in the world. You can download the app here, it was released in 2017, shortly after ARKit.

We had two primary ambitions; demonstrate that AR could ‘come with you’ (at the time most AR demos were table-top) and secondly that stories could adapt to your location.

HotStepper haircut changes outside any barbershop in the world

But he also had to be a pretty good guide in AR. This unearthed a whole new world of pain for as we learned about the inaccuracies of GPS (it’s obviously still a great technology) and that Apple compass...

I could say a lot about this, but to keep things short — you have to walk (let’s say 3-5m) before GPS has any usable data to work with. Therefore we don’t know where you are or which direction you might be walking until you’re somewhere down that street.

Funky GPS readings

still learning the high-way code

Ultimately HotStepper was partially fixed through “creative hacks”, my favorite, a ‘confidence score’ based on the GPS data, a sort of invisible dog leash that connects HotStepper to the phone; if he’s confident he walks further ahead, if not, he’s more like a badly behaved dog who needs to be by your side.

Enter Visual Positioning

We were happy with the reception the funny looking fella received but disappointed we couldn’t have exactly what we wanted.

So we started looking at Computer Vision solutions or as it’s called, VPS, a Visual Positioning System.

We started to deep-dive on the ability to use Computer Vision to locate you on a worldwide scale.

The promise was centimeter accuracy from a standing start.

A global VPS would solve Hostepper’s inaccuracy almost overnight (hence the beta-release this year of Google Maps AR (minus the wolf). The ability to locate a device with an absolute position in real-world space is really what underpins the future of location-based AR.

However, while the HotStepper might be able to stop walking into the road, he still wouldn’t be able to occlude behind a corner. It was during this time that we began to realize that if we could combine a dense visual map of the location, with visual positioning, not only could Hotstepper occlude around a corner, or walk through a door, we could build an entire virtual world on top of ours.