Imaginarium Studios

Behind the doors of Imaginarium Studios, the London-based company founded by Andy Serkis that specialises in performance capture, a brand-new animated character is bouncing off a coffee table. He is a small, not-so-scary monster, about half the height of a human being, called Grishnek. He can run, dance or talk – in fact, his speech is vaguely reminiscent of ape leader Caesar’s in Rise of the Planet of the Apes.

That's because Grishnek, like Caesar, is performed by Serkis and brought to life using the same performance capture technology. The difference? While you can only watch Caesar onscreen, Grishnek walks around your living room. The monster appears on the Magic Leap One augmented reality headset; he is the first result of a partnership between Magic Leap and Imaginarium.


Magic Leap launched its long-awaited headset, which superimposes AR animations onto the real world, in August. Unlike some competitor devices, Magic Leap doesn’t need to be connected to a computer with a cumbersome tether. The headset is instead connected to a small controller and processing unit that clips to your pocket. Magic Leap's executive creator director Andy Lanning describes it as "a sort of wearable Mac". Ultimately, the idea is that the real world could be seamlessly augmented with everything from a dancing Grishnek to a WhatsApp notification.

The partnership with Imaginarum started five years ago, when Richard Taylor, the founder of special effects company Weta Workshop, introduced Serkis to Magic Leap CEO Rony Abovitz. As Abovitz explained his grand idea of an augmented reality headset that could integrate dynamic virtual objects – from emails to animated penguins – into a user’s natural environment, Serkis saw the potential for a new platform for performance.

Read next Harry Potter: Wizards Unite is just Pokémon Go with magic Harry Potter: Wizards Unite is just Pokémon Go with magic

The actor has made his name for embracing performance capture technology (also known as motion-capture) in roles such as Gollum in Lord of the Rings. Performance capture records an actor’s body movements and facial expressions through sensors to create animated characters that reproduce human performance. It means that characters like Gollum are still driven by an actor’s authentic performance, where traditionally they would have had to be animated by hand. “I have always seen that the magic of performance capture is its potential to bottle up performance and let it live on as any given avatar,” says Serkis. “Until now, those avatars exist on cinema screens or TVs. So the idea of taking it to the next level by bringing avatars to your living room got me incredibly excited.”

Serkis’ team came up with Grishnek as a demo of what augmented reality headsets may be able to offer for actors. The idea is that performance capture tech could collect and store a vast library of actors’ movements and expressions, which could then be used to animate characters as people interact with them through the Magic Leap headset. The characters could draw from this library to adapt to different scenarios – for example, if a character needs to jump off a coffee table, they could use a “running” performance and then switch to a “jumping” one.


It's not just a passive performance. The AR experience alternates between scenes where the character tells their story and moments when the viewer can interact with them. If Grishnek decides to walk from your TV to your sofa, for instance, you can pick up digital elements to help him along the way. Alternatively, he might suddenly decide to run around your house and hide behind a door. “It’s not about technology, it’s about storytelling,” says Serkis. “Magic Leap for me is a key into one of the things I have been looking for: new ways of connecting with an audience, by taking performance directly into the audience’s world.”

Not everyone has been so impressed with Magic Leap. When the company released its Magic Leap One headset in August, many found it underwhelming, especially in light of the long wait, staggering amount of funding and extensive publicity, which built up expectations for a truly exceptional product. Oculus Rift developer Palmer Luckey was particularly brutal in slamming it as “a flashy hype vehicle” in an article titled: “Magic Leap is a tragic heap”.

Actors carry out motion-capture work at Imaginarium Studios in London. The studio, founded by Andy Serkis, has partnered with Magic Leap to create applications for augmented reality Imaginarium Studios

Read next Tuesday briefing: IKEA wants you to shop using augmented reality Tuesday briefing: IKEA wants you to shop using augmented reality

Serkis, however, thinks it lives up to its promise. “There are some things to work on, like the limited field of vision, but [Magic Leap is] by far the world leader in making this product a reality,” he says. “The level of resolution of characters and the use of light is mind-blowing.”


Lanning agrees that there are still many improvements to be made. “We are already working to develop the field of vision,” he says. “But you have to remember that this is the starting point. This is the equivalent of the brick-type mobile phone. We’ll be looking back at this in ten years, when we will have developed it into a cool pair of sunglasses, thinking of how embarrassing it is that we were using it.”

While other AR headsets such as Microsoft's Hololens have emphasised enterprise applications, Lanning says that Magic Leap is focused on entertainment. He recalls that the original idea for Magic Leap came after he was working on a story with Abovitz. “Rony likes to say, ‘We are a comic book company that took a weird technological turn somewhere down the way’,” he says.

This approach is what won over Serkis, who believes that AR could be a revolution in the film industry. “It is like the birth of cinema,” he says. And Imaginarium is committed to working out exactly what that future could look like.

More great stories from WIRED

– The science of how to hack your brain to remember anything

– The free school that innovated itself to death


– How the very meaning of being human will change by 2050

– The football fans who used a videogame to beat the bookies

Don't miss out. Sign-up to WIRED Weekender to get the best of WIRED in your inbox every weekend