Mae Jemison, the first woman of color to go into space, stood in the center of the room and prepared to become digital. Around her, 106 cameras captured her image in 3-D, which would later render her as a life-sized hologram when viewed through a HoloLens headset.

Jemison was recording what would become the introduction for a new exhibit at the Intrepid Sea, Air, and Space Museum, which opens tomorrow as part of the Smithsonian’s annual Museum Day. In the exhibit, visitors will wear HoloLens headsets and watch Jemison materialize before their eyes, taking them on a tour of the Space Shuttle Enterprise—and through space history. They’re invited to explore artifacts both physical (like the Enterprise) and digital (like a galaxy of AR stars) while Jemison introduces women throughout history who have made important contributions to space exploration.

Interactive museum exhibits like this are becoming more common as augmented reality tech becomes cheaper, lighter, and easier to create. A few years ago, the gear alone—a dozen HoloLens headsets, which visitors can wear as they file through the exhibit—would have been out of reach. Now, as the technology becomes easier to use and the experiences easier to create, museums are increasingly turning to them as a way to engage visitors—whether that's fleshing out the skeletons on view at the Smithsonian's National Museum of Natural History, or taking a tour of Mars with astronaut Buzz Aldrin (as a hologram, naturally).

'There’s a tremendous opportunity, especially around technology like augmented reality, to engage visitors.' Chris Barr, director of innovation at the Knight Foundation

At the Intrepid, the holographic Jemison isn't just the docent of the future. She's also a part of the exhibit, a chance for visitors to come face-to-face with an important figure from space history. “I hope that me taking them on this tour, that it makes it a little bit more real,” she says.

State of the Art

Museums have long relied on technology to give context to their exhibits—whether through informational videos, audio guides, or smartphone apps. Augmented reality, in some ways, is just the next iteration of that. It gives curators a chance to layer more information on top of existing exhibits, and to get visitors more involved with what's on view.

Microsoft

"Cultural institutions are asking, ‘How do we ensure our relevancy in the future?’" says Chris Barr, the director of arts and technology innovation at the Knight Foundation, which gave over $1 million this year to support museums using new forms of technology. "We’re looking at tech as part of the toolset that they use to do that. There’s a tremendous opportunity, especially around technology like augmented reality, to engage visitors."

Some museums have experimented with AR to bring damaged or broken artifacts back into their collections, or to remix the collections on view. This year, the San Francisco Museum of Modern Art worked with the design agency frog to create an “augmented reality gallery” to showcase some of René Magritte’s works, currently on view. The Smithsonian National Museum of Natural History put on an exhibit, called Skin and Bones, which lets visitors animate the museum’s collection of skeletons with an AR app on their phones. Even the U.S. Holocaust Memorial Museum has brought one of its exhibits to life, allowing visitors to learn more about the Lithuanian villagers featured in its Tower of Faces display with a companion AR tool.

"Museums are starting to get smarter and smarter about how do we personalize [the experience of visiting a museum], and how do we make those experiences just as magical as the art that you’re seeing," says Barr.

The Intrepid’s exhibit takes it one step further, using HoloLens headsets to bring Jemison alongside visitors as she guides them through the space shuttle. “We want to make sure that while our artifacts create this exciting and tactile opportunity, we want to make sure we’re capturing our current generation in the language they’re speaking,” says Susan Marenoff-Zausner, President of the Intrepid Sea, Air, and Space Museum.

Behind the Scenes

The Intrepid collaborated with Microsoft, which filmed Jemison at its Mixed Reality Capture Studio in San Francisco. The studio space holds a combination of RGB and infrared cameras that capture scenes in 360 degrees, then render a mesh map in 3-D. “The infrared camera see a very densely speckled version of what’s in that scene, which the computer vision algorithms eat for lunch,” says Steve Sullivan, who heads up Microsoft’s Mixed Reality Capture Studios program.

Microsoft

When Microsoft first started licensing its mixed reality capture tech, it expected most of its business to come from celebrities, sports figures, and the entertainment sector in general. But Sullivan says educational and instructional institutions have become another fast-growing part of what the studio creates. “It’s way richer than video, but not radically more expensive,” he says.