Sometime in the next thirty years, it is likely that humans will set foot on the surface of Mars. The mission that takes them there might be an international government-sponsored mission, or it might be the result of private endeavour. However it comes about, it will be the culmination of decades of planning, hopes and dreams stretching back beyond the birth of the space age.

There is much that a crew on such a mission will be taking with them in terms of hardware, equipment and technology. And it is very likely that when looking down the list of technologies they’ll take with them, one will be able to find virtual reality, virtual worlds and augmented reality – an in a variety of roles and uses.

Take the crew’s psychological health and well-being for example. A round-trip mission to Mars will take between two and 2.5 years to complete, depending upon the “class” of mission undertaken.

Throughout that entire time, they’ll be completely isolated from everything we take for granted here on Earth – the freedom to wander outdoors, the sight of a blue sky, green hills, rivers, the sea, cities, lakes, people; they’ll be confined to enclosed spaces which really don’t offer too much in the way of privacy. They’ll even be confined to meals from a menu set months in advance, with no real option to give into a whim for a particular delicacy if it isn’t on their vessel.

For the majority of the mission time, the only people they’ll be able to directly converse with are their fellow crew members – with a minimum round-trip time delay in communications between Earth and Mars of 8 minutes (and potentially as much as 40 minutes through parts of the mission), having real-time conversations with loved ones on Earth simply isn’t going to be possible; they’ll have to rely on pre-recorded messages and video and e-mail.

In these circumstances, stresses are bound to develop, both for the individual members of the crew and, potentially, between team members, no matter how carefully selected for compatibility ahead of the mission or how well-trained. One way of potentially dealing with them is through the use of VR and virtual environments, as NASA and other organisations have been investigating for much of the last decade.

It’s not hard to imagine, for example, a crew going to Mars with a library of pre-filmed environments and events which they can then explore and enjoy individually or together through the use of personal headsets – or for such a library to be updated with new items beamed via something like OPALS to their craft. Such environments and activities could provide psychological relief from the confines of the space vehicle.

Similarly, high fidelity virtual world environments which support direct interaction, such as through haptic feedback mechanisms, might provide the means by which crew members can “remove” themselves from the confines of their vehicle and enjoy a variety of activities, including something we take for granted in VWs today – the ability to create and build.

ANSIBLE (A Network of Social Interactions for Bilateral Life Enhancement) was an initial attempt by NASA, working with SIFT and All These Worlds, to explore how virtual worlds might be leveraged to provide astronauts with environments which could be shared or used individually, and which might offer a range of AI interactions as well.

An intriguing element with ANSIBLE was the exploration of the idea that virtual world environments could be asynchronously “shared” between crew members and their friends and family on Earth, allowing them to engage in shared content creation activities, for example, through the swapping back and forth of OAR files, the ability to engage in “shared” immersive games and so on. ANSIBLE researchers even suggested that used in this way, a personal virtual world space could enable an astronaut and their family “share” special occasions more personally than could be done via e-mail, radio or video.

Commenting on the used of immersive environments and haptic technologies in Moving to Mars: There and Back Again (Journal of Cosmology, 2010, Vol 12), Sheryl L. Bishop, Ph.D, noted, “Telepresence and full fidelity audio/video/3-D communication replay capability will provide for more effective psychological support and interaction for crew members and to families and friends back on Earth.”

In terms of crew welfare, virtual reality has another potential use: assisting in matters of fitness. Most current mission scenarios involve the crew travelling to and / or from Mars in a “weightless” environment. Such an environment can be detrimental to many aspects of human physiology – muscles, bones, heart, lungs, etc. It is therefore essential long exposure to weightlessness is countered by routine exercise of up to two hours every day.

In the confines of a space vehicle, the opportunities for exercise tend to be limited and potentially boring. How much more pleasant it might be for an astronaut who, after lugubriously strapping themselves into a treadmill harness and making all the required tension adjustments ready for 30 or so minutes of going nowhere while staring at a bulkhead, could slip on a VR headset, and go for a run through a woodland park or along a beach, the sounds of nature or the waves in their ears?

VR, AR and immersive environments have other applications within a mission as well.

NASA already employs virtual reality and haptic / kinetic systems to train crews heading for the International Space Station. not only would such techniques be used to train crews selected for a Mars mission, there is no reason why portable versions of these systems couldn’t be carried aboard their vehicle, offering additional opportunities for training and for “refresher” courses in various activities.

Further, data gathered via orbital and ground reconnaissance could be combined to create 3D representations of the landing site, allowing the crew to familiarise themselves with it at ground level, practice “driving” their rovers, etc. Equally, pre-defined immersive simulations could be used by the crew to rehearse critical parts of the mission, such as atmospheric entry, descent and landing at either end of the mission.

Another area of VR use is in what NASA calls “immersive telepresence”. This allows human operators to take direct control of rover vehicles used in support of the mission, giving them added flexibility of use. While the rovers might operate largely autonomously, carrying out remote survey work or sample-gathering, should one find something of potential interest, a crew member could take command, using VR to “see” what the rover sees, and teleoperation to manipulate its robot arms.

Augmented reality is also likely to play a variety of roles in long duration space missions. For example, it is likely to be incorporated, Google Glass-like, into space suits, providing the wearer with access to a wealth of additional data which can be called upon according to situation.

Another use for augmented reality already under development is again related to crew healthcare. A crewed mission to Mars may experience any number of medical problems arising from the likes of accidents or unforeseen health issues, some of which will need immediate medical intervention. While crews will undoubtedly include a surgeon-internist-astronaut, it is unlikely that they will be trained to handle every medical procedure they may have to perform. Nor, due the communications delays already noted, are they going to be able to rely on direct support from Earth.

To help deal with such situations, the European Space Agency is in the process of developing CAMDASS, the Computer Assisted Medical Diagnosis and Surgery System.

Utilising an augmented reality headset CAMDASS precisely combines computer-generated graphics with the wearer’s view. Thus, the medical expert on a mission can don the headset and carry out a procedure, precisely guided by a 3D “overlay” and images presented on screens above the operating table. CAMDASS currently uses ultrasound, already use as an examination / diagnosis tool on the space station, but ESA note it could be enhanced and used in surgical procedures.

A final intriguing aspect of the use of VR and virtual environments in a humans to Mars mission is related to public engagement in such a mission.

We might not be able to all go to Mars ourselves, but as Richard Terrile, director of the Centre for Evolutionary Computation and Automated Design at NASA’s Jet Propulsion Laboratory noted in 2012, virtual environments using data and footage returned from Mars during a mission, could allow “Earthbound humanity to take the journey as well and sense the thrill of walking on an alien world. Three-dimensional surface models with real physical properties and used with avatars further allow the simulation of surface deformation. Thereby, the public can virtually explore, dig holes, move rocks and leave foot prints on Mars.”

We’re still most likely at least two decades away from seeing humans on Mars, even allowing for ambitious plans such as Dutch-based Mars One (where the use of VR and VW environments might be even more vital for crew welfare). Obviously, in that time, VR, AR and virtual worlds are going to mature enormously – and so is their likely relevance for long duration space missions as well. As such, what is presented here merely scratches the surface. It’s going to be an interesting future!