HoloLens2 debuts virtual cockpit

It’s a hot morning in Sydney and a Qantas engineer is about to carry out an engine run on a Qantas 737-800.

But the engineer’s not in an airplane, or a flight simulator, or even in Sydney. The engineer is in Darwin, and dons a HoloLens2 to replicate exactly what they would see if they were sitting in the cockpit of the plane on the tarmac in Sydney.

The engineer works through the engine run process, prompted by on screen cues displayed in the HoloLens. At the same time, the system is collecting data that creates a feedback loop for Qantas about what they are doing and how well they are doing it.

The proof of concept, which was developed with Altoura over a period of around five months, digitises a sample of steps within an engine ground run (EGR) test. The next phase will look to build out the full 45 step process, including responding to an engine fire. It’s a dangerous scenario, that’s difficult to test and train for in a realistic environment.

The HoloLens2 however lets Qantas engineers train for emergency situations safely so that passengers stay safe.

It’s a solution that can also bridge time and space – allowing engineers to be trained anywhere, anytime, for any plane.

Engine Ground Runs are a critical maintenance activity that follow strict standards and procedures for the safe operation of aircraft. Qantas engineers receive an in-depth two-day initial training and four hours re-currency training every two years covering operation, and emergency and safety procedures.

One instructor delivers the training and assessment for one or two engineers per session. Initial training consists of theory and practical assessment in a simulator. Engineers spend approximately eight hours in a simulator with the session conducted overnight due to scarce simulator availability.

Recurrent training is carried out by an experienced engineer assessor on-the-job, but without the use of a live aircraft to replicate the emergency safety situations that would be possible in a full simulator.

Click here to load media

In 2018 Qantas engineers undertook over 170 initial two-day courses and nearly 500 four-hour recurrent annual sessions. Training and assessment are regulated by the Civil Aviation Safety Authority (CASA).

The proof of concept HoloLens system uses mixed reality to build a digital twin of a 737 Flight Deck. The learner sits in either the Captain’s or First Officer’s chair to run through procedures and checklists.

According to Amanda El Bahou, Group Learning Technology Manager for Qantas, “The solution is not intended to replace an instructor or current physical assessment practices for Engine Ground Runs.

“The key purpose is to allow learners to undertake an engine ground run in a safe, simulated environment – demonstrating responses to high risk events.” And the system has been designed mindful of the need for Qantas to be able to manage the platform itself.

As El Bahou explains; “The airline industry is changing rapidly and the needs of the business are changing very, very frequently. So from a learning organisation’s perspective, we need to be responsive to how we deliver learning to our team members.” Initial feedback on the trial is proving the HoloLens is effective and meaningful for users and presents opportunity for value.

The technology also ensures simulators are used to maximum advantage.

That’s important because each simulator can cost tens of millions of dollars, is anchored to one fixed location and the logistics of getting people into them are complex not least because of rostering requirements and jostling priorities. If engineers – and longer term, even pilots – are able to use mixed reality to get familiar with a digital cockpit first, then the time they do spend in the simulator can be made far more effective.

Peter McDonnell, head of Qantas Group technology, said that the airline’s HoloLens 2 trial was further evidence of the extent to which innovation was core to Qantas’ DNA.

“I think HoloLens is really exciting. I had the opportunity to experience it the other day, and it takes that virtual reality concept one step further – that ability to for our engineers and crew to be able to experience (high risk situations) in a safe and more stable environment.”

He said that HoloLens gave Qantas personnel the opportunity to “touch, control and move and change. I think we will see much more application in probably a number of our different training areas, or even in technology.”

It’s another demonstration of Qantas’ deep commitment to using state of the art technology to empower all its 29,000 workers and ensure customers get the best and safest experience in the air.