Adding ALL the Audio

If you ask any player or fan of Halo, I believe players would say one of the most iconic and integral parts of all Halo titles is the audio. From the first time you heard the monks chanting in Halo: Combat Evolved, to mid-mission hearing Cortana say, “This is not a natural formation,” or the sounds of combat across all of the games; audio is something that makes Halo, well, Halo.

As we have talked about previously, MCC is one of the most unique game collections in the world where each title within is made up of, in some cases, multiple unique game engines spanning two previous generations of consoles. Working to bring MCC to the Xbox One and now PC came with some unique technical challenges which are actively being worked on to ensure the best experience possible for all. This includes ongoing efforts to further improve audio in Halo: Reach and Halo: CE Anniversary.

In the previous section Max touched on the fact that we are bringing the original multiplayer sounds to MCC H:CE, a top community request since Halo: Combat Evolved Anniversary released originally in 2014.In addition, we are going to touch on ongoing audio work for Halo: Reach and other titles within MCC. There’s a lot to dig into on the topic of audio – the current status of work, how audio is handled by the overall code and individual engines, and the unique processes to play back audio with different hardware architecture.

All with the goal of making the games sound right on both the Xbox One and PC.

Since being introduced to one of partners working on MCC and hearing more about what is being done, pun intended, I am super pumped to discuss the partnership that 343 Industries has formed with the world-renowned audio postproduction studio, Formosa Group! One fun detail is they are not only partnering with us on updating H:CEA’s and Halo: Reach’s audio, but many other titles in the collection for both Xbox One and PC!

Well, enough from me let’s look at what’s going into this epic auditory transformation from Paul Lipson.

Postums: Hi! Thank you for taking some time to talk with me Paul. Some people might recognize your name in the Halo community, but for those who don’t, let’s start with the basics. Who are you and what do you do?

Paul Lipson: Great to be here, and hello Halo community! I'm Paul Lipson, Senior Vice President at Formosa Group - the leading audio post production team headquartered in Los Angeles, CA. We work across all screens, from large feature films like Mad Max: Fury Road, Blade Runner 2049, Deadpool 1&2, John Wick series to broadcast favorites like Game of Thrones, Family Guy, Better Call Saul, and leading video game franchises like the Last of Us Part II, Call of Duty, The Legend of Zelda: Breath of the Wild, God of War - and of course, Halo! Before joining the team at Formosa Group, I was the Senior Audio Director - Publishing at 343 Industries and at the Central Media Team with Xbox Game Studios.

So not your first rodeo is what I’m hearing. With having that much exposure and creative industries being worked on, what does life look like at the studio? How many people in the studio are long time Halo fans? Any heated rivalries during playtests?

PL: There is never a dull moment at our studios (laughs). We are mixing hours of cinematics, recording all kinds of weapons and vehicles, crafting bespoke sound design, recording and capturing amazing voice and stunt performances, and recording the world's most elite orchestras to make music that drives emotion to the core. We have a bunch of hardcore Halo fans on our team, so everyone loves to take a break and test out new MP features or do their own Q/A on campaign levels. Our love of Halo shines as fans, so we are striving to get the best results in lockstep with the community. It is a dream job to love what you make and make what you love.

I know you mentioned earlier that your teams work across many different mediums, but what has the studio been up to recently that is not Halo related? What are some of the other video game related projects your teams have worked on lately?

PL: Our teams here at Formosa Group have been quite busy, and we have enjoyed some big wins over the past year. Players have loved our recent work on God of War, Call of Duty: Black Ops 4 and Call of Duty: Modern Warfare, Days Gone, Borderlands 3, Concrete Genie, and are excited to experience our upcoming work on Ori and the Will of the Wisp, and The Last of Us Part 2. We have been thrilled to return to Halo and support the launch or Halo: Reach and the upcoming MCC titles, it’s like coming back to the holy grail!

That’s an impressive resumé with some amazing games on it! With your long relationship with Halo, 343 Industries, and Microsoft, what do you personally do? What are some of the games you have worked on in the Halo Universe?

PL: I've been involved with Halo audio for many years now, both internally at 343 and externally. Being a day-one Halo fan has kept me glued to the franchise, and it has been exciting to contribute to 343i's earliest franchise efforts, way back with the animated content for Halo Waypoint in 2008. You might remember the story projects like Mona Lisa, The Return, and Headhunters, which I functioned as Audio Director and Composer. I then went on to Music Director and Audio Director roles for Halo Combat Evolved Anniversary and then subsequently joined the team at Microsoft as Senior Audio Director. I was happy to work on The Master Chief Collection, Halo 2 Anniversary, Halo Wars 2, Halo Spartan Assault, Halo Spartan Strike, Halo Fall of Reach, The Halo Channel, Halo Online, and a host of internal initiatives and projects. I continue to support 343i and Halo in any way I can, now with our broad teams in Los Angeles at Formosa Group.

Just a slight background working on Halo it sounds like. How has development work been going on MCC?

PL: Development is going well considering the size, scope, and depth of the MCC. We are working diligently to upgrade the audio profile of all titles and it is a large coordinated effort from content experts, coders, producers and franchise leads to make it happen. The code side requires a heavy focus as we advance the experiences to PC; we want fans to feel the same joy as the original release of the games and feel good about the optimization and fidelity gains available to us today. Fixing bugs is never easy, but we are confident on how things should sound and appreciate the feedback and insight from the community of Halo faithful.

Each title presents challenges and we have some exciting things to rollout now and in the near future that will certainly delight fans. We are taking all the community feedback seriously and engaging point by point on the areas we know we can improve. One challenge is to present both the original audio content and the new Anniversary content side by side and in an organic way that gives players the options they crave. Bringing back original sounds options to multiplayer sessions has been a high-priority and we are thrilled to be delivering that back to the community.

I may have seen several hundred or so conversations on social media overflowing with joy of having the option to hear those original Multiplayer sounds brought back in our recent work-in-progress flight of H:CEA. What has it been like going back into H:CEA and creating the ways to bring the original classic sounds back?

PL: Bringing the original audio back to H:CEA Multiplayer is a primary request from our players, and we are glad we can answer that call. Being able to choose audio sets for MP is an exciting feature and has required some deep code-work to make it happen. We have learned much over the years supporting MCC and this addition brings us closer to our ideal experience and provides more choice for players who crave MP in its original format.

I know for me personally it was very nostalgic hearing them in MCC again. As far as bringing them into MCC, what areas has your team had to dive deep into for updating and implementing the original audio? How complex is the whole process?

PL: The good news is we already have two identical sound sets for H:CEA, the original and the Anniversary updated content. The trick has been to have these two sound trees live side by side, and then create a user-selectable option that loads one or the other on-demand. This is more complicated than it sounds but it is something we are delivering.

In terms of the process used of new codecs, formats, etc., what liberties have been taken to preserve the audio quality from Halo: CE to make it sounds as close to if not exactly as it did in the original game released nearly 20 years ago?

PL: We have needed a very detailed strategy to preserve and advance these landmark titles to our new platforms. We are looking at thousands and thousands of sounds that all have iconic status, so we have an almost forensic approach to the audio archives. A few key details involve the original sound source content, and its legacy .aif file format. We cleaned up some of the older files and brought them forward to an optimized .wav source format. We have also advanced the underlying plumbing and audio code architecture to modern audio engine solutions that interface efficiently with the overall updated code-base. The end results are a cleaner and tighter legacy experience and a side-by-side anniversary update that takes advantage of our modern hardware. Some of these sounds are over twenty years old now, so this work is equal parts preservation and equal parts pure gameplay goodness for fans.

Well I know it’s beyond that even. I think it’s time to get into a question that the community really wants to know more about. Halo: Reach audio has been a hot topic since its release into MCC. People within the studio are apprised to what’s going on, but what has been the team’s focus in restoring it and having it sound more like the original audio?

PL: Reach has some challenging technical hurdles - the foremost being the update and transfer of the audio signal path and processing scheme from the original release on Xbox 360. Since the underlying hardware architecture of the Xbox 360 is fundamentally different than our current Xbox One and PC platforms, our first internal pass involved an approximation of the PowerPC audio code and DSP.

We need to push farther, and so we have a special announcement! We are pleased to reveal a renewed partnership with Waves Audio, the leading audio DSP plugin manufacturer, who provided the original DSP plugins used in the original Halo 3, Halo 3 ODST, and Halo Reach releases.

Waves Audio is working hard with our teams to recompile and update their world-famous audio plugins and preserve the exact audio profile we know and love. It is a commendable effort from our Microsoft teams and partners - and we are excited to extend this level of detail and commitment to preserving the original experiences in their purest form.

When working on MCC and the rich history of audio contained within it, what would you say is the most unique thing the teams have had to do to bring some of the older audio recordings to today’s standards?

PL: There are so many experiences to highlight when making Halo, but two things really stand out - the first being recording actual tanks, large-caliber guns, and all sorts of military vehicles for both Halo 2 Anniversary and Halo Wars 2, and the other is recording the best orchestral musicians in the world for the Halo Combat Evolved and Halo 2 Anniversary Score. It is such a treat to bring back people like Steve Vai and Misha Mansoor for shredding guitar, or elite vocal ensembles like Chanticleer to sing the famed monk chants. Nothing is as iconic as that!

With all this discussion about audio for games it only makes sense to ask the question; what is it like to work on game audio as a professional? What is a tip you have for aspiring game developers out there who want to break into audio production?

PL: After 21 years working in games - I'm still figuring it out day by day (laughs). Interactive Audio is challenging in that sound adapts and changes over time as players move through the narrative - Halo has always blazed the trail as far as iconic audio quality is concerned. The best way to start is to study music, record lots of sound, and understand interactive systems and tools - play with everything you can get your hands on. Find a game you love and ask yourself "what makes this so special and how can I recreate this style of audio in my own unique way?" Audio is 50% of the experience for any great game, so jump in and make your mark!

Thank you so much for taking the time to have this interview. Also, a huge thank you for partnering with us to ensure the legacy of Halo sounds the best it can on the platforms MCC is on and heading to! Before we go though, I have one last question. Of all the sounds across all of Halo – what is your favorite in game audio you have been a part of?

PL: There have been many amazing Halo audio moments for me over the years, but the absolute jaw-dropping holy-c$4p moment was the one-buttons mode switch in Halo 2 Anniversary. It is insane to be spooling up a Plasma Pistol or be mid-fire with the AR or thrusting with a Ghost and be able to switch from original audio to updated audio (along with the graphics) 1:1 on the fly. There is not a single hiccup, it just changes back and forth mid-stream. I have never seen anything like it. My colleagues Brian Fieser, Paul Stoughton, Robert Ridihalgh and I are super proud of the content fidelity and it is a code triumph to have two entire sound-trees running in parallel and in perfect sync. I still can’t believe it (laughs). The response from the fans has been awesome, and I still get high-fives from game teams when they ask about it.

Thank you again so much for taking the time to go in-depth into what the Formosa Group is cooking up for MCC and its future! Now let’s dive on into all the other goodness being worked on for MCC. This next section is directly from the Senior Producer on the Publishing Team, Michael Fahrny. Take it away brother!