Patrick Baltzell has been the sound engineer for most of America’s most-watched events in the past few decades. He’s sitting alone in the convention halls of NAMM, a trade show for the music making industry. Though I instantly recognize his signature thin frame and curls of white hair, no one looks as Baltzell stands to greet me with unbridled enthusiasm. It’s likely everyone in this room has no idea who he is. But Baltzell was not only in charge of the audio for the past 19 Super Bowls (excluding this year’s) — he also currently designs and mixes sound for the Grammys, Oscars, and presidential inaugurations.

One hundred-eighty countries will be watching this Sunday’s Super Bowl 52 between the Eagles and the Patriots. And, while everyone will be cheering on the players, the halftime performer, and the person who sings the national anthem, we often forget there is someone, unseen, who has spent months making sure you can hear every word and note. Baltzell talks with The Verge to give an insider’s look at how to make sure everything is heard without a hitch during one of the country’s most-anticipated broadcasts.

The Verge: How does the planning process begin with the Super Bowl halftime show?

Baltzell: There are two sides to that. The first is the site itself, the nuts and bolts. That starts in June for a Super Bowl in February. We make our first site survey in June [to] take a look at the big picture of the space. Where are we going to put broadcast trucks? Where are we going to put the audio compound where we build all the speakers? And we have to have trailers that are offices. We figure out inside the stadium where we’re going to ultimately put amplifier racks where we can store the speaker carts. We have to put the whole show on the field in six minutes.

Hold on. Six minutes?

Six. Minutes. From the end of the first half of a football game to the beginning of the halftime show is between six and seven minutes. That’s how much time you get to build the entire show on the field.

Then we’ll make two more trips out there and say, “How do we get wires?” We’ll drill big holes in the concrete floors and run cable trays where we need them for the Super Bowl, which isn’t like a regular game. [At] a regular game, there are two TV trucks for the two broadcasters they invited. We have 180 countries and 25 languages. So we have to accommodate 25 announcers broadcasting in Japanese and Russian and German and so on.

So, we use any part of the PA system that’s already in there and we have to work with their engineers that work in-house and create a way for me to take control of all their speakers with a push of a button for the pre-game, which is the national anthem, and the halftime show, and then, of course, it goes back to their control for the game itself, the referees, announcements, all that.

I heard that the performers lip-sync during the halftime show. Is that true?

No. Most of the lead vocals are live. Background vocals are always pre-recorded. Most of the band is pre-recorded. The Rolling Stones was an exception in Detroit. They refused to do any backing tracks so that was all live. So we still had to push their stage out in six minutes, and then also connect it all up with actual drum mics, guitar mics, keyboard inputs, the whole thing. But Madonna, for example, was just live vocal, and everything else was tracked. Katy Perry, same thing, but even some of her vocals, the really difficult parts when she was flying, she got really nervous so she pre-recorded and we switched to that.

Ah, so you gauge on the fly when someone needs a little support.

Yes. Most of the solos, like all of Prince, his guitar playing was all live. With Springsteen the sax was live, the guitar was live. Some people just aren’t good lip-syncers and some are. Madonna is spot on, probably because she’s been doing it for so long. But some, like Bruce Springsteen, he changes the way he sings it each time he sings it. So it would be embarrassing if you had a close up of him and it didn’t match.

But do they all pre-record the vocals just in case?

Yes. We call it a protection track. We always record a protection track for the halftime lead singer and for the national anthem. The big notorious one that I play for everybody was my very first Super Bowl in 1998. The national anthem was done by Jewel. She was brand-new then. She got so nervous because she was a kid and she’d just been nominated for a Grammy. The Super Bowl was in San Diego, she’s from San Diego, so they thought oh, we’ll get a local.

The morning of the Super Bowl she said, “I don’t want to sing live.” But, it starts acapella and the music comes in later on. And we said, “Well then if you’re not going to sing live then you’re going to wear in-ear monitors because it starts acapella.” And she said, “Nope. I can’t do this. I’m not used to it. Just give me the speaker wedges and give me the count off in the wedges you know like, “One, two, oh say can you see.”

We said, “When they say ‘Ladies and gentlemen, Jewel’ the crowd is going to scream.” With in-ear monitors, you’re going to hear it... but with speakers, whatever you set in an empty stadium, you might not hear it when that crowd screams. So watch here, she misses the first word because she couldn’t hear.

Did anyone notice?

Oh god yes. We took a ton of heat. The only other time in my 20 years something happened with the national anthem was when Christina Aguilera forgot the words. She sang the first verse twice and the same thing, that’s all [the press] talked about.

So to finish up about the timeline, the creative starts in September. In September they hire the halftime talent... because we need that many months to build these gigantic sets and visions of what the show is going to be. The creative team gets together with the artists, producers, and the NFL and they all sit down. Mostly, the producers, the NFL just kind of signs off yes or no. So they start developing the show. It’s only 14 minutes so you have to condense your best, biggest songs. No ballads. You have to put them all together into a high energy 14 minutes.

Then once you have the songs picked you have to think about scenery and cast. Are kids going to be everywhere, do you want a marching band? What exactly do we want to include to make this thing bigger than last year? So that process takes place over September, October, and November. By December that has to be locked down because they have to build these big elaborate stages that are built just for the Super Bowl. Then they just throw them out afterward. Each one is custom. It’s crazy how much money they can spend. The artists don’t get paid but the production costs are covered, which can be in excess of 10 million dollars.

What were some of the more challenging halftime shows you’ve worked on?

The most difficult was Prince. The way I do the Super Bowls are these speaker carts. There’s 16 to 18 of them and they’re placed in this big ring on the sidelines aimed up in the stands very carefully. We spend days tuning them and optimizing what energy gets to the top.

I use these for each Super Bowl and I don’t change the type based on the artist. That’s not that critical. But I will make changes based on the stadium. Some might be taller and some are flatter. Some I can get my speakers further away, and some they’re right up against the sidelines, which makes it trickier. But, Prince was the most difficult because he’s nuts about sound. His team didn’t want to do the speaker carts. They said, “We’re not doing that. That doesn’t work for Prince. We have done stadiums for decades. We know how to do it. You need these big giant speaker arrays.”

Sure, this is the most efficient way to do it with a big tower [but] we have to start in the parking lot and in six minutes we have to get all of this equipment onto the field in position plugged in, in the dark, checked and working. So I know what you do when you have two days to load a show in for a stadium. This is a football game. The Super Bowl is a big deal, even for U2 and Bruce Springsteen and Prince. And they get nervous [because] this is in 180 countries. Even though it’s only 14 minutes, the history of people that have done it have set a bar that is very high. You want to have your best show so in the future when they talk about the 10 best Super Bowl shows it includes you.

So I had to talk them off the ledge. Like, I know what you want to do and what you normally do. Keep in mind that this first and foremost is a championship football game. Also, I don’t even know if I can design something that can fit through the stadium tunnel, which, by the way, is 10-feet wide maximum.

All the sound has to go through the tunnel the players go through?

That’s the only way you’re going to get it on the field. Everything comes through the tunnels. Some have two, some have three, some have four.

That’s not a lot of space.

Right. For something as big as Prince wanted, the legs alone would be way bigger than that so it would all have to collapse down, go through the tunnel, get the legs put out into position, and get this tower set up.

Would towers like that actually sound better if it could theoretically be done?

It would sound as good. Maybe a little bit better. Certainly, you could use fewer speakers because there’s efficiency when you couple them together. When you break them into little piles of four or five on the carts you sacrifice a lot. So I do agree with that part, but, this is a TV show. How are you going to shoot this with cameras when you have six big towers with speaker clusters?

And people at the Super Bowl are looking at the show from 360 degrees.

Correct! Everybody sitting down low would be looking through a tower at his performance. My little speakers are about six feet tall; 99.99 percent of the people with my design are going to see the entire show. Nobody could assure me that we could do towers and make it work and I was starting to run out of time.

So I went back to the Prince guys and I had to do a whole acoustic modeling study where you take each frequency band and what the sound pressure level would be. I had to prove to Prince’s team that it would work, that I could deliver the same sound pressure level that the big towers could do. Also, the towers would wreck the field. They would weigh 5,000 pounds and leave marks on the field. He was worried it wouldn’t have impact. That these little piles were bullshit.

Just particulars they’ve never had to consider.

Yeah. When you’re trying to squeeze onto a championship football game, get your show on, and get it off, it’s a whole different thing.

How many speakers do you use in total for the halftime show?

It’s probably 120 of those full-size line array, and 36 subwoofers. Sometimes I’ll put additional piles of subwoofers for effects just along the sideline that aren’t traveling out on the carts. Like if there’s a big thunder or earthquake effect built into the tracks and they really want the seats to rumble.

Thirty-six subwoofers doesn’t seem like a lot to me for an entire stadium.

It isn’t. You’re right, but in a stadium, the low end tends to hang around a lot for a while.

How much of the artist’s sound are you controlling during the performance?

A lot. For most of them, we get the tracks. It’s about 16 to 24 tracks from a Pro Tools session. And they will be divided into drums, sometimes kick drum, snare drum, bass, guitars, keyboards, special effects, background vocals in stereo, and a protection lead track. Some of them we go back and forth on how much of the protection track is fed in. I’m mixing quite a lot even in just those 14 minutes.

Some of them probably favor certain styles of mixing.

Yes. Some of the engineers want to tuck the lead vocal lower because they do it every single night with the same songs. But a lot of this audience are big football fans and you need $15,000 dollars to go to the Super Bowl. They’re older guys usually, who might not know every Beyoncé lyric. So I try to convince them that we need to lean on the vocal just a little bit for those who don’t know the lyrics and actually want to hear what the lyric is.

What was your favorite Super Bowl halftime show to do?

Prince.

Even though you said he was difficult to work with?

Yes. It was torrential rain. We built the beginning of the show in CG. We showed the outside of the stadium with these big lightning bolts cracking down and then the ribbon board around the stadium was all electrified. Then, out of the smoke comes Prince like he was shot down out of the heavens to start the show. That was the concept. We took exterior shots of stadiums and they built this big lightning effect. Turns out on the night of the show it really was a lightning storm.

Does rain affect the speakers at all?

The rain doesn’t affect the speakers. The real functioning amps with the mics on them are under the stage. But it affects the mixing board! Guys were holding plastic over my mixing board because I’m outside so I can hear everything.

You’re not covered normally?

No, because there are people behind me and they don’t want their sight lines blocked. We made a quick wooden structure out of boards and wrapped it in visqueen. We kept it down low during the game, but during the halftime show I had two guys hold it up over me so I could see but the rain wouldn’t hit the [mixing board]. Otherwise, we’re in trouble. It’s the end of the show for us.

So the fact that he performed in this torrential rainstorm, I give him so much credit. That’s my favorite halftime show because it was almost prophetic. We see this lightning bolt start the show, and it was a lightning storm on the night of the show, and the fact that he said “Rain? I don’t care. Bring it on.”

And, all the speakers survived, everything worked, the console didn’t stop.

What’s your process to tune a system for a stadium?

What does it mean to tune an audio system? The goal of tuning a sound system is to have the same quality of sound no matter where you are within a venue. Essentially, trying to match the acoustic characteristics of a space with the speakers you are using. Ideally, you want to get as close as possible to having the same levels, frequency response, and clarity for every seat. Certain things that can be changed to affect sound include location and direction of speakers, levels, driver polarity, phase and delay, and filter / EQ. Audio engineers often use software like SMAART to help them analyze the frequency responses throughout a venue and make changes accordingly. Often, pink noise is blasted through speakers, which is then picked up by reference mics set up at different spots within a venue, and measured using the software to make adjustments.

Well, there are two types of stadiums and they’re very different. One is the open-air stadium, which most of them are. So all I have to deal with are my speaker carts on the field and maybe speakers that are in suites. Sometimes in a nicer stadium, the suites have speakers built into the ceiling. I’ll take those over for the halftime show and I have to send a signal to them.

In those open-air stadiums, it’s pretty basic. I’ll take a quadrant and set up 10 microphones near the field, then at the top of the lower bowl, then the next level up, and then the next level up, and take a sampling vertically. It takes me usually eight hours... sometimes two sessions. I book two evening sessions from 6 at night to 2 in the morning when everybody else is done working on the field. It’s got to be quiet. If it rains, or it’s very windy, I can’t make measurements. It’s a bust.

So you’re basically measuring the ambient noise in different portions of the stadium.

Yes, I energize the sound system with pink noise and then I just keep making measurement after measurement because this mic in front is gonna be indicative of all the front row seats. Next one farther up, it’s going to be the measure for that whole section. So I’m actually taking snapshots of each of the audience areas and optimizing it.

The problem is if you make it perfect for the people at the top you probably are going to compromise the people in the front rows. So that’s just a juggling act. You make it the best that you can for all of the seats and that’s the best you can do. You take a tolerance, plus or minus 3dB in every seat.

What is pink noise? There’s a reason why pink noise is used to calibrate audio equipment. Pink noise is closer to how the human ear and brain perceive sound, which is in terms of octaves, and not absolute frequency. White noise has equal power per hertz across all frequencies, while the power per hertz in pink noise decreases as the frequency increases. This means white noise sounds more hissy and treble-laden in higher pitches, and lower frequencies in pink noise are louder and have more power. For example, the difference between 100Hz and 200Hz, and 5,000Hz and 10,000Hz are both one octave. But, in terms of frequencies, the difference between 100Hz and 200Hz is 100Hz, while the difference between 5,000Hz and 10,000Hz is 5,000Hz. The relative relationship is the same, but the actual mathematical difference is quite large. With white noise, all these frequencies contribute to the overall level in octave change, but with pink noise, every octave contains the same amount of energy. Because pink noise distributes energy according to how humans hear, in octaves, it’s used to measure how the sound is perceived at different places within a venue.

Now, it gets really tricky when we do a stadium that’s covered. In the NFL, they encourage the teams to build new stadiums. If you build a new stadium, you go right to the top of the list for Super Bowls. That’s a big incentive for the owners to spend the money and put in a new billion dollar stadium. The only way you’re going to get the Super Bowl in the north is to build a stadium with a roof.

If you have a roof, then this is a fundamentally different stadium for me because you’re already going to have a sound system. But, the sound system isn’t going to be up to par for a Super Bowl halftime show. Why should it be? It’s designed for announcements. But, I can still make use of parts of them. So there are my speaker clusters on the field and they’re covering this lower portion. Then these hanging clusters that are already in the stadium, I use parts of them to cover this middle seating section and the very upper deck seating. All of the speakers are already in place and they’re high quality because it’s a new stadium. They’re much closer to the seats. They’re not going to reverberate by shooting sound all the way up there from the field.

That’s the approach to stadiums like this, but it takes a very long time because I have to go and measure every one of the speakers because I don’t trust that everything is correct. There’s layers and layers of signal processing before it goes from the mixing board, whether it’s mine or the stadium one, to actually coming out of the speaker. There are lots of electronics. So I’ll go physically listen to every single one of those in-house speakers because I always want it perfect. I’ll take at least two full days to do the tuning on a system that’s a joint venture between my portable system and the in-house system.

What speakers do you prefer to use?

The best speakers out there now are the JBL VTX speakers and the L Acoustics K1s. Those two are both great speakers.

What’s the most challenging space you’ve done audio for?

The worst show I’ve ever done was the Super Bowl in 2004 at Reliant Stadium in Houston. The stadium is just unworkable because it’s got a roof and the seats stop and the sidewalls go up for another 20 to 30 feet and then there’s the metal roof and there’s no treatment [for audio] anywhere. So it’s notorious that tours don’t want to stop there.

Really, they avoid it?

Yes. Well, they do a show once and half the audience asks for the money back because you can’t understand a single lyric at a concert.

It is because the sound is bouncing around everywhere?

Yes. It’s reflections. Now, it’s a football stadium so the owners deliberately do not put acoustic treatment because in football, the home crowd screams at the inappropriate times for the opponent. On a critical play, when they’re trying to talk to each other, they can’t hear at all and they make mistakes. It’s actually a big advantage for the home team to scream ungodly at certain key moments in the game where they’re trying to communicate. So they deliberately build the stadiums without acoustic treatment.

Most of the smarter owners split the difference and they go, “Okay well don’t treat all the surfaces with acoustical material but let’s have a concert where people want to come.” Or they do rodeos in the South in stadiums like Houston and Dallas. You should at least be able to hear the announcer, and understand them. But this one, it wasn’t like that.

What is acoustic treatment? Once sound leaves a speaker, any number of things can affect it, including the dimension of the space, and the material of the surfaces it bounces off of. Stadiums in particular are often designed with hard and very reflective materials, like glass and concrete, and are large, open spaces with high ceilings. In short, they’re not designed for optimal acoustics. Sound will ping-pong around, causing excessive reverb trails which muddies what you hear, and makes speech hard to understand. Acoustic treatment, whether it’s foam wedges or suspended baffles, are used to absorb excessive acoustic energy. Even coverage of just 10 to 20 percent can help bring the reverb trail time down to one second, meaning there is an increase in audio clarity and intelligibility. This not only makes the stadium “sound better” but can also help isolate the sound and prevent it from leaking outside of the main arena.

So it’s not that they’re ignoring that part of the design process with football stadiums, they intentionally skip it?

Both. It’s sometimes ignorance, and it’s money because the final thing when you build a new stadium or arena [is] acoustic treatment. Everything else is finished. By then you’re over budget and out of money and say, “We’ll come back and visit that in five years let me make some money on the investment first.”

That Super Bowl in 2004, I wanted to fly the speakers. I had a solution, but they wouldn’t let me fly the speakers because they were blocking big brand-new scoreboards. I ran around with my design, I kept trying to move and change them and minimize the number of seats that would see my speakers blocking them and they said I couldn’t do it. So I did it with speakers on the ground and it was a disaster like I knew it would be. And the only reason I survived to tell the story is because that was the Super Bowl where Janet Jackson’s boob came out.

Are you serious?

So all the e-mails that the sound was the worst, the NFL could care less about that because they were doing such damage control over Nipplegate.

Most of the jobs I do now are in decent arenas that have acoustic treatment. The arenas usually do because they’re primarily for concerts or basketball, or hockey where it’s not the same in terms of the screaming crowd.

It seems, even down to the smallest level in say, nightclubs, that the quality of sound and where speakers are placed is the last thing owners worry about but it’s the first thing people complain about.

Yes, that’s exactly right. When I argue with say, the Oscars, about where I want to put my speakers, I get they want to have a big, clear shot of the work the art director did. But then they just keep wanting me to be higher and higher, further out of the way, so that they can do these other lighting effects on the scene. At the end of the day, if every light went out in the show except for a follow spot, we would still do the show and people would sit at home and watch Meryl Streep walk out and announce the winner, Jack Nicholson get up and get his Oscar, nobody would leave. [But] if the audio went out, there is nothing. The message is the audio message. Not how well it’s lit, and not how spectacular the scenery is.

Is there a higher level of expected quality in the room when it comes to a music awards show versus something like the Super Bowl?

Sure. And it’s tricky because you’re serving so many masters on a show like the Grammys. The hip-hop acts want it incredibly loud because they want it like their shows, which is stupid loud, but they feel like that’s the way the energy communicates, with subwoofers out of control and all that. Then you go to a classical piece that sounds ridiculous at that level. So we’re trying to get some thread of consistency.

Beyoncé did that on the Country Music Awards. The Dixie Chicks did a remake of one of her songs and she heard it and loved it. So she reached out to the Country Music Association said, “You know what would be really cool is if I do a mash-up with the Dixie Chicks. Them doing my song the way they’re doing it with fiddles and banjos and stuff and then I come out and kick it my way.” Of course, you’re not saying no to Beyoncé.

Then she tried to hijack the show. I’ve done that show for 28 years. Country guys are way chill. The whole show kind of sits in a pocket. Once in a while, you get Keith Urban or somebody who wants to be a little louder, but by and large, they leave me alone. And then Beyoncé’s camp shows up and all they kept doing is saying it needed to be louder than what we’ve done for almost 30 years. The producers came over to me and said, “Just turn it up.” That’s what I did because otherwise, she won’t leave. You know she’ll stay all night saying it’s still not loud enough.

So where did you have the levels during her actual performance?

Exactly where all the other acts were. Who’s complaining about the sound during the show? No one. It’s so smooth. Did anybody but her want that? No. Just her. But she comes in with that kind of tour de force.