For many of the millions of viewers worldwide who will tune in for Super Bowl LI on Sunday, the halftime show featuring Lady Gaga will be a bigger draw than the game.

It's hard to blame the casual NFL fans or the revelers who are watching only because they were invited to a Super Bowl party. The halftime shows -- with their pyrotechnics, elaborate stage production, hordes of costumed dancers and megastar musicians -- have become can't-miss, cultural happenings of their own.

Younger viewers and international fans attracted to the sport during the NFL's global push in recent years might not realize it, but the halftime show wasn't always a big deal. In fact, for a long time, it was just another break in the action.

For instance, the first time Houston, the site of Sunday's game, hosted the Super Bowl was in 1973, and the halftime show featured the University of Texas marching band and Miss Texas playing the fiddle. Yes, the fiddle! That's a far cry from the next time it was in Houston, in 2004, when Justin Timberlake briefly exposed Janet Jackson's nipple in the infamous "wardrobe malfunction" that is probably remembered by more people than the actual game. (Reminder: The New England Patriots edged the Carolina Panthers 32-29.)

In the Super Bowl's early years, the halftime show was about filling time while fans and TV viewers went to the concession stands or their refrigerators to get more beer -- or took care of other necessities. As former NFL commissioner Pete Rozelle once said about halftime: "That's when everybody goes to the bathroom."

Beyonce, Chris Martin of Coldplay and Bruno Mars took center stage during halftime of last year's Super Bowl in Santa Clara, California. Robert Deutsch-USA TODAY Sports

Thus, marching bands and drill teams often were the main halftime performers during the Super Bowl's first two decades, like at a typical U.S. college football game. The Grambling State University band played at six Super Bowls -- or one less than New England quarterback Tom Brady will have after Sunday's game. Fortunately, Grambling was never cited for deflating the music.

Another frequent halftime act in those days was called Up With People. If you're not familiar with Up With People, it is an admirable and uplifting global education and music organization that has had more than 20,000 members and performed at the 1972 Munch Olympics as well as four Super Bowls (X, XIV, XVI and XX). Given how few people know about Up With People today, the group's lasting Super Bowl mark is similar to the Buffalo Bills and Minnesota Vikings, who also played at four Super Bowls -- but never won.

There were a few notable celebrities during the early halftimes -- or at least notable at the time. Ella Fitzgerald and Carol Channing sang at Super Bowl VI, Andy Williams at Super Bowl VII, and Mickey Rooney pranced with Mickey Mouse at Super Bowl XXI. Elvis Presley never appeared, but Elvis Presto, a strange mixture of music and magic tricks, entertained at Super XXIII. Yeah, you might have wanted to go to the bathroom during that show.

Bigger names began to hit the field in the 1990s, with New Kids on the Block at Super Bowl XXV and Gloria Estefan at XXVI. Halftime still wasn't what it is today, though. For instance, before Estefan singing briefly at Super Bowl XXVI, U.S. Olympians Dorothy Hamill and Brian Boitano figure skated and rode on snowmobiles during a celebration of Frosty the Snowman and cold winters. (Well, the game was played in Minnesota.)

Halftime changed radically the next year when Michael Jackson sang, moon-walked and danced along with 3,500 kids at Super Bowl XXVII. Fortunately, unlike with his sister Janet, MJ merely grabbed his crotch rather than exposed anything. He was a massive hit, and while there were a few odd halftime shows the next couple of years -- Indiana Jones fought snakes and villains to retrieve the Lombardi Trophy at Super Bowl XXIX -- music superstars eventually took over.

Lady Gaga, this year's halftime performer, is no stranger to the Super Bowl; she sang the national anthem before last year's game. Timothy A. Clary/AFP/Getty Images

Now halftime isn't about bathroom breaks. It's about watching some of music's greatest legends -- at least for the 12 to 13 minutes of performing rather than the time it takes to set up and take down the stages. Since 2002, the Super Bowl has presented the likes of Paul McCartney, the Rolling Stones, U2, Bruce Springsteen, Prince, Madonna, The Who, Coldplay and Beyoncé. Not even Brady -- or even his wife, Giselle -- can match that level of fame.

Oddly, the NFL doesn't pay the stars for the halftime show. They perform anyway because they benefit from the huge publicity and viewership. Madonna's performance five years ago at Super Bowl XLVI, in fact, drew higher ratings than the game itself.

That's a pretty good indication that most fans will make sure to have finished going to the bathroom and getting their beers in time to watch whatever spectacle Lady Gaga delivers.

Even if there's not another wardrobe malfunction.