Billed as Outkast by Letterman’s intro, The Love Below-emboldened André 3000 takes the lead, featuring a Big Boi verse in a 2004 "Late Show" performance that’s basically a re-enactment of the "Roses" video. Shaffer gets a saloon-ish little intro on keys but the best part is maybe the Dungeon Family theater troupe mugging to the camera at the end. Eleven years and something like five Kanye Wests ago, West gave his likely first "Late Show" appearance, performing "All Falls Down" with a fraction of the live showmanship he would later possess. Look closely and you’ll spot a goateed John Legend as a pre-stardom sideman. Also, big shout out to the College Dropout mascot bear just vibin’ out in the background.

It’s not easy to find decent performances from Letterman’s "Late Night" days on YouTube. Often, the video/audio quality is extremely poor because, hey, the '80s. But these performances—if only for the ruthless diversity of which they were booked—are treasures. "Late Night" gave U.K. pop group the La’s its U.S. network television debut in 1991 with "There She Goes". Murmur-era R.E.M. made its American TV debut with "Radio Free Europe" in 1983. That same year, "Late Night" viewers got a sneak peak at Talking Heads’ Stop Making Sense-era live show with its performance of "I Zimbra". Sammy Davis Jr. performed "I Can’t Get Started" with Shaffer and his band’s "Late Night" incarnation the World's Most Dangerous Band in 1989. It may well be Davis’ final TV performance a year before his death. Recent Rock'N'Roll Hall of Fame inductee Joan Jett appeared on "Late Night" at the peak of her 1987 fame, with a badass cover of the Modern Lovers’ "Roadrunner". Yma Sumac’s performance—the "Peruvian nightingale," as Letterman called her– that same year was emblematic of the kind of fiercely varied music guests "Late Night" would book. Bruce Springsteen and Shaffer closed out Letterman’s final "Late Night" show in 1993, going way over time as a screw you to the network that passed him over in favor of Jay Leno for the Tonight Show. "You’re movin’ on up!" Springsteen yelps to Shaffer at one point.

No frequent musical guest embodied Letterman’s eccentric side more than Tom Waits. Waits and Letterman’s rapport goes even further back than the oldest of these clips: Waits performing selections from Swordfishtrombones ("Frank’s Wild Years") and Heartattack and Vine ("On the Nickel"). Most recently, he performed "Chicago" off 2011’s Bad as Me on a 2012 episode of the "Late Show". Waits has appeared on Letterman’s talk shows at least seven times and he’ll appear on Letterman’s final "Late Show" episode.

"The Late Show"’s special themed weeks of music made for moments you couldn’t see anywhere else. Last year’s Beatles Week—celebrating the Fab Four’s 1964 American TV breakthrough at Letterman’s Ed Sullivan Theater—was stellar, peaking with McCartney performing to a packed Manhattan street corner atop the Sullivan Theater sign. Also notable was the Flaming Lips and Sean Lennon’s take on "Lucy in the Sky With Diamonds", a particularly bizarre performance for network television.

Speaking of bizarre, how fucking trippy was that Hatsune Miku performance last year? For viewers of Hatsune Miku’s "Late Show" performance last year, it was screen-on-screen meta, a possible glimpse into a future where television audiences watch talk-show performers on a screen, on a screen. After the holographic pop star waved goodbye to Letterman and digitally disintegrated, the host cracked wise, "It’s like being on Willie Nelson’s bus."

Feist’s 2007 "Late Show" performance of "1234" was a classic moment as soon as it hit the music blogosphere, with half of the year’s indie vocalists—including members of Broken Social Scene, Grizzly Bear, New Pornographers, the National, and Mates of State serving as her backing choir.

Leave it up to rowdy live acts Rage Against the Machine and the Beastie Boys to take their performances outside the confines of the Sullivan Theater. The Beasties’ 2004 performance was as fun and breezy—entering from subway stairs and Manhattan streets—as Rage’s 1999 performance was aggro and chaotic. Plus, the band’s stage and moshpit blocked Broadway at 53rd Street. Jay Z and Eminem played it smarter, taking their performance to the theater’s roof for a rare performance of "Renegade" off The Blueprint.

__

"My Favorite Band Playing My Favorite Song"__

Letterman summoned Foo Fighters mid-tour to come play a 2000 welcome back show after his quadruple-bypass heart surgery, featuring Shaffer on keys. Letterman called the band’s performance of "Everlong", "my favorite band playing my favorite song."

"I’m no beginner," Letterman wisecracks at the end of this supercut of him admiring musical guests’ instruments. After watching this, it wouldn’t seem shocking if Dave took up an instrument of his own as a hobby in retirement. Who knew that Dave was a gearhead?