Madlib is running late—not just a little late, but late late. But there’s no way to check his proximity to Rappcats, the Los Angeles record store that’s also the headquarters for the producer’s record imprint, Madlib Invazion; the prolific hip-hop producer doesn’t have a cell phone. For about an hour, I just have to trust that he actually left the house and will show up at some point.

This isn’t a bad place to wait if you’re a music nerd. You’re not gonna find these albums in Urban Outfitters: There are rare jazz records like Mulatu Astatke’s Mulatu of Ethiopia and Mtume Umoja Ensemble’s Alkebu-Lan - Land of the Blacks, and avant-orchestral records like David Axelrod’s Songs of Experience. Around the corner is a bigger room with rows of esoteric Brazilian LPs. There’s a good chance that Madlib has used some of the records in Rappcats to make his own beats across the last 20 years: His signature sample-based style combines disparate vocal clips and staggered drums into an open-ended sound that feels off-centered yet somehow on time. His music feels vintage, like a blend of Blaxploitation flicks, early-’70s spiritual jazz, and global music from old record shops in faraway lands, and he has created so much of it that he’s launching a subscription service to give it away.

Yet in the late-’90s, when Madlib was first gaining steam as a producer, his style wasn’t as popular. It was the era of Puff Daddy and Bad Boy Records, and even the toughest street anthems felt glossy and gleaming. Madlib’s beats, on the other hand, have always been dirty and spaced-out—theme music for the kind of kids who like rap, funk, and doing shrooms on the weekends. Back then, producer James “J Dilla” Yancey was perhaps the only hip-hop producer who could navigate the mainstream and underground, crafting hypnotic tracks for Common, Erykah Badu, and A Tribe Called Quest while retaining an air of mystery. Dilla found solace in the work; making great beats was more important than celebrity. Madlib studied Dilla and carved a career in his image. The two joined forces in 2002 and started trading instrumentals for a joint album called Champion Sound that was released on L.A.-based Stones Throw Records a year later. In 2004, the label released another album that Madlib had been working on—Madvillainy, with the enigmatic rapper MF DOOM. The record was dubbed an instant classic by critics and it remains Madlib’s most popular album to date.

Still, there’s a notion that his beats are a little too weird for casual listeners, and too challenging for rappers not named DOOM, Yasiin Bey, Talib Kweli, or Kendrick Lamar. Turns out Freddie Gibbs, a rugged street rapper from Gary, Indiana, was up for the task. The two met through Gibbs’ manager, who wanted to see if Freddie could rap over Madlib’s instrumentals; in turn, the producer sent eight CDs worth of beats. The rapper admits he was a bit apprehensive about working with Madlib at first. “For me to get with Madlib, I can’t go weird. I’ve got to keep it gangsta,” Gibbs told Red Bull Music Academy in 2014. Ultimately, though, he saw it as a way to broaden his fan base, and Madlib’s, too: “I brought the streets to the Madlib fans.”

Indeed, there was an element of surprise with their 2014 LP, Piñata, the duo’s first joint album. But now the duo are set to release their new album, Bandana, in June, to a public that’s much more familiar with Madlib’s sound than ever before. The veteran Oxnard, California producer’s influence is catching on with a new generation of beat-heads like Knxwledge and Earl Sweatshirt, as well as with young jazz artists including Makaya McCraven and Standing on the Corner.