Kendrick Lamar’s major-label albums play out like Spike Lee films in miniature. In both artists’ worlds, the stakes are unbearably high, the characters’ motives are unclear, and morality is knotty, but there is a central force you can feel steering every moment. The "Good and Bad Hair" musical routine from Lee’s 1988 feature School Daze depicted black women grappling with colorism and exclusionary standards of American beauty. Mookie’s climactic window smash in 1989’s Do the Right Thing plunged its characters into fiery bedlam, quietly prophesying the coming L.A. riots in the process. In these moments, you could feel the director speaking to you directly through his characters and their trajectories. Lamar’s records, while crowded with conflicting ideas and arguing voices, have a similar sense of a guiding hand at work.

Lamar’s new album, To Pimp a Butterfly, doesn’t explicitly bill itself as a movie like good kid, m.A.A.d city did, but the network of interlocking dramas explored here feels filmic nonetheless, and a variety of characters appear across the album’s expanse. The opener, "Wesley’s Theory", turns the downfall of action-star-turned-convicted-tax-dodger Wesley Snipes into a kind of Faustian parable. Snoop drops by on "Institutionalized"; Dre himself phones in on "Wesley". The mood is wry, theatrical, chaotic, ironic, and mournful, often all at once: On "For Free? (Interlude)" an impatient woman ticks off a laundry list of material demands before Kendrick snaps back that "This dick ain’t free!" and thunders through a history of black oppression, spoken-word style, as if to say, "This money you crave, it’s blood money." The album is dotted with surreal grace notes, like a parable: God appears in the guise of a homeless man in "How Much a Dollar Cost", and closer "Mortal Man" ends on a lengthy, unnerving fever-dream interview with the ghost of 2Pac.

The music, meanwhile, follows a long line of genre-busting freakouts (The Roots’ Phrenology, Common’s Electric Circus, Q-Tip’s Kamaal the Abstract, André 3000’s The Love Below) in kicking at the confines of rap music presentation. There’s half a jazz band present at all times; pianist Robert Glasper, producer/sax player Terrace Martin and bass wizard Thundercat give Butterfly a loose, fluid undertow every bit as tempestuous and unpredictable as the army of flows at Kendrick’s disposal. The rapper’s branching out, too, exploding into spastic slam poetry on "For Free?", switching from shouty gymnastics to drunken sobs on "u" and even effecting the lilt of a caring mother on "You Ain’t Gotta Lie (Momma Said)". It turns out Kendrick’s new direction was every direction at once.

Despite all this, he’s still toying with a narrative on the sly: Just beneath the surface lies a messianic yarn about avoiding the wiles of a sultry girl named Lucy who’s secretly a physical manifestation of the devil. Kendrick refuses to dole out blame without accepting any, however, and on the chaotic free jazz excursion "u" he turns a mirror on himself, screaming "Loving you is complicated!" and suggesting his fame hasn’t helped his loved ones back home. Kendrick’s criticisms, as they did on good kid, come with powerful, self-imposed challenges. As Bilal quips on the chorus to "Institutionalized": "Shit don’t change until you get up and wash your ass, nigga."

Kendrick’s principle of personal responsibility has treaded dangerously close to respectability politics lately, especially after a prickly remark about the Mike Brown shooting in a recent Billboard interview that seemed to pin the death on the victim, but To Pimp a Butterfly avoids that trap. (Mostly.) "Complexion (A Zulu Love)" is a tender note of appreciation for women of all skin tones with help from North Carolina rapper Rapsody (whose slickly referential guest verse contains a nod to "Good and Bad Hair"). This is an album about tiny quality of life improvements to be made in the face of seemingly insurmountable odds. It might not be the message we want in a year where systemic police and judicial inequality have cost many the ultimate price, but that doesn’t bankrupt it of value.

To Pimp a Butterfly pivots on the polarizing lead single, "i". Upon release last autumn, the sunny soul pep talk came off lightweight and glib. When it appears deep in the back end of Butterfly, though, "i" plays less like the jingle we heard last year and more like the beating heart of the matter. To push the point, the album opts for a live-sounding mix that ditches out midway through, giving way to a speech from the rapper himself. In tone, the speech is not unlike the legendary 1968 concert where James Brown waved off security and personally held off a Boston audience’s fury after news broke that Dr. Martin Luther King Jr. had been assassinated. "How many niggas we done lost, bro?" Kendrick shouts over the crowd. "It shouldn’t be shit for us to come out here and appreciate the little bit of life we got left." Underneath the tragedy and adversity, To Pimp a Butterfly is a celebration of the audacity to wake up each morning to try to be better, knowing it could all end in a second, for no reason at all.