Everyone’s heard the famous maxim, generally accredited to legendary music producer Brian Eno: while the Velvet Underground’s debut, The Velvet Underground & Nico, sold a paltry 30,000 copies upon release in 1967, every person who bought one of those 30,000 copies started a band. Though a slight exaggeration, the line is a testament to the album’s far-reaching influence trumping its commercial failure. Lou Reed, John Cale, Sterling Morrison, and Maureen Tucker merged raw rock and roll with musique concrète and the avant-garde to create an untamed and menacing sound that perfectly underscored their poetic tales of drug deals, sadomasochistic sex and other snapshots of the urban underworld.

Emboldened by manager and patron Andy Warhol—who linked them up with featured vocalist, Nico—the Velvet Underground’s brand of leather-clad Lower East Side cool emerged onto vinyl with all of its grit and daring intact, serving as a beacon to generations of young artists unwilling to conform to pop music niceties. Decades ahead of its time, it planted the seeds for punk, glam, goth, and a host of others genres to flourish.

Get push notifications with news, features and more.

In honor of the groundbreaking album’s 50th anniversary this month, Cale spoke to PEOPLE about his memories recording The Velvet Underground & Nico. Read on for his exclusive track by track commentary.

Image zoom

“That happened one Sunday morning at Lou’s friend’s house. We were out boozing and running around the Lower East Side and Lou suddenly had a great idea. He said, ‘Hey, I’ve got a friend who lives around the corner, let’s go see him.’ And it was like three o’clock [in the morning]. I said, ‘Yeah, ok!’ We ran over, and he had a harmonium in the corner of his living room. Generally what we did when we went anywhere, we just zeroed in on the instruments and started playing. It was kind of manic—anywhere you’d go, if you saw an instrument you’d just pick it up and start playing. Lou saw the guitar, I saw the harmonium, and off we went writing ‘Sunday Morning.’

“I remember the first gigs we did with just him and me —I had a recorder and a viola, and he had an acoustic guitar. We’d go sit on the sidewalk outside the Baby Grand [bar] up in Harlem on 125th and see if we could make some money. Every time we got moved on the cop always had a suggestion of where we should go. ‘Try 75th on Broadway! That’s a good spot.’ So we’d go down there and make a little bit more money.”

“Andy saw that Lou was moping around the Factory [Andy Warhol’s studio], and he gave him a list of words. He said, ‘Here are 14 words, go write songs with these words.’ And Lou was never happier. He had a task in hand and he sat down. That was a lot of fun for him. We had our own thing going [before Warhol] but he showed up and was more of a guy helping us not forget who we were. He would always say things like, ‘Tell Lou, don’t forget to put little swear words in that song.’ He was reminding us of who we really were. And he didn’t have to say very much to do that, he could just be around and it would be like that because he’d notice what was going on around you. He’d notice the art that was going on. We didn’t understand it. We were just flabbergasted by it, but we loved it at the same time.”

“Lou wrote ‘Venus in Furs’ while we were playing around when we met at Pickwick [Records]. He told me that the label wouldn’t let him record all of the songs he really wanted to do. That sort of pissed me off. I asked him what they were and he showed them to me. He’d play them on acoustic guitar and I said, ‘These are rock songs. These can be really big and orchestral if you want them to be.’ Then I said, ‘Let’s just do it ourselves, let’s get our own label and get our own recording situation—not here.’ So we put a band together. That was a signature number for us.”

“’Run Run Run’ was always the first number to do, because it was up-tempo and got everybody going. It was great.”

“We had made the arrangement for ‘All Tomorrow’s Parties’ before Nico came along. That was the result of a year of weekend work—sitting around on the weekend and just playing and playing and playing and playing until you slowly gradually moved out of the folk music side of things.

The record was all done with just us playing, there were no effects involved in that. We tried a version where Nico doubles her vocal, but the vocal just became too heavy. “But the noise of putting paper clips in between the strings of the piano gave it a ring that made it a little more orchestral. We were trying to make orchestral stuff. We were trying to be Phil Spector, really. Phil Spector would mix Wagnerian orchestrations with R&B. That was a really unique combination. We had the drone. The viola wasn’t wasn’t used, so the piano became the drone. Whenever we’d try to do something, we’d always try to find something that would be the drone.”

“’Heroin’ is really special. At that point it was kind of a resident of the band because it was so important to the set. Everybody had heard of it. It was one of the attractions of the set, apart from the attitude of the band. Whatever we were doing, we were trying to get more people in the door. But we had a lot of different ideas of how to do that. My idea of getting people in the door was doing something experimental. I tried to get Lou to see that we don’t have to do the same set every night. That was a direct result of all these club owners in New York saying, ‘You’ve got to play one or two songs that are in the top 10, otherwise you won’t get a gig.’ We said, ‘We’re not doing that. We’ve got our own numbers.’ And until Andy showed up we barely got any venues at all. I thought, ‘One selling point that we can have is that we never do the same set twice.’ We improvised songs every night, which was rather fun with Lou. I said, ‘We can give Dylan a run for his money if we just improvise every night, because our lyrics are just as good.’”

“That was probably the easiest one, with a soul riff from Marvin Gaye. You could hear Lou’s time at Pickwick writing pop songs.”

“Lou was writing songs for Nico, and some of the best songs he’d written were written for her. That was one of them. She was becoming more interested at that time in being her own songwriter. She’d sit down and write poetry, and to her it was in a foreign language. She was trying to find poetic language in a foreign language, because she was German-speaking. But she was determined, she bought a harmonium for herself and was really single-minded about doing all that.”

“’Black Angel Death Song’ no one ever got. It would go over everybody’s head. But in general, I think what people responded to, even if they didn’t understand it, was the energy that we had. Lou and I, we knew we could play these songs, but we were never genuflecting to each other about how to play them. The performances were more done as a bald statement of fact: ‘This is what we do. Whether you like it or not, we don’t care.’ And we didn’t care whether we played it well. We really were on top of that. And we were excited about what we were doing. And then the band gets a record deal right away? Come on, that’s great. Really exciting.”

“’European Son’ in my mind was purely for improvisation. Whenever we played anywhere, we couldn’t wait to get to the point where we’d improvise and do ‘European Son.’ It was always different. That was the fun part for us, doing those improvisations. And those improvisations would really get the best of us in the end, because they’d go on and on and on and on. We’d be up there for an hour just improvising before we’d even done a song! In San Diego we did that. That’s kind of the rep we had when we got to San Francisco and L.A.