As is only right for a man who packed such diversity into less than half a century, it is impossible to choose just one detail from the career of the Beastie Boy Adam Yauch that sums up why his death last week at the cruelly young age of 47 feels so especially sad.

There's his film company, Oscilloscope Laboratories, which was behind some of the best indie films since its inception in 2008, including Banksy's Exit Through the Gift Shop and the breathtaking documentary about Maurice Sendak, Tell Them Anything You Want. Or there's the very funny 30-minute movie he made last year in which he satirised his own image, Fight For Your Right (Revisited). Seth Rogen, Danny McBride and Elijah Wood play the Beastie Boys, circa 1987, having just left the video for (You Gotta) Fight For Your Right (To Party) and acting as bratty and hedonistic as the Beasties were thought to do then and, undoubtedly, did.

Then there was the letter he wrote to the New York Times in 2004 after it ran a poor review of the video he directed for the Beastie Boys' song Ch-check It Out. He wrote using the same memorable pseudonym he favoured when he directed videos, Nathaniel Hornblower, a man of indistinct nationality and such indomitable self-belief he may well have been the inspiration for Borat. I don't want to spoil the pleasure you'll derive from reading the letter in full by selectively quoting it so let's just say this is surely the only time a letter featured in the New York Times had berated the paper for its fondness for U2 and told the editor that he owed the writer a goat.

This last anecdote, with its caperish cleverness, strikes me as the second most telling one about Yauch. We'll get to the first soon but before that, the music. Growing up in New York in the 80s, the Beastie Boys felt like the soundtrack of the city, with that very Lower East Side mix that they coined of rap, rock and sampling. It is a rare day that I don't listen to Pass the Mic, So What'cha Want or Shake Your Rump at some point.

"In those days, hip-hop was truly from the streets, and everybody rapping was black. All of a sudden, these punk rock white kids crossed into hip-hop with the shock of Jackie Robinson in reverse," Public Enemy's Chuck D once said of the Beastie Boys. And because they never pretended to be anything but "punk rock white kids", they taught hip-hop fans who weren't black, who weren't from the street, that they shouldn't play act at being anything other than who they were when listening to hip-hop. They weren't revolutionary, as Chuck D said, they were evolutionary.

Yauch's vocals are easy to recognise, the husky belt among the yelps of Mike Diamond and Adam Horowitz, and he generally has the best lines: "What's running through my mind comes through in my walk/True feelings are shown from the way that I talk"; "If you try to knock me you'll get mocked/I'll stir fry you in my wok/Your knees start shaking and your fingers pop/Like a pinch on the neck from Mr Spock."

The Beastie Boys were hilarious, talented and arrogant. But that was just the beginning of Yauch. As well as setting up his film company, he became a sensitive and effective political campaigner, organising concerts for the people of Tibet and victims of 9/11. So for me, the most tell-tale Yauch anecdote comes from his acceptance speech at the 1998 MTV awards when he took to the podium to speak out against – calmly, and without any smugness – America's racism against Muslims.

One can sneer at celebrity activism and, heaven knows, I have indulged in that in my time. But this prescient stand of Yauch's came across as heartfelt and could hardly be seen as self-serving. Three years before 9/11, racism against Muslims was not exactly a credibility-enhancing topic in America. But then, I'm biased. The truth is, I liked Yauch a lot more than I do most celebrity activists, because of his work, yes, but also because of his personality.

About six or seven years ago I was in LA for work and stuck at some dreadful celebrity event. I was the only non-famous person and so, as night follows day, absolutely no one spoke to me. Except one person. Adam Yauch was there and, seeing me standing awkwardly in the doorway, came over and said hello. We chatted about how much we both missed New York, how weird we both found LA, how one never got any proper food at these things, and he then went out of his way to make sure I got something to eat. Taking pity on my obvious jet lag, he then sorted out my ride back to my hotel. He shook my hand and said goodnight and, I'm sure, never thought about it again. But I did, especially whenever I stood in a doorway, ignored at another celebrity event. There are few celebrities – few people, for that matter – as kind as Adam Yauch. Not long after, he was diagnosed with cancer and the man from the band of eternal teenagers began to age, too soon, too quickly.

The comedian Ben Smith, better known as Doc Brown, also met Yauch, on the street on a trip to New York when he was a 16-year-old hip-hop-obsessed British teenager: "He shook my hand and talked to me and my mum like he knew us. The guy was a true gent when you think about how many fans he must have met on a daily basis," he told me.

It's easy to get oversentimental about celebrities when they die. It's even easier to mock that tendency as silly self-indulgence. But Yauch was not just a celebrity: he was a musical pioneer, a champion of independent films, a man who was true to himself. But, most importantly, Adam Yauch was a real mensch.