When Humphrey Lyttelton was alive, and I'd meet fans of Radio 4's I'm Sorry I Haven't a Clue, and they discovered I wrote his scripts, the conversation would invariably go something like this:

"What a genius."

"He's the funniest man on radio."

"It must be such a joy to work with Humph."

"Hang on - what do you mean you write his scripts?"

"He has scripts?"

There can be no greater compliment to the performance of a script than for the audience to fail to notice there is one. At recordings of Clue, even I could forget there was a script. Such was the transformation from the words I'd cobbled up into the masterful performance provided by this octogenarian, Old Etonian trumpet-player, I could collapse with laughter along with the rest of the audience, as if I'd never heard my own words before. In fact, they no longer were my words. From the moment Humph took the two small envelopes containing the scripts printed on postcards, the words became his.

When I am asked how a comedy writer is inspired to write new jokes every week, there is a pat answer: I get up and sit down at my computer with a cup of strong coffee. I then think about the show I have to write: its form, its style, its performers. I then begin to type, inspired by the thought of how the hell I'm going to pay the mortgage. That was generally the case - except when I had the luxury of Humph's voice in my head. The inspiration was always there to do something more quirky, more daring, more stupid and, occasionally, even more outrageously knob-gag-laden. People often talk of the rich, slightly posh, authoritarian tone of Humph's voice. There was more than that for me. I could hear the cheeky glint in his eye.

Most comedy performers prefer to have some involvement in the writing process. Not Humph. In 15 years, we never had a script meeting, nor even a quick discussion about a single word he was going to read. His attitude was that the writing of the script wasn't his job. His job was to read it. And yet it seems never to have occurred to many of his listeners that Humph actually had a script, even though he'd often point out during the broadcast that he was reading it for the first time.

During his silent, solitary pre-show read-through in a dingy corner of the theatre, I would occasionally spot him scribbling notes when he discovered a typo or a grammatical error. He was making notes to remind himself to point them out. He would delight in stopping, mid-performance, to announce that when he was supposed to say "genteel", for example, the script said "gentile". Sometimes he would even read the stage directions. After a set-piece game, he might say: "'If that dies on its arse, make them do another one.' I don't think I was meant to read that bit." Or: "... the title will be provided by the technical wizardry of the multi-pixelated laser display screen. 'Wait while someone comes on with piece of cardboard.'"

It was as if Humph occasionally felt the show was going too well, was too professionally slick, and he felt the need to remind us not only what a rank amateur he was, but that all those around him were even worse. As he didn't get involved in the writing, he could make great play of being dismissive of his script and bolster his "couldn't care less" attitude. This wasn't an act - he really couldn't have cared less. That's not to say he wasn't the complete professional, but he didn't need to prove anything. He had no future comedy career to worry about. It didn't even occur to him that he was funny. He often said that the others on the show were professional comedians, so why would he, a trumpeter, try to compete? He just found certain things funny, and he shared them with people around him whom he hoped would also find them funny.

Sometimes I wondered if producer Jon Naismith and I took too much advantage of his easygoing insouciance. Much play was made of Humph as the "purveyor of blue-chip filth", and he liked that title. Indeed, he often said he wanted to drive between gigs in a tradesman's van with "Humphrey Lyttelton: Purveyor of Blue-Chip Filth" written on its sides. He also said he had never been so proud as when fellow jazzman George Melly, probably the filthiest raconteur ever to have scandalised an audience, claimed to be shocked at what Humph was getting away with.

Even though I was responsible for what I like to describe as "post-feminist irony" (known in the comedy business as "knob gags"), I was sometimes equally amazed by what we asked Humph to try to get away with. Most centred around his assistant and scorer, the ever-delightful Samantha. "Samantha has been working down in the gramophone library today, where the archivists have been engaged in a heated argument about who sits at which desk to get the best view of Samantha's shapely legs. To calm things down, she had to keep them apart all morning." "Samantha has recently taken up beekeeping with a small hive, housing just three dozen or so. This evening she has an expert beekeeper coming round to show her a few tricks of the trade, and he says he'll quickly have her 38 bees out and flying round his head." "Samantha is off to see a chef gentleman friend who is renowned for his fine-quality offal dishes. While she's very keen on his kidneys in red wine and his oxtail in beer, Samantha says it's difficult to beat his famous tongue in cider." That's the problem with a knob gag: the next one only seems as funny if it's filthier. More than once, between tears of helpless laughter, the producer and I asked each other: "Can we actually broadcast that?"

The rule was that the joke was always clean on the page, with the laughter coming solely from the mis-hearing - literally, in the double entendre. We could therefore, if required, defend the material, in that it could only appear filthy to someone with a dirty mind - so what were they complaining about? However, given the graphic nature of the sexual acts that were "inadvertently" described, that is at least borderline disingenuous. In fact, I don't think anyone ever did complain. There is a paradox in that although many listeners didn't realise Humph had a written script in front of him, they were nonetheless happy to accept that he clearly didn't understand what he was reading.

To claim that Humph didn't know what he was doing on Clue is, of course, complete nonsense. As to his not being a comedian: that is also untrue. It may not have said so in his passport, but he was one of the wittiest comedians I ever met - one who could go off-script with the sharpest ad-libs. While being mocked by the others for his age and incompetence at trying to read a tricky retake, he stopped, gazed to the side for a moment, squinted, and asked: "Has the man in the end bed died?" Following a great roar of appreciation at the introduction of a favourite round, he once told the audience: "Now go off and invade Czechoslovakia." Then he pointed up to the circle and said: "I was only joking. Sit down, Madam." Approaching what would have been his 87th birthday, he told an audience in Eastbourne: "It makes a nice change to be one of the youngest people in the room."

Before Chairman Humph, panel games had been chaired by nice chaps (and they invariably were chaps) who hoped we'd all enjoy the show as much they were going to enjoy delighting each other with their wit and wisdom. After Chairman Humph, we got Have I Got News for You, They Think It's All Over, Never Mind the Buzzcocks and countless other comedy panel games, where the chairman appears not to want to be there and isn't too fond of either his guests or the subject matter. Humph redefined the role of the comedy panel game chairman.

In the months before he died, Humph and the team toured a stage version of I'm Sorry I Haven't a Clue. He became quite frail towards the end and it was obvious he was frustrated by his own body giving up on him. At the end of one of the last performances, he was handed his trumpet to close the show. An audience of 3,500 rose as one to express their appreciation, as Humph improvised a powerful, note-perfect rendition of We'll Meet Again.

I wish we could, Humph.

• Lyttelton's Britain: A User's Guide to the British Isles As Heard On BBC Radio's I'm Sorry I Haven't a Clue, by Iain Pattinson, is published by Preface.