Dave Lamb never wanted to be a cult voiceover artist. So tell that to the 63,501 Facebook fans who have declared that the narrator of Channel 4's Come Dine With Me is a "legend". The mild-mannered comedy actor who played the "token white bloke", as he puts it, in Goodness Gracious Me has an appetisingly wry line in tart asides, savouring contestants' culinary cock-ups (he has been compared to Harry Hill). In fact many only watch the show to hear him witheringly dismiss a would-be Gordon Ramsay's inedible efforts.

Lamb modestly disagrees: "My wife watches it for the cooking actually." She is surely in the minority. What started as a part-time side dish to fill time back in 2005 has turned into a proper job. While he recently appeared in the sitcom Miranda and played a scriptwriter in ITV1's Moving Wallpaper, it is CDWM that tops his CV. His excitable, sarcastic commentary – "Hopefully things are getting better for Emily now . . . or maybe not" – is merciless. He never sees the basic script in advance, so his ad libbed horror at a misguided meringue or a burnt boeuf bourguignon echoes our own schadenfreude-filled reactions. He also likes to squeeze in the word "ruddy" wherever possible. "They won't let me swear so that's the nearest I can get."

Maybe Lamb is an acquired taste. The 41-year-old Londoner has also made cyberspace enemies. "There are people out there who sent tweets about the way I pronounce dauphinoise. They said I should put a 'z' on the end of it."

Lamb insists that the show's success is more about the participants than him – "the gap between who they are and who they think they are." He suggests that the vital ingredient is a generous pinch of eccentricity. His favourite episode was set in Preston, where "one woman cooked the starter then decided to go to bed".

Whatever the reasons for its popularity, CDWM has spawned a host of wry variants, from BBC2's eerily similar Instant Restaurant to ITV1's May the Best House Win, both of which boast a Lamb-flavoured commentary, but will never match the original for its cocktail of irreverence and deadpan putdowns.

Lamb is more than happy to keep doing the series as long as they keep asking him. With a new run imminent – starting at 8pm on Sunday – plus specials planned, those acidic asides will be peppering the screens for some time yet.

The man behind them might be a natural, but perhaps it was not the voice that landed him the job but his appropriate surname? "I don't know about that. But I do get a little frisson every time I mention lamb."