Ba-ba-ba booom: it's a musical motif so familiar, so distinctive, we recognise it written out in baby words. Did Beethoven realise, when he composed the opening bars of the first movement of his Fifth Symphony, that he had come up with a riff that spoke of fate knocking at the door, the dawn of German Romanticism, a people's search for a national identity, genius standing up against the world? And similarly, did Ritchie Blackmore realise, when he jammed the introduction to Smoke on the Water (duh duh duuuh, duh duh de-duuuh), that he had come up with a riff as emblematic for his generation as the opening of Beethoven's Opus 67?

As a former member of a pop band (the Communards) and now chaplain to the Royal Academy of Music, I am relatively well-placed to ask what makes a riff a riff, and if there's any real difference between ba-ba-ba booom and duh duh duuuh, duh duh de-duuuh. A catchy motif serves the interests of Beethoven or Debussy just as nicely as Deep Purple or Kylie. Most composers, certainly those working within tonality and patterned forms, would love to come up with something that is not only arresting and recognisable, but leaves the hearer wanting more. The opening of the Fifth Symphony is a brilliant example. The first few bars, the ba-ba-ba booom ... ba-ba-ba booom, seem to come out of nowhere, as startling and as unexpected as a violent knock on the door. (In fact Beethoven's pupil, Karl Czerny, maintained that the motif was inspired by the song of a yellowhammer heard in Vienna's Prater Park.) The whole of the first movement could be described as a development of that motif, which, in the imagination of Beethoven, produces extraordinary richness and complexity.

I am not so sure Deep Purple would make a similar claim. The merit of their riff lies in doing nothing much. It is just there, like a dependable friend, so persistent, so dogged, that nearly four decades after it was first heard, you cannot walk down Denmark Street in London without hearing it clumsily reprised from every guitar shop. It does not generate complexity and richness - but who wouldn't give their right arm to have come up with it? Or the little guitar hook in the James Bond theme? Again, not much on its own - there is a lot more technical stuff happening with strings and horns - but it is the guitar figure that tells us what we need to know about Bond, and it is the guitar figure everyone recognises.

Why do certain riffs work so well? Is it the patterning of pitch and rhythm, or the means by which the sounds are made? In Verdi's opera Don Carlos, the Grand Inquisitor's entrance is marked with a marvellously sinister theme. It is extremely simple, involving no complex chords or suspensions, and could quite properly be described as pedestrian (the blind and feeble Inquisitor is not fleet of foot); but growling basses and haunting horns turn it into something much more than an everyday arrangement of pitch and pulse. It works the other way on a track such as the Smiths' This Charming Man. Johnny Marr plays an ordinary little figure on the guitar, nothing special, but it adds something I would describe as irresistibly perky, if perky didn't sound so awful.

Hearing a good riff once makes you want to hear it twice; there are those you crave like cigarettes. I remember being so taken with a figure in the Prince track Alphabet Street, I replayed it obsessively all weekend; by Sunday afternoon, I was the only person left in the house. The most innocent or artless of things can, with repetition, sound significant and compelling. The composer Jocelyn Pook, who 20 years ago played viola in the Communards, is best known now for her work with film-makers such as Stanley Kubrick and Peter Kosminsky. Her piece Portraits in Absentia began when she replayed messages left on her answerphone. Something about the pitch and timbre and rhythm of people saying hello and goodbye impressed her so she re-recorded them, looped them, edited and played and replayed them until they acquired a kind of pathos, or hopefulness. Perhaps we cannot now hear people saying hello and goodbye on an answerphone without thinking of 9/11. But Portraits in Absentia was made in 1999 and sounded significant then.

Pook was already alert to the potential of taking sound from one genre and exporting it into another. She once recorded a Romanian priest singing a passage from his church's liturgy, looped it, played it backwards, and made a piece out of it that was chanced upon by Kubrick, who invited her to use it in the music she composed for the masked ball scene in his last film, Eyes Wide Shut. It is extraordinarily affecting, like a refugee lost in an incomprehensible world trying to sing his way back to a comprehensible world and not doing too well.

At its best, a riff has the power to express a feeling, or a sense of nostalgia, or a sense of community, with such economy that it takes on a life of its own. I remember an edition of Face the Music when Joseph Cooper played a single note on the piano, which Joyce Grenfell correctly identified as the beginning of Debussy's piano prelude La Fille aux Cheveux de Lin. How did she know from a single note? Was it a lucky guess, or a memory of something half-heard at a rehearsal? Was Debussy's riff so deeply imprinted that the merest hint caused it to emerge fully formed in the conscious mind? Whoever works out the answer to that will never be out of the top 10.

· Twenty Minutes - Ba Ba Ba Bum: The Art of the Riff is on Radio 3, next Monday at 8.15pm