You can try any number of tricks but even the best songwriters don't know how they do it, as a new BBC2 show proves

How do you write a hit song? Through the decades, many books have been written on the subject – usually by people who aren't songwriters themselves – using analyses of previous hits to come up with solutions. Music industry publication Billboard recently revealed the stats of all the songs that had featured in the magazine's Hot 100 charts.

Since the 50s, songs have become longer, from an average of 2.36 minutes to 4.26 minutes this decade. If you want a hit, it may be best to stay away from writing ballads – ever since the 40s, the average tempo of chart entrants has hovered between 117bpm and 122bpm (ballads usually play at around 90bpm). You should also stick to major keys, with C major being the most popular. Of the top 10 most successful songs of all time, only Kanye West's Gold Digger is in a minor key.

Billboard shows it has become increasingly important to get a cut with artists who are already successful. The number of hits in this category has steadily risen, and is now 25% higher than in the 50s. The most successful artist of all time, chart-wise, is Mariah Carey – followed by Madonna, Elvis Presley, Michael Jackson and someone called Patti Page (glad to see Hall & Oates coming in eighth).

Author Jay Frank says the way people consume music in the digital age has changed what makes a hit. In his book Future Hit.DNA he argues that people are discovering music online and not always via radio, so song intros need to be shorter. He recently used Adele's Someone Like You as an example of how the theories in his book are correct. "The intro is five seconds long, it's at walking tempo (105bpm), contains repetition of many lyrics with a choral counter-chorus, has a very sly shift in the chord progression at the bridge, and contains many dynamic shifts throughout the song," he concludes.

This doesn't exactly tally with stats provided by Billboard, although Adele's hit is in a major key. So who's right? Maybe it makes more sense to look to songwriters who have had plenty of hits. BBC2's brilliant current series Secrets of the Pop Song is trying to shed light on the issue. In it, successful songwriters talk about the craft, and we see hit-maker Guy Chambers in action as he co-writes with a selection of artists.

If anyone was hoping to stumble on a secret formula, that hope was quickly shattered. As Motown legend Lamont Dozier once said: "I've written about 78 top 10 songs, and I still don't know what a hit is. I can only go by what I feel." His point was brought home as Chambers got together with Mark Ronson (most famous for his collaboration with Amy Winehouse) and budding artist Thalia to write her a breakthrough hit. After trying out a few ideas they settled on an African highlife sound. Ronson noted that very few hits had that particular sound, bar Vampire Weekend, which probably should have been a reason to abandon the idea. A view echoed by the radio pluggers who thought it could possibly be a fourth single, adding: "It's not bad, but it's not great."

As a fellow songwriter once told me: "The world doesn't need any more good songs. What we need are great songs." Or, to take the idea a bit further, the enemy of great is good.

One of the advantages of being a songwriter instead of a performer is that, while artists have a hard time recovering their reputation when a record bombs, the only time the public pays attention to who wrote a song is when it's a hit. Most people don't realise that, even for successful writers, the good-v-great ratio is low: Chambers has written more than 1,000 songs in the last 15 years, of which 21 ranked in the top 10 – that's one hit for every 47 songs. That may sound like a frustrating process, but most writers would agree it's necessary to write non-hits to get to the nuggets. As with athletes, it's important to exercise the writing muscle.

It can also be detrimental to throw away the seed of a song too early. "As you get older, your filters are much more refined," Sting has said. "The critical mind takes over from the creative mind, and I think they are opposed." He admits that, nowadays, it can take him months – even years – to write one song, as he feels every idea he has is too much like something else he or somebody else has written.

When I first heard of the BBC's intention to film the process of songwriting, I wondered if it could work. The creative process is intensely personal, and you need to feel relaxed and comfortable enough to talk about intimate experiences, and to throw some ridiculous ideas into the mix. The prospect of being judged by a TV audience would surely make writers curb their ideas.

But while I found the Ronson session quite painful to watch, the Rufus Wainwright session – in which the goal was to write a ballad – was inspiring. He started out playing some ideas he'd recorded on his Blackberry, including a rap called Everybody Wants a Piece of the Action, and so proved my fear about filtering unfounded. Even though he'd just met Chambers, he revealed personal things, which later were incorporated into a beautiful ballad called World War Three (shame that ballads rate so low on Billboard's hit barometer). He used an old songwriter saying as the first line of the chorus – "don't bore us, get to the chorus" – as a metaphor for life.

In the third and final episode, to be broadcast this Saturday, Chambers and the Noisettes attempt to write an anthem. It features some of the songwriters behind past anthems, analysing why they think their songs became so popular. What is apparent is that none of them knew they were writing an anthem at the time. I Will Survive, for example, was originally a B-side track before a Studio 54 DJ picked up on it. Yet it ended up generating over $100m. Many songs have been written with the same ingredients – a hook that makes people feel good, a repetitive chorus, basic chords – and yet they've failed to set the world alight.

There's one thing successful songwriters have in common: they all love the music they write. They're not cynical about their craft. But while there are tricks of the trade to deliver songs in a more palatable way, other key ingredients are more elusive. Don Black believes lyrics should say something new about the human condition; Björn and Benny of Abba said you should have at least five hooks in a track. But how do you come up with a great hook? If Black or the guys from Abba knew that, they might still be churning out hits.

Even if Secrets of the Pop Song had been able to capture the moment Björn and Benny came up with The Winner Takes It All, we'd be none the wiser as to how to write a hit song – and neither would they.