Twenty years ago, after appearing in two phenomenally successful, visually opulent and generally brilliant Batman movies, Michael Keatondecided he didn't want to make any more Caped Crusader films. So he walked away. It was a disastrous move that effectively ended Keaton's career as a leading man, the actor learning the hard way that the only unforgivable crime in Hollywood is to walk away from a phenomenally successful franchise.

Today's superhero films have not yet reached the point where they flat-out suck. But they are getting there. Iron Man 2 was a huge disappointment, The Avengers (Iron Man 2½) an aimless hodgepodge and The Dark Knight Rises a pretentious, incoherent mess. And now this week we have yet another Superman movie, Man of Steel. Those of us who would like to see an end – or at least an extended pause – to the hegemony of superhero films would be very pleased if Robert Downey Jr, Christian Bale and the nebbish who plays Captain America would make a similarly audacious artistic decision and walk away.The next two Batman films starred Val Kilmerand George Clooney. Batman Forever was not very good and Batman & Robin was terrible. And for the next few years Batman dropped out of the global conversation. This was good because it gave society a breather. The Dark Knight thing was played out: the excitement moviegoers felt when Tim Burton made the first Batman film had evaporated under the tutelage of Joel Schumacher. In retrospect, Keaton's catastrophic decision to walk away now seems heroic, because he was the last actor to go through a script, take a cold, hard look at the superhero genre and say: "Enough. These films are starting to suck."

As Steven Soderbergh recently complained, these films are sucking the life out of motion pictures, diverting virtually all of the industry's resources into insanely expensive "tentpole" films that supposedly prop up other projects. It is unlikely that any of these actors will make such a courageous decision as Keaton, though: they saw what happened to him, they saw how Sean Connery's career stalled when he quit 007. But it's still okay to dream, isn't it?

This thing is starting to get old. There are too many superhero films: their storylines are all beginning to run together. It is a genre dominated by the thoroughly unoriginal notion that you cannot trust the government. Even when you can trust the government, you cannot trust all of it. And even the branches you can trust aren't much help, because they are incompetent. To save humanity, one must rely on a bootstrap operation headed by a dedicated go-getter and self-starter. At heart, all superheroes are Republicans.

In superhero movies, women are almost always accessories. This is true even if they themselves are superheroines. The men do the heavy lifting; the women serve an ornamental function. This is why we are all the way up to Iron Man 3 and Batman 7, but have not seen a Supergirl film since 1984, or a Wonder Woman film ever (supposedly, it is coming this year). The 12-year-old boys for whom superhero movies are made are not interested in women. They may not even be interested in girls. They are certainly not interested in girls with superpowers.

Superhero films increasingly rely on a structure where the hero thinks he is fighting one villain when he is actually fighting another. In The Dark Knight Rises, Batman thinks he is up against the crypto-fascist Bane, when he is actually locked in a deadly struggle with a mysterious fellow philanthropist played by Marion Cotillard. In Iron Man 3, the hero believes he is going toe-to-toe with a terrorist called the Mandarin, when the villain is actually a mad scientist who bears a striking resemblance tothe dead but not forgotten US rock star Warren Zevon.

In Thor, the bodacious nordic deity spends most of the movie worrying about a race of tall, antisocial creatures called The Frost Giants of Jotunheim, and does quite a bit of jousting with the testy emissaries of the US government, when the person he should really be worrying about is his brother, Loki. In days gone by, a superhero only had to worry about the Joker or the Silver Surfer or Lex Luthor. Now he has to worry about mysterious philanthropists. No wonder he's so neurotic.

In fact, the rise of superhero movies signals the triumph of the neurotic over the maverick. In the classic Hollywood movie, whether the hero is cop, cowboy, private eye, rebel or drifter, there comes a moment when this solitary, self-sufficient loner faces the bad guys all by himself. The bad guys are usually trying to destroy a ranch, a town, a portion of the high chaparral, or in some extreme cases, a flourishing ethnic group. They are rarely seeking to destroy an entire planet. These villains have limited aspirations, and the man in the white hat has a limited arsenal of era-appropriate weaponry: a gun, a bow and arrow, a few grenades, maybe even a tank. He does not have any weapons of mass destruction to fall back on, nor any supernatural powers. He has to rely on brains, brawn and guts, nothing else. Sometimes, this is not enough: more often than we would like to think, he ends up like Spartacus or Braveheart.

In the classic superhero movie, the situation is quite different. Here, the bad guys are trying to destroy entire societies, cities or planets, and the good guy is rarely self-sufficient. Instead, somehow or other, he has come into possession of a preternaturally phantasmagoric suit of armour, complete with zany high-tech accoutrements; or a hammer that can call down lightning from the heavens; or extendable fingernails; or laser eyesight; or implausible (and non-steroid-related) abs; or the ability to change shape.

And these superpowers aren't just good news for all the societies, cities and planets that need saving: most superheroes are nerds or geeks or losers or screw-ups or pixies or marooned orphans from deep space who can't get their personal lives functioning properly – until they come into possession of some mystical power or magical weapon. Nothing in their pitiful lives works out until they are bitten by a spider, or start sporting a remarkable piece of jewellery, or are handed a large, seemingly radioactive hammer by their father.

"Being a superhero is a way of working out your personal problems," my 26-year-old son told me when I asked him about the popularity of the genre among his age group and younger. "You're an ordinary person with no special skills – and suddenly you wake up one day and you're awesome. So if you're asking me if the superhero genre is going to fade away soon, the answer is no."

You wake up awesome. Not because you did something special like beat Hitler or cure polio. All you did was wake up. And suddenly you were awesome. It is the dream of the fame-hungry X Factor generation.

If movies are a reflection of society's most cherished hopes and deepest fears, then superhero movies perfectly capture the planet's current mood of uncertainty and dread. Today's global economy is a disaster, unemployment is ravaging the economies of both the developed and the developing world, and the threat of terrorism stretches from Kabul to Moscow, from London to Boston. Superman debuted during a particularly dark time in the world's history, the 1930s, so it is not surprising that the franchise is being rebooted now, with Man of Steel directed by the prolific action hack Zack Snyder (The 300, Watchmen). There is no clearer indication that this is a dark time in the world's history than the fact that the director who made the slovenly, inept Watchmen is now getting to reboot Superman. Is nothing sacred? No.

Superhero movies are made for a society that has basically given up. The police can't protect us, the government can't protect us, there are no more charismatic loners to protect us and the euro is defunct. Clint Eastwood has left the building. So let's turn things over to the vigilantes. Superheroes need not obey laws or social conventions; they go where they please and do what they want. They pose simple – usually violent – solutions to complex problems. Superheroes operate in a netherworld just this side of fascism.

Still, it would be a mistake to say that all superhero movies are the same. The Dark Knight movies are dark. The Iron Man movies are funny. The Hulk movies are goofy. The X-Men movies are complicated. Captain America was camp, Thor a bit silly, The Avengers sillier still. The Spider-Man movies are closest to conventional movies, placing ordinary people in difficult situations. The Spider-Man movies also feature a romance that seems quite believable, unlike Iron Man.

Although superheroes are archetypal, each succeeding generation of filmgoers demands a more up-to-date hero. And so each generation gets its own reboot of the Batman/Spider-Man/Superman franchise. One day, there may even be a Daredevil reboot, though hopefully not soon. The films reflect the values of the decade in which they appear. The Batman movies of the 1990s were camp and jokey; the Dark Knight movies, appearing a decade later, were not. The Superman movies of the 1970s were over the top, like the comic books they were based on – it was, after all, the era of Nixon, Ford and Carter, clowns to a man. Iron Man, a more recent creation, is recognisable as a sneering, insincere slacker: nothing heartfelt ever passes through his lips; the very thought of saying something honest and authentic would mortify him. You cannot imagine Iron Man talking like Batman, Wolverine or even Thor. He is the superhero as wise guy. He is Ironyman.

One thing that is puzzling about modern-day superhero movies is that the skill set of the individual hero is often poorly defined. I am not sure what it would take to put Iron Man out of commission. I have not been able to figure out whether the Dark Knight can actually fly: he certainly seems light on his feet. And I have no idea what powers Thor possesses. I know that his hammer has miraculous potencies, but I am still not sure precisely how miraculous they are. I have no idea what it would take to kill Thor; nor for that matter does Loki. In films featuring Dracula, Tony Montana, Orcs or even Achilles, the parameters are more clearly drawn.

The most interesting thing about the popularity of superhero movies is that they are insanely expensive to make, yet they spring from a plebian, populist artform. Comic books, at least until recently, were cheap. They were beautifully drawn and exciting, but they were still basically cheap. That was the point. Movies are not cheap, especially not in 3D. Comic book heroes, like football players, have lost all contact with their proletarian roots.

Some people will read all this and say: "You're over-intellectualising. You're reading too much into it." This may be true. But these charges are always made by people who never over-intellectualise anything, who never read too much into things. They are made by people who want you to take The X-Men seriously, as legitimate fiction. And then when you do, they say that you are over-intellectualising.

After all, they say, it's only a movie. That's exactly right. It's only a movie. But it's the same movie – over and over and over again.