Atlético Madrid fans are half in love with failure – long-suffering loyalty setting them apart from their glitzy city rivals. But now they have a European semi-final of their own

A car pulls up at the traffic lights. A small boy sits in the back and gazes silently out of the window, a pensive look on his face, while his father drums his fingers on the steering wheel. The hush is broken when the boy, a pleading, inquisitive tone in his voice, says: "Papa?" A sigh, a hint of resignation: here comes another one of those questions your kids ask that you can't answer. "Yes, son." "Why," asks the boy, "do we support Atleti?" The silence deepens, his father is stumped. There is no answer, no logical explanation. Why do we support Atlético Madrid?

Ask a Real Madrid fan to explain his loyalty and the answer is as easy as it is long: for Di Stéfano and Raúl, Sánchez and Puskas, Zidane and Butragueño, the two Ronaldos, for the 31 league titles and nine European Cups. The glory.

Ask Atlético Madrid fans and the answer is different – especially over the last decade. It is inexplicable, intangible. It is about sentiment, commitment, passion. "Atlético fans are prisoners of a feeling, of their colours," says Fernando Torres, the former captain who returns to his boyhood club with Liverpool this week. "Madrid's fans are prisoners of results and if results don't follow, nor do the fans."

Until they moved it to a different part of the stadium, the entrance to Real's museum used to boast a sign that said: "Trophies tell the whole story, defining the holder." Real were simply: "The Greatest Club of All Time." Atlético's museum has a necessarily more plebeian approach, while – forget players and trophies – the club shop's best-selling T-shirt is splashed with the slogan: Bendita afición. Blessed fans.

As Torres says, Atlético fans would follow Atlético to the ends of the earth. All too often that's exactly where they are heading.

There is a certain mythology about it, but the image pervades, deliberately embraced and projected. The scene in the car has become the most famous expression of Atlético, an illustration of unfathomable fidelity. It comes from an advert made in 2001. Atlético had been relegated from the first division for the first time at the end of 1999-2000. The club's owner, Jesús Gil, declared it "one little year in hell" but Atlético were already into a second season. Yet season-ticket sales increased to over 40,000, the 55,000-capacity Vicente Calderón stadium packing out weekly.

Would it have happened at the other end of town? "No way," Torres says. "The Bernabéu is like a theatre. They don't feel their football. Atlético is different, special." Anyone who has witnessed the Vicente Calderón as a spine-tingling roar of Atleeeeeeeeeeeeeeeeti takes hold can vouch for that and Atlético fans revel in it. As one says simply: Atlético hasta la muerte; Madrid hasta la próxima derrota. Atlético until I die, Madrid until the next defeat.

It was an Atlético fan, Miguel García Vizcaíno, who dreamed up the advert. "When you talk Atleti's defining values, the recurring words are passion, authenticity, realness. Arrogance just doesn't fit," he says. "People identified instantly with the advert; it summed up their feelings. Some sympathise more with Wile E Coyote than Road Runner – who always wins by cheating."

The facile assumption is that, like Wile E Coyote, Atlético always lose, no matter what. Especially against the team they desperately want to beat, in whose shadow they live – Real Madrid. When Liverpool supporters travel to the Calderón on Thursday, most will stroll along the appropriately named Paseo de los Melancólicos. Melancholics' Way. From high in the main stand, the view is stunning. Towering over the Calderón is Madrid's cathedral, and you imagine it looking down piously, almost pityingly. You poor lost souls.

It is too easy to forget that Atlético won the double in 1996. Or that they are historically the country's third-biggest club, with nine league titles. That, having been fused with the air-force team and been rebranded Atlético Aviación at the end of the Spanish civil war, they won the league in 1940 and 1941; that, now plain Atlético Madrid in 1947, they won two more leagues at the start of the 1950s, and four trophies in the 1960s – including back-to-back Copa del Generalísimo successes against Real. To forget them beating Real in the 1992 cup final, after Luis Aragonés's team-talk: "Forget tactics," he screamed. "It's Real Madrid at the Bernabéu. They've been sticking it up our arses for so long, now it's our chance to stick it up theirs!"

It is easy to forget that in the 1970s Atlético won three league titles, two cups and the World Club Cup. And it is easy because losing the 1974 European Cup final was more painful and, ultimately, more memorable, because of the way that defeat has been clutched to their hearts. It has been easy, too, because of Real's extraordinary success and because since 1996 Atlético have been relegated and won nothing. Since Atlético's demotion in 2000 more than 60 teams have beaten Real, but the club in red and white are not one of them. Not even once. The city derby has become Atlético's own private Groundhog Day.

Atlético lost the 1974 European Cup final against Bayern Munich to a freak late goal from 40 yards. The club president, Vicente Calderón, dubbed them El Pupas, the jinxed ones. It has stuck; in recent years it has become even more immovable. A sense of fatalism hangs about. Atlético have made a cult of collapse; defeat is a way of being. There's even a supporters' club called "The Suffering".

"Atlético are the only club who presume to lose," says Michael Robinson, the former Liverpool player who now works on Spanish television. "Atlético are a stray dog that has wonderful eyes to stare at you. A dog that will never win Crufts, whereas Madrid won't even partake in Crufts if they don't think they're going to win it. Not winning proves they care more. It's almost a masochism, like an Easter procession, flogging themselves."

Some don't swallow it, Juanma Trueba from the newspaper AS insisting: "Atlético are a romantic myth – Atlético are Humphrey Bogart, Madrid are Cary Grant." Most Madrid fans are dismissive of them but others are irritated at Atlético's willingness to play the martyrs – as if that gives them a moral superiority. The identity survives, though.

Torres's commitment to Atlético came from his grandfather, Eulalio, who complained that Fernando's class-mates were all Real fans. "He told me what being an atlético was all about," the Liverpool striker explains. "He didn't tell me about players, but about what it means to wear the Atlético badge, with the bear and the strawberry tree that symbolise the city. About hard work, humility, sacrifice, overcoming adversity, about resistance to Real Madrid, the city's football giants."

Real are inescapable. Atlético define themselves against their rivals; the differences make them special. Anti-Madridismo is fundamental. The recent injection of cash may have ruined the analogy but until recently, Madrid's rivalry could be compared to Manchester's. They started life as a branch of Athletic Bilbao, but Atlético consider themselves the more authentically Madrileño team – the people's club battling overbearing neighbours with designs on a higher calling.

Atlético like to think that favouritism, bent referees and the Franco regime explain their rivals' success. A victim complex pervades. "The directors' box at the Bernabéu is like Franco's hunting trips," Atlético's late president Jesús Gil said, yet it was he who had a life sentence commuted by the dictator. Severino Lorences, author of Red and White, insists: "The Bernabéu trophy room is like Aladdin's cave: everything's been stolen. Robin Hood stole from the rich to give to the poor, Madrid are the Sheriff of Nottingham, stealing from the poor to give to the rich."

In 1959 Atlético reached the European Cup semi-final against Real. They would have won on away goals, but there was no such rule and they lost a play-off in Zaragoza with a debated goal – debated at the Calderón, anyway. "That European Cup should have been red-and-white," laments one Atlético fan. "History would have changed."

Torres says: "Atlético represent a permanent battle against the odds; being an atlético means never giving in. Atlético are on their own, fighting the establishment. Kids might say: 'Why should we support Atlético when they always lose and Madrid fans are always happy?' But Madrid fans aren't happy. Supporting Atlético makes you suffer but makes you stronger. That's why my grandfather will always be an atlético and why I will be, too."

He adds: "Living in Real's shadow is hard. For years Madrid have been the big, rich team in Spain and we've been the poor team, the working-class club. Atlético belong to the city; Madrid are the world's team."

As one Madrid fan puts it: Real Madrid and Atlético Madrid are caviar and cocido. Cocido is Madrid's hearty, heavy stew. Atlético fans agree. They like to present Real versus Atlético as power against people; aristocracy against proletariat; right against left (even though the hardcore are on the far right); the smart north against the real city of the south; arrogance against humility; entitlement against loyalty; favoured against persecuted; implacable winners against romantic losers.

The Bernabéu stands imposingly alongside businesses on the Paseo de la Castellana, the Calderón is locked in by the grubby M30 motorway, which runs right under the stand. It is an exaggeration, sure, but by the Bernabéu you'll see stylish bars packed with slick hair and Barbour jackets; by the Calderón it's beer in gigantic plastic glasses, shiny tracksuits and mullets. Former Atlético player José Movilla was a bin-man; ex-goalkeeper Germán "Mono" Burgos a fan of AC/DC. "I couldn't play at Madrid because of how I look," he insists. "I'm not joking. They'd make me cut my hair. Atlético is synonymous with workers. Atlético fans are brickies, taxi drivers, churro sellers ..."

Their centenary celebrations underline the difference. Madrid's involved the United Nations, the pope, and the king; Atlético's a huge paella for the fans. Madrid's anthem was sung by Plácido Domingo, lauding a "field of stars"; Atlético's was called Reasons for a Feeling and was sung by gruff-voiced folk singer Joaquín Sabina, in which he declared "war on arrogant [Madrid]". Its opening lines exalted "a feeling that cannot be explained", insisting: "To understand you have to have cried at the Calderón." The chorus goes: "What a way to win/What a way to lose/What a way to go down/What a way to come up/What a way to draw/What a way to suffer!" Typically for Atlético, because of an argument over the rights, it could not be played on their centenary. Instead they used the Rolling Stones' "You Can't Always Get What You Want".

But Atlético's perceived "identity" does not convince everyone. Especially those of an older generation – those that remember success. Asked about the legend of El Pupas, the former keeper and coach Abel Resino responds simply: "Pah! Bayern Munich lost the European Cup in the last minute and no one says they're jinxed." Worse, talk of El Pupas, of blessed suffering, unexplainable loyalty, and a deep feeling that resists disappointment and collapse, has become an excuse for failure – a kind of smokescreen for the disastrous handling of the club. The chance to shrug and say: hey, losing is just what Atlético do.

It should not be. One supporters' group produced an alternative video in which, over a period of time, the dad in the car has fewer reasons to give his son. "Everything we are, they're taking away from us," it concludes – from success, to dignity, to the Calderón itself, which they will soon leave, crumbling but with character, still one of the country's great football arenas.

There is a brewery alongside the Calderón; Atlético couldn't organise the proverbial drinks there. Under Jesús Gil, they ditched the youth system because he did not see the point. Among the first to seek a new home was a 13-year-old by the name of Raúl. Atlético went through 37 coaches in 16 years. Ron Atkinson lasted 94 days and joked that he deserved a testimonial. Gil's successor, Enrique Cerezo, went through eight in seven years. Since 1996, with Gil's son Miguel Angel Gil Marín as chief executive, they have had 18 coaches. Meanwhile, the debt tops €300m (£263m).

At long last, Atlético have a chance to lay the jinx to rest. They are already in the final of the Copa del Rey; now they have a European semi-final against Liverpool. Win and the party will be huge; maybe the biggest the city has ever seen. After all, as Torres puts it: "Atleti's successes are ours and ours alone – that makes them more real." Who knows, next time the kid in the car asks "Papa, por qué somos del Atleti?" his father may just have a logical answer.