"I play tennis for a living even though I hate tennis, hate it with a dark and secret passion and always have." So writes Andre Agassi in his new autobiography, Open, published this week. It is 2006 and one of the world's most feted sports stars has just woken up in a New York hotel room, poised to play his last tournament.

But why would a great sportsman hate his sport? Why wouldn't he love everything about it and all it brings to his life – travel, glamour, money, mass adoration, endless free tennis rackets and barley water, not to mention the surely sustaining thought that he is doing something for a living that makes many of us sick with envy?

"But it becomes more than a job, it takes over your life," says former British tennis professional Barry Cowan, perhaps best known for taking Agassi's nemesis, Pete Sampras, to five sets in Wimbledon in 2001. "If you're at the top of tennis, you're on tour 30-plus weeks of the year – and when you're doing that, everything revolves around tennis. Every decision you make, tennis is at the back of your mind. That's the main reason for burnout among tennis players in their 20s.

"I know this for myself – it's something you've done since you were six years old, and there's a sense that if you stop giving 100% you are doomed to failure, and that is unacceptable. No wonder so many players hate their sport – the surprise is that so few admit it."

And despite all the kudos, money and silverware, there's a reason it's the top players who suffer most – because they're the ones playing the most tennis, as they don't get knocked out in the first or second round. So they have the least free time, the most mental stress and suffer the most physically.

Agassi's avowed hatred for his sport is far from exclusive to tennis. British cyclists Chris Boardman, the former Olympic pursuit champion, and Tour de France star David Millar have both admitted to not really liking cycling. "In Boardman's case," says William Fotheringham, the Guardian's cycling correspondent, "he liked the winning not the cycling itself, and he drove himself to win."

That need to win can become a miserable addiction. Olympic gold-winning track cyclist Victoria Pendleton gave an insight into this in a brutally frank Guardian interview after winning gold at Beijing last year. "I was an emotional wreck beforehand," she admitted. "I worried that I would be the one person who let down the team. So winning was just a relief. And even that felt like a complete anti-climax. It was very surreal on the podium and as soon as I stepped off it I was, like, 'What on earth am I going to do now?' I found it quite hard to deal with. It was, like, I've got no purpose any more."

But it is her answer to the question of how to get out of this psychic void that is most telling: "I soon worked out that the only thing I could do was to get another gold medal. I need one. If 2012 goes to plan, winning the Olympics on my home turf, I might finally feel I've achieved the ultimate for me."

Pendleton's pleasure-free, angst-ridden drive to win is almost a defining characteristic of the greatest sports stars. "People say the pressure on top stars such as Andy Murray is unbelievable," says Cowan, "but I feel the pressure is from the stars themselves. They expect the best and if they don't deliver, it is horrible for them. With a sport like tennis, where at any tournament there can be only one winner, there are going to be a lot of perfectionists having to deal with disappointment. You need to be incredibly mentally strong."

Not all are. Former England cricket all-rounder Vic Marks has a poignant insight into the realities of being an athlete. "Sometimes as a cricketer," he says, "you just long for it to rain." But why? "So you don't have to play. I'm not saying cricketers hate cricket, but when you're playing a county game and the sky darkens and it starts to piss down, it doesn't half fill everybody in the dressing room with joie de vivre."

But surely top-flight players long to show the world how marvellous they are at their chosen discipline? "Not always. When it pissed it down, you knew were not going to fail that day. Lovely thought. With cricket, perhaps more than any other sport, everything you do is measured and analysed for all time – your failures are a matter of enduring public record."

Former professional footballer Stuart James echoes that thought: "Lots of players I know would travel to the ground hoping the game would be cancelled," says the ex-Swindon Town regular. "Fans say: 'You've got it good, you're on hundreds of thousands of pounds a week, so how can you moan?' – but most football players think the fans don't really understand what their lives are like."

A terrible fear of failure is one reason the life of the sports star can be rather less than the realisation of a beautiful dream. But there are others: horrendous training schedules, endless travel, foul fans, boredom and lack of privacy. "I remember being underwhelmed when I was selected to go on tour for England," Marks recalls. "People said what a bloody cynical and churlish response that was – but the prospect of being away for four to five months is not necessarily very appealing. Everybody thinks it must be so wonderful to spend the winter in the Caribbean or Australia, but it's not when you're away from your family and you're standing outside for eight hours five days straight."

There have been many English cricketers who have refused the supposed delights of the winter tour, but none more celebrated than Marcus Trescothick, the England batsman whose stress-related illness forced him to pull out of the national squad in 2006. "With Trescothick, there's no one who was more consumed by cricket than him," says Marks, the chairman of Trescothick's county, Somerset. "It had been his life since he was six, and that may well have made the stress worse to the point he had to take drastic measures to get away from Test cricket."

Mental stress

Agassi's biography reveals that he snorted crystal meth from a coffee table at his home in 1997, when suffering a lack of form and worrying about his impending marriage to actor Brooke Shields. "There is a moment of regret followed by vast sadness," he writes of the drug-taking experience. "Then comes a tidal wave of euphoria that sweeps away every negative thought in my head. I've never felt so alive, so hopeful – and I've never felt such energy."

As this passage implies, mental stress isn't the only major reason sports stars suffer more than the rest of us are generally prepared to admit. In his autobiography, Agassi describes the sheer difficulty of getting out bed one morning towards the end of his tennis career. "I'm a young man, relatively speaking. Thirty-six. But I wake as if 96. After two decades of sprinting, stopping on a dime, jumping high and landing hard, my body no longer feels like my body. Consequently, my mind no longer feels like my mind."

That passage will resonate for any player nearing the end of their career, with a body once in prime condition now a bundle of aches and pains that prefigures more intense physical suffering in later life.

"Freddie got a sense of that before he retired," says Vic Marks of the England all-rounder Andrew Flintoff, whose Test career ended earlier this year. "He could still do the bowling, but the batting suffered."

"The incentive to play for England is so high you'd do anything," Flintoff admitted recently. "Some mornings the missus had to get me out of bed and put my shoes and socks on for me. You then get the anti-inflammatories inside you, and a painkiller, and off you go . . . For me, a big achievement was just actually getting out on a cricket field. I've had six operations in four-and-a-half years – and two-and-a-half of those years were in rehab. I've been injured since I was 13. I had back problems all the way through."

Flintoff, of course, is a national icon, all-but-universally liked. The same isn't true of Derby County captain Robbie Savage, who earlier this week went public about some of the more horrible things that he has endured from football fans off the pitch. In Britain, football stars more than any other kind of sportsman or woman are likely to suffer foul abuse (think of what England fans chanted at David Beckham after a match against Portugal: "Your wife's a whore, and we hope your kid dies of cancer"), but none more so in recent years than Savage.

The former Welsh international told Radio 5 Live that he could put up with what he called "dog's abuse" from the terraces and conceded it even fired him up to play better. What he couldn't tolerate was death threats, having the windows at his home broken, having coins thrown at him as he left the pitch. He recalled that once, when he was playing for Birmingham City, he was visiting the NEC with his son when an Aston Villa fan spat at him in the face. "I was out with my little boy. That's got to be out of order, hasn't it?" You'd hope so, but the horrible truth is that many of us who aren't sports stars are immune to taking their feelings or lives seriously.

And even the former England and Aston Villa manager Graham Taylor takes an unsympathetic view of Agassi's revelations. "I'm not certain writing about how he doesn't like playing tennis is a good idea. We're all human beings, but generally speaking I have not got a lot of time for those people who complain about playing professional sport for a living."

There is, a horrible coda to this story of sporting misery. In his 2007 book Silence of the Heart: Cricket Suicides, historian David Frith wrote that cricket has a suicide rate that exceeds the national averages for the respective cricketing nations, and estimated that more than one in 150 professional cricketers have taken their own lives, among them the great Yorkshire and England wicketkeeper David Bairstow, who killed himself in 1998. Why? Frith concluded that cricket is an all-consuming endlessly absorbing sport and after retirement the thought of life without cricket is intolerable.

The mental and physical pain of playing sport and being at the top of your game may be bad enough, but the existential horror of realising at the end of your career that you are no longer part of that world is surely worse. Perhaps, unlike Agassi, these players didn't hate their chosen sport. More likely, they loved it too much.