Bradley Wiggins, hunched in a chair and speaking quietly through his exhaustion, remembers the suffering during his hardest day on the Tour de France. It is a chilling moment. He has already revealed that he sometimes thinks of himself as a "fraud" for finishing fourth in the race, despite his compelling courage and resilience, and in a contrasting statement has said that he plans eventually to win the Tour. But this is different.

This is a downbeat murmur that causes goosebumps to rise. For a second Wiggins looks as though he has seen the ghost of Tom Simpson again, the haunting British cyclist who died on the slopes of Mont Ventoux during the 1967 Tour de France.

"I don't want this to sound cheesy," Wiggins says, fearing that a surreal experience might be misconstrued as the delusions of a man driven mad after 2,200 miles on his bike. "But when I reached Ventoux on the second-last day it felt as if Tom was waiting for me. As I began the climb it felt as if his spirit was riding with me. It started on the early slopes and I imagined how Tom must have been feeling, riding towards his death, and the feeling grew as I climbed.

"There were times when Andy Schleck [who finished second behind Alberto Contador, and just ahead of Lance Armstrong and Wiggins] was attacking and it was horrible. I thought, 'I can't go on. I can't do this anymore …' But I then thought more vividly of Tom and how he must have felt that last day. It was like a reason not to give up. I felt like I was doing it more for his memory than anything."

Wiggins pauses, ducking his head shyly, as if he might have gone too far. But the 29-year-old, who has turned himself from a three-times Olympic gold medal-winning track pursuit rider into one of the world's great road cyclists, smiles wryly when I stretch out my arms to show him the sudden gooseflesh.

"That's the feeling I had," he says. "A lot of riders will take the piss out of me for saying this – but I don't think they fully respect the history of the race. A lot of young riders in the peloton think they're 'it' at 22. And I'm not referring to Mark Cavendish [the young British sprinter who won an extraordinary six stages of the Tour] in any way. But half the peloton don't even know that a person, Tom Simpson, died on the Ventoux. That's terribly sad."

Carrying a small photograph of Simpson with him, to steel himself for a climb that he feared might break him after three weeks on the road, Wiggins was aware that "90% of them rode slowly up the Ventoux, chatting to each other, knowing their race was over. It was different up front. We had to hit it hard – because the next few hours would decide where we ended on the Tour. I was exhausted, and emotional, but there were times when I felt fantastic."

Just as Simpson's drug-streaked death was a tangled affair, Wiggins's intense emotions are complicated. "It's difficult to describe because I feel a bit of a fraud in some aspects. Tom Simpson is like the Bobby Moore of British cycling. I wouldn't say he was a hero of mine because he was dead a long time before I was born. But I hold him in such high esteem. So it doesn't feel real. Great names finish up there in the Tour. People like [Carlos] Sastre …"

The Spanish winner of the 2008 Tour finished 13 places behind Wiggins this year. And yet, as I remind him, Wiggins outclimbed Sastre and Cadel Evans. "Yeah," he grins, "and [Andreas] Klöden [a former Tour runner-up who finished sixth]. I beat all those guys. The first time I realised I could do it was on Verbier [on the middle Sunday]. I attacked Armstrong's group and I had an out-of-body experience there – where I saw myself leaving Klöden, Evans and Armstrong in the Tour de bloody France. It didn't feel right. It was very strange."

He laughs quietly – but it's a rasping half-laugh, half-cough. It echoes again later when we walk through central London, and Wiggins talks about feeling so poorly, with his depleted body no longer able to fight the sniffles, sore throat and muscle-eating fatigue that followed his drastic weight-loss and ravaged immune system. He shed 20 pounds before the Tour started so that he could swap his power on the track for a skeletal lightness in the mountains. It is just one consequence of his epic tilt at sporting greatness. "I've been running at 4% body fat the last few weeks – and that's quite dangerous. I'm starting to get sick now, a little ill, just from being run down. I'm just glad I'm now free to put a few pounds back on."

Wiggins is aware of the irony that Simpson rode Ventoux the last time on amphetamines, chased down by brandy, with the drugs costing him his life. "I'm sure there were also a few grumbles about me, and suspicions, because I've come from nowhere. That's partly why I released my blood tests for the past two years last week. The evidence is there for people to see. My attitude is that if you have nothing to hide why not show it?"

The Tour's doping history is so grim that misgivings about others remain. Do the peloton have doubts in regard to Contador – who proved himself to be on a different level from every other rider in both the mountains and the time-trial? "I don't think so. I didn't hear any mention of doping on the whole Tour – which is amazing. I'm just hoping we don't get any drug revelations over the next few weeks."

Yet Greg LeMond, a former three-times winner of the Tour, suggests that Contador's VO2 max [his body's ability to absorb and use oxygen] must exceed that of any other athlete who has ever lived. "The burden is on Contador to prove he is physically capable of performing this feat without performance-enhancing products," LeMond argues. Contador's supporters retort that LeMond bases his measurement of VO2 max on flawed data, which fails to take altitude into account.

"I don't know much about any of that stuff," Wiggins says, shrugging. "I just know Contador's going to be the man to beat for the next five years. I don't think there was a big divide between me and Fränk Schleck [who finished fifth] and Lance. There is a gap between me and Contador that will be harder to close. But who knows how far I can go? Coming into this Tour I thought privately that I could make the top 10 – and my only doubt was on the mental side. I thought I would crack first mentally, rather than physically. But it's different now.

"Winning the Tour has to be my goal now. I would never say, 'OK, this is my goal: to finish third.' My goal is to win the race. Logically, I've got to be in with a real shot. I was a definite contender this year and I'm only going to get better. And Lance is only going to get older …"

The throaty cackle comes again – and Wiggins checks himself briefly. "But Lance is the only bike rider I've ever met who has that aura. He's like no one else. But we got on fine because I treat him like a normal person. It was different between him and [Armstrong's team-mate] Contador. It got to the point where I wouldn't have been surprised if they got off their bikes and ended up fighting each other. Everyone knew there was bad blood and it looks like, reading the war on Twitter between them, it's not yet over."

Armstrong is likely to be interested in Cavendish and Wiggins when he finally retires and concentrates on building his own team. If Cavendish speaks of his awe for Armstrong, Wiggins is more measured. "It would be interesting to ride for someone like Lance but Andy Schleck is going there. And after this Tour I've reached the point where I need to be the leader. I've gone up to another level – of Tour contender status – and once people wave chequebooks at you it changes a lot. I'm quite happy where I am and I've had success here. But we'll see what happens once my contract [with Garmin-Slipstream] ends next year."

The determination of Dave Brailsford and the British Sky team to entice Wiggins away from Garmin – if not next year then certainly in 2011 – clearly intrigues him. He praises his current team-mates, especially the previous leader, Christian Vande Velde, who stepped down to support Wiggins, but his relationship with British cycling is so strong that it is hard to imagine him not joining Sky.

"That idea definitely appeals to me. I'd like to ride the Tour one day with a British team. It would be like the track – no detail will be missed. The Tour is all about detail and this year, because I didn't expect to be in my position, we didn't recce the course or study the climbs. Contador and Lance, and the Schlecks, checked out the entire course. They knew the gradients of every climb and when it comes to gear-selection that helps a lot."

The prospect of Wiggins working with meticulous coaches such as Brailsford and Shane Sutton on future Tours – and fulfilling predictions of a British winner – might be made still more fascinating should Cavendish eventually complete the dream package. "We did something special this year, me and Cav," Wiggins says. "He said he was very proud of me and likewise me of him. Six stage wins is phenomenal – and he's now won 10 in two years. We're on to something incredible with British cycling. We've done it on the track but the medals we won in Beijing felt almost business-like – because it was so expected. The Tour was more magical."

Wiggins is in thrall to the Tour, rather than the 2012 Olympics, but feeling hungover and shattered he is still not in a position to make any coherent statement as to how he might balance his new ambition to win the Tour with the old desire for a few more gold medals in his home city. That decision will only emerge in the coming months, once his head has cleared and his body has recovered. But on a sunlit morning in London, with his glasses perched on his nose, as he remembers how they played the Sex Pistols's Pretty Vacant on the team bus every morning – "because that's exactly how we felt" – Wiggins looks to have shaken the ghost of Ventoux.

"I'll never forget that day," he says softly, "because it told me how far I've come. This time last month I didn't know what lay in store for me. I couldn't even guess how far I could take this. But now I know there is much more to come – because I just keep surprising myself."