THE LESSONS OF THE HANOI HILTON

Asked how he survived seven years of torture and isolation in the Hanoi Hilton prison, US pilot James Stockdale explained: "I never lost faith in the end of the story, I never doubted not only that I would get out, but also that I would prevail in the end and turn the experience into the defining event of my life, which, in retrospect, I would not trade."

Asked about the captives who died inside, Stockdale explained that they were: "The optimists. The ones who said, 'We're going to be out by Christmas.' And Christmas would come, and Christmas would go. Then they'd say, 'We're going to be out by Easter … They died of a broken heart."

Stockdale was a fighter pilot, a lecturer in Stoic philosophy at Stanford, and one of the least successful candidates for the vice-presidency of the USA in modern history. He was a serious man, and one worth listening to. The first time his captors tried to force him to appear in a propaganda film, he beat his face to a pulp with a wooden stool. The second time, he sliced his scalp with a razor blade. The third time, he cut his wrists. After that they stopped trying to make him appear in propaganda films.

It seems a little ridiculous to take the thoughts of such a man and turn them to such mundane matters as sport, or business, but that was exactly what the writer James Collins did.

"This is a very important lesson," Stockdale told Collins. "You must never confuse faith that you will prevail in the end – which you can never afford to lose – with the discipline to confront the most brutal facts of your current reality, whatever they might be." Collins called it the Stockdale paradox.

Collins turned that dichotomy into a central tenet of his best selling book on management, Good to Great, describing how firms can make the transition from one to the other. If Andy Flower had more than three days between these two Tests, he'd do well to get himself a copy and read it.

England have done it the other way. They have gone from great to good. Or perhaps worse. They still have three matches to play in 2012, but right now this has been their least successful year of Test cricket since 1999. In the last 11 months, England have played 12, won three, lost seven, and drawn two. That record leaves them with a win/loss ratio of 0.42, their lowest since '99, when they fell to the bottom of the unofficial world rankings. That year it was 0.25. The particular perversity of their current predicament is that the slump follows two of the most successful calendar years in the team's history: in 2010 and '11, England played 22, won 15, lost three, and drew four. And this is part of their problem.

When Andy Flower took on the job of head coach, and whether he knew it or not, he stuck closely to Stockdale's formula. He believed that he and his team would get to No1 in the world, and that they would do it by sticking to certain principles that they had absolute faith in. He had a core group of players, and he stuck by them – 13 men who have won their first Test caps under Flower, but only Jon Trott has gone on to become a fixture in the side. And he arranged them into an XI that had a steadfast top-three, and, more often than not, a four-man attack which included one spinner. Their strategy, put simply, would be to bat steady, bowl dry, and field sharp.

At the same time, Flower never shied away from "confronting the brutal reality" of the situation the team was in. He did it in Jamaica, after England had been bowled out for 51. Ian Bell was dropped, simply to make the point that more was expected from the senior players. The excruciatingly honest meeting the team held the next day at another

Hilton, Kingston this time, not Hanoi, set the tone for the next three years. They did a similar thing in 2009, after they were hammered by Australia at Headingley in the fourth Ashes Test. Flower introduced an ethos of accountability and responsibility. England cultivated an atmosphere where fielders who fumbled drives could expect to get sledged by their own team-mates. And it worked. England became so remorseless, so ruthless, so professional, that they were almost unrecognisable to their own fans. Then, things fell apart.

England have only won one Test in India in the last 19 years, so there is nothing shocking about the fact that they lost again in Ahmedabad. It was the manner of the defeat that was alarming. Flower and the "intelligent men", as Alastair Cook calls them, in the management team still cleave to the old principles that took them to No1 in the rankings, as they have done all year.

Strategically, England prefer to play to their own strengths rather than adapting to the circumstances they are in. So they picked four bowlers and three seamers, and left out Monty Panesar. Tactically, they set themselves up to do what they have always done – the seamers were supposed to bowl dry, and look for reverse swing. The top three were supposed to bat for a long time, allowing the middle-order a little licence to accelerate, with the strong tail-end batting there as a bulwark against a collapse.

It didn't work. The three fast bowlers could not conjure any reverse swing, and, for a large part of the first innings, bewildered and bludgeoned by Virender Sehwag, they seemed unsure whether they were supposed to be taking wickets or trying to contain the batsmen. The result was that too many otherwise good overs included bad balls, and their economy rates were too expensive. The Indian seamers displayed more skill, and more speed. England's quicks were outbowled. The established batsmen, with the conspicuous exception of Cook and Matt Prior, seemed to be suffering from similarly clouded thinking. And the fielding was lacklustre. England's bowling, fielding, and batting coaches, David Saker, Richard Halsall, and Graham Gooch must also, like Flower, ask themselves whether what worked once still works now.

It feels like England have become too entrenched in their thinking, too dogmatic in their approach. That happened to England's last successful coach, Duncan Fletcher, who became so hell-bent on having a 'keeper and a spinner who could bat, and a pair of 90mph bowlers, that he stuck with Ashley Giles and Geraint Jones, and picked Saj Mahmood and Liam Plunkett. Success can blunt the ability to carry on asking difficult questions, especially when it comes to analysing your own performance.

Now, Flower must do something Fletcher never quite did, and re-confront that "brutal reality". Do England want to stand by their policy, seemingly so fundamental, of picking four bowlers and three quicks? Do they want to drop Stuart Broad, who is vice-captain, to make room for Steve Finn, who bowled so well on England's ODI tour to India last year? Are they prepared to go into the match with Graeme Swann batting at No8, shorn of the security, which has saved them so often, of having a team that bats all the way to No10? Do they want to be consistent in how they treat the batsmen, as they always have been before, or should they be more expedient? Jonny Bairstow is the next man in line, having scored 95 and 54 in his last Test. But is he really a better bet than Eoin Morgan to replace Ian Bell? Should England drop Samit Patel after just a single Test at No6?

The certainties that served England so well in 2009, '10, and '11, must all be examined with the same honesty that Flower brought to that team meeting in 2009. It's possible that some of his principles – the refusal to play a five-man attack, for instance, or the insistence on batting down to No10, or on playing Broad – were, as the footballer Steve Archibald once said of team spirit, only illusions glimpsed in the aftermath of victory. With a new captain in place, England have a chance to start over again. They must ask which of those policies is still fundamental to their faith that they will prevail in the end, and which have become hindrances, and must be discarded.

• This is an extract from the Spin, the Guardian's free weekly cricket email. To sign up, click here.