Lords reform is closer than Nick Clegg thinks – provided he can jump over the series of ten-foot high hurdles that blocks his path to the finish line.

Most academics who have been following the Lords reform story are in a gloomy mood. They see the largest coalition rebellion yet, with 91 Tory MPs revolting against the very idea of an elected second chamber. They see the politicians failing even to agree on the process by which the legislation is going to be debated. And they've learned the lessons of their history books: for a century different generations have tried, and failed, to end the unaccountable, unelected second chamber which helps make Britain's laws.

Not much to raise their hopes of a genuine reform being achieved, then.

But if the coalition has learned the lessons of the past, this latest attempt doesn't have go the same way as all the others. There are signs that the ministers pushing for this reform have the right approach. They're playing a long game, using the full length of this five-year parliament to make progress. By doing so they might be able to give themselves at least half a chance of succeeding where so many before them have failed.

The first hurdle is winning over Labour. Mostly governments can get their way without getting the approval of the opposition. That doesn't work on fundamental constitutional change. If the reform is to survive in the decades to come it has to be agreed by both left and right, because consensus is essential if the Parliament Act is to be used in earnest against peers.

Ministers face some seasoned political operators among their counterparts on the opposition benches, however. So far this is all playing out exactly according to Labour's script. They're offering the carrot of their support for the legislation, but by causing trouble over the timetable of debate are giving themselves the leverage they need to wring concessions from the coalition.

That's why the private talks taking place in the next few weeks will be about more than just how many days parliament spends debating the reforms. Ed Miliband and co are likely to withhold their cooperation until they get big concessions on their big sticking points - like the referendum, for example.

Labour can afford to strike a hard bargain. It's no skin off Miliband's nose if the Lib Dems don't get their Lords reforms through. Equally, the opportunity for endless filibustering on the floor of the Commons might be in Labour's interest – it opposes the rest of the coalition's legislative agenda so would be happy to see that all held up.

Yes, the opposition is committed to Lords reform in principle – we saw that last night. But when all these other political factors are added into the equation, it faces a more complex dilemma. Ministers will be willing to make major concessions – but given the unlikely chances of them winning a referendum on the changes, agreement may prove elusive. In which case, Lords reform fails.

This isn't just about Labour. Once the opposition have finally been bought onside, ministerial attention has to turn to the government's backbenches. This isn't going to be easy, either.

Cameron can't afford to squander his political capital with the right of his party, which is already irritated by his equivocal position on the EU. So he has to do as much possible to placate the 91 backbenchers who rebelled yesterday. They are not numerically significant in terms of the Commons – as the whopping 338 majority showed – but their concerns have to be addressed if the sense of an uneasy truce is actually going to be achieved. The time to address their concerns will be when the bill's proposals are examined in detail on the floor of the Commons – assuming agreement is reached with Labour, of course.

Concessions will have to be well-timed and sufficiently bold to address the biggest sticking points. The 15-year non-renewable term is already being identified as a likely area where retreat could prove in the coalition's interest. In any normal piece of legislation that would be far too large a point to seriously contemplate abandoning. But this is Lords reform, where the bigger principle of securing an elected second chamber trumps all. The timing, as much as the nature, of the concessions is vital to success. Clegg stands half a chance of winning back some of the rebels – at least cutting their numbers down to a manageable level – this autumn. If he can't, Lords reform fails.

There is a third hurdle to be dealt with: the problem of where Lords reform fits in the broader strategic decision-making of David Cameron and Nick Clegg. Both party leaders have invested large chunks of personal credibility in ensuring the coalition goes the distance to 2015. So, having abandoned the idea of a drive towards 'midterm renewal', they need to do all they can to minimise the negative impact the Lords reform debate will have on relations between Tory and Lib Dem MPs.

Already the signs are not good. Bickering over what was said in the party manifestos and the coalition agreement resembles a divorcing couple arguing over the wording of their pre-nup, one commentator told me. He's right: the signs are not good. The longer this goes on, the more coalition relations will be stretched.

Cameron and Clegg's problem is that abandoning the changes won't lead to an outbreak of harmony, either. If the coalition collapse takes place it will be because of a withdrawal of support from the Lib Dem grassroots – a possibility made much more likely if Lords reform fails.

If this broader context has a measurable impact, it's going to be to increase the chances of ministers making substantial concessions just to get this nightmare past them. That actually makes success more likely: for whether it's dealing with Labour, or Tory rebels, or coalition malcontents, flexibility is ministers' secret weapon.

Don't listen to those saying that Lords reform has been kicked into the long grass after last night's debacle. The coalition's ministers, taking their time and ready to retreat on the details, can still achieve Lords reform. Just so long as they can get over those hurdles, that is.