If Sarkozy loses the second round of the French presidential election on 6 May, the implications for the French right will be massive. The UMP party, which has ruled France for 10 years, and which Sarkozy took over in 2004, could face years of turmoil. The far right, led by Marine Le Pen, is already lurking in the shadows hoping to take over as the main opposition party.

The first UMP was founded in 2002, by Jacques Chirac's Gaullist RPR and Valéry Giscard d'Estaing's centrist UDF: the motivation for the "Union pour une Majorité Présidentielle" ("Union for a Presidential Majority") was to support Chirac's bid for re-election. The idea was to create a French equivalent to Germany's CDU: a vast centre-right movement to end divisions once and for all and win presidential elections.

To an extent, that tactic worked: for 10 years, the new UMP ("Union pour un Mouvement Populaire") triumphed at a local level and failed in local elections. Nowadays, the Socialist party controls most of the major cities, nearly all local powers and the Senate. François Hollande's victory on 6 May would leave France in the hands of a single party and the UMP in a very difficult position.

If Sarkozy loses, the battle for the UMP will start on 7 May and whoever wins control of the party is likely to be the party's presidential candidate for 2017. The main competition is likely to be between the current party leader Jean-François Copé, the prime minister François Fillon, (58) and the foreign affairs minister Alain Juppé (67). Copé has already announced his intention to run in 2017, whereas Fillon has never made it clear. Juppé joked about running but his age (he will be 72 in 2017) seems to rule him out.

Copé is often seen as a younger version of Nicolas Sarkozy, but with better manners. He has the same raw energy, the same blunt way of putting forward his ideas, the same cunning ways with the media; but he will have to convince supporters that he is able to unify the party, and the country, around him. He has already gathered the support of several young ministers (eg François Baroin, Bruno Le Maire, Valérie Pécresse), his own "musketeers".

Fillon's situation is a paradox: discreet, traditional and softly spoken, he will have spent five years as Sarkozy's prime minister but still manages to come across as his nemesis. Surprisingly, Fillon has never built his own group of supporters within the party and is mainly backed by those who do not wish to see Copé carry on as party leader. Fillon will now try his luck in Paris, where he believes he can build a new power base to propel him to the French presidency. We have already witnessed skirmishes between Copé and Fillon in recent months and this could degenerate into all-out war if not prevented.

To avoid a trench war and the implosion of the UMP, a third man could emerge and settle the dispute. This man could be Juppé, France's "comeback kid". Prime minister in 1995, Juppé was the UMP's first president in 2002 before he got struck down by a 14-month suspended sentence in a corruption case. Juppé returned to frontbench politics in 2010 when he joined Sarkozy's government. Thanks to his good results as foreign secretary, Juppé has reacquired his senior status within the party as well as a strong popularity with the French public. Many see in him the best way to avoid a brutal internal competition without upsetting the presidential hopefuls for 2017.

If Sarkozy loses, the UMP will be given a clear choice: unite or die. The former will allow the right to rise again and start winning back local powers as soon as 2014; the latter would alter France's political landscape dramatically for many years.

• Follow Comment is free on Twitter @commentisfree