The lights dim and a dramatic Lord of the Rings-style score fills Strasbourg’s Palais de la Musique. The 1,200-seater auditorium might only be three-quarters full, but the crowd is fired up. People jump to their feet applauding, waving plastic French and EU flags, as the big names of France’s centre-right Les Républicains party file into the hall. In their midst is a little-known German politician.

Manfred Weber, a 46-year old Bavarian MEP who wants a European FBI and “a European master plan” against cancer, has been on the campaign trail for months. But he is not standing for office in France. Welcome to the race to lead the European commission.

Armed with Barack Obama-inspired images of his face on T-shirts and biscuits, Weber has crisscrossed the continent. In Ireland, he donned green wellies to chat to farmers. In Portugal, he inspected a port wine factory and fish market. In Cyprus, he admired hand-embroidered linen and did a little stitching.

What are the European elections and European parliament? Read more

Then came Strasbourg, the French city near the German border that is a symbol of Europe’s postwar reconciliation. A few hours before the rally, Weber visited an asparagus farm with French conservative allies – Laurent Wauquiez, the hard-right president of Les Républicains, and François-Xavier Bellamy, a boyish-looking philosophy teacher, who was a surprise choice to lead the party’s MEP list in this month’s elections. Surrounded by a thicket of cameras on the edge of a muddy field, the farmer Jean-Charles Jost outlined to his guests his problems in attracting Polish workers to pull the thick white asparagus out of the soil.

It never used to be like this for those who wanted to lead the EU civil service, which has 32,000 officials. In the past, a conclave of European leaders haggled over the man for the job (in 62 years, the EU executive has never had a female leader).

The backroom dealing was challenged in 2014 when the European parliament seized the initiative by devising a system to favour the “lead candidate” of the political group that won the largest share of the vote in European elections. Known by its German name, the “spitzenkandidaten” system enabled the triumph of Jean-Claude Juncker, a former prime minister of Luxembourg, well-versed in the arts of late-night EU dealmaking. Juncker succeeded despite the misgivings of EU leaders, such as Germany’s Angela Merkel, as well as outright opposition from David Cameron.

“The leaders of the European council completely underestimated the power of the European parliament,” recalls Luuk van Middelaar, a European council adviser at the time. “Merkel still wanted to dump [Juncker], but she couldn’t do it,” says Van Middelar, who points out that by winning the public relations battle, the parliament outmanoeuvred the EU leaders.

The EU leaders do not want to be caught out again: the European council president, Donald Tusk, reminded parliament last year that there was “no automaticity” that the spitzenkandidat would get the top job in 2019. Under EU law, European leaders propose a candidate and parliament approves him or her by a simple majority vote. The 48 hours following the election results on 26 May will be decisive in determining which institution takes control. “It will be a power struggle,” predicted Van Middelaar.

Facebook Twitter Pinterest Weber (centre) alongside Annegret Kramp-Karrenbauer, the leader of Germany’s Christian Democratic Union and a Merkel ally. Photograph: Wolfgang Rattay/Reuters

It will also be another chapter in the decades-long battle over who runs Europe: the federal institutions, such as the parliament, or the inter-governmental club of leaders.

“Tell me a better concept,” Weber says, sitting in a Strasbourg barn after sipping chilled Alsace wine with local farmers. “The old concept was to go to the European council, they close the door and they have chosen the leader for tomorrow. And that is not a democratic process.” He argues the EU cannot go back to the old ways. “The power of the argument that we tell people the truth before the elections is so strong.”

EU governments reject the charge that the choice of 28 democratically elected leaders lacks legitimacy. Parliament’s claim to have devised a more democratic system was “bullshit”, one minister said last year.

And while Weber has to convince an electorate of 28 people, millions of European voters have no idea who he is.

Only 26% of Germans know their compatriot, according to a recent poll for the German Press Agency. French voters at the Les Républicains rally were equally nonplussed. “Honestly, I don’t know who this is,” said Jean-Emmanuel Robert, a 43-year-old Strasbourg municipal councillor, on seeing a photo of Weber. “Is it Martin Schulz?” he said, referring to the German Social Democrat politician. He came to hear Bellamy, the rising star of the French right, who has given the party a boost in the polls.

Honoré Mansella, a 70-year-old holding an EU flag, also did not recognise Weber from his photo. He thought the spitzenkandidaten process sounded “a little bit abstract”. He said: “This is not a direct election; there is no president or chancellor … it is rather complex and we have absolutely no influence on who gets chosen.”

Another man, who declined to give his name, did recognise Weber, but said he could not say if he approved of him. “It depends on his position on Strasbourg,” the man said, referring to a recent proposal by the German CDU leader, Annegret Kramp-Karrenbauer, to abolish the “anachronism” of the European parliament’s second seat in Strasbourg. “Madame Mini Merkel is against Strasbourg, so I will wait to see what he says,” said the man.

Back in Brussels, EU watchers are more concerned about Weber’s lack of executive experience. After 15 years in the parliament, Weber has run neither a government nor a big national ministry. “He is a candidate that doesn’t come with the credentials that seem to have been important for European commission presidents in the past,” said Corinna Hörst, a senior fellow at the German Marshall Fund thinktank. “I could see the heads of state and government throwing in a wrench.” Hörst thinks Weber has the “lukewarm” support of Merkel, who wants her centre-right ally in the commission at a time of political fragmentation.

The shattering of political certainties means the traditional blocs of centre-right and centre-left are set to fall below 50% for the first time in 40 years. Any commission president will need a bigger, multi-party coalition in order to gain the backing of a majority of the parliament’s 751 MEPs. “I am already in talks with the Liberals, with the Greens, with the Socialists, with the broad European parties to establish afterwards a majority in the European parliament,” Weber said.

In the race to succeed Juncker, he faces competition from the Socialist candidate, Frans Timmermans, a multilingual former Dutch foreign minister and commission vice-president. Many fancy the chances of the charismatic EU competition commissioner, Margrethe Vestager, one of seven lead candidates for the Liberals. Rumours persist about the chief Brexit negotiator, Michel Barnier, who stood unsuccessfully against Juncker in 2014 but insists he is “not a candidate today”.

Facebook Twitter Pinterest Margrethe Vestager is among those vying with Weber for the top job. The EU executive has never had a female leader in its 62 years. Photograph: Anadolu Agency/Getty Images

Some observers think Weber’s cross-party appeal could be more limited than that of Juncker, who liked to call himself a “Christian Social Democrat”. “That was the ultimate seduction across the aisle,” recalls one senior official.

By contrast, Weber’s stress on Europe’s Christian heritage – a legacy, he says, that “should guide us for the future” – could be seen as divisive.

Weber’s biggest applause lines in Strasbourg were when he promised to “cut European regulations” and abandon Turkey’s EU accession talks (in reality, going nowhere). Yet a bonfire of EU laws – Weber has promised to cut more than 1,000 “outdated regulations” – and a needless, potential conflict with Ankara could deter supporters on the centre and left.

The Brexit delay has heightened uncertainty. If 73 British MEPs are returned to the parliament, that will boost Weber’s rival in the Socialist camp, who gain from the return of Labour MEPs.

At the Palais de la Musique, Weber went down well. His closing appeal to “defend our European way of life” received a standing ovation. But he was not the star that night. It was “Bellamy, Bellamy”, as the young activists chanted again and again. As the evening closed the politicians took to the stage to sing the Marseillaise. Joining them, Weber looked a little awkward. He was smiling, but did not know the words.