The 40-year grip of the two main centrist political groups on the levers of power in Brussels looked set to be broken as voters in the European elections turned out in record numbers to bolster radical alternatives including the Greens and the far right.

A populist Eurosceptic surge failed to emerge on Sunday but they were on track to be returned to the European parliament in larger numbers than ever before, with Marine Le Pen’s National Rally celebrating a narrow symbolic victory over Emmanuel Macron’s En Marche in France.

There was also major success for the Greens across Europe, with the group jumping from 50 MEPs in 2014 to around 70, and almost doubling their result in Germany from the previous election to leapfrog the Social Democratic party (SPD) into second place with 22%, the exit polls suggested.

As votes for the centre parties fall away, the results will trigger tortuous negotiations over the key jobs in the EU institutions, including Jean-Claude Juncker’s replacement as European commission president.

The makeup of the parliament will be used by the 28 heads of state and government to guide their choice of replacement for Juncker and his counterpart in the European council, Donald Tusk.

The parliament will have a veto over any choice of commission president and on the whole of the new commission team.

“I guess that some marginal parties will be less marginal tonight,” Juncker said as he cast his vote in Luxembourg where he was prime minister for 18 years.

The estimated results based on exit polls leave the centre-right European People’s party as the largest in the parliament, but down from 221 seats to 179.

The Socialists and Democrats group also appeared set to drop from 191 seats to 150, leaving the two main groups looking likely to need help from Alliance of Liberals and Democrats (ALDE) with about 107 seats, bolstered by Macron’s En Marche, and the Greens to form a stable majority.

The Europe of Nations and Freedom group, which combines populist and far-right parties in countries including France and Italy, looked set to secure 58 seats, up 18 from five years ago. The Europe of Freedom and Direct Democracy, home of the Brexit party, increased its projected seats from 48 to 56.

After polls suggested it would secure up to 34%, increasing its seats from 5 to 28, Italy’s far-right League, headed by deputy prime minister Matteo Salvini, was vying with Nigel Farage’s new party as to which will be the biggest single party in the parliament.

Speaking in Milan, Salvini said: “A new Europe is born. I am proud that the League is participating in this new European renaissance.”

Manfred Weber, the EPP’s lead candidate for the commission presidency, said the EU was facing a “shrinking centre” but that he would not seek to build a majority in the parliament with the extreme right.

“For the EPP today the feeling is not one of victory because we are losing seats,” he said. “We are happy to be the biggest group in the European parliament. That is what we are happy about that but we also see we lose seats.”

Weber added that the Greens were “the winners” of the night and he would be open to holding talks with their leaders and the ALDE group over building a majority in the parliament to pass legislation and approve the commission team.

The Greens’ Philippe Lamberts (left) and Ska Keller said they would use the party’s success to push for climate action. Photograph: Olivier Matthys/AP

The Greens co-leader, the Belgian MEP Philippe Lamberts, said: “To make a stable majority in this parliament the Greens are now indispensable.”

Guy Verhofstadt, the leader of the Alliance for Liberals and Democrats for Europe party (ALDE), said: “For the first time in 40 years, the two classical parties, socialists and conservatives, will no longer have a majority. And that means that no solid pro-European majority is possible without the help, without the participation of our new centrist group.”

Turnout across the EU27 member states was looking to have hit 50.95% – the highest since 1994 and breaking a four-decade downward trend. In 2014, 42.6% of the electorate took part.

But the surprise rise in the number of voters going to the polls did not lead to success for the traditional parties in the major states.

In Germany, both the large centrist parties had a sobering night, with Angela Merkel’s Christian Democratic Union dropping from 35.3% of the vote in 2014 to 28%, and the centre-left Social Democratic Union facing the worst European results in its history, down to 15.5% from 27.3% five years ago. The exit polls suggested the populist Alternative für Deutschland (AfD) was up marginally on a projected 10.5%.

While staunchly pro-EU parties are still slated to win 493 of the 751 seats in parliament, Macron’s attempt to pitch himself as the saviour of the EU also looked to have backfired.

Le Pen’s rebranded National Front party secured about 24% of the vote, compared with 22.5% for the French president’s En Marche.

Macron had called the elections “the most important since 1979”, warning the EU was facing “an existential risk” from nationalists seeking to divide the bloc.

Speaking as the exit polls emerged, the head of En Marche’s list of MEPs, Nathalie Loiseau, insisted the “fight isn’t over, we will continue the fight in the parliament”.

With more than 98% of the vote counted, Spain’s ruling socialist party (PSOE) won an emphatic victory taking 20 seats and 32.8% of the vote less than a month after finishing first in the general election. The far-right Vox party, which picked up 24 seats in the general election, finished fifth with three seats and 6.2% of the vote.

The Greek prime minister Alexis Tsipras called for an early national election after the conservative New Democracy party secured 33.5% of the votes ahead of his Syriza party on 25%.