Sinn Féin will try to form a government in Ireland after apparently winning more votes than any other party in Saturday’s general election – a historic result that upended the political system.

The party leader, Mary Lou McDonald, told cheering supporters on Sunday that a “revolution” had occurred and she would try to form a ruling coalition with other parties. “This is no longer a two-party system,” she said.

Ireland election: latest results Read more

Sinn Féin, once a pariah for its IRA links, won almost a quarter of first-preference votes, possibly pipping Fine Gael and Fianna Fáil, two centrist rivals that have taken turns ruling Ireland for a century.

It rode a wave of anger over homelessness, soaring rents and hospital waiting lists as well as disillusionment with the traditional political duopoly.

McDonald, speaking over rapturous, deafening chants at a Dublin count centre, said she had spoken to the Greens and small leftwing parties in hope of forming a coalition without Fine Gael or Fianna Fáil – an unlikely scenario. She did not rule out a deal with either Fine Gael or Fianna Fáil.

Parliamentary arithmetic may exclude Sinn Féin from power, however. Fianna Fáil and Fine Gael ran more candidates and are expected to each win more seats than Sinn Féin in the 160-seat Dáil Éireann, the Irish parliament’s lower chamber, leaving unclear which parties – if any – will be able to form a viable coalition. Deadlock could lead to another election.

Q&A What are the main political parties in Ireland? Show Hide Fine Gael

Its name can be translated as family or tribe of the Irish. A centre-right party with a socially progressive tilt. In office since 2011, first led by Enda Kenny, then Leo Varadkar, with support from smaller coalition partners. Traces roots to Michael Collins and the winning side in Ireland’s 1922-23 civil war. The party traditionally advocates market economics and fiscal discipline. Appeals to the urban middle class and well-off farmers. Fianna Fáil Its name means Soldiers of Destiny. A centrist, ideologically malleable party that dominated Irish politics until it steered the Celtic Tiger economy over a cliff, prompting decade-long banishment to opposition benches. Under Micheál Martin, a nimble political veteran, it has clawed back support and may overtake Fine Gael as the biggest party and lead the next coalition government. Founded by Éamon de Valera, who backed the civil war’s losing side but turned Fianna Fáil into an election-winning machine. Sinn Féin Its name means We Ourselves, signifying Irish sovereignty. A leftwing republican party that competes in Northern Ireland as well as the Republic. Traces roots to 1905. Emerged in current form during the Troubles, when it was linked to the IRA. Peace in Northern Ireland helped Sinn Féin rebrand as a working-class advocate opposed to austerity. Under Mary Lou McDonald, a Dubliner without paramilitary baggage, Sinn Féin has become the third-biggest party, and its vote share surged in the 2020 election. Others Partnership with Fine Gael during post-Celtic Tiger austerity tainted the centre-left Labour party. The political arm of the trade union movement, it is led by Brendan Howlin, a former teacher and government minister. The Social Democrats and Solidarity-People Before Profit are part of an alphabet soup of smaller, more leftwing parties. The Greens, wiped out in 2011 after a ruinous coalition with Fianna Fáil, have campaigned on the back of climate crisis anxiety and youth-led protests. Independent TDs have prospered in recent elections, turning some into outsized players in ruling coalitions. Rory Carroll

During the campaign Leo Varadkar, the taoiseach and Fine Gael leader, and Micheál Martin, the leader of Fianna Fáil, both ruled out entering government with Sinn Féin, citing ethical and policy reasons.

On Sunday evening, Varadkar told journalists in Dublin: “For us, coalition with Sinn Fein is not an option, but we are willing to talk to other parties.”

Facebook Twitter Pinterest Fine Gael leader and taoiseach Leo Varadkar arrives at the vote count centre in Dublin. Photograph: Donall Farmer/Getty Images

Varadkar, Ireland’s first gay taoiseach, had hoped a healthy economy and his record on Brexit would deliver a third term for Fine Gael, but he encountered voter fatigue and anger over the cost of living and state of public services.

Varadkar’s party colleague Richard Bruton, the outgoing environment minister, said Fine Gael’s poor but not disastrous results gave it a chance of forming another government.

Asked if Fianna Fáil would now consider sharing power with Fine Gael or Sinn Féin, Martin appeared to leave the door ajar, citing a need for stability amid political fragmentation. “The country comes first … there is an onus and an obligation on all that such a functioning government is formed after this.”

Many in Fianna Fáil, stung by the backlash over their confidence-and-supply deal with Varadkar’s outgoing government, would prefer a deal with Sinn Féin than another centrist alliance.

Ireland’s single transferrable vote system of proportional representation means it could be Monday, Tuesday or even later before all Dáil seats are allocated.

With 96% of first-preference votes tallied on Sunday, Sinn Féin had 24.1%, with Fianna Fáil on 22.1%, Fine Gael on 22.1%, Greens on 7.4%, and small leftwing parties and independents comprising the rest.

It was a stunning result for Sinn Féin, which was the IRA’s political wing during the Troubles and remained a fringe party in the republic until well after the 1998 Good Friday agreement.

It surpassed its 2016 election result of 13.8% by appealing to voters – especially the young – who felt left behind by a booming economy and chafed at soaring rents, homelessness, insurance costs and hospital waiting lists.

Preliminary vote tallies suggested Sinn Féin could win around 36 seats, up from 22 in the outgoing Dáil, far exceeding its own expectations.

Many candidates ratcheted up huge surpluses in urban heartlands while others appeared on course for unexpected victories in Galway, Tipperary, Roscommon, Mayo and Wexford.

Gerry Adams, who stepped down as party leader in 2018 and as a Dáil member in this election, credited McDonald’s leadership and said he had not foreseen the extent of the gains. He said Sinn Féin would use its mandate to plan for a united Ireland – a defining tenet for the party.

On Sunday night he tweeted: “I’m disappointed that [deputy prime minister] Simon Coveney says Fine Gael won’t talk to Sinn Féin. Obviously a misguided effort to wrong foot Fianna Fail. But I thought he was better than that. Incompatible policies fair enough. But has he learned nothing from the DUP? Sinn Fein voters lesser voters?”

The issue of a united Ireland barely featured in the campaign, but an exit poll of voters found most supporting a border poll in the next five years. Sceptics say that could destabilise both sides of the island. Unionist leaders in Northern Ireland gave no immediate response to the results south of the border.

People across the political spectrum said the election was seismic, even if its consequences remained unclear. The former Labour leader Pat Rabbitte said a tectonic shift had killed the old system of Fianna Fáil and Fine Gael alternating in power with Labour or other small, establishment parties.

The commentator Fintan O’Toole wrote in the Irish Times that young voters had shattered the taboo of backing a party associated with terrorism. “They have gone where they were warned not to go and in doing so they have redrawn the map of Irish politics to include territory previously marked ‘here be dragons’.”