The Iranian president, Hassan Rouhani, has won a sweeping endorsement for efforts to end Iran’s international isolation and bring greater freedoms at home, with an unexpected landslide victory in a fiercely contested re-election bid.

His powerful mandate protects the nuclear deal, which has been his landmark achievement to date, and his courting of foreign investment. It could also have much longer-term implications for Iran’s future, by giving reformists a greater influence over the looming battle to choose a new supreme leader.

Polling stations were forced to stay open until midnight in parts of the country because so many Iranians wanted to vote, defying fears of voter apathy. Rouhani claimed 23.5 million votes, while his rival Ebrahim Raisi trailed on 15.8 million, after nearly three-quarters of the electorate cast their votes, the interior ministry said.

In a victory speech to the nation on live TV, Rouhani promised to rule for all Iranians. But while celebrating his huge mandate in the election, which he labelled the “most competitive ever”, he also described his opponents as dangerously backward-looking.

Iran presidential elections: everything you need to know Read more

“Yesterday, you said no to those who wanted us to return to the past,” he told the nation. The scale of his victory provides a strong platform to challenge hardliners who still hold ultimate control in a Iran’s unwieldy hybrid of theocracy and democracy.

And in a signal that he planned to turn an outspoken campaign into a combative second term, Rouhani also thanked reformist figurehead Mohammad Khatami, his most important ally and backer. Security forces have banned any mention of the hugely popular former president’s name in the media, meaning Rouhani crossed a red line just hours into his new term.

“Millions and millions of people are happy because Rouhani won,” said businessman Ahad Esmaili, 31, one of a crowd breaking into dance at a spontaneous celebration in the heart of Tehran’s crowded bazaar, when the final figures were announced.

The election was a tense showdown between Rouhani and hardliner Raisi, both senior clerics but with little else in common. The challenger consolidated conservative support behind his initially lacklustre bid for power, by mounting a campaign that mixed economic populism with religious conservatism and an isolationist foreign policy.

Raisi’s last-minute surge may have unwittingly helped Rouhani, as moderates spooked by the prospect of slipping back into international isolation and stricter controls at home raced to the polls.

“I’m even happier than I was four years ago when he won the first time,” said tailor Mariam Farmayeshi, 34. “My husband voted for the first time in 20 years, because he thought it was necessary to keep out Raisi.”

Watch salesman Yousef Khaleghi said he spent the entire day driving dozens of friends and relatives to the polls. He had gone bankrupt during the government of Mahmoud Ahmedinejad and was determined to do everything he could to prevent another hardliner coming to power.

Iran election: frantic campaigning as high-stakes vote goes down to wire Read more

“I love Mr Rouhani,” he said with a grin, before adding he was grateful to Iranian women for their part in the re-election success. “We should acknowledge that Iranian women did a lot for him to win. At each polling station (in areas that supported Rouhani) they made up more than half of the voters waiting in line,” he added.

In an apparent nod to that vital support base, Rouhani thanked voters on Instagram with a picture that is likely to particularly outrage conservatives smarting from their loss.

It shows a family group celebrating their vote, with three of the younger women wearing colourful clothes and headscarves set back so far on their heads that they are barely visible. “Great people of Iran, you are the true winners of this election,” he wrote underneath.

Many of the women who turned up to vote for Rouhani felt their personal freedoms were under threat from Raisi, whose supporters frequently accused the president of abandoning Islamic values. Many were particularly exercised about women’s dress, at one rally even handing out suggestions about covering up to women who they deemed not appropriately clothed.

“I felt much better with Rouhani, more secure and freer. If his rival had come to power there would have been more restrictions on women,” said Tehran housewife Pantea Mehrabadi, 46. “I voted for him first because I wanted to support him, but also to combat Raisi.”

But victory also comes with a heavy weight of expectations that Rouhani may find it hard to fulfil, given the constraints of Iran’s complex government system and the weight of a US sanctions regime that Washington is in no hurry to lift.

The end of nuclear sanctions that followed his landmark deal was not followed by the hoped-for flood of foreign investment because unilateral US sanctions stayed in place, making doing business in Iran complicated or illegal.

For Rouhani to meet the expectations generated by his victory he will need western governments to push for those sanctions to be rolled back, or to step up investment in the areas they allow.

Although victory has tilted the political balance towards reformists in the short term, Raisi secured a face-saving vote tally high enough to mean that he is not finished politically, and lying ahead is the contest over who will be the new supreme leader.

Ayatollah Ali Khamenei holds the position for life, but he is 77 and thought to be in poor health. A hardliner keen to preserve his legacy, he is believed to have tacitly backed Raisi as president, and possibly favoured him as a possible successor.

After the vote he issued a statement addressed to the Iranian people in which he praised the “massive and epic” turnout. However in contrast to the 2013 elections, he offered no congratulations to Rouhani.

While Raisi lost the election, he won enough support to preserve his political career. His 16 million votes, combined with success in persuading hardliners to back him, could put him in a good position to run in 2021 when Rouhani will be barred from seeking another term in office.

“Mr Rouhani should not forget that more than 16 million people did not vote for him,” Reza Gholami, a cleric allied with the hardliners, was quoted as saying by the semi-official Fars news agency. “So he should respect their right to criticise him.”