Thailand's Democrat government has conceded defeat in Sunday's election – putting Yingluck Shinawatra on course to become the country's first female leader, five years after her brother Thaksin was toppled as prime minister in a coup.

Preliminary results suggest a remarkable turnaround for Thaksin, a billionaire now living as a fugitive in Dubai. Although Yingluck is putatively leader of the Puea Thai party, she is regarded as his proxy.

Yingluck, a 44-year-old businesswoman who entered politics just six weeks ago, cautioned that she was waiting to see the results on Monday. But she added that she had already spoken to the Chart Thai Pattana party, whom she said would take a coalition past the halfway mark in the 500-seat parliament.

"We have tough days ahead and all of this is just the beginning," she said. "I'll do my best and will not disappoint you."

Five hours after polls closed, the election commission projected Puea Thai would win 261 seats with prime minister Abhisit Vejjajiva's Democrats taking just 162. An absolute majority would prevent the weeks of horse-trading to form a coalition that many had expected.

Hundreds of redshirt supporters squeezed into the party's Bangkok headquarters and crowded outside – chanting, cheering and applauding as results came in. Many waved pictures of Yingluck and Thaksin and some let off firecrackers to celebrate.

Moments before Yingluck spoke, Abhisit congratulated the opposition "for the right to form a government". He said he wanted to see unity and reconciliation, and added that the Democrats were ready for opposition.

Analysts have warned that the election could lead to further turmoil in Thailand, after years of intense political conflict. Last year more than 90 people died in clashes as the military cracked down on Thaksin-supporting redshirt protesters in the centre of the capital.

Exit polls had predicted a landslide for Puea Thai, but Dr Andrew Walker, an expert on south-east Asian politics at the Australian National University, said even an absolute majority would be remarkable.

"If they get [one] they will be only the second government in Thailand's history to do so, the first being Thaksin Shinawatra's [in 2005]. It shows this is still a very strong electoral brand," he said.

Earlier, Professor Thitinan Pongsudhirak of Chulalongkorn University said: "If they win at all it's a big statement.

"[It means] the ideas and policies that made [Thaksin's] original Thai Rak Thai party so electable are unstoppable and indestructible … This is a party that has been dissolved twice; its leading politicians have been banned twice; it's being led by a deposed exile and former prime minister a six-hour flight away."

Redshirt leaders have warned they will take to the streets again if Yingluck wins the vote but does not become prime minister. They have said they fear that opponents could attempt to mount a legal challenge to her, or even another coup.

Analyst Chris Baker said it was likely that opponents of Thaksin had a plan B. "[Most] likely, I think, is that it will look calm for a short time. But there have been several things said in the last week that suggest the Democrats and their supporters will accept this result – but not what it means," he said. "If Puea Thai do what they say they will – bring Thaksin back and change the constitution – [opponents] will resist that in the same way as before, with street demonstrations and so on. I don't think we can avoid it … We are in a changing political society and there's big resistance from the old institutions: the bureaucracy, monarchy, military and a lot of the middle class."

Thaksin told the Thai PBS television station: "I have wanted to come back since yesterday, but I do not want to create problems."

He spoke in a telephone interview from Dubai, where he lives to avoid a two-year prison sentence for abuse of power, which he says was politically motivated.

Puea Thai leaders have repeatedly indicated they plan an amnesty allowing Thaksin to return to Thailand, though his sister has said it is not a priority and that an amnesty would not be about one person.

The billionaire has polarised Thai politics. He draws his support largely from poorer residents in the north and north-east – who see him as a champion – while the Democrats are dependent on the urban upper and middle classes in central Thailand and the south. They regard the former prime minister as corrupt and autocratic.

"I have been waiting five years [since the coup] for this moment," said Sompoon Tamakaew as he watched the results at Puea Thai's headquarters. "Bangkok is full of the elites and upper classes, so they don't really understand how much of an impact Thaksin had."

Sompoon, a 50-year-old gardener from the north-eastern province of Ubon Ratchathani, added that he was not worried that elites would seek to interfere again.

"I don't think [another coup] will happen, but if it does I will keep fighting," he added.

Police said more than 170,000 officers were on duty throughout the country on Sunday to monitor voting by 47 million eligible Thais. Britain was among the countries warning its nationals of potential violence, urging visitors to avoid demonstrations.