India’s ruling party has conceded defeat in a provincial election seen as a test of the vote-winning abilities and political strategy of the prime minister, Narendra Modi.

The loss in the populous and poor state of Bihar is the most significant domestic setback for Modi since he won a crushing victory in a general election in the emerging economic power last year, after a campaign promising rapid development, modernisation and opportunity combined with a defence of conservative cultural and social values.

The failure to win Bihar for his Bharatiya Janata party (BJP) will hinder Modi’s push to pass crucial economic reforms because he needs to win such elections to gain full control of parliament.

So far, the economic takeoff Modi promised during last year’s election has proved elusive.

More broadly, Sunday’s defeat in Bihar, which has a population of 105 million, might indicate that though Modi, a Hindu nationalist who started his career with a rightwing religious and cultural revivalist organisation, still retains significant national popularity and momentum, his appeal to voters has begun to wane.

An anti-Modi alliance was ahead in 125 seats in the 243-seat regional assembly by noon, enough for an outright majority, unofficial tallies compiled by television channels showed.

“The writing is on the wall. Modi’s magic has failed and we have clearly won the battle,” said Sanjay Singh, a spokesman for the regional parties led by the incumbent leader of the state, Nitish Kumar of the Janata Dal party.

Kumar was supported by Lalu Prasad Yadav, a former chief minister of Bihar who has served time in prison for corruption and is one of the most colourful figures in Indian politics.

Coalition party workers danced in the street and set off firecrackers in celebration in the state capital, Patna.

Kumar relied on carefully managed appeals to groups defined by caste, the ancient Indian social hierarchy, as well as his own record of improving infrastructure and investing.

Modi turned the election into a referendum on his leadership after he addressed at least 30 campaign rallies, a departure from tradition in state elections, which usually centre on local issues and regional leaders.

The poll came against a background of concern in India at incidents in which Muslims have been targeted by Hindu extremists. There have been protests by prominent intellectuals at what they call a climate of rising intolerance.

It is unclear to what extent such concerns are more widely shared, though defeat in Bihar may indicate that the BJP strategy of simultaneously stressing both development at a national level and using more sectarian rhetoric at a local level may be becoming less effective.

During the campaign, election authorities criticised the BJP for advertisements “of an offending nature” and banned several party posters they said could incite hatred.

One showed a young Hindu woman embracing a garlanded cow, an animal sacred to Hindus. In one of the most notorious recent incidents of hate crime in India, a Muslim man was lynched by a mob that suspected he had eaten beef.

Authorities also criticised opposition parties in the state for making what they claimed to be unsubstantiated and injurious personal allegations against opponents.

BJP officials sought to protect their leader. GVL Narasimha Rao, a spokesman, denied the loss was a personal blow for Modi, saying the odds were stacked against their party after regional rivals joined forces.

“This election was loaded against us. It is a defeat of the arithmetic,” Rao told India Today TV.

The BJP is in a minority in the upper house of parliament, where seats are allocated according to a party’s strength in the states, enabling the opposition to block Modi’s reforms, including the biggest overhaul of taxes since independence in 1947.

Writing in the Hindustan Times, analyst Prashant Jha said a defeat for Modi’s BJP would mean the “return of the opposition” in India.

“The opposition will roar. Parliament will witness a renewed vigour. The setback will be used to attack BJP for both its willingness to turn a blind eye to religious chauvinism, as well as its failure to meet the promises of 2014. Narendra Modi will have to do a drastic review of operations in government,” Jha wrote.