NEW DELHI — Prime Minister Narendra Modi’s governing party suffered a major setback in elections in New Delhi on Tuesday — winning far fewer seats in the capital district’s Legislative Assembly than expected after a bitter campaign that it tried to turn into a referendum on Mr. Modi and his Hindu nationalist platform.

On Tuesday, election results showed that Mr. Modi’s Bharatiya Janata Party, or B.J.P., picked up five more seats in the assembly than it won in the last election here, in 2015. Still, the showing was a major disappointment, falling short of expectations and the seats needed to beat the opposition, the incumbent Aam Aadmi Party, or A.A.P., to take over the critical office of chief minister for Delhi.

Taking over that office was a priority for the B.J.P., amid nationwide anti-government protests against Mr. Modi’s policies that are now entering their third month. Mr. Modi swept national elections last year and quickly set his sights on winning big in the capital.

The protests are directed at the citizenship law passed by Mr. Modi’s government that opponents say discriminates against Muslims. The unrest has posed the biggest challenge yet to the prime minister and has taken on more issues, evolving into an expression of resistance against what many demonstrators see as a long-term plan by his party to redefine India’s secular foundation and turn it into a Hindu-centric state.