The former tax inspector and activist who is to be Delhi's new chief minister, the top elected official in the metropolis, has refused police protection, saying that his "biggest security" is God.

Arvind Kejriwal, leader of a new populist political party "dedicated to improving the lot of the common man", announced on Monday that he would form a government to run the sprawling, troubled and increasingly wealthy city of 15 million people.

The 45-year-old novice politician, who has vowed to end the "VIP culture" of the capital, also said that new ministers would refuse to accept the large government bungalows that are customary perks of such posts as well as cars that use flashing red beacons to force their way through traffic.

Even minor dignitaries in the capital travel with a large police escort, adding to the already acute congestion and enraging ordinary citizens.

Kejriwal's Aam Admi (Common Man) party stunned political analysts and established parties when it won 28 out of 70 seats in local assembly elections in Delhi earlier this month. The newcomer to the capital's cutthroat machinations, who launched his party a year ago, beat the former chief minster of the city, a veteran of the ruling Congress party who had dismissed his challenge as "not even on our radar". Congress suffered a catastrophic defeat, being reduced to eight seats.

Almost all the candidates of the AAP were political debutants and included a rickshaw driver, a lawyer and a TV actor.

Their key pledge was to clean up politics and the endemic corruption that has crippled the provision of public service for the many millions who cannot afford to pay for private healthcare, schooling or even basics such as water.

The party's message and symbol – a broom – proved popular with urban voters also struggling with runaway inflation, chronic youth underemployment and slowing economic growth. Kerjiwal, who called the party's victory a historic win, had initially said it would not form a minority government. But after lengthy negotiations in recent days, Congress has now decided to support the AAP in the Delhi local assembly.

"It's a great day, a very exciting time," said Aathishi Marlena, a senior AAP activist and policy adviser. Marlena said the challenges were daunting but that the party would meet them. "Tackling the rampant corruption, the lack of transparency and accountability, the problems faced by the common people is a matter of political will and that's what we bring to this."

The main opposition Bharatiya Janata party won 32 seats but no other political group was willing to offer it support. Sachin Pilot, a Congress minister, described the emergence of a new political outfit – the AAP in Delhi – as "absolutely astonishing".

"For my party of course it has been a setback but generally speaking the fact that indiviudals can come together in a very difficult space and find themselves relevant and meaningful has been a new phenomenon. In the last three or four decades it hasn't happened at all. That's something one has to recognise reconcile and then deal with," Pilot told the Guardian, speaking shortly after the poll earlier this month.

Senior Congress officials told reporters that the party's support for the AAP in Delhi would not be unconditional and would depend on the party's performance.

Cleaners outside the parliament compound in New Delhi. The AAP's symbol is a broom - illustrating its representation of the working man. Photograph: Prakash Singh/AFP/Getty Images

The election in Delhi was one of four state elections in the last month, which saw Congress lose heavily. The polls were dubbed a semi-final for the general election due next spring, by analysts.

Though India's population remains largely rural, the proportion of seats in the national assembly determined by urban voters has steadily risen in recent years. Some observers see the new middle classes in cities and, particularly, tens of thousands of small towns, as determining what is likely to be a tough battle between the Congress party and the BJP next year. Also crucial are likely to be between 120 million – 150 million first time voters.

The AAP has said it now wants to focus efforts on expanding into the huge northern state of Uttar Pradesh, which has a population fo 180 million, and Bihar, another poor northern state. It hopes to become a national party however.

Kejriwal's announcement ended two weeks of speculation and fears of a hung assembly in Delhi, a city which has become much richer in recent years but faces huge problems of crime, sanitation, housing and pollution .

Though the outgoing Congress government in Delhi had been praised for successfully building a metro system and holding the Commonwealth Games in the Indian capital in 2010, it had also been hit by graft charges and suffered from the general unpopularity of the Congress-led coalition government at a national level.

The new government will be sworn in on Thursday. The BJP called the AAP's decision a betrayal. "The AAP accused the Congress of being the most corrupt. Today they have compromised on their principles. This is gross betrayal," said Dr Harsh Vardhan, the BJP's chief ministerial candidate.