On 3 May, at around 4.45pm, a short, trim Indian man walked quickly down London’s Old Compton Street, his head bowed as if trying not to be seen. From his seat by the window of a nearby noodle bar, Anuvab Pal recognised him instantly. “He is tiny, and his face had been all over every newspaper in India,” Pal recalled. “I knew it was him.”

Few in Britain would have given the passing figure a second look. And that, in a way, was the point. The man pacing through Soho on that Wednesday night was Nirav Modi: Indian jeweller, billionaire and international fugitive.

In February, Modi had fled his home country after an alleged $1.8bn fraud case in which the tycoon was accused of abusing a system that allowed his business to obtain cash advances illegally from one of India’s largest banks. Since then, his whereabouts had been a mystery. Indian newspapers speculated that he might be holed up in Hong Kong or New York. Indian courts issued warrants for his arrest, and the police tried, ineffectually, to track him down.

It was only by chance that Pal spotted him. A standup comic normally based in Mumbai, he happened to be in London for a run of gigs. “My ritual was to go to the same noodle bar, have a meal, and then head to the theatre,” Pal said. “I always sat by the window. And then suddenly Modi walks past. He was unshaven, and had those Apple earphones, the wireless ones. He looked like he was in a hurry.”

It was another month before the press finally caught up with Modi, as reports of his whereabouts emerged in June, along with the suggestion that he was planning to claim political asylum in the UK. (Modi denies wrongdoing, and did not respond to requests for comment.) In the process, Modi also gained entry into one of London’s more notorious fraternities: the small club of Indian billionaires who seem to end up in the British capital following scandals back at home.

The most prominent among these émigré moguls is India’s “King of Good Times”, Vijay Mallya, the one-time aviation magnate and brewer, who transformed Kingfisher beer into a global brand. A few years ago, Mallya was one of India’s most celebrated industrialists, famous for his mullet haircut and flamboyant lifestyle. But in early 2016, Indian authorities filed charges relating to the collapse of his Kingfisher airline, which went bust in spectacular fashion in 2012, leaving behind mountainous debts and irate, unpaid staff. And so, facing allegations of financial irregularities and of refusing to repay outstanding loans, Mallya quietly boarded a plane for Britain, too.

Like Modi, Mallya denies wrongdoing. Last month he released a long statement accusing India’s government of conducting a witch-hunt against him. And to the extent that this claim has some merit, it is because Indian prime minister Narendra Modi (no relation to Nirav Modi) has of late been under great pressure to bring supposedly errant tycoons such as Mallya to book.

Men like Mallya and Modi were members of India’s expanding billionaire class, of whom there are now 119 members, according to Forbes magazine. Last year their collective worth amounted to $440bn – more than in any other country, bar the US and China. By contrast, the average person in India earns barely $1,700 a year. Given its early stage of economic development, India’s new hyper-wealthy elite have accumulated more money, more quickly, than their plutocratic peers in almost any country in history.

A cardboard cut-out of billionaire jeweller Nirav Modi at a protest against him in New Delhi in February. Photograph: Chandan Khanna/AFP/Getty Images

Narendra Modi won an overwhelming election victory in 2014, having promised to put a stop to the spate of corruption scandals that had dogged India for much of the previous decade. Many involved prominent industrialists – some directly accused of corruption, while others had simply mismanaged their finances and miraculously managed to escape the consequences. Voters turned to Narendra Modi, the self-described son of a poor tea-seller, hoping he would deliver a new era of clean governance and rapid growth, ridding India of a growing reputation for crony capitalism.

Narendra Modi pledged to end a situation in which the country’s ultra-wealthy – sometimes called “Bollygarchs” – appeared to live by one set of rules, while India’s 1.3 billion people operated by another. Yet as they continue to hide out in cities like London, men like Mallya and Nirav Modi have come to be seen as representing the failure of that pledge; the Indian authorities “have a long road ahead”, as one headline put it in the Hindustan Times last year, referring to a “long and arduous” future extradition process in Mallya’s case.

And as Narendra Modi gears up for a tough re-election battle next year, he is fighting the perception that India is unable to bring such men to heel, and that it has been powerless to respond to the rise of this new moneyed elite and the scandals that have come with them. “This ongoing battle to get India’s big tycoons to play by the rules is one of the biggest challenges we face,” says Reuben Abraham, chief executive of the IDFC Institute, a Mumbai-based thinktank. “Getting it right is central to India’s economic and political future.”

India has long been a stratified society, marked by divisions of caste, race and religion. Prior to the country winning independence in 1947, its people were subjugated by imperial British administrators and myriad maharajas, and the feudal regional monarchies over which they presided. Even afterwards, India remained a grimly poor country, as its socialist leadership fashioned a notably inefficient state-planned economic model, closed off almost entirely from global trade. Over time, India grew more equal, if only in the limited sense that its elite remained poor by the standards of the industrialised west.

But no longer: the last three decades have seen an extraordinary explosion of wealth at the top of Indian society. In the mid-1990s, just two Indians featured in the annual Forbes billionaire list, racking up around $3bn between them. But against a backdrop of the gradual economic re-opening that began in 1991, this has quickly changed. By 2016, India had 84 entries on the Forbes billionaire list. Its economy was then worth around $2.3tn, according to the World Bank. China reached that level of GDP in 2006, but with just 10 billionaires to show for it. At the same stage of development, India had created eight times as many.

In part, this wealth is to be welcomed. This year India will be the world’s fastest-growing major economy. During the last two decades, it has grown more quickly than at any point its history, a record of economic expansion that helped to lift hundreds of millions out of poverty.

Nonetheless, India remains a poor country: in 2016, to be counted among its richest 1% required assets of just $32,892, according to research from Credit Suisse. Meanwhile, the top 10% of earners now take around 55% of all national income – the highest rate for any large country in the world.

Put another way, India has created a model of development in which the proceeds of growth flow unusually quickly to the very top. Yet perhaps because Indian society has long been deeply stratified, this dramatic increase in inequality has not received as much global attention as it deserves. For nearly a century prior to independence, India was governed by the British Raj – a term taken from the Sanskrit rājya, meaning “rule”. For half a century after 1947, a system dominated by pernickety industrial rules emerged, often known as the Licence-Permit-Quota Raj, or Licence Raj for short. Now a system has grown in their place once again: the billionaire Raj.

The rise of India’s super-rich – the first and most obvious manifestation of the billionaire Raj – was propelled by domestic economic reforms. Starting slowly in the 1980s, and then more dramatically against the backdrop of a wrenching financial crisis in 1991, India dismantled the dusty stockade of rules and tariffs that made up the Licence Raj. Companies that had been cosseted under the old regime were cleared out via a mix of deregulation, foreign investment and heightened competition. In sector after sector, from airlines and banks to steel and telecoms, the ranks of India’s tycoons began to swell.

Nothing symbolises the power of this billionaire class more starkly than Antilia, the residential skyscraper built in Mumbai by Mukesh Ambani, India’s richest man. Rising 173 metres above India’s financial capital, the steel-and-glass tower is rumoured to have cost more than $1bn to build, looming over a city in which half the population still live in slums.

The Antilia building, at right of photograph, in Mumbai. Photograph: Alamy

Ambani owns Reliance Industries, an empire with interests stretching from petrochemicals to telecoms. (His father, Dhirubhai, from whom he inherited his company, was one of the main beneficiaries of the economic reforms of the 1980s.) At Antilia, Ambani entertains guests in a grand, chandeliered ballroom that takes up most of building’s ground floor. There are six storeys of parking for the family’s car collection, while the tower’s higher levels feature opulent apartments and hanging gardens. Further down, in sub-basement 2, the Ambanis keep a recreational floor, which includes an indoor football pitch. Antilia became an instant landmark upon its completion in 2010. The city had long been a place of stark divisions, yet the Ambani’s home almost seemed to magnify this segregation. (A spokesman for Reliance did not respond to a request for comment.)

The emergence of the Indian super-rich was bound up in larger global story. The early 2000s were the heyday of the so-called “great moderation”, when world interest rates stayed low and industrialised nations grew handsomely. This was also when the fortunes of India’s new tycoons began to change. Pumped up by foreign money, domestic bank loans and a surging sense of self-belief, industrialists went on a spending spree. Ambani dumped billions into oil refineries and petrochemical plants. Vijay Mallya spent heavily on new fleets of Airbus jets. Nirav Modi began building a global chain of jewellery stores. Stock markets boomed. From 2004 to 2014, India enjoyed the fastest expansion in its history, averaging growth of more than 8% a year.

The boom years brought benefits, most obviously by reintegrating India into the world economy. Yet this whirlwind growth also proved economically disruptive, socially bruising and environmentally destructive, leaving behind what the writer Rana Dasgupta describes as a sense of national trauma. India’s new wealth has been shared remarkably unevenly, too. Its richest 1% earned about 7% of national income in 1980; that figure rocketed to 22% by 2014, according to the World Inequality Report. Over the same period, the share held by the bottom 50% plunged from 23% to just 15%.

Unsurprisingly, some feel resentful. “You walk around the streets of this city, and the rage at Antilia has to be heard to be believed,” Meera Sanyal, a former international banker turned local anti-corruption campaigner, told me in 2014. Six years before that, in 2008, as the scale of India’s billionaire fortunes were becoming clear, Raghuram Rajan – an economist who would later become the head of India’s central bank – asked an even more pointed question about his country’s tycoon class: “If Russia is an oligarchy, how long can we resist calling India one?”

Back in London, Vijay Mallya feels unjustly targeted by India’s recent attempts to shed its reputation for crony capitalism, the second defining characteristic of the billionaire Raj. “I have been accused by politicians and the media alike of having stolen and run away with Rs 9,000 crores [90bn rupees, or $1.3bn] that was loaned to Kingfisher Airlines,” he wrote in his open letter last month. His case had become, he suggested, a “lightning rod for public anger” over the alleged misbehaviour of his fellow tycoons.

In spring 2017, I met Mallya at his London home, a Grade I-listed town house a short walk from Baker Street tube station. A variety of Rolls-Royces and Bentleys were parked along the mews at the rear, alongside a fat silver Maybach with the number plate VJM 1, which idled outside Mallya’s back door. Inside, he sheltered behind a grand wooden table, a gold lighter and two mobile phones lined up in front of him. At one point I asked to be excused to visit the toilet. A flunky ushered me into a golden bathroom, with a shiny gold seat to match its golden taps and loo-roll holder. The fluffy hand towels were white, but each one came embossed with the letters “VJM” in gold thread.

On the surface, Mallya still seemed every bit the ebullient tycoon of old: a bulky man in a red polo shirt, with gold bracelets on each wrist and a chunky diamond ear stud. But by then he had been stuck in Britain for more than a year, and grew downbeat as the afternoon wound on and our conversation turned to his business troubles and the state of his homeland. “India has corruption running in its veins,” he said with a sigh. “And that’s not something one is going to change overnight.”

With his shoulder-length hair and taste for bling, Mallya had long honed an image as the most piratical of India’s generation of entrepreneurs. A specially kitted-out Boeing 727, its bar well-stocked with his own Kingfisher beer, whisked him between parties and business meetings – a distinction that was, in any case, often hazy. “It was all a bit ridiculous,” one ex-board member at a Mallya company told me.

Vijay Mallya, who fled India for London in 2016. Photograph: Mark Thompson/Getty Images

Now marooned in London, Mallya had plenty of time to ponder his missteps. Once a member of the Rajya Sabha, the Indian upper house of parliament, his diplomatic passport has been cancelled. As a long-time UK resident, he was permitted to stay in the country, but without travel documents he was unable to travel, curtailing his notorious jetsetting lifestyle almost entirely. Earlier this month, a UK court issued an order allowing authorities trying to recover debts to enter his various British properties.

In his pomp, Mallya seemed to represent a new India. In a country whose old commercial elite had been dominated by cautious, discreet industrialists, Mallya was different: rich, powerful and not inclined to hide it. Not all of India’s pioneers behaved in this way – its software and IT billionaires, for instance, were typically less flamboyant figures. But while Mallya continues to deny that he did anything wrong, he admits that he has become what he calls the “poster boy” for a moment of public anger against India’s rich, as many newly wealthy business figures found themselves mired in allegations of wrongdoing.

India’s old system created fertile ground for corruption, forcing citizens and businesses alike to pay myriad bribes for basic state services. But these humdrum problems were trivial compared with the grand scandals that emerged during the 2000s. Assets worth billions were gifted under the table to big tycoons by senior politicians and bureaucrats in what became known as the “season of scams”. Giant kickbacks helped businesses acquire land, bypass environmental rules and win infrastructure contracts. Headlines filled up with fresh outrages, from fraudulent public housing schemes to dodgy road-building projects.

Many of those who backed India’s economic reforms hoped that a more free-market economy would lead to more honest government. Instead, crony capitalism infiltrated almost every area of national life. Hundreds of billions of dollars were siphoned away, according to some estimates, by a shadowy alliance of colluding politicians and business tycoons. India’s old system of retail corruption went wholesale.

Many politicians also became astoundingly rich, and would have made the Forbes list had their holdings not been hidden carefully in shell companies and foreign banks. Rapid economic growth increased the value of political power, and what could be extracted from it. Political parties had to raise more money, to fight elections and fund the patronage that kept them in office. One estimate suggested that India’s 2014 election cost close to $5bn, a huge increase over the cheap and cheerful polls of the pre-liberalisation era. Election experts believe most of this money is brought in illegally from favoured tycoons, in exchange for unknown future favours.

Politicians spend the money to fund campaigns, but also on handing out favours, jobs and cash to constituents. “It’s sort of an unholy nexus,” as Raghuram Rajan put it to me during his tenure as head of India’s central bank. “Poor public services? Politician fills the gap; politician gets the resources from the businessman; politician gets re-elected by the electorate for whom he’s filling the gap.”

This nexus between business and politics lies at the heart of the third problem of India’s billionaire Raj, namely the boom-and-bust cycle of its industrial economy. In recent decades, China went on the largest infrastructure building spree in history, but almost all of it was delivered by state-backed companies. By contrast, India’s mid-2000s boom was dominated almost exclusively by its private-sector tycoons, giving the industrialists and the conglomerates they run a position of outsized importance in India’s economic development.

Bollygarchs borrowed huge sums from state-backed banks and invested with gleeful abandon, in one of the largest deployments of private capital since America built its railroad network 150 years earlier. But when India’s good times came to an end after the global financial crisis, the tycoons’ hubris was exposed, leaving their businesses over-stretched and struggling to repay their debts. In 2017, 10 years on from the crisis, India’s banks were still left holding at least $150bn of bad assets.

Since taking office, Narendra Modi has tried, often ineffectually, to fix this corporate- and bank-debt crisis, alongside the related problems of cronyism and the super-rich that contributed to it. Watching developments such as these, some argue that the power of India’s tycoon class is fading. Yet India’s ultra-wealthy are still thriving, while its ranks of billionaires keep swelling, and will continue to do so.

There is every reason to believe that on its current course, the country’s the gap between rich and poor will widen, too. Perversely, the closer India comes to its achieving its ambitions of Chinese-style double-digit levels of economic growth, the faster this will happen. On most measures, it should already be ranked alongside South Africa and Brazil as one of the world’s least-equal countries. Even more importantly, poor countries that start off with high levels of inequality often struggle to reverse that trend as they grow richer.

Many experts believe India needs to act. “The main danger with extreme inequality is that if you don’t solve this through peaceful and democratic institutions then it will be solved in other ways … and that’s extremely frightening,” as French economist Thomas Piketty has said of India’s future, pointing to likely rising future tensions between the wealthy and the rest.

Protesters burning an effigy of Nirav Modi in New Delhi in February. Photograph: Chandan Khanna/AFP/Getty Images

India is now entering a new phase of development, as it tries to follow Asian economies such as South Korea and Malaysia out of poverty and towards full “middle-income” status. There is no reason this cannot happen. But as we’ve seen in Latin America, the economies with the widest social divides have tended to be the ones that are most likely to get stuck in the “middle-income trap”, achieving moderate prosperity but failing to become rich. The more successful countries of east Asia, by contrast, grew prosperous while managing to stay egalitarian, partly by building basic social safety nets and ensuring that their wealthiest citizens paid their taxes. Of the two models, it seems clear which India should want to follow.

Much the same is true of corruption. India’s old system of cronyism, with its political favours and risk-free bank loans, has came under intense scrutiny, but the battle against corruption is at best half-won. Kickbacks still dominate swathes of public life, from land purchase to municipal contracts. State and city governments are just as venal as ever. Surveys report that India remains Asia’s most bribe-ridden nation. “For any society to lift itself out of absolute poverty, it needs to build three critical state institutions: taxation, law and security,” according to the economist Paul Collier. All three in India – the revenue service, the lower levels of the judiciary and the police – still suffer endemic corruption. Perhaps most importantly, the country’s under-the-table political funding system remains largely untouched.

Progress in fixing India’s problems of corporate and bank debt has also been frustratingly slow. Modi has introduced some important measures, including a new bankruptcy law and a series of bank recapitalisations. But more radical options have been ignored, notably the privatisation of struggling public-sector lenders.

If these struggles sound familiar, that is because they are. India is far from the first country to enjoy a period of rampant cronyism and wild growth, and then grapple with how to respond. In Britain, the onset of the industrial revolution in the mid-19th century kicked off such a moment, as captured in the novels of Charles Dickens and Anthony Trollope. But the more obvious parallel is with America, and the era between the end of the civil war in 1865 and the turn of the 20th century: the Gilded Age, or the era of “the great corporation, the crass plutocrat [and] the calculating political boss”, as one historian put it.

India’s own Gilded Age is different in many ways, but it shares at least one characteristic – namely, that such a period of early industrialisation is also a time of rapid political and economic change, in which it should be possible to invoke what the philosopher Richard Rorty once called the “romance of a national future”, the sense of hope that infuses powers on the rise.

India is set to grow in economic might throughout this century, as America did during the 19th. By some accounts, it has already overtaken China as the world’s most populous nation; in others, the baton will pass during the next decade or two. Whatever the case, the fate of a large slice of humanity depends on India getting its economic model right. Meanwhile, as democracy falters in the west, so its future in India has never been more critical. To make this transition, India’s billionaire Raj must become a passing phase, not a permanent condition. India’s ambition to lead the second half of the “Asian century” – and the world’s hopes for a fairer and more democratic future – depend on getting this transition right.

The Billionaire Raj: A Journey Through India’s New Gilded Age by James Crabtree is published by Oneworld, and available at guardianbookshop.com

• Follow the Long Read on Twitter at @gdnlongread, or sign up to the long read weekly email here.