In the centre of Delhi, one of the world's biggest, dirtiest, noisiest cities, is an island of calm. Here, government ministers live in vast, state-owned villas; judges, generals and senior bureaucrats walk their dogs across well-watered lawns as servants scrub their government cars; top politicians confer in compounds and the wives of unimaginably wealthy industrialists hold lunch parties catered by top chefs. You live here and visit India.

Last week, India visited this island in the shape of a giant power cut.

Such outages are a daily occurrence for the rest of the population – or at least the two-thirds of India's 1.2bn inhabitants who actually have any electricity supply. But they are not for India's elite. For the latter, power guarantees power. The bureaucrats in charge of Delhi's grids switch off the supply to hospitals before they plunge the homes of top politicians into darkness. But this time the lights did go off. And so the residents of the most upmarket parts of the city – so confident of their power supplies that they do not have generators – had to sit in the fetid monsoon temperatures of 35 degrees like everyone else.

The north Indian outage, possibly the biggest in the history of mankind, affected an estimated 700m people. It was a global news story. It revealed the parlous state of Indian infrastructure and provided a dramatic example of how public institutions have failed to keep pace with economic growth. And it also revealed quite to what degree the conclusion, so deeply rooted in the west, that India is not only "shining" but will only get progressively shinier, is complacent in the extreme.

All things are relative and the "India Shining" slogan, coined for a marketing campaign in heady boom days almost a decade ago, can easily be justified when the current situation of the country is compared to the old days of famine, political instability, riots and disease. India's wealth, though more and more unequally distributed, has grown very substantially in the last 20 years. If the environment has suffered hugely, at the proportion of Indians living in poverty has dropped. A whole world of top-class hotels, coffee chains serving cappuccino, glitzy domestic airlines, shiny new airports, information technology companies, quality private clinics, art galleries and the odd mass urban transit system has been dropped on top of the old India and partly obscures the heat, the crowds, the filth and the misery, as well as the cultural riches, for which this extraordinarily varied country has long been known.

But it is increasingly difficult to reconcile the optimism surrounding India with the reality. In a recent book surveying the developing world, analyst and investment banker Ruchir Sharma says that India has, at best, only a 50% chance of becoming what he calls "a break-out nation". A reversal of recent fortunes is also a possibility, he argues. Other countries have suffered decline after a period of rapid growth. But this thought barely appears to have occurred to western policymakers. Few stop to interrogate the narrative of inevitable, inexorable Indian success. Take, for example, the famous Indian "middle class". If defined in western terms, as people with salaried jobs, a car, the odd overseas holiday and an apartment or even house, then Indians fitting this category cannot number more than a couple of per cent of the population at best. A local definition, given to me by a young man in a cheap restaurant, are those who can afford an 11p cup of coffee. The cappuccino-sippers are infinitesimally few, albeit more numerous every year.

Or take the vision of what a fully developed India might look like. This is imagined as a vague mixture of Rajasthani forts, new campuses packed with elite IT specialists and picturesque traditional villages.In fact, the new India being created by economic growth and urbanisation consists largely of thousands of square miles of bad quality apartment blocks sprawling around almost every major city.

And yes, 6.5% GDP annual growth rate, even if down from nearly 10% in earlier years, still sounds impressive. But many economists estimate India needs an economy growing at around 7% annually simply not to lose ground.

India and Indians are famously flexible and extraordinarily resilient. Much of the impact of the power cut was cushioned by the precautions – generators, battery back-ups etc – that many businesses and households have already taken against the daily, shorter, outages.

But, if other countries can live with a few slow periods, India cannot. Stasis is not an option. Hundreds of millions of young people will be pouring on to the labour market in coming years. They will need to find homes and healthcare as well as jobs. Most have been only partially educated in substandard establishments.

They suffer a grave lack of skills. More than a third will have been malnourished when infants. Horrific gender imbalances due to the pre-or post-natal killing of girls will mean a serious shortage of partners for young males.

Driving through the poor, if improving state of Bihar, on a late summer evening a year ago, the sight in every town I passed through, of large numbers of young men, some evidently drunk, out on the streets in the gathering dusk, brought home quite how easily India's "demographic dividend" could turn out to be the opposite. In worst-case scenarios, the consequence of a combination of tens of millions of people with unrealistically high aspirations, deteriorating prospects of improvement and a consistently mediocre standard of living will be high levels of social unrest.

Finally, there is the assumption that India, because it is a democracy, will become a compliant and constructive ally of the west. This is to underestimate the country's determination to pursue what it sees as its own interests, the legacy of decades of "non-aligned" international positions and an entrenched suspicion of America and Europe.

Currently, the muscles of the political leadership of India are so atrophied by long-term paralysis as to be incapable of reaction to even the most aggressive stimuli. An election in 2014 may change this. All know it cannot continue. If the promotion of the minister responsible for power in the wake of the outage inspires despair, the angry reaction of media and middle class is hopeful.

India usually has to wait for a crisis for genuine reform. It was a desperate financial crunch that inspired the reforms of the early 1990s that made the growth of the last decades possible. It is likely, too, that after a few difficult years, the elephant will resume its slow lumbering march towards prosperity and stability. But the path is far from as smooth and straight as many in the west would like to think.