Misha Glenny tells of his prison meetings with Nem of Rocinha, the slum crime boss who channelled some of his cocaine profits into running a welfare state for 100,000 people

On the morning of 10 November 2011, the people of Rio de Janeiro woke up to some riveting footage being played over and over on all major television networks. Editors had cleared the decks to concentrate on one story.

Just after midnight, three competing police forces had converged on a black Toyota Corolla. After a scuffle and mutual recriminations, they all pointed their weapons at the car’s locked boot. Illuminating the scene was a searchlight mounted on a police helicopter to the delight of the surrounding camera crews from the headquarters of Brazil’s largest news channels, Globo and Record.

The hysteria mounted as federal police officers hauled out a lanky, dazed and disoriented man into the car park of Rio’s Naval Club by the side of Lagoa, the lagoon at the heart of the South Zone where most of the city’s swankiest residential areas lie. Cops and journalists vied with each other to take snaps of the man, one yanking back his curly hair to facilitate the digital gawking.

It was a moment that José Mariano Beltrame, secretary of security in Rio State, could savour despite being far away in Berlin, where he was discussing the policing of mega-sporting events. He could boast, not for the first time, that his radical policy of “pacifying” Rio’s lawless slums was working. His forces were clearing the city’s favelas of drugs and guns and restoring the authority of the Brazilian state. The country could look forward to the prospect of a safer Football World Cup and Olympic Games.

The man in the boot of the car was known across the country as Nem of Rocinha, after the favela in the South Zone where he was born and raised. Nem was Rio’s public enemy No 1; for six years he was the undisputed boss of the drugs trade in Rocinha which police claim is responsible for 60% of all cocaine consumed in Rio.

A favela, once earmarked for pacification, would be prepared for an invasion of highly trained special forces. The aim was less to halt the drugs trade and more to flush out the heavy weaponry that had been used by the gangs since the early 1990s. Then a more community-friendly police force and enhanced social services, such as creches, health-care centres and schools, would be established.

Beltrame is a sympathetic character. A former senior officer of the federal police, he exhibits none of the money-grubbing, influence-peddling traits that Brazilians associate with so many of their politicians. And when he explained to me the motivation behind pacification, he was highly critical of his country’s past.

“For 50 years, the state chose to abandon the favelas,” he said. “The whole world knows that Rio has been a divided city with these islands of criminality. Everyone knows that we have to occupy those islands. And this occupation has simply given an opening for the formal city to enter into the informal city. Taxi drivers know this; politicians know it; sociologists know it; journalists know it; everyone knows it. But until we chose pacification, it hadn’t been done – because of politics and because of corruption.” Favelas, he argued, were allowed to wallow in poverty, corruption, drugs and violence with the state seemingly indifferent and unwilling to provide social or economic assistance.

It was just such deprivation that changed the life of Antônio Francisco Bonfim Lopes – aka Nem – forever in the year 2000. In his mid-20s, he had a respectable job distributing a TV listings magazine around the South Zone. Then his baby daughter developed a rare autoimmune disease and his tiny salary was not enough to cover the complex treatment she needed.

The only person who would lend him the money was the then boss of the favela, and to pay back the debt, Bonfim had to leave his job and begin work for the cartel. Over the next few years, he became the most powerful and successful drug lord in Rio.

After his arrest, I wrote to Nem in prison and asked if he would speak to me. He agreed. The story that emerged was fascinating: once he reached the top, Nem was, in effect, mayor, police chief and director of the chamber of commerce for a community estimated at 100,000 residents. With the receipts from the cocaine trade, he ran a business that supported nearly 1,000 people. He also channelled some of his profits into a basic welfare state. He could do this because he paid close attention to accounting and budgetary matters.

“The food baskets and the support we gave to extracurricular school activities, such as the Thai boxing or capoeira classes, were all accounted for as part of our business expenses,” he explained. “But the burials, prescription costs or if anyone who couldn’t afford it needed gas, these were all extra payments.”

In the absence of any regular police, law was maintained by 150 armed men, most in their teens and early 20s. But while the man known locally as Mestre, or master, decided over life or death, he usually opted for the former. Under his rule, homicide rates dropped by more than two-thirds.

This was part-calculation, part-intuition. Rocinha was so profitable for the cocaine trade because it is surrounded by the three richest areas of Rio – Leblon, São Conrado and Gávea. By turning Rocinha into the safest and most attractive favela in Rio, business boomed. “He was not a man of violence,” said Detective Bárbara Lomba, who led the three-strong team that patiently investigated the Rocinha drugs operation for four years. “He had a policy of avoiding confrontation wherever possible and of not facing down the police. Rather the opposite, he was in contact with them in a corrupt relationship.”

Nem’s policy paid off. Rocinha became a fixture on the tourist route; Brazil’s biggest pop stars such as Ivete Sangalo and Claudia Leitte were happy to include the favela on their tours, boosting their popularity with Brazil’s poor. Politicians including former president Luiz Inácio Lula da Silva and the current incumbent, Dilma Rousseff, were keen to tour, as were members of Brazil’s national football side. Above all, the youngsters from the surrounding middle class areas went to buy coke.

Beltrame knew that he would have to “pacify” Rocinha because of its symbolic power and its location. As the World Cup and the Olympics approached the pressure grew. But by taking Nem out of the equation, Rocinha’s character has changed. The relationship between the police and residents is uneasy at best. In July 2013, a group which included the chief of Rocinha police murdered an innocent bricklayer, and the favela came close to open insurrection.

Since then the drug cartel has been edging its way back and there are sporadic shootouts with the police. Homicides remain at historic low levels but domestic violence, rape, assault and burglary have increased fourfold.

The international press has been awash these past few months with stories about Rio’s lack of readiness for the Olympics: facilities not yet built and concern about the chronic sewage contamination of Guanabara Bay where the sailing is due to take place. Brazilians believe that Rio, as it did with last year’s World Cup, will snatch victory from the jaws of defeat and put on a great show.

Perhaps it will. But the country is facing a monumental political and economic crisis that is leading to budgetary cuts in every direction, including that earmarked for security in Rio.

Beltrame is faced with an immense challenge to prevent violence resurfacing in the favelas and then spilling out into the tourist areas. With mistrust between ordinary residents and police growing and a surge in demand for cocaine, part of Rio may start to feel nostalgic about Nem’s rule of Rocinha.

Over the total of 28 hours that I spent with Nem, I received an unprecedented insight into how an uneducated but naturally intelligent man grew to manage a huge, complex business and a small city, and how he dealt with profound moral dilemmas that were often literally a matter of life and death.

Nemesis: One Man and the Battle for Rio by Misha Glenny (Bodley Head, £18.99) is published on 17 September. Click here to order a copy for £15.19 from Guardian Bookshop