Brazil, by both area and population, is the fifth-largest nation on earth. Its economy is perhaps the sixth- or seventh-largest and will soon surpass those of France and Britain. Yet this great state has barely registered its presence globally. In the complex flux of globalised popular culture or the rarefied circuits of high culture and the sciences, Brazil is an undercurrent.

Music and carnival are, in their picture-postcard form, perhaps the most widespread, if glib, images of the nation. Denuded of their social and political context, they serve, alongside Copacabana and the palm-fringed beaches of its northern coasts, as code for languid tropical hedonism, the brand identity of Brazil in the global tourist market.

Alluring as these traditions may seem, the global popularity of samba is dwarfed by that of salsa or Jamaican reggae, a now-global musical genre from an island with 1% of Brazil's population. And – though we need not take the views of the Nobel Foundation and its judges as the definitive word on the nation's sciences and arts – it is notable that not a single Nobel prize has been awarded to a Brazilian.

In one realm, however, Brazil is not only visible but ubiquitous, not merely competitive but the clear winner: football. Nike, whose commercial judgment on the value of global brands should be respected, has been prepared to pay more money than any other company for any other kit deal in pursuit of the yellow and green shirt.

Since the 1970s, when film and television coverage of the team first reached Africa and Asia, the Brazilians have been supported across the global south, often alongside or even in preference to national teams. Brazil is the tribune of those football cultures that have never qualified for the World Cup. Except in Argentina and Uruguay, Brazil is almost everyone's second team when the tournament rolls around.

In this realm, Brazil's standing rests not merely on the value of the brand or the breadth of its support, but because Brazilian football has become, in the collective imagination, if not the daily practice of professional football, the gold standard of the game. Since 1938, when Brazil dazzled Europe at the World Cup in France, the European and then the world's media have framed Brazilian football as exotic and otherworldly, a unique blend of the effective and the aesthetic.

Cliché it may be, but very few Brazilian phrases have entered the global lingua franca like Pele's notion of O Jogo Bonito – the Beautiful Game.

Brazil's style of play and the manner of its victories and disasters have been seared more deeply into global football culture than any other. Just as importantly, long before the rest of the world came to consider football the exemplar of Brazilian national identity, Brazil had done so itself.

Thus from the moment in 2007 when Brazil acquired the hosting rights to the 2014 World Cup, it was almost inevitable that the staging and winning of the tournament would, once again, be defined as the conclusive proof of Brazil's transformation and modernity – a measure of the progress of the Lula years. This was a double-edged sword, for the futebol nation has displayed both the successes of the era and its limits.

In 2002, Luiz Inácio Lula da Silva, or Lula as he is known, and his Workers' party won the Brazilian presidency at the fourth attempt, forming Brazil's first unambiguously social democratic and popular government. Football was a key strand of Lula's popular persona and vocabulary. Certainly he had a relationship to football far closer to that of the average Brazilian football fan than any of his predecessors. He was unambiguously a supporter of São Paulo team Corinthians, often commenting on its performances.

The social kickabouts he held at the presidential palace were well attended and keenly contested, as evidenced by the minister of finance, Antonio Palocci, entering Congress on crutches after a crunching game between the president's team and the ministry of fisheries. When speaking to the national team – the Seleção – by videolink before the 2006 World Cup, Lula asked whether the nation's beloved centre-forward, Ronaldo, was still fat. His language was peppered with football metaphors. Justifying a cabinet reshuffle, he argued: "Pele was the best player in the world, and he was replaced."

Cordiality, an easy intimacy with the Brazilian people and the game of football could only get a president so far. Lula's success rested on the economic, social and political changes his two governments helped bring about. The state of the economy was perhaps the most significant of these. While never achieving the relentlessly high growth rates of the Far East or Chile, Brazil experienced a decade of unbroken growth and historically low levels of inflation, which delivered across-the-board increases in wages and living standards.

Social policy was constrained, however, by the macroeconomic conservatism of the regime. Public healthcare and education, in particular, remained woefully underfunded. But in one area Lula brought decisive change: consolidating a whole series of poverty and hunger-reduction programmes into the bolsa família social welfare programme and then massively extending it across the country. Thus his government presided over the biggest decline in hunger and absolute poverty that the country had ever seen.

Small steps were taken in the direction of transparency and openness, and the detection and prosecution of corruption, but in football, as in other areas of political life, the Lula governments' direct interventions in the game were often blunted by the government's reluctance to take on the powerful resistance of entrenched elites. The binge of infrastructure spending that has accompanied the World Cup has become emblematic of all the most problematic elements of Brazil's political economy – corruption, kickbacks and conflicts of interest. The political limits of the Lula years were starkly demonstrated by the problems of racism and violence in Brazilian football and beyond.

The Cardoso governments of the 1990s had, to their credit, called time on any official notion that Brazil was a racial democracy. They and the Lula governments acknowledged the deep-rooted and pervasive racism of Brazilian society, appointed African-Brazilians to important state positions (under Lula this included the first black judge at the supreme court) and introduced affirmative action policies. But significant progress in these realms must be set against the country's enduring racial inequalities: African-Brazilians – who make up at least half the population – are overwhelmingly concentrated in the poorest classes and under-represented in the ranks of nearly all elite occupations.

Precisely who is black, or African-Brazilian, remains an open question in the country's unspoken and complex racial codes, where one's position is only partly determined by skin colour or genetics. When Ronaldo, clearly a man of mixed African and European heritage, was asked what he thought of racism in Brazilian football, he acknowledged its existence but replied: "I'm white, so I am really ignorant of these matters."

This was the same Ronaldo whose black mother was denied access to the residents' elevator and directed to the service lift in her son's exclusive apartment block.

Brazilian football and its media have long been sensitive to racist abuse by foreigners against Brazilians, dating back to an infamous Brazil-Argentina game in 1937 when the Argentinian crowd howled racist insults down on to the pitch. It has, however, been rather slower off the mark in exposing the same kind of behaviour at home.

In this regard, at least, there has been progress recently. The long-standing use of racial epithets by players and racial abuse from crowds have been exposed by a number of incidents in which the authorities actually prosecuted perpetrators: Juventude supporters were barred from their ground after racially abusing Internacional's Tinga in 2005. Palmeiras's Danilo was actually sentenced to a year in prison for abusing the Atlético Paranaense defender Manoel, though this was later reduced to a fine.

Despite the massive presence of African-Brazilian players, and the game's long association with the struggle against social and racial exclusion, there have been precious few black coaches and even fewer black club directors or football administrators. Again, the Lula years have seen advances and retreats. In 2009, Rio's Flamengo team won their first national championship for 17 years, and they did it under a black coach, Jorge Luís Andrade.

Television pundit Telmo Zanini said: "Hopefully this will become a symbolic day for Brazilian football and help to open doors for black coaches."

But Andrade's reward was to be made the scapegoat for the next season's disappointments and fired, leaving the league without a single black coach.

In May 2006, in a string of incidents orchestrated by the leaders of the criminal gangs who ran São Paulo's bursting prisons, 13 banks were attacked and 56 buses torched, and revolts broke out in 73 of the 144 prisons in the state. The police counterattack saw more than 100 people killed on the streets of the city and in the jails.

Among the demands of the Primeiro Comando da Capital, the leading criminal gang, were for 60 televisions in the jails to ensure that they could watch the national team at the World Cup in Germany. The population now included almost half a million people held in prisons designed for a half or a third of this capacity, most in connection with organised crime and drug trafficking.

Kidnapping became a growth industry, and 2004 saw four high-profile footballing families targeted. The mother of Santos star Robinho drove her new Mercedes to a barbecue with old friends in a notoriously troublesome district of town. She was captured and held for 40 days. Campinas, a small and wealthy town north-west of São Paulo, became the centre of a small crime wave, with the kidnapping of the mothers of São Paulo forward Grafite, and of Luís Fabiano and Rogerio, who were playing in Portugal. All were freed after ransoms were paid. In 2008 Pele was robbed at gunpoint. He told the gang who he was, but they took his phone and his jewellery anyway.

Accustomed to a world in which violence is pervasive, life is cheap and the public authorities – police and judiciary – cannot be relied upon to keep the peace or administer justice, many of Brazil's young men go armed and ready to use their weapons. Moreover, in a world that constantly strips them of economic dignity and offers them little but enduring marginalisation, humiliation in public becomes simply intolerable.

This same rage and embarrassment fuels pitch invasions when a team is losing or attacks on players who have let them down. From acts of grotesque rural revenge to the urban riots of the national championships, unchecked by the police and ignored by the sport's authorities, Brazilian football has been a conduit for the mental and emotional pathologies of a still-brutalised society.

As the nation staged its World Cup warm-up in June 2013, the Fifa Confederations Cup, Brazil experienced the largest wave of social protest the country had ever seen. At their peak, demonstrations took place simultaneously in 120 cities.

The protesters' demands were remarkably heterogeneous. From what appeared to begin as a protest over an increase in bus fares in São Paulo in early June, the demonstrators, with their tens of thousands of handmade placards, had a huge range of grievances: the state of the public health and public education systems, a more general revulsion against the systematic corruption of business and political elites, and the brutality of many police forces.

The backdrop to these events was the tremendous economic boom Brazil had enjoyed. This had principally benefited two key groups: the very rich and powerful had enjoyed a massive surge in their wealth; and the very poor had benefited from the bolsa família that was the centrepiece of Lula's eight-year presidency. Neither of these groups was present in any number at the demonstrations.

A few organised bodies from the favelas who were actively opposing housing relocation were present, and some youths from the peripheries took part in some demonstrations, but the crowds were overwhelmingly made up of the urban middle classes – a category that stretches from downtown junior office workers to university professors.

As a class they had swelled under Lula and his successor, Dilma Rousseff, especially as enrolment in higher education had expanded, but their living standards had only inched forward. They were paying, in their own words, "European taxes to get Mozambican services". They took their own toilet paper and blankets to public hospitals and went into debt to get some private health cover. Deep-seated resentments were multiplied by the contemptuous manner in which Brazil's elites treated the rule of law and indeed the other classes.

This had all been the case for some time. The question, then, is why the protest should have erupted in the winter of 2013.

Significantly, perhaps, participants and observers struggled to give the uprising a name. The Vinegar Revolution was tried – a reference to its use as a teargas antidote and the arrest of a number of protesters merely for carrying vinegar in their bags. Others opted for the Brazilian Spring.

The comparisons with the Middle East were real but limited: the events' reliance on social media networks and the collective disbelief of rulers and ruled alike that this was happening had shades of Egypt. However, Brazil had been booming economically, it had no really problematic foreign entanglements and it was not under military rule.

Perhaps the best option was Copa das Manifestações – the Demonstrations Cup. For what gave rhythm and focus to the protests was the staging that month of the Confederations Cup; and what allowed the many grievances of the Brazilian public to coalesce into this wave of outrage were the economic costs and the social impact of staging the World Cup in 2014.

Earlier this year, the Brazilian government announced that the prospect of protests at the World Cup would be met with beefed-up security, on a budget of over £500m, and the formation of a national police force 10,0000-strong, deployable anywhere in the country. Reassuringly, they said the force would be trained to the same standards as UN peacekeepers.

Perhaps they will be taking courses with the Brazilian forces who served in this capacity in Haiti, and who treated the exercise as a cross between a military occupation and a favela pacification programme. Perhaps they will be needed to quell the gun battles and protests that have been emerging in Rio's "pacified" favelas, where the rule of the drug gangs has yet be replaced by the rule of law.

There may still be victories to be won on the football field next month. but it is hard to imagine that they will unite the nation in the way they did in the past, for they have been bought at the cost of making Brazil's injustices starker than ever.

David Goldblatt's book Futebol Nation: A Footballing History of Brazil is published by Penguin. His radio series, The History of Brazil Is Round, starts on Radio 4 on Monday