The run-up to the 2016 Olympics exposed issues with infrastructure, crime, and poverty in host city Rio de Janeiro, yet during the games, the worst predictions of unrest have for the most part failed to materialize. That may be because Rio is heavily secured—some 85,000 officers and soldiers are on duty, the second-largest contingent among the four games held since 9/11. But all eyes should be on what happens this week, when the Olympics end and normalcy returns to Rio.

Throughout the games, muggings have occurred regularly in Rio. On August 5, four men with knives tried to rob Felipe Seixas, the federal police delegate that oversaw the security of the opening ceremony, as he walked to his car. A day later, Portugal education minister Tiago Rodrigues Brandão was mugged by two men with knives in Ipanema. And on August 7, two members of the Australian Paralympic team had their bicycles taken at gunpoint.

Petty crime is old hat for Rio: Although homicides have been declining for at least a decade, theft—including stolen cell phones, cars, cash machines, and houses (link in Portuguese)—is on the rise. But more blatant illegal activity has been somewhat sidelined during the games, as security forces, the government, and criminal factions succumb to what British journalist Misha Glenny calls an “unspoken truce” during major events. “There’ll be a lot of muggings,” Glenny told Quartz ahead of the games. “[But] what I don’t think is that there’ll be street fight.”

While police and soldiers are charged with keeping the peace in Rio, their ability to do so rests on the tacit cooperation of would-be serious offenders—typically drug dealers and members of organized crime outfits—many of whom have at least one reason to be on their best behavior. Roughly 500,000 foreigners are in Rio for the games, and the city’s dealers, who sell R$317 million ($100 million) in product annually, are keen to capitalize on their larger-than-usual customer base.

But that all changes once the Olympics are over, at which point the government must decide what to do with the massive security forces put in place ahead of the games. Many of those forces are in the city’s slums, known as favelas.

A history of violence

Throughout the 2000s, one Rio favela was famous for being outside the influence of organized crime. Called Tavares Bastos, it was a favorite setting for movies (from Brazil’s Last Stop 174 and Elite Squad to The Incredible Hulk) and music videos (Snoop Dogg and Pharrell’s “Beautiful” was shot there). Tavares Bastos was peaceful for one reason: It was home to a feared special command of Brazil’s military police—the same group that inspired Elite Squad.

In 2008, what seemed to work at Tavares Bastos—having a permanent police force stationed inside its borders—was standardized as part of a municipal project that distributed Police Pacification Units throughout Rio. Since then, 38 police units have been installed in the city—not coincidentally, favelas around Olympic sites were prioritized. At Santa Marta, the first favela to join the project, murders dropped to zero in the first four years.

Today, 75% of those living in the “pacified” favelas approve of the police presence (link in Portuguese), but over the past three years, peace has lost its foothold. In Santa Marta, members of criminal factions can now be seen walking around with guns. Earlier this year, a gun battle between police and members of the Red Command (a Brazilian gang) resulted in the first violent death in the favela since 2008.

A matter of money

The health of Rio’s pacification project depends on its finances. Just two months before the Olympics, the state government declared a financial emergency, saying there was no money to pay salaries or pensions. That default has been attributed to fluctuations in tax collection and oil revenue, but also to overspending and corruption.

Although the city said municipal finances were unaffected by state-level problems, the pacification units are partially maintained with state money. Due to the crisis, the state invested R$8 billion ($2.4 billion) in public safety this year, R$2 billion less than planned. In March, the installation of one police unit, at Complexo da Maré, was cancelled (link in Portuguese).

Criminal factions have been returning to the favelas in part to prepare for Olympics-re”partly because they wa for business during the Olympics, but also because the police units are losing authority,” says Glenny. He notes that Rocinha, Rio’s largest favela, is currently divided between police (who control the main roads and the commerce areas) and organized crime (who control everything else).

When and if the pacification project recovers, and the police are able to restore peace, there will still be many flaws (link in Portuguese) to address. Favelas need sanitation, schooling, and health care resources, in addition to military ones.

All of these issues are likely to emerge in force once the glow of the Olympics has worn off. After the games, “the depth of the economic crisis [will really] hit Rio with full force,” Glenny says; just as “the gaze of the world goes elsewhere,” the people of Rio will face an uncertain future.