"Modern architecture died in St Louis, Missouri on July 15, 1972, at 3.32pm (or thereabouts)." So wrote the polemicist Charles Jencks in the course of launching its replacement, postmodern architecture, and he was referring to the Pruitt-Igoe housing development, a 57-acre array of 11-storey slab blocks which, less than 20 years after they were completed, were destroyed by controlled implosion.

The architect of the project was Minoru Yamasaki, who also designed the twin towers of the World Trade Centre, and might be considered unlucky in the amount of violence done to his works. Yamasaki professed humanity, harmony and his opposition to prejudice, beliefs born of his experiences as a put-upon Japanese American. He laid out Pruitt-Igoe according to the best principles of the modern movement: an orderly plan in which cars and pedestrians were separated, ample open space was provided between the blocks, and flats were oriented to catch daylight and views.

By the time they were destroyed, the blocks were notorious for violence, vandalism, chaos and squalor. Police and firemen refused to go there, as missiles would be hurled at them from high levels if they did. If Yamasaki, in common with Le Corbusier and other modernists, believed that rational architecture could make people behave better, Pruitt-Igoe seemed to prove the opposite. So, in Jencks's words: "It was finally put out of its misery. Boom, boom, boom." Its fate became the prototype for the detonation of similar projects in the United States and elsewhere. By the 1990s, in Britain, the great kerflumpf of a collapsing tower block became a form of civic festival, in which politicians would preside over bacchanals of cascading masonry.

It was obvious. It became a certitude of the age that some silly architects had had a fantasy, which turned into the opposite, and appropriate measures were taken, but as with many certitudes the truth now seems to be not quite so simple. According to a film recently released in the United States, the causes of Pruitt-Igoe's end were many and complex.

The Pruitt-Igoe Myth: an Urban History directed by Chad Freidrichs, is built around interviews with people who lived there, archival clips, and footage of the Blair Witch-like woodland that has grown up on the site. The residents recall their utter joy when they moved in, not only at the plumbing, heating and electricity, but also at views and the "warmth of community". "When I moved in, it was one of the most exciting days of my life," says one interviewee. "My memories of Pruitt-Igoe are some of the best I have," says another. One calls her flat a "poor man's penthouse". People remember "a wonderful building with so many different smells of cooking" and "so many kids to play with". To start with at least, Yamasaki's modernism was not alienating.

The film charts the degeneration – the lifts that stopped working, the rubbish incinerators that failed, the winter that the pipes burst, the rise of bullying and gangs. Eventually there is horror. A man describes how, as a nine-year-old, he watched his mother vainly trying to shove his brother's guts back inside, after he had been shot in the stomach with a sawn-off shotgun.

"It would be here today if it had been maintained like it was when it opened up," says one of the voices, "but it went down and down and down and down."

While government money paid to build the blocks, running costs were to be funded out of rents which, as the residents were poor, were not enough. Rents were raised, even as the blocks declined, such that they consumed three quarters of some residents' income. They were paying more for less.

21 April 1972: part of Pruitt-Igoe is demolished. Photograph: Getty Images

Behind maintenance lay politics: public housing projects were viewed with suspicion, as un-American and socialist. They were bad for business, in that they deprived private landlords and real estate companies of opportunities. They were also good for business in that they made work for construction companies, but only while they were being built, which helps explain the decision to invest in their creation but not their maintenance.

And, with politics, went race. Pruitt-Igoe's residents were black, many of them recent immigrants from the depression-hit countryside, whose first homes in St Louis had been appallingly overcrowded slums, with nine people in three rooms, to give one example remembered in the film. The lifespan of the development coincided with the rise of white flight, the evacuation of white people to suburbs (whose growth was encouraged by government policy) beyond the city limits.

The film shows old footage of housewives saying "I moved here because it is a white neighbourhood" and "I could just not live alongside them… they want mixed marriages and to be equal with us". With white flight went the loss of tax income to the city. Factories and jobs also moved out which, combined with discrimination in hiring, left the residents of places like Pruitt-Igoe with minimal prospect of work.

Recipients of welfare, meanwhile, were treated brutally: there was a rule that families would lose their entitlements if there were a man in the home and one-parent families were thereby forcibly created. An adult recalls how, as a boy, he had to lie to inspectors when his father paid a visit. There were also bans on owning televisions and telephones. The attitude, says an ex-resident, was "we're giving you money so we control you". For another the effect was to "make people feel isolated and restricted". Their treatment was "void of humanity caring, like a prison environment". The "mindset for inhabitants was that they weren't cared about".

And so the place fell to pieces, despite the heroism of many of its residents. Vandal-proof fixtures were installed, but "the fact that it was indestructible made you want to destroy it". There were "bitter people, angry people" who would attack emergency vehicles, because "they wanted to make a statement". Drugs, prostitution and shootings took over. A rent strike by residents brought some concessions from the authorities, but it was a short-lived victory. Not long after, they were ejected, and the blocks destroyed.

Apart from the effects of policy and poor maintenance, Pruitt-Igoe was also a victim of events nobody foresaw. When it was planned in the early 50s, cities like St Louis were expected to keep on growing, as they had done for decades. As it turned out, the population halved. If the scale of the development made some sense in an expanding city, it did not in a shrinking one.

It is a striking feature of the film that no one mentions Yamasaki's architecture as a major factor in Pruitt-Igoe's calamity, except to praise the quality of the flats. No one calls it grim and inhuman, and when it is compared to a prison it is because of the management regime, not the design. There is no suggestion that a postmodernist treatment could have saved it. Perhaps if it had been lower rise, the problems with lifts could have been avoided, and perhaps the wide-open spaces contributed to the eventual lawlessness. Then again, there are perfectly successful social housing projects, in America and elsewhere, that have both lifts and open spaces.

If it was vain of modernist architects to think they could change people for the better, there is also vanity in their critics to think that design alone can change them for the worse. If there was an element of grand gesture, rather than close attention to the human detail, in the creation of vast projects like Pruitt-Igoe, there was equal gesture in their destruction. The effects of architecture alone are rarely as significant as people think: it can certainly affect your sense of wellbeing for better or worse, and it can aid or hinder the larger forces that it serves. But politics, economics, and such things as the presence or absence of prejudice are what really change people's lives.