In the understated but highly effective new film Spotlight, the city of Boston, Massachusetts, is depicted as a grey, inbred small town dominated by the Catholic church, one major ethnic group, a cabal of ethically malleable lawyers, and a small group of civic leaders who all seem to know each other and who join together to keep a lid on things. In new film Black Mass, the city of Boston, Massachusetts, is depicted as a grey, inbred small town dominated by the Catholic Church, one major ethnic group, a cabal of ethically malleable lawyers, and a small group of civic leaders who all seem to know each other and who join together to keep a lid on things. Not all the villains in these films are my fellow Irish Catholics. But an awful lot of them are. An awful lot.

Spotlight and Black Mass, which share a somewhat subdued emotional tone, come on the heels of such Beantown fare as The Town, Mystic River, The Boondock Saints and, most importantly, The Departed, all of which depict Boston as an inbred small town that … well, you get the general idea. The obvious question is: what did the great city of Boston ever do to end up so black and blue?

To answer this, it is important to know a few things about the city. Boston, like San Francisco, occupies a distinctive place in the hearts of the American people. It is a city that is both beloved – the Pilgrims, Paul Revere’s Midnight Ride, the Boston Tea Party – and resented. Boston, like San Francisco, looks down on the rest of the country. This attitude has become particularly annoying in the past decade-and-a-half as its four sports franchises have come to dominate American sports, a distinction previously held by New York or Los Angeles.

Boston is a city that imagines itself locked in an epic battle with New York, a battle of which New York is oblivious. A useful analogy is Edinburgh versus London. Edinburgh is small, charming, cultured, historic, mythical. It has great architecture, great museums, great parks, great restaurants, a wonderful ambience. It is one of the world’s truly great cities. London, however, is London.

It cannot be denied that cities such as Philadelphia and Chicago, and perhaps even Washington, also have a chip on their shoulder, because they are great cities that simply are not vast and varied enough to go toe to toe with New York, or, for that matter, Los Angeles. But Boston is unique in that it has several chips on its shoulder. The white Anglo-Saxon Protestants who used to run the city deeply resent the Irish and Italian immigrants who took it over a century ago. The Irish and the Italians resent being resented. And the minorities who never seem to make their way into Boston mythology wonder when they’re going to get a chance to call the city their own. This sort of stuff passed into the history books in other parts of the United States decades ago.

Paul Newman in The Verdict, 1982. Photograph: Allstar/20 CENTURY FOX

Boston is the city where immigrants literally deposed the “Brahmins” who dominated it for the first three centuries of its existence. The Brahmins and those who still fancy themselves vaguely Brahminish have still not forgiven them. I once met a White Anglo-Saxon Protestant, who lived not far from where the Kennedys grew up, and asked her if she had ever crossed paths with John, Bobby or Teddy. She replied: “The Kennedy family resonated more at the national level than at the local level.” Nope, the locals never forgave the Irish for taking over their town. They never will.

The most attractive feature of Boston as a film location is the sense of inbred cultural values that other cities do not have. If you want to make a movie in a town where a small group of implacable white men still run everything, you can mostly forget about Philadelphia, Detroit, Cleveland, St Louis, Washington, Baltimore, Miami and a dozen other cities where the Irish and Italian ethnic enclaves have greatly shrunk in size or simply disappeared. If you want to make a film set in a town that simultaneously has a superiority and an inferiority complex, a town that has an us-versus-them mentality toward the rest of the country, a town that clings desperately to its traditions, even its worst ones, Boston is your ticket.

Boston is a small town that wants to be thought of as a big city, and a big city that wants to think of itself as a small town. For a big city, it certainly has a meagre skyline, and the fungible part of Boston is surprisingly tiny. It is, however, a city with neighborhoods where it is still possible to imagine someone saying, “I’ll put the word out on the street,” as if the neighborhood tribal drums still pounded. This sort of thing used to be true of Manhattan’s Little Italy and Philadelphia’s Kensington and various parts of Chicago. But it is true no more.

Not all movies set in Boston deal with the tough neighborhoods in South Boston – “Southie” – and not all films set in Boston end badly. The Equalizer takes place in Boston, but it could just as easily have been Chicago, Philadelphia, Detroit, Miami or Los Angeles. The late-90s charmer Next Stop Wonderland is a sweet little film that has nothing to do with ethnic rivalry or gangsters. Nobody gets their legs broken in Good Will Hunting, though it is worth noting that the basic message of the film, conveyed in a memorable conversation between Matt Damon and Ben Affleck, is: if you want to make something out of your life, get out of South Boston.

Nobody gets chopped up and tossed into the river in Ted or Ted 2, and in many other films set in Boston the principals generally manage to avoid confrontations with thugs. Still, most of the classic Boston films, starting with The Friends of Eddie Coyle and proceeding through The Verdict and on to Mystic River and Gone Baby Gone focus on the idea of Boston, and particularly South Boston, as a dark, brooding, dangerous place, a closed community inhospitable to outsiders, where bad things happen to people who don’t play by the rules. The subject is treated at great length in the 1999 film Southie, starring Mark Wahlberg’s brother, Donnie.

Denzel Washington in The Equalizer. Photograph: Scott Garfield/Publicity image from film company

While it is impossible to ignore tax incentives that have lured film-makers to Boston, other factors are at work. Some of the US’s biggest movie stars – Ben Affleck, Matt Damon and Wahlberg – hail from the area. Damon and Affleck, however, grew up in Cambridge, where Harvard University is located. Real Bostonians do not regard movie stars from Cambridge as authentic sons of the city. This is why memorable films such as Love Story and The Paper Chase and The Social Network do not usually appear on lists of famous films set in Boston. Real Boston films are set in the old neighbourhood; Cambridge is the preserve of rich kids and outsiders. People are very touchy about this stuff up in Boston. Even though Cambridge is literally right across the river.

Except for a few dissolute cowboy shut-ins wasting away out in Wyoming, Americans know all this. The legend of Southie is not the world’s best-kept secret. There is a scene early in Spotlight, where Liev Schreiber, playing a Jewish editor who has just been redeployed to Boston from Miami, after working in New York, is seen reading a famous book called The Curse of the Bambino. The book deals with the disastrous 1919 trade that sent Babe Ruth, the greatest baseball player of all time, from the cash-strapped Boston Red Sox to the hated New York Yankees. The Red Sox did not win another championship for 85 years; the Yankees won 27. Schreiber is reading the book in an effort to familiarise himself with Boston’s values and traditions. This is the one scene in the film that rings false; everybody in America knows about Boston, everybody knows about Babe Ruth and the Red Sox, everybody knows how much Bostonians hate New Yorkers, everybody knows about Southie. It would be like an editor from London needing to read up on the Beatles to become more familiar with Liverpool.

Women are basically invisible in both Spotlight and Black Mass; the only recent film set in the city where a woman plays a major role is The Heat. An even more notable feature of the two films is that minorities are almost never seen. This would surprise few Americans sensitive to the issue of racial relations, where Boston has both a good and a bad history. The abolitionist movement to end slavery in the southern states of America was especially intense in Boston, a fact for which the city deserves great praise. But Boston was the site of some of the ugliest confrontations in the 1960s when whites objected to the federal government’s desegregation program that involved bussing schoolchildren in and out of neighborhoods like Southie. This is a scar that has never healed. In films like Spotlight and Black Mass, the protagonists – whether journalists, gangsters, priests or lawyers - are so uniformly white that you’d never know minorities even lived in the city. That’s Boston to a tee.

Black Mass is on release; Spotlight is out in the UK on 29 January