It’s a mild Monday evening in April and 56,795 people, officially at least, have come along to London Stadium at the Olympic Park in Stratford to watch West Ham United play Stoke City in a bottom-of-the-table Premier League match. That’s a large crowd, by any standards, but judging by the fans milling around the huge concourse outside the ground, it’s not a happy one.

Of course, football fans are not renowned for their cheery optimism. By and large, it’s a grim business being a supporter, a forlorn struggle between daydreams and despair. As Nick Hornby wrote in Fever Pitch: “The natural state of the football fan is bitter disappointment, no matter what the score.”

In March, during a match against Burnley, some West Ham fans were sufficiently disappointed to invade the pitch in an act of protest. A much larger number gathered beneath the directors’ box to hurl insults and objects at the joint chairmen of the club, David Sullivan and David Gold, who were ushered away by security as the protesters – in a show of magical thinking of which diehard supporters are reliably capable – demanded that the board sack itself.

It’s highly unlikely that this scene would have taken place had West Ham been at the top end of the table, yet the animosity was about more than a poor performance (they went on to lose 3-0 to Burnley).

West Ham are no strangers to setbacks. They’ve twice been relegated in this century, and endured several spells in the second tier in the last century. Such reversals are understood to be part of the twists in fortune that inevitably attend competitive sport. What has upset the fans this season is not the loss of a football match so much as the loss of an identity. As far as the West Ham faithful are concerned, the move in 2016 from the old Boleyn Ground in Upton Park, three miles to the east, to the London Stadium in Stratford feels like a painful exile from their spiritual home.

You’ve got all these trendy bearded people and they’ve got all these 13-hop ales I don’t understand Tom Girling, West Ham fan, 52

A typical complaint I heard before the Stoke game came from Tom Girling, a 52-year-old who has been following West Ham since he was 13. “This place is soulless,” he said, gesturing at the stadium and the empty expanse in which it’s located. “It’s got nothing. I used to go to Upton Park, grab a programme, nip in the pie and mash, have a bet, into the boozer, meet my pals, all good, have a laugh, then out afterwards. I’ve got nish here. I’m out in the elements drinking beer out of a plastic glass.”

I told him that I’d seen a pie and mash kiosk outside nearby Hackney Wick station, and there is one on the perimeter of the stadium. He looked at me as if I’d suggested becoming a Spurs fan.

“You mean the fashionista place?” he asked scathingly. “No, no, no. I’m not being funny… you’ve got all these trendy bearded people and they’ve got all these 13-hop ales I don’t understand. I’m old school. I like a pint of Carling. I’m a working-class boy. Football’s a working-class man’s game.”

As we know, there are few tribes who feel more endangered than the white working-class male. And while many will mock, the plight of disgruntled West Ham fans is in many ways a symbol of a culture that is rapidly vanishing from London life. Among the capital’s big football clubs, West Ham long enjoyed the reputation as being the most grounded in its community – the working-class East End. It was the home of heroes such as former players Billy Bonds and the late, legendary Bobby Moore.

But east London is in the midst of a huge transformation, and nowhere is that process more evident than from the vantage point of the Olympic Park, a concrete wilderness sitting between the advance forces of gentrification: consumerism and hipsterism.

On the east side of the park looms the giant glass edifice of the Westfield shopping centre, a glittering cathedral dedicated to the disposal of excess income. On the west is an old industrial quarter now inhabited by artists and canal-side bars with their craft beers and moodily hirsute clientele. Neither place feels like home to people like Girling, and nor does the “sterile wasteland” – as another fan, Mark Reynolds, describes it – that lies between them.

“I don’t smell any burgers and onions,” Girling complains. Reynolds, a 56-year-old supporter of 43 years’ standing, agrees: “You go down to Upton Park and there’s boozers right by the ground. It was all locals and the market, and you had the Asian community. There was Nathan’s, the pie and eel shop, people used to queue up round the corner. It was just part of the match-day experience.”

The new ground in Stratford. Photograph: Mike Hewitt/Getty Images

But pie and eels don’t transform a football team’s fortunes, and football, as we’re constantly reminded, is a business. If you’re going to be successful you need to generate enough money to buy the best players. To do that it helps if you can attract a large attendance. The capacity at Upton Park, a tight little ground, was 35,000. It’s 57,000 at the London Stadium. As the saying goes, do the maths. Sullivan and Gold did, and they argued that the new ground was the first step towards a bold, bright future. West Ham fans understand that logic; they just believe that that future is a cruel illusion.

“The move to Stratford was made on the promise of a world-class team in a world-class stadium,” says Brian Williams, another long-term fan and author of Home from Home: A West Ham Supporter’s Struggle to Reach the Next Level. “After two seasons at the new place we know that that’s not true, and that we have given up our history, our heritage, our legacy, all of which was focused on Upton Park. So we have a very poor team playing at a stadium that is not fit for football.”

History, heritage, legacy? Aren’t those rather inflated concepts. After all, it’s only a football pitch surrounded by some seats. What’s the big issue?

Back in the summer of 2009 I was taken around a vast building site at Stratford by members of Sebastian Coe’s London Organising Committee for the 2012 Olympic and Paralympic Games. At its centre was a giant stadium in the mid-stages of construction, though its design – a rather bland modern bowl – was even then clear. It was built to hold 80,000 spectators and then to be disassembled to leave a capacity of around 25,000. Its legacy, which was the buzzword of Olympic planning, was to provide an athletics stadium to replace the decrepit National Sports Centre at Crystal Palace. Coe told me that there was a “complicated stakeholder landscape” that included the government, the mayor’s office, five London boroughs, UK Sport and Sport England, but everyone agreed that there should be no “white elephants” left behind by the Games. He explained that what distinguished the Olympic Stadium from previous ambitious public projects, like the Dome (now the O 2 Centre), was “clarity of purpose”.

Shortly afterwards it was decided that the idea of a national athletics stadium should be shelved. Suddenly the stadium’s longterm purpose was looking distinctly murky. The talk turned to a multi-purpose venue, which in fact was the concept originally discussed when London was bidding for the Games. At that time West Ham had expressed an interest in an occupancy, but were discouraged when a decision was made to design the stadium specifically for athletics.

Once the Olympics were over, the stadium, having been built as an athletics arena, became the subject of complex tenancy competition between two football clubs – West Ham and Tottenham Hotspur – for what seemed like a cheap way of using a large stadium with great transport links. E20, the public company set up to run the stadium, offered a deal to the potential occupants in which taxpayers would pay many of the running costs, like policing, stewarding and turnstile operators. Tottenham’s bid involved knocking down the Olympic Stadium and building a football-specific arena in its place, which they estimated would cost a total of £320m.

After much legal wrangling, and accusations of behind-the-scenes deals, it was West Ham who emerged victorious. And rather than knock down the stadium, the agreement was made to adapt it for football, at an estimated cost of £190m. Last year an independent review commissioned by the London mayor, Sadiq Khan, found that costs had escalated to £323m – more than it would have cost to build a new stadium.

Inside the London Stadium, the first thing you notice is the low rake of the terraces and the wide space between the pitch and the stands, where the athletics track is placed. The space is filled by a green cover which, according to West Ham’s previous manager, Slaven Bilic, left his players confused about the dimensions of the pitch. There was talk of changing the colour to claret – West Ham’s main kit colour – but that was blocked by the stadium’s owners.

Green or claret, from where I’m sitting in the 35th row watching the Hammers take on Stoke, that gap makes the action looks oddly remote, as if disconnected from those watching it. From the 73rd row, at the back of the stadium, it’s more like looking through the wrong end of a telescope. The atmosphere is a mixture of anxious and surly, with very little singing by the home team. The exception is a couple of hearty renditions of West Ham’s club anthem, I’m Forever Blowing Bubbles. The lyrics, written by three Americans in 1918, are filled with a telling pathos.

I’m forever blowing bubbles

Pretty bubbles in the air

They fly so high

Nearly reach the sky

Then like my dreams

They fade and die

Fortune’s always hiding

I’ve looked everywhere

I’m forever blowing bubbles

Pretty bubbles in the air

Who could sing with such conviction about fading and dying dreams but football fans?

“We can make a din here,” Girling had told me, but he felt that it was a rare occurrence rather than the regular experience at Upton Park, where opposing teams were “intimidated” by the noise.

Perhaps the only people who have been intimidated at the London Stadium this season are the owners, Sullivan and Gold. In the club programme for the Stoke match Sullivan wrote a defensive column, calling on the fans to “produce the goods in front of the cameras” (the match was live on Sky). The pair acquired a controlling stake in the club in 2010, buying out the previous owners – the rump of an Icelandic consortium that was bankrupted by the 2008 financial crisis. They had previously owned Birmingham City FC, where they proved to be less than popular. Having made their money in the porn business, then Ann Summers (Gold) and the Daily Sport (Sullivan), it’s fair to say they have seldom enjoyed a glowing public profile.

But it could be argued that they are well suited to West Ham. They’re both working-class blokes made good. Gold, the son of an East End criminal, actually grew up a few yards from the Boleyn Ground and even played for the West Ham youth team.

As the protests showed, however, that counts for little with the fans. There is a belief that Sullivan and Gold are prone to grand statements on which they can’t or won’t deliver. Williams suggests they’re “too hands-on in football matters”, in which they are not expert.

This criticism, perhaps unsurprisingly in the still largely unreconstructed world of football, has been particularly addressed to West Ham’s vice-chairman, and sometime sidekick of Alan Sugar on The Apprentice, Karren Brady. One fan, who insisted on anonymity, told me: “I think she is probably the most malevolent influence there is at West Ham.”

Disgruntled fans make their feelings known at the Burnley game in March. Photograph: Christopher Lee/Getty Images

She did not endear herself to the fans when she tweeted 15 minutes before the Stoke match that her new TV show was starting at 8pm – the same time as kick-off. “Give It a Year starts on ITV,” she wrote, “hope you watch it and I hope you enjoy it!”

“She seems more worried about her own ego than what’s going on at the clubs,” says Reynolds.

“I’m not her biggest fan,” says Williams, with ironic understatement.

Still, when all is said and done, at the time Sullivan and Gold bought the club it was in financial crisis. Following relegation, they guided it out of the Championship (the second tier league) and into mid-table Premiership security, and moved it into a brand-new stadium with good transport links in a part of London that is set to thrive.

Among the fans I spoke to, Upton Park was a link to a disappeared past for the East End’s white working class

The old Boleyn Ground was not suited to expansion. It has since been demolished and new Barratt Homes are going up in a development called Upton Gardens – more East End gentrification. Nor is the area around the ground quite the pie and mash heaven of old cockney folklore; in fact it’s largely populated by Londoners of subcontinental origin, with Halal butchers and sari shops. Reynolds says that this was another element of life that made match days so appealing by contrast with the “soulless” experience of the new location.

But perhaps it’s significant that many of the West Ham fans lament the loss of community to which they did not in any geographical sense belong. Williams, for example, lives in Brighton, though his wife hails originally from the East End, and Girling is a west Londoner with family roots in the east London meat trade. Reynolds lives in St Albans, “out in the shires”, as he put it. Of half a dozen fans I spoke to at the Stoke game, none lived near the old ground.

From that admittedly small sample, Upton Park represented a link to a disappeared past for the East End’s white working-class diaspora. Not that West Ham followers are exclusively white or working class. You do see different ethnicities, but nowhere like the make-up of either the West Ham team itself or Newham, the borough that contains Upton Park and some of the Olympic Park, which – at 16.7% – has the lowest percentage of white British residents in London.

None of that is to suggest West Ham is unwelcoming of other ethnicities; it’s to note that football, like east London, is in the midst of social transformation, and as with all change, some will find the new environment more alienating. “Detached” was how Girling characterised his fellow West Ham fans’ feelings about their new home. But it’s possible that to the next generation of fans, which is likely to come from a more diverse base and for whom the names of Billy Bonds and Bobby Moore may not possess the same totemic power, the new setting could well appear more open.

One such candidate might have been 18-year-old Sami Sidhom from Forest Gate, a West Ham season ticket holder who was at the Stoke game. After the match he was stabbed to death outside his home in an apparently unprovoked attack, the latest victim of London’s epidemic of teen knife murders.

As they often say in football, such tragic events lend a sense of perspective. Or as Williams acknowledges in his book, “relocating from one football stadium to another is not the most serious crisis facing humankind in these worrying times”.

In truth, the London Stadium is not a ground best suited to watching football or generating an atmosphere of intense – or intimidating – togetherness. A restless, resentful feeling swirled around the bowl until Stoke, against the run of play, scored a goal.

Then something more urgent and angry kicked in. The crowd bayed and cursed and in the last minute, the substitute Andy Carroll hit a superb volley to equalise. There was a sudden release of pent-up tension and the fans went home, if not happy, then mollified.

West Ham will probably survive in the Premier League, but the culture and community with which it has for so long been associated may be destined, like those pretty bubbles, to fade and die.