The turmoil that enveloped Serena Williams and Carlos Ramos during the final of the 2018 US Open had little or nothing to do with racism or sexism as it was widely portrayed at the time and is lingeringly perceived.

However, from the player’s perspective it did have distant roots in prejudice and struggle, which exist beyond tennis. Also, it was made possible by an uneven battle of wills, the lauded and entitled champion bumping up against the inconvenience of authority, which is part of the culture of modern sport.

When Williams began to lose control of the match and her composure she trained her frustrations not on her dignified opponent, Naomi Osaka, but on the one person who could not fight back in kind. Williams wanted an argument; Ramos, the chair umpire who swung no racket, wanted peace and order – so he reacted with the only weapon at his disposal, the laws of tennis.

Ramos, the only person to have umpired singles finals in all four majors, is widely regarded as firm but fair, irrespective of a player’s gender, race or reputation. He brought all that experience to bear in imposing three code violations, which resulted in Williams forfeiting a whole game in the second set and throwing away a grand slam final she should have won. Victory would have moved her alongside Margaret Court on 24 majors, the most any player, male or female, has won. To do so in front of her home crowd, at 37, a year and a week after giving birth to her first child, would have been the crowning achievement of her career. Her disappointment in losing to a player making her debut in a grand slam final was immense.

From a distance there was an obvious inclination to side with Williams, several columnists and commentators arguing that this would never happen to a man of similar standing in the game, while heaping opprobrium on Ramos, the Portuguese martinet du jour. Close up it was more nuanced.

Williams, down a set and up 1-0 and 15-40 in the second, was stunned when Ramos warned her for a coaching violation from the stands by Patrick Mouratoglou [which he admitted in a post-match TV interview]. She told the umpire, “If he’s giving me a thumbs-up, he’s telling me to come on. We don’t have any code, and I know you don’t know that, and I understand why you thought that was coaching but I’m telling you it’s not. I don’t cheat to win. I’d rather lose.”

The matter seemed done. But Williams, still agitated, sought assurance on the next changeover that Ramos understood what she had said. He replied, pointedly, “I understood your reaction [to Mouratoglou] as well.” This unsettled her further.

She broke Osaka to lead 3-1 but then double-faulted twice in a row from 30-15. On break point she dumped a backhand and turned to her box, shouting “I can’t get over the top of the ball”, before smashing her racket into the court. Ramos issued her with a second code violation and docked her a point. He had done no more than follow protocol. “This is unbelievable!” she screamed at Ramos, who remained calm. “Every time I play here I have problems.” It was true.

Aggrieved at calls against her and delivering high-grade invective at the officials, Williams also erupted when losing the 2004 quarter-final against Jennifer Capriati, the 2009 semi-final against Kim Clijsters, and the 2011 final against Sam Stosur. Her past crowded in on her again in 2018. Incandescent, she told Ramos, “You owe me an apology, I have never cheated in my life. I have a daughter and I stand for what is right for her.” It was an incongruous point to make.

Osaka held and broke, to lead 4-3. She was two games from becoming the first able-bodied Japanese player, man or woman, to win a major and, at 20, the youngest at Flushing Meadows since Maria Sharapova in 2006.

Williams’s tirade continued on the changeover: “You will never, ever, ever be on another court of mine as long as you live!” She called Ramos, “a liar” and, “a cheat”. Pausing to absorb the insult, he waited until she had almost returned to her position at the baseline to receive serve, then shocked the packed Arthur Ashe Stadium by awarding the game to Osaka without a ball struck.

Williams, needing to hold to stay in the match, halted play and pleaded for understanding from the tournament referee, Brian Earley, and the WTA supervisor, Donna Kelso. “There are men out here who do a lot worse than me,” she said, “but, because I’m a woman, you are going to take this away from me? That is not right.”

For her supporters that was Williams’s golden bullet of defence. She was being picked on because she was a woman. But Kelso could do nothing for her. Nor could Earley. History was not much help either. The New York Times later listed the men and women who had incurred code violations in the four majors in the 20 years to 2018. In the categories for which she was punished 646 men were fined for racket abuse, 99 women; for coaching, though, it was 87 men, 152 women.

Williams held to love, eyes blazing, and threatened a fightback. But she could not stop Osaka closing out the match to win 6-2, 6-4 in an hour and 19 minutes that none of us present will easily forget. To witness the volcanic explosion first-hand was to feel and hear the wail of resentment, a sense of injustice that had gathered in her soul over years and was distilled into an almost blind rage.

What happened that afternoon began not when Ramos spotted Mouratoglou gesturing to Williams to come to the net but on the streets of Compton in south Los Angeles, where Richard Williams took his family away from the relative calm of Saginaw, Michigan – where Serena was born in 1981, a year after Venus – to give his children an appreciation of struggle in a tough environment.

For Serena and Venus it was a brief education and they left Compton in 1990 for the warmth, comfort and coaching facilities of Florida, where their tennis journey continued. One of their three half-sisters, Yetunde, stayed, however, and was murdered in a drive-by shooting in Compton in 2003. The family were devastated.

Late in the 2018 season Serena was reminded of the tragedy shortly before going on court to play Johanna Konta in the first round of the Silicon Valley Classic. She learned that Yetunde’s killer had been released from jail. “I couldn’t shake it out of my mind,” she said. She won just one game in the heaviest defeat of her career. Her path to New York was thus a troubled one. Although she had played well in her comeback season – reaching the fourth round at Roland Garros and losing to Angelique Kerber in the Wimbledon final – Williams revealed shortly before the US Open she had been coping with post-natal depression.

What is curious about the incident that September evening is that it sprang from an arcane and fiddling point of tennis legality. Williams could have ignored it. But she was a prisoner of her past.

Another curiosity: while coaching is allowed outside the majors, nobody could recall ever previously seeing Williams look to her team for advice or support to take advantage of it. She always fixes her gaze across the net – except on the worst night of her career, when she chose to engage a determined umpire in a futile exchange that cost her a title she should, and could, have won.