The Champions League final this year will be the eighth in a row without Pep Guardiola, which perhaps should not feel as extraordinary as it does. This is, after all, knockout competition; odd things happen. Nobody, not even in the superclub era, has a divine right to be in the final.

And yet given Guardiola has won seven league titles in his nine completed seasons as a manager – and may next month make it eight out of 10 – and has managed three of the most powerful clubs in the world, it is notable.

So, why? Why, since Guardiola won the Champions League with Barcelona in 2009 and 2011, do his sides keep falling a fraction short?

Spurs prevail as mind-bending match sees Guardiola's City lose control | Barney Ronay Read more

Narrative, Will Storr’s new book The Science of Storytelling argues, is a machine for making sense of the world; it is natural to try to find reason behind the basic incoherence of existence. Nobody works harder on that quest than Guardiola, a manager renowned for his research who is forever striding through the pages of Martí Perarnau’s books clutching a portfolio of stats and diagrams under his arm.

And yet even a genius such as he must struggle to find much sense in the chaos of what happened at the Etihad on Wednesday. Yes, he could point to the way Tottenham’s midfield diamond put pressure on the heart of his side, prompting the errors that led to Son Heung-min’s two early goals. And he could note that same diamond left Tottenham’s full-backs exposed, which was a contributory factor in each of City’s first three goals. And he could observe the injury to Moussa Sissoko and Tottenham’s dearth of midfield alternatives changed the balance of the match late in the first half.

Quick guide Follow Guardian sport on social media Show Hide Twitter: follow us at @guardian_sport Facebook: like our football and sport pages Instagram: our favourite photos, films and stories YouTube: subscribe to our football and sport channels Photograph: Chesnot/Getty Images Europe

But none of that was what decided the game, which was a corner bouncing in off the hip of Fernando Llorente, who probably would not have been on the pitch had Sissoko not had to go off, and had Harry Kane not been injured last week. And before it bounced off the hip, that corner brushed Llorente’s arm, not that the referee, Cuneyt Cakir, appeared to be shown the replay that demonstrated that most clearly. He probably would still have allowed the goal under the law as it stands, but perhaps not when it changes in June.

And after that there was Raheem Sterling’s 93rd-minute effort that VAR chalked off. Guardiola was gracious about that, acknowledging Sergio Agüero had been offside in the buildup, but it was a tight-run thing, a matter of a foot or so, and it was impossible for the mind not to go back to last season’s quarter-final when an erroneous offside call denied Leroy Sané a goal that would have reduced the deficit to 3-2. Had VAR existed then, that goal would have stood and who knows how Liverpool would have reacted in the second half? It was reasonable also to wonder just how different the Llorente situation was to the handball for which Yaya Touré was penalised in the buildup to what would have been a winner for Barcelona against José Mourinho’s Inter in 2010.

Facebook Twitter Pinterest What if this Leroy Sané ‘goal’ for Manchester City against Liverpool last season had not been ruled out? Photograph: Simon Stacpoole/Offside/Getty Images

Guardiola has gone out of the Champions League on eight occasions as a manager. Three times it was on away goals. Three times his players missed penalties. Three times his side battered their opponents in the second leg and inexplicably fell a goal short. And that is before you begin to consider implausible events that have worked against him: the eruption of the Icelandic volcano that forced his Barcelona squad to travel to Milan by bus in 2010, Ramires scoring after a 40-yard run and brilliant chip for Chelsea in 2012, the defensive glitch that allowed Tiemoué Bakayoko to head the decisive goal for Monaco in 2017 …

If Guardiola has developed a complex about the Champions League, it would hardly be a surprise. He is, after all, of the generation of Barcelona fans who grew up with a sense the competition was not for them but for Real Madrid; he was even a 15-year-old ballboy for their astonishing comeback against Gothenburg in the 1986 semi-final, only to see his side lose on penalties to Steaua Bucharest in the final, even though it was played in Seville and only 200 fans were allowed to travel from Romania.

Pep Guardiola hits out at VAR after Llorente’s disputed goal for Spurs Read more

And that creates a fascinating tension. On the one hand there is the accusation Guardiola is too committed to attack, that he is yet to come to the conclusion Sir Alex Ferguson eventually did that, at the highest level, against the most clinical attackers, it is safer to have five chances and deny your opponent any than to have 20 and allow your opponent three. And yet on the other there is the fact that whenever he tries to compromise, it goes wrong.

But beyond such theorising is the more basic sense that at key moments, at least since the 2009 semi-final against Chelsea, Guardiola keeps being unlucky. How he could do with an opposing goalkeeper to help him out as Sven Ulreich and Loris Karius did Zinedine Zidane last season. The impact of fortune can be mitigated to an extent but never absolutely. What is critical is how that has generated a narrative of failure that may or may not be legitimate but with which Guardiola is nonetheless wrestling.