Shortly after Jeremy Corbyn finished his speech to the Labour party conference, his closest colleagues gathered in a small bar at the end of Brighton Pier. Among the attendees were the leader’s aides, as well as those of John McDonnell and Diane Abbott. Abbott’s birthday coincided with the end of the conference and a birthday cake was produced in her honour. As is almost inevitable when Corbyn’s allies are gathered together, the discussion turned to his football team, Arsenal, and the fortunes of its long-serving manager, Arsène Wenger.

Corbyn might have been a serial rebel in parliament but he is a loyalist when it comes to his football team, whose home ground is in his constituency. He is one of the dwindling band who will argue for retaining Wenger’s services no matter what.

The Labour leader’s support for the Frenchman is attributable in part to the latter’s success in overseeing Arsenal’s move from Highbury to the state-of-the-art 60,000-seater Emirates Stadium a decade ago without saddling the club with unsustainable debt. Perhaps Corbyn sees a parallel with his stewardship of Labour: he has transformed the party’s finances and moved it to the left. But Corbyn has yet to claim the major trophies that marked Labour’s and Arsenal’s 1990s and 2000s golden years.

Corbyn’s support for Wenger has an emotional element, too: the two men have known each other for a long time, and they discuss not only football but politics and philosophy. It might seem counter-intuitive, given his long record of voting against official Labour policy, but Corbyn puts a great premium on personal loyalty. That’s why he will not use his enhanced internal clout to conduct a wide-ranging reshuffle, even if Theresa May refreshes her team, as seems likely. Corbyn feels a sense of obligation to those MPs who stuck with him even during his most difficult period as leader.

That frustrates some of his inner circle, who fear that they may come to regret their leader’s excessive sense of fidelity. One aide blames Corbyn’s loyalty to long-standing allies for the party’s underwhelming response to anti-Semitism among its rank and file. Many believe that cost Labour votes in areas with large Jewish populations. In Chipping Barnet on the outskirts of London, where Wenger has long been rumoured to be a party member, Labour came within 353 votes of unseating Theresa Villiers, a prominent Brexiteer and former Northern Ireland secretary.

Factor in five additional seats in which the Jewish population was larger than the Conservative majority – Harrow East, Hendon, Chingford and Woodford Green, Finchley and Golders Green, Cities of London and Westminster – and, they believe, a more robust intervention against anti-Semitism would have helped put Labour in office, albeit in a fragile coalition.

Some in the leaders’ office also worry that there is too much dead wood on the front bench, particularly at junior ministerial level. One shadow cabinet minister who is the target of particular ire is Jon Trickett, who was stripped of his responsibility for the election campaign, in favour of Andrew Gwynne and Ian Lavery, but retains his role as shadow lord president of the council.

Trickett, it is said, was no good as campaign co-ordinator, is not a reliable presence on television or radio, and acts as a bed-blocker to more talented and younger MPs with impeccable Corbynite credentials. (It’s worth noting that although Trickett has reinvented himself as a left-winger, he came into the Commons after a stint as a fairly right-wing council leader and served as a parliamentary private secretary to both Peter Mandelson and Gordon Brown.)

Others disagree with this analysis, however. As one of Corbyn’s aides put it to me, the trouble with reshuffles is that “they always make more enemies than you had before”. The feeling is that the MPs to Corbyn’s right who stepped up to fill holes during the 2016 coup attempt against him have been the biggest successes: Jon Ashworth at health, Angela Rayner at education, Barry Gardiner, notionally at international trade but in reality Corbyn’s minister for the Today programme. Gardiner’s appeal in the leader’s office is that he does what Tessa Jowell did for Tony Blair and Caroline Flint did for Ed Miliband: he is a serene presence on the airwaves, even in times of turmoil. As for the lingering debate about anti-Semitism, allies of the leader note that Labour has now decisively tightened its rules.

Nevertheless an underlying division remains among the inner circle over Labour’s prospects at the next general election. Corbynite optimists believe that the Conservatives, rather like Labour in the 1970s, are defenders of “a status quo that is failing”. The passage of time will therefore wear away what remains of the Tory electoral advantage. At the next election, there will be fewer homeowners, more people in insecure work and an even longer period of sluggish-to-stagnant wage growth, all of which favour the opposition’s electoral chances.

Pro-Corbyn pessimists are less sanguine. They worry about what one calls “a Tory reset”: the emergence of a new leader who can appeal to the socially liberal young or who can convincingly make the case that Brexit is not working out. They believe that Corbyn, by refusing to reshuffle and bring in new faces, is only adding to their vulnerability should the Conservative Party pull itself together. Adding to their sense of unease is Labour’s failure to build a more commanding poll lead at a time when Theresa May is so weak and the Tories are so divided.

It would be a mistake to see the split as factional or as being between McDonnellites and Corbynites. Both Corbyn and his shadow chancellor have aides who think the best is yet to come. But others quietly fear that we have reached “peak Corbyn”. It is too soon to say which side is right.