To Arsène Wenger it is the set-backs that remain with him, gnawing away, impossible to scratch. “It is the problem of my life, basically,” the Arsenal manager says. “I’m competitive and every disappointment stays with me forever. It is a scar in your heart and reduces your life expectancy.”

When Wenger considers his record against Sam Allardyce, who visits the Emirates Stadium with his new Crystal Palace team on New Year’s Day, it ought to be with his glass half-full. In 15 years or so he has faced Allardyce 31 times in all competitions, winning 18 and losing only five. But it is the ones that have got away that Wenger recalls with the greatest clarity. His curse is that victory over an Allardyce team, be it Bolton Wanderers or Newcastle United, Blackburn Rovers, West Ham United or Sunderland, loses much of its cachet because it has tended to be expected.

The dropped points hit home with force and none have done so more than those from the 2-2 draw at Allardyce’s Bolton in the fourth last game of the 2002-03 Premier League season. It was the day when Wenger’s team surrendered a 2-0 lead late on and, with it, control of their destiny at the top of the table. Manchester United found themselves two points clear and they did not look back, winning their remaining fixtures to take the title.

On that day at the Reebok Stadium Wenger was close to meltdown on the touchline. He ripped off his tie early in the second half and whirled his arms with increasing ferocity as the game wore on. The commentators at the time said it was Wenger as had not been seen – the cool, scholarly exterior stripped back.

Allardyce loved it and, even though plenty of matches have since been played, the images have endured. This is how Wenger’s relationship with Allardyce has come to be considered – the Frenchman as the sophisticate but vulnerable to being knocked from his perch by Allardyce, the great friend of Sir Alex Ferguson and, for so long, a member of the cartel of British managers in the north-west of England.

“Of course, I remember the 2-2 at Bolton,” Wenger says. “Football is like real life but in a more condensed way, more intense. At some moments it catches you suddenly and it can be very cruel. In competition it is like that.”

Wenger’s relationship with Allardyce has improved over the years. “Slowly, slowly,” Wenger says, with a smile. Back in the Bolton days Allardyce scored a number of successes over him, most notably during a five-match purple patch in the league from 2004-06, when he won three and drew two. Allardyce frequently out-tactic-ed Wenger, to borrow one of his famous phrases.

The pickings for him have been slimmer in more recent times. Since Allardyce’s Blackburn beat Arsenal at Ewood Park in 2010 he has lost in all bar one of his 10 meetings with Wenger’s team. The exception was the 0-0 draw that he earned with Sunderland at the Stadium of Light last April.

And yet Wenger will be wary of the Allardyce Factor on Sunday, not least because his old rival knows how to organise a team; how to make them tough to break down. “The level of confidence or urgency always goes a bit up when a new manager comes in,” Wenger adds. “That makes the game more difficult.”

Allardyce’s Palace will present a familiar challenge at the Emirates – a compact unit that will seek to profit on the counter-attack or set pieces and for whom a point would be a terrific result. Wenger has lamented the lack of “classic” games in this season’s division; the dearth of “spectacularity” despite the presence of so many elite managers.

And he puts it down to the fear that every club below the top six feels to varying degrees about relegation. Going down is unthinkable for financial reasons and so the caution has come to be pronounced. “Everybody is under pressure, first, not to lose,” Wenger says.

Tony Pulis’s West Bromwich Albion were the last team to come to the Emirates and they frustrated Arsenal on Boxing Day until Olivier Giroud scored the only goal of the game in the 86th minute. “I don’t blame West Brom [for their approach],” Wenger says. “You can understand them. They want their 40 points. And, after they get them, you will see a different West Brom.

“At the start of the season you have the six or seven teams that will fight at the top and the other teams say: ‘Let’s make sure we are not in the seven that will fight not to go down and get 40 points as quickly as possible.’ The problem today is that for these teams to play in the Premier League, they need to pay the wages of the Premier League. And if you go down with the wages of the Premier League, it is a massive problem.”

Wenger is not surprised to see Allardyce back in the Premier League, after his ill-fated stint with England. He talked in general terms about why chairmen at struggling clubs would always favour experienced managers who gave “certain guarantees to stay in the league”. Allardyce is one of those managers.

Wenger’s preoccupation is simply to get the better of him and continue the response to the back-to-back defeats at Everton and Manchester City before Christmas. “We responded well against West Brom and we now have to show who we are – which is the team that went 19 games unbeaten [after the first match of the season]. We have to show we can bounce back. That will make our season or not.”