Arsène Wenger has admitted his team have had to play in “a very difficult climate” at the Emirates Stadium this season but suggested the mutinous atmosphere is being whipped up by a few egos with a personal agenda against him and actually has “not a lot behind it”.

Three of Arsenal’s supporters groups intend to protest at Saturday’s televised game against Norwich City, calling for a “Time for Change”. Fans from the Arsenal Supporters’ Trust, the Black Scarf Movement and Red Action will hand out placards pre-match and will urge the crowd to hold them up after 12, 78 minutes and, in particular, after the final whistle, with their ire aimed at the hierarchy and, by implication, the manager for the way the club is being run.

Wenger’s side are currently fourth and attempting to deflect Manchester United’s challenge at their back, with the Frenchman admitting his own exasperation at the fizzling out of their title challenge. “It is very frustrating because we were in a position for a very long time where we could compete for the title” he said. “We won the FA Cup in 2014, the Charity Shield [sic] and FA Cup in 2015, and we wanted to go another step up and win the league. It wasn’t possible, which is why we are frustrated. I can understand the frustration because no one is more frustrated than we are. But we must protect what we have and go for what we can still achieve. Our disappointment can’t go too far.

Facebook Twitter Pinterest Arsenal supporters protest against Arsène Wenger, and the general running of the club, ahead of the recent victory over West Bromwich Albion at the Emirates Stadium. Photograph: BPI/REX/Shutterstock/BPI/Rex/Shutterstock

“We lost the championship at home against the lower teams – we have the best record against the top clubs – but we played at home in a very difficult climate. We have to realise that, away from home, we are championship winners [Arsenal actually boast the third best away record in the division at present]. At home, against the smaller teams, we lost the league. This club does have special values though and one I’ve experienced over the years is to stick together and support the team. There’s no success without that.

“But some groups of people try to manipulate our fans, and I believe apart from an agenda – a personal agenda, a big ego – there’s not a lot behind it.” Piers Morgan, who has been vociferous in his criticisms of Wenger on social media, immediately suggested on Twitter that the Arsenal manager was directing his criticism at him, though the Frenchman would not elaborate on whom he was commenting.

Piers Morgan (@piersmorgan) He meant me, and that's fine.

Very happy to debate ego & manipulation with Wenger any time, any place. https://t.co/0d5Gvqvlws

Wenger defended his team’s “character and attitude”, which has been questioned not least by former players now working as pundits. “Some people question them,” he said. “I know [the critics] well, and they have less character than this team have.

“I saw them play and I know them very closely, and they shouldn’t question this team’s character. Attitude-wise, we’ve been impeccable from the first to the last day. But when a club cannot enjoy anything any more, it’s in trouble. In football you go down very quickly and come up very slowly. We have to stick together.

“I think this club has special values, and we care about the club and we care about our fans as well. We try to keep everybody happy. But we have to put things a little bit into perspective and see how the club’s evolution has gone over the years. It was not always easy. The quality of the work we’ve done at the club has got us into a strong position where expectations are very high, and where we are frustrated when we don’t get what we want.”

Arsenal’s manager of almost 20 years did not speak like a man planning to walk away from the last year of his contract at the Emirates Stadium, and pointed to the way he had transformed the club over his time in charge. “When I do go I hope the club goes higher up and wins the Champions League and the championship every year,” he added. “I’ll be the biggest supporter in the stands because I will feel as if I have contributed a bit to that with the basis we’ve built.

“When we built the stadium, we had five to seven difficult financial years where we had to pay the money back: we had to be three years in the Champions League out of five and have an average of 54,000 people, and we didn’t know we would be capable of that. We had to sell our best players every year to survive, but we didn’t do three years out of five in the Champions League. We did five out of five. Now the club is in a stronger position and we can compete again with our main opponents. But during that time it was very difficult. Now the club is in a very strong situation. You want your fans as well to be behind your team.”