A Tottenham fan who threw a banana skin at the Arsenal striker Pierre-Emerick Aubameyang has been handed a four-year football ban and fined after magistrates ruled there was a racial element to the incident.

Averof Panteli admitted throwing the banana skin on the pitch after Aubameyang celebrated a goal but denied there was any racial motivation to the incident, with his lawyer claiming the 57-year-old picked it up because it was the closest thing to him.

But magistrates decided there was a racial element after an application was made by the prosecution.

Panteli was fined £500, which includes £100 for the racial element, and told to pay £135 in costs at Highbury Corner magistrates court on Tuesday.

The magistrate, Mervyn Mandell, said: “We find that it was a targeted gesture to throw a banana after a goal was scored by a black player.”

Also handing him a four-year football banning order, Mandell said: “The racial element was there.”

Panteli, who lives in Norwich, pleaded guilty after being charged with throwing a missile on to a football playing area in contravention with the Football (Offences) Act 1991.

The court heard the van driver had been caught on CCTV after Aubameyang, who captains the Gabonese national team, had scored in the 4-2 victory for Arsenal and celebrated in front of the Tottenham away fans at the Emirates stadium on 2 December.

The court heard Panteli, who said he had been attending games since he was seven years old, told a steward who removed him from the stand: “It was not a racial thing. It just happened.”

After being arrested, he told police: “Sometimes you do things in life and say: ‘Why did I do that for?’”

Defending, Leila Rasool claimed Panteli did not throw the skin directly at the Arsenal player, saying: “It really was a spur of the moment thing. He picked up the closest article to him. It could have been a plastic cup. It could have been an apple core.”

The Arsenal players were “goading” the away fans with their celebrations, the lawyer argued. Panteli did not bring the banana to the match himself and was sober, she said.

He was not charged with a racist offence, Rasool said, claiming that police had accepted his account. She said Panteli was from a large Greek Cypriot family and had a son-in-law who was mixed race.

She said Panteli had faced intense scrutiny since the incident, including suffering death threats. She told the court: “This is so abhorrent and disturbing to him that he’s been branded a racist.”

Outside of the court, Rasool said she was “very disappointed” with the court’s decision, adding: “I think they have bowed down to media pressure. At the end of the day, Mr Panteli has been charged and pleaded guilty at the earliest opportunity to an offence of throwing a missile or object, namely in this case unfortunately a banana skin on to the pitch.

“The crown today sought to add a racial element to it through the backdoor by asking for an uplift … It simply affected the nature of the fine.”

Asked by the Guardian outside of court how he would defend himself against critics who say he could not have unthinkingly thrown a banana skin at a black player, Panteli said: “No, I threw it at the group of players, not at one particular player.”

He added: “If there’s anyone I need to apologise to I will. Ian Wright, Aubameyang, Arsenal players, Tottenham players, and especially Tottenham the football club and my fellow football fans.”

After reports of the verdict surfaced, Aubameyang posted a picture of an article about the case adding a emoji of a sad face and thumbs up, writing: “Four years and we start again.”

Separately, an Arsenal fan who threw a plastic bottle at Spurs players celebrating a goal in the same game was banned from attending football matches for three years.

Charlie Watts, 27, from St Albans, received the ban and a £650 fine, and was ordered to pay £85 costs and £65 victim surcharge, after pleading guilty to throwing a missile.