A man who came to Australia as a toddler 61 years ago has lost a high court bid to have his visa reinstated after being convicted of cannabis trafficking, leaving him at risk of deportation.

The high court has dismissed John Falzon’s fight to regain his visa, meaning the Victorian grandfather could be deported to Malta, a country he left when his parents migrated to Australia in 1956.

Falzon, who never gained Australian citizenship, had his visa cancelled on character grounds after serving eight years’ jail for trafficking a commercial quantity of cannabis. He also had convictions for previous drug and other crimes.

Falzon has been in immigration detention since his release from jail, fighting through the courts to get his visa back.

Meanwhile his family has run social media campaigns, saying deportation would be an unbearably cruel act for a man who knows no one in Malta, and who would be forced to abandon a large and loving family in Australia.

Falzon argued in the high court that he had served his time for his crime and ejecting him from Australia was an attempt to punish him further.

But, in a decision handed down on Wednesday, the high court dismissed that argument.

The court found the immigration minister must cancel a visa held by a person if the minister was satisfied that they failed the character test because of a substantial criminal record. His application was dismissed with costs.

Falzon’s family in Australia includes two sisters, four brothers, four adult children and 10 grandchildren.

At one point his daughter Kim Falzon started a change.org petition, seeking support to ward off her father’s eviction from the country.

“Please help save my dad from deportation,” Kim Falzon posted two years ago. “John made some bad decisions, & the law convicted him & he is now getting towards the end of his payment to society for his wrong doing.

“That should be that but it is not. There are moves afoot to deport him to Malta … He knows no one there, his whole life and family is in Australia … this cannot be allowed to happen.”

The petition was backed by 249 supporters and other relatives also set up a Facebook page in a bid to keep him in Australia.