A Banksy sculpture called The Drinker, a comic riff on Rodin’s The Thinker, has been withdrawn at the last minute from a major London auction – two days after an allegation it was stolen.

The artist Andy Link, also known as Art Kieda, said the sculpture, which is more than 180cm tall was taken from his garden a number of years ago and he regarded himself as the rightful owner.

On Tuesday it was due to be a highlight of a contemporary art sale at Sotheby’s main auction room in London with an estimated top price of £1m.

But at 9am Link received a phone call from Sotheby’s to say the sculpture was being withdrawn. The auction house said the decision was not related to his claim.

In a statement Sotheby’s would only say: “The work has been withdrawn in agreement with the consignor.”

The statue was originally left by Banksy in a small square off Shaftesbury Avenue in central London in 2004. At first glance it looks like The Thinker, only drunker and with a traffic cone balanced on his head.

Link “kidnapped” the statue and contacted Banksy for a ransom of £5,000. The anonymous street artist responded with an offer of £2 towards a can of petrol to set it on fire.

The statue ended up in Link’s garden but was later removed while he was away. According to the sale catalogue the consignor acquired the work from a dealer in 2014.

Link said he did not know why it had been withdrawn from auction. It could have been the result of publicity after the original story in the Observer or that the consignor had got a better deal. “I’ve no idea. Maybe they’re going to sue me for causing loads of grief over it,” said Link.

He said the saga had affected his mental and emotional health. “I’ve been in tears, I’m up, I’m down, I’m angry … it has been two weeks of hell.”

Link, who still has the original traffic cone from the sculpture, said it was not and never had been about the money.

His ideal outcome would be to meet the present owner of the statue “and come up with a deal that would reunite the cone with the statue and we get 50% each. I want to give a million quid to a local youth charity in Hackney. If it’s worth £2m I would gladly give half that to a local charity.”