Art curators aren’t usually the targets of hate mail and death threats.

But then Tony Baxter (who has received messages informing him “there's a bullet with your name on it” and whose staff are regularly accused of being thieves who “are going to die of cancer”) isn't your average custodian of culture.

The former model, investment banker and cross-Channel swimmer has become the go-to guy for anyone wanting to shift – and flog – a Banksy mural.

It all started when “Slave Labour” was painted on the side of a Poundland shop in Wood Green, north London, just before the Queen's diamond jubilee in May 2012. A few months later, the work, featuring a boy hunched over a sewing machine stitching union flag bunting, had been chainsawed off the wall before vanishing.

“No one knew where it was, and our whole goal as a concierge is that we're supposed to be the best-connected network in London,” says Baxter, in his first proper interview.

The director of The Sincura Group, which aims to fulfil every whim of its VIP members, adds: “It just so happened that a client said, ‘find the answer to ‘Slave Labour’’. The council did bugger all; they just sat there and drank tea.”

Sincura located the work in a Miami auction house and flew it back to London to see if a buyer with at least £900,000 in spare change could be found so the street art could be kept in the UK.

The Sincura name was splashed across the media and Baxter says he now gets about ten emails a week from people asking him to remove what they assume to be Banksys from their buildings.

The 39-year-old son of a Cambridge academic has since overseen the “salvaging” of three other murals across Britain and has got hold of a further five.

They form what is the most expensive collection of the artist’s work ever assembled under one roof – the Stealing Banksy? exhibition, which opened at the ME London hotel today. The show concludes on Sunday when all the pieces go under the hammer (total estimates stand at £5m).

Baxter does not own any of the works, and he insists he has never made a penny in profit from the sale of any piece taken from a building, nor has he ever approached anyone to remove one.

What he does get is “a small management fee that covers just a fraction of my staffing and insurance costs”; a strange kind of kudos for his company; and “a good way to entice people” to buy the Banksy canvases he sells, where he does get a cut.

Baxter also has ethical conditions that must be met. The owners must be intent on removing the piece regardless; they must have at least one non-financial motive (for example, fearing a grade II listing as a result of the graffiti); and he insists on some kind of charitable donation.

Has Baxter had any contact with the man himself? “I, I, I can't comment on any of my involvement with Banksy,” he says. It is the only time he hesitates.

The works are being taken from communities and will likely end up in an plutocrat's mansion. But he claims: “We're restoring these for ever. In 100 years’ time, or 1,000 years’ time, when these are the old masterpieces they may have become, they’ll still be around. We can’t stress enough – if you're going to do it, do it properly. You wouldn’t take the Mona Lisa and chop it in half.

“At the end of the day, we sleep easy at night knowing that what we’re doing is legal. It may not be the most ethically sound – but it is the lesser of two evils.”