An outspoken former Conservative party donor has said he is prepared to go to prison to keep a giant “Bollocks to Brexit” poster above his London office after he was told to remove it by his local council.

Charlie Mullins, the founder of Pimlico Plumbers, accused Lambeth council of interfering with his freedom of speech and succumbing to people’s sensitivities over a word that was proven in a Sex Pistols case in 1977 not to be obscene.

The council said it had received complaints from commuters about the sign near Waterloo station and, as it was not related to his business, Mullins would have to remove it or apply for planning consent, which he would be unlikely to get.

“We had a sign there previously saying, ‘Nobody voted to be poorer’ for six months and Lambeth did not object,” Mullins said. “They say it has nothing to do with our business, but quite clearly it has everything to do with our business. If there is Brexit, we won’t have a business.

“This is just bullying and the advert is not coming down. They will need to put me in prison before that comes down. I am just a business standing up for my rights.”

The millionaire, who has a workforce of about 440, is one of the country’s best-known backers of the campaign to stop Brexit.

In 2016, he supported Gina Miller’s successful high court bid to give parliament a meaningful vote over the final Brexit deal and he donated to her End the Chaos campaign to publish facts about Brexit.

Bollocks to Brexit is the slogan of a grassroots campaign group, which handed out stickers at the recent Labour and Conservative conferences.

Mullins has said he accepts that some commuters may find the word offensive but points to the trial over the Sex Pistols’ Never Mind the Bollocks album cover. In 1977, the punk band’s reputation soared after a police officer confiscated albums in a shop in Nottingham, arguing that the appearance of the word “bollocks” contravened the 1899 Indecent Advertising Act, forcing shops around the country to sell the album under the counter.

The record was ruled not indecent after the defence barrister John Mortimer QC got an English professor to testify that the word had appeared in medieval Bibles, veterinary books and literature and was a colloquialism for nonsense.

Mullins said that a media company had offered to provide him with 22 electronic sites for a similar advert since the sign had gone up three weeks ago and attracted attention on social media.

Lambeth council told Mullins that “as the sign is not specifically related to the business of Pimlico Plumbers and is well in excess of the size limits, the sign does not benefit from deemed consent and will therefore either need to be removed ASAP or an application for advert consent submitted”.

It said it would be an offence under the Town and Country Planning Act 1990 to continue to display it without consent.

Although the council received a number of complaints from the public about language, it pointed out that the content of adverts was a matter for the Advertising Standards Authority.

The ASA said it had a number of complaints about the billboard, but it was outside its remit which does “not include shop window displays, point of sale, in-store ads or material on the company’s own property” . There are no categorical rules against swearing in ads, though the ASA takes into account context, medium and audience, as well as wider standards.

A Lambeth council spokesman said it was considering “whether or not the physical advertisement requires advertisement consent under planning controls and as part of this process, we contacted the company setting out the normal rules and options available to them”.

It said planning-related advertising legislation was “complex and adverts can be displayed without the need for the council’s consent” but this was subject to various conditions and limitations.

The council said it had made an informal approach to Pimlico Plumbers and, if it did not breach the rules, no action would be taken.