The government is considering "next steps" after Britain First, an offshoot of the British National party, failed to remove an image of the crown from its uniform and publicity materials.

The Cabinet Office, which is responsible for authorising use of official royal symbols, said it had written a letter to the far-right party asking it to remove all images of the crown from its website, marketing materials, stationery and stock "with immediate effect". The Cabinet Office said it had received a number of complaints about Britain First's use of the official symbol, which it said it had not authorised. A spokesman added: "It appears that Britain First has not fully complied with our requirements and we are considering the most appropriate next steps."

The action against Britain First emerged after the party was this week rapped by the advertising watchdog for using royal symbols on its website without permission. The Advertising Standards Authority (ASA) ruled that all images of the royal crown must be removed from the group's websites. But the body has no power to rule on the group's use of the symbol elsewhere.

This year, Britain First activists campaigned in Yorkshire wearing flat caps and green jackets embossed with the party logo, which features the crown and the words "Taking our country back" and "Rule Britannia". The jackets, as well as other clothing including a beanie hat and hoodie, are for sale on the group's website.

Paul Golding, the party leader, dismissed the ASA as a "toothless quango with no power which no one takes any notice of".

He said the group would continue to use its logo, including the crown. "We've checked it out with our solicitors and it's used by multiple organisations up and down the country. It's not an official crown, it's something we've designed with a graphic designer."