Radioactive contamination that leaked for more than two decades from the Dounreay nuclear plant on the north coast of Scotland will never be completely cleaned up, a Scottish government agency has admitted.

The Scottish Environment Protection Agency (Sepa) has decided to give up on its aim of returning the seabed near the plant to a "pristine condition". To do so, it said, could cause "more harm than good".

At a board meeting in Stirling on Tuesday, the Scottish government's environmental watchdog opted to encourage remediation "as far as is practically achievable" but to abandon any hope of removing all the radioactive pollution from the seabed.

Tens of thousands of radioactive fuel fragments escaped from the Dounreay plant between 1963 and 1984, polluting local beaches, the coastline and the seabed. Fishing has been banned within a two-kilometre radius of the plant since 1997.

The most radioactive of the particles are regarded by experts as potentially lethal if ingested. Similar in size to grains of sand, they contain caesium-137, which has a half-life of 30 years, but they can also incorporate traces of plutonium-239, which has a half-life of over 24,000 years – meaning that is the time period for half of the material to break down.

The particles are milled shards from the reprocessing of irradiated uranium and plutonium fuel from two long-defunct reactors. They are thought to have drained into the sea with discharges from cooling ponds.

In 2007, Dounreay, which is now being decommissioned, pleaded guilty at Wick sheriff court to a "failure to prevent fragments of irradiated nuclear fuel being discharged into the environment". The plant's operator at the time, the UK Atomic Energy Authority, was fined £140,000.

Since 2008, over 2,300 radioactive particles have been recovered from the seabed, with 351 removed by a remotely operated underwater vehicle this summer. Since 1983, over 480 particles have also been found on three local beaches and the Dounreay foreshore.

Sepa recommended in 1998 that the seabed around Dounreay should be returned to a "pristine condition". Since then, it pointed out, the contamination had been extensively investigated and new regulations on radioactively contaminated land had come into force.

"It is now widely accepted that a literal return to a pristine condition is a far from simple or even achievable concept," a Sepa spokeswoman told the Guardian.

"Trying to achieve it might also cause more harm than good. There is the potential that ecosystems may be destroyed on trying to get to something which does not pose a significant hazard."

An expert committee set up by Sepa warned in 2006 that disturbing the seabed could cause particles to escape and be swept ashore, putting members of the public at risk. The most radioactive particle found "could have had life-threatening consequences if it had been ingested", the committee said.

Sepa's board agreed to change its policy to encourage further remediation "provided that this achieves more good than harm and accepting that at some sites it will not be practical to return the land to a pristine condition".

Dounreay, which is now managed by a consortium including the UK engineering firm Babcock, welcomed Sepa's new policy. It was still aiming to remove "the majority of the most hazardous particles, together with the removal of any other particles encountered," said the site's senior project manager, Phil Cartwright.

"The best practicable environmental option, which was welcomed by the government agencies, is focused on doing more good than harm and was publicly discussed on the basis that it would never be possible to retrieve every particle."

Friends of the Earth Scotland, however, attacked the development. "Once again, we see the nuclear industry causing a problem it can't solve, and dumping the cost and consequence on the rest of us," said the environmental group's chief executive, Stan Blackley.

"Nuclear power is neither safe, clean, cheap nor low-carbon and it continues to cause problems and cost the taxpayer a hidden and open-ended fortune. Let's learn from our past mistakes and consign it to a lead-lined dustbin."

Nuclear leaks in the UK

Windscale, Cumbria, 1957: Fire at a military plutonium reactor spread radioactive contamination over large parts of England and Europe

Dounreay, Caithness, 1963-84: Tens of thousands of radioactive particles from old reactors contaminated the shoreline and the seabed

Sellafield, Cumbria, 1983: The government advised people not to swim or use beaches along 10 miles of coastline after a radioactive leak from a reprocessing plant

Chapelcross, Dumfriesshire, 2000-05: 126 radioactive particles from defunct reactors found on the shore of the Solway Firth

Sellafield, Cumbria, 2006-11: 1,233 radioactive particles and pebbles contaminated by historic leaks found and removed from nearby beaches

Dalgety Bay, Fife, 1990-2011: Hundreds of radioactive remnants from the luminous dials of second world war aircraft removed from foreshore