British taxpayers will still be paying fines to the EU over the mishandling of farming subsidies after the country has left the bloc, the National Audit Office warned on Friday.

At least £660m has already been paid in fines, owing to delays in implementing the rules of the common agricultural policy in the six years to 2013. More fines will follow for the intervening years, as they are levied retrospectively, and leaving the EU does not absolve the UK from responsibility.

The issue adds to the problems facing Andrea Leadsom, the surprise appointment to the Department for Environment, Food and Rural Affairs (Defra), as she struggles with the looming end of EU farming subsidies. Farmers and conservationists are tussling over what form any new subsidy should take, while the new secretary of state will have to explain her views on subsidies having previously rejected them.

The National Farmers Union (NFU) told the Guardian: “We will obviously be looking for guarantees from government that the support given to our farmers is on a par with that given to farmers in the EU, who will still be our principal competitors.”

Conservation and health groups have also lobbied the government this week, ahead of Leadsom’s appointment, to ensure that any future subsidies from the taxpayer are given only on condition of farmers fulfilling strict requirements to protect the environment. At present only a small proportion of payments are directly tied to environmental stewardship, and some of the current rules are unpopular with sections of the farming community.

Big farms do best out of the current subsidy system, with many gaining millions while small farmers make do with a few thousand pounds. Conservationists are concerned because intensive farming has led to big declines in the number of farmland birds and other wildlife, and want any new system to try to halt or reverse that. They are hoping to swing public opinion behind them.

Leadsom will have to negotiate between farmers anxious to preserve their subsidy – currently about £3bn a year, depending on the euro exchange rate – and civil society and conservation groups seeking to ensure current protections are not lost, and are strengthened as far as possible.

The NFU is currently holding a consultation with its members over what form post-Brexit trade agreements for the agricultural sector will take, which will inform its discussions with Defra. However, the organisation was unable to say whether questions on what role environmental protection requirements should play in any future subsidy system would be included in the consultation programme with members, which is still being finalised.

Meurig Raymond, president of the NFU, said he was seeking a meeting with Leadsom as soon as possible to discuss the issues. “At the forefront will be a new domestic agricultural policy,” he said. “This should encourage growth, innovation, productivity and profitability.”

Farmers are also hoping that a national system of support for farmers would be better administered than payments have been under the Common Agricultural Policy (CAP). Delays to the single farm payment were behind the fines from the European commission, and have caused hardship for thousands of farmers who have frequently had to wait months for payment, which often makes the difference between profit and loss on small farms.

A Defra spokesman told the Guardian that the department was catching up on the backlog and that the distribution of EU subsidies was proceeding more smoothly: “We understand the importance of rural payments to farmers, and over 87,000 farmers successfully registered on the rural payments system in 2015. It has been used to process and pay over 86,700 farmers – over 99% of all those eligible – their basic payment scheme payment. We are taking action to tackle the causes [of the problems] and are making a significant investment in our mapping data. This will be used to better administer CAP payments.”

Raymond insisted that problems with the existing system – the underlying structure, data and technology of which is likely to form the basis of any replacement subsidy regime – must be remedied. “We need a simpler application process and improved delivery for [payments]. We need a guarantee from government that agreement for farmers in agri-environment schemes will be honoured in full,” he said.

But there are likely to be headaches far bigger than payment systems and EU fines facing Theresa May’s government when it comes to agricultural subsidies. Taken together, farming and food represent one of the UK’s biggest export industries, and any trade agreement with other countries, including EU partners, will be fraught with national concerns on each side. This will constrain the UK’s ability to subsidise its farming industry.

Small farmers may be particularly vulnerable to the uncertainty to come.

Nigel Akehurst, a small farmer in Sussex, who voted remain but whose parents voted to leave, worries that whatever happens the smaller sector will lose out. “Sadly I do think a lot more family farms will go to the wall as many of them don’t have the skills to market their product direct and without subsidies they will make even bigger losses than many small farms are making currently,” he said.

“The only positive is that land prices may come down – though I appreciate most politicians see agricultural land, especially in the south east, as building or development land – and that may give more people the opportunity to buy small plots of land to grow and rear niche products to sell direct.”

“There are many uncertainties to overcome,” said Raymond. “Trade agreements, labour, financial support, legislation are all up in the air. [We want] a domestic agricultural policy that is science-based, adapted to the needs of British farmers, and promotes a profitable, productive and sustainable future for British food and farming.”

Less likely to cause problems is Leadsom’s own previous slamming of subsidies, as many iron political principles have become malleable under the heat of the referendum.