James Stevens has spent 37 years fishing out of Peterhead. When he began, some 450 boats frequently filled the granite harbour on Scotland’s rugged northeast coast. Today only about 100 trawlers regularly leave Peterhead to ply their trade in the North Sea.

Like almost everyone in Peterhead, Stevens blames the European Union – and particularly the unpopular common fisheries policy – for his industry’s decline. In June, the skipper of the Harvest Hope voted for Brexit “for my children and my grandchildren”.

Bargaining chip

Peterhead has weathered many storms over the years. A ring of imposing granite houses overlooking the port and a name, Blubber Quay, are all that remain of the whaling industry that dominated in the 1800s. The 20th century saw the death of the herring industry and the arrival of motorised trawlers. Now fishing representatives hope Brexit will reinvigorate the industry by redesigning the quota system – which limits annual catches – and placing restrictions on foreign fleets’ access to British waters.

In early September, Bertie Armstrong, chief executive of the Scottish Fisherman’s Federation, told a special House of Lords EU select committee that Brexit offered the chance for the UK to become “a world leader in sustainable seafood”.

“Fishing was considered expendable and British waters were given to the EC as part of the accession negotiations. It was a deliberate act but an act of folly. Now we have the opportunity to right it,” Armstrong said.

Trawlerman Jimmy Buchan agrees. Brexit “is a great opportunity we have long waited for”, says Buchan, who mainly harvests langoustines, a popular delicacy on the continent.

“It now gets down to the negotiation,” he says. “It’s a game of chess but we are hopeful of coming out on top at the end of it.”

History, however, is not on the fishermen’s side. In the early 1970s, then Conservative prime minister Edward Heath accepted full access to British waters by continental fishermen as the price for entering the Common Market.

Today, fisheries comprise barely 0.5 per cent of GDP in Scotland, where about half of the total UK fleet is based. “There is concern that we will be sold down the river again by government,” says fisherman Peter Bruce. He would like to see article 50 – the mechanism to trigger Britain’s formal exit talks with the EU – invoked quickly but that is unlikely.

Scotland voted overwhelmingly to stay in the EU. Within hours of the announcement of the result, Scottish first minister Nicola Sturgeon declared a second referendum on independence “highly likely”, to protect Scotland’s place in Europe. The SNP’s commitment to Europe could place it on a collision course with the Scottish fishermen.

Some 93 per cent of Scottish fishermen voted for Brexit, according to research by Craig McAngus, a lecturer in politics at Aberdeen University. But the North Sea trawlermen have limited political clout.

“Fishing is a tiny proportion of the economy. There aren’t many [fishermen], they are a group that you would rather annoy than perhaps other groups when it comes to Brexit negotiations,” says McAngus.

If Scotland does ultimately leave the UK, post-Brexit, control over fishing policy would switch to the Edinburgh parliament. “How would that work with English boats coming unto Scottish waters? We would have to wait and see,” says McAngus.

Customs tariffs

There is concern about the prospect of customs tariffs, particularly among fish processors. Brexit will also mean the end of access to EU cash. Later this year, a £49 million redevelopment of Peterhead port will begin, part funded by the EU.

Ian Laidlaw, Peterhead port authority chief executive, is wary of the impact of leaving the EU and a second independence vote. “Right now we have a double doubt over our future. Regardless of your political colours uncertainty is never good for business,” he says.