Over the last year the decisions of the British public and, subsequently, the UK government have been observed with growing dismay by the people of Ostend. A slowdown in the number of Britons seeking a cheap bucket-and-spade holiday on the wide, soft-sanded beaches of the blowy North Sea resort ushered into administration the operator of the once popular Ramsgate to Ostend ferry four years ago, closing down the only direct link from the UK to the Flemish port. The Brexit vote last June, and the subsequent crash in the value of the pound, made it that bit more expensive for Brits who still wanted a break in Belgium.

Staff at British Brands, a seafront souvenir shop that once did a brisk trade in flogging British-made goods to homesick tourists, admitted business had been slow recently. Ostend is increasingly becoming a resort for retired Belgians, drinking glasses of beer in the mid-morning sun. Now it is the suggestion that the British want to “take back” their fish that has caused Ostend’s fishermen, and all those who depend on the EU’s smallest fishing fleet, fresh concern.

Last month the Conservative manifesto pledged to withdraw the UK from the common fisheries policy (CFP), fulfilling the Brexiters’ vow to take back control of British waters. The fisheries minister, George Eustice, has suggested that the British fleet will have access to hundreds of thousands more fish in the Brexit paradise to come. While the election result puts this, and other policies, into disarray, the signs are not good for the Belgian fishing industry. Three-quarters of the country’s fleet hunts in the 200 nautical miles from the UK coast over which Britain is to take sovereign control, be it in the North sea, the Channel or the Irish sea. Those waters account for half of the Belgian catch.

As it stands, under the CFP Europe’s seas are effectively common to the 28 EU member states. The 100 fish stocks that swim in European waters move around, after all, and know no national boundaries. The total number of each species of fish that can be taken from various zones in the EU’s seas each year is settled by the member states on the advice of scientists, to ensure sustainability. Each country has a species-by-species quota they can take from that European haul. The quotas have been fixed since 1983 on the basis of the recorded catches of the various national fleets between 1973 and 1978. According to recent estimates, 33% of the catches of the European fishing fleet are caught in what could soon be claimed as British waters. Belgium’s 402 fishermen and their 65 boats, whose main catch is dover sole, are more dependent on those waters than most.

It was not, however, because of fishing rights that Ostend once viewed the UK with such warmth that some would proudly advertise their home as “the most British town in Europe”. A grainy photograph of Field Marshal Bernard “Monty” Montgomery, who liberated the badly bombed port from the Nazis in 1945, still has pride of place on the seafront between the stalls selling escargots and whelks. But a few are now questioning their faith in perfidious Albion.

Sitting in the quayside offices of the Rederscentrale, the organisation that represents Belgium’s fishermen, Emiel Brouckaert, its managing director, and Urbain Wintein, a 60-year-old fisherman who seeks catches in Britain’s seas, admitted their people were nervous. “Everyone is thinking about Brexit and the consequences,” Brouckaert said. “Just under 50% of our catches are in our waters, and we hear what our [British] colleagues’ requirements might be, though we realise there are some extreme views there.”

The Belgian fleet is currently enjoying a healthy living, thanks to cheap fuel and higher prices being charged for their produce in the shops. But the last three decades have seen an unnerving decline. In 1990 there were 200 Belgian vessels. Economic realities hit, and there was a stream of mergers, as fishermen sought to squeeze the most out of the industry. “Then the last serious decline was from 100 vessels to the level we are now, and that was since the crisis in 2008,” said Brouckaert. Around 40% of the fleet today is owned by Belgian companies whose major shareholders are actually Dutch. Brouckaert is not yet linking the two challenges, but once individual Belgian fishermen start to struggle in a tighter post-Brexit reality, the bigger foreign companies could again swoop – bringing an end to the Belgian fishing industry in all but name.

Facebook Twitter Pinterest ‘Nervous’: Emiel Brouckaert in Ostend. Photograph: Daniel Boffey/The Observer

“We realise that, in a small fleet in a small sector, besides Brexit we have another important challenge for our sector’s survival, and that is succession planning,” he said. “There is a huge interest for any fishing opportunities we might have, not only from Holland, but from all countries, and successful companies that are restricted in their growth. They are constantly saying, ‘Where can we find more growth?’ If we don’t have Belgian succession for fishing possibilities, there’ll be lots of European colleagues on the quayside to take it.”

The Belgians have joined eight other fishing communities in Europe with an interest in British waters to make their case in the Brexit talks. They have compiled a manifesto of sorts, which they have presented to the EU’s chief negotiator, Michel Barnier, a former French fisheries minister, who was said to be highly sympathetic. But fishing disputes have a tendency to get messy quickly.

During the cod wars in the 1970s, Icelandic gunboats harassed British boats out of their waters in a row over access to their seas, and its coastguard took knives to their fishing lines. More recently, a row over scallops in the Channel saw French fishermen pelt British boats with flares and rocks. There has been talk of blockading English produce from continental Europe, should the British seek to restrict European fishermen’s access post-2019, when the UK leaves the EU. Olivier Leprêtre, president of the regional fisheries committee for the Hauts-de-France region, covering Nord Pas-de-Calais and Picardy, feared tempers would flare up again on the open seas. “It’s a very emotional issue,” he said.

The Belgians aren’t talking about an uprising just yet. “We’ve had up to now very good relations with our British colleagues,” Brouckaert said. “A decision has been made by the UK to leave the EU and it is making people worried, but despite all the political things happening, we still hope we can have business as usual. If that’s in a common fisheries policy or a policy jointly negotiated, that is the first thing we are thinking about. Similar access, similar distribution of the quota.”

One option, to fish in European waters elsewhere, was not possible, said Brouckaert. “They [the fishermen] are used to operating on fishing grounds they know inside out, from generation to generation.” And, he can only hope, generations to come.