Up to half a million refugees are gathering in Libya to attempt the crossing to Europe on the deadly boats that have killed thousands already.

The toll of misery was revealed by senior Royal Navy officers leading Britain’s Mediterranean rescue mission off the Libyan coast. Britain’s amphibious assault ship HMS Bulwark has helped save around 4,000 refugees before they drowned having set sail in unseaworthy boats.

The ship’s 350-strong company of sailors and Royal Marines is bracing itself to rescue a further 3,000. Captain Nick Cooke-Priest said: “Indications are that there are 450,000 to 500,000 migrants in Libya who are waiting at the border.”

As he spoke, intelligence reports were warning Bulwark’s crew, which is among about 11 rescue vessels in the central Mediterranean, of thousands more migrants in the water. Britain is working with the Italian navy on Operation Weald to rescue migrants who have fallen prey to people smuggling gangs in Libya.

Migrants sit on the ground after being detained by Libyan coast guard near Garabulli, Libya. Photograph: Xinhua /Landov/Barcroft Media

The passengers on some of the overcrowded boats are thought to have been told to scupper them as soon as they see a rescue ship, to ensure they will be saved, even though many of them cannot swim. Commodore Martin Connell, the Royal Navy’s senior officer commanding Operation Weald on board Bulwark said: “This is just the most grim situation you could possibly find. It is a grim business … most of these boats won’t make it. If I could get my hands on some of these smuggling gangs. They have no regard for human life.”

The defence secretary, Michael Fallon, visited Bulwark, flown in on one of the ship’s two Mark II Merlin helicopters, which are used to scour the seas for stranded boats. He urged the rest of Europe to help as Italy and Britain are taking on the brunt of the rescue workload.

Asked about the half-a-million already preparing to make the hazardous journey by boat from Libya he told reporters: “Well you have seen today 3,000 people trying to cross in a single day, we could see hundreds of thousands trying to cross this summer. We all have an interest in tackling this much further back. The issue here is poverty and conflict in west Africa and poverty and conflict in east Africa.”

He said more work was needed to smash the ruthless people smuggling gangs who organise the trips across treacherous waters.

Fallon added: “We can pool intelligence, get after the gangs themselves, we can try to cut off their financing. People are making money out of misery and we can do more there to track down the money.

“The Royal Navy stepped up to the plate straight away as David Cameron promised at the emergency summit so we are doing our bit with Bulwark and the helicopters, obviously we want other European countries to do more as well.”

So far this year 76,000 migrants have made the 260-mile crossing into Europe from Libya, with 40,000 remaining in Italy.

The passengers on some of the overcrowded boats are thought to have been told to scupper them as soon as they see a rescue ship, to ensure they will be saved, even though many of them cannot swim. Photograph: Xinhua /Landov/Barcroft Media

On Saturday, Bulwark turned dramatically into the wind as once more it launched a Merlin helicopter to investigate reports of thousands of migrants who needed to be rescued.

Meanwhile, Cameron is set to clash with Angela Merkel at the G7 summit over her plans for a pan-EU distribution of the migrants coming across the Mediterranean from north Africa, with the British prime minister insistent that such measures will only encourage the traffickers.

The German chancellor has said that finding a way forward on the migration crisis will be a priority during the two-day talks starting in Bavaria on Sunday. She has previously said there should be a new EU system that distributes asylum seekers to member states based on their population and economic strength. Merkel is expected to make further such calls in the days to come.

Downing Street, however, insists that it will not go along with the plans. Government officials claim they would deal only with the symptoms and not the cause of the humanitarian disaster. One government official said: “The more the traffickers see that people are being resettled, the greater the incentive there is for them.”

As part of his freshly announced agenda of tackling corruption, officials said Cameron would instead argue that attempts to dismantle the human trafficking networks should remain the focus, although the idea of an EU military force destroying boats in the Mediterranean has been rejected by the Libyan authorities.