Three Home Office ministers this week summoned the Labour peer campaigning for the rights of child refugees to a meeting to urge him to stop pushing for a key amendment to the EU withdrawal agreement bill.

But Alf Dubs has refused to shift his position and will continue to oppose a government attempt to drop a crucial right to family reunion for unaccompanied child refugees.

Lord Dubs, who has been fighting for improved protections for child refugees travelling alone since 2016, said it was disgraceful to use child refugees as a “bartering chip” with the EU, describing the move to drop the family union rights from the withdrawal legislation as a “betrayal of Britain’s humanitarian position”.

Dubs said he had been surprised to be called in to meet the Home Office ministers Brandon Lewis, Lady Williams and Victoria Atkins, as well as seven officials, for a 45-minute meeting on Monday afternoon.

The size of the group assembled to talk to him was likely to be a reflection of their concern that the government could face a defeat in the House of Lords next week, he said. “They know there is a wave of support for this. I have spoken to quite a few Tories, and there are enough who don’t understand why the government is doing this,” he said.

Ministers stated that the government’s position had not changed on offering child refugees travelling alone through Europe the right to be reunited with family members in the UK after Brexit, a right enshrined in EU law as the Dublin regulation.

Dubs had successfully led a campaign for those rights to continue after Brexit in an earlier version of the withdrawal bill, but the protections were removed in the latest version, published after the December election. The ministers told Dubs that despite the decision to drop the measures from the bill, they would be included in the long-delayed immigration bill later this year.

“They asked me to accept the government’s commitment that they were still on side with this, to believe them,” he said, but he remained sceptical. “If they are not departing from the policy, then why not keep it in the bill?”

In a letter to MPs sent last week, Lewis said ensuring that unaccompanied children seeking protection in an EU member state could continue to be reunited with family members “remains a negotiating objective of this government”.

Ministers told Dubs that including the family reunification right in the withdrawal bill “weakened their negotiating flexibility”, the peer said. “That suggests they are going to barter unaccompanied child refugees for something else, which is absolutely disgraceful.”

Removing the right to family reunification for refugee children would force more young people to attempt to travel to the UK illegally, exposing them to considerable dangers, he said.

Sir Alf Dubs came to the UK as a refugee himself during the second world war. Photograph: Christopher Thomond/The Guardian

Dubs, who himself came to the UK as a Kindertransport child refugee after fleeing Prague in 1939, said the welcome he received continued to motivate his campaign. “This country gave me fantastic opportunities. I would hardly be human if I didn’t have a feeling for young people trying to find a home in Britain as I did.”

Beth Gardiner-Smith, chief executive of Safe Passage, an organisation working with unaccompanied child refugees in Europe, said: “We have already been contacted by families who are desperately worried about the impact of the government decision to remove this law on their families and the prospect of being reunited with family members across Europe.”

There are believed to be more than 4,000 unaccompanied asylum-seeking children in Lesbos, Greece, the majority of them living outside formal camps, and a few hundred in northern France.

“Without access to a safe and legal route many of them will turn to smugglers, who will promise them passage that is not safe and they will risk their lives,” she said.

MPs voted 348 to 252 against an amendment on child refugees last week, which had previously been accepted by Theresa May’s government. “It is very disappointing that the first real act of the new Boris Johnson government is to kick these children in the teeth,” Dubs said afterwards.

The Lords is expected to vote on Dubs’s new amendment next Monday or Tuesday.

Lewis, minister of state for security, said: “I welcomed the opportunity to meet with Lord Dubs and reaffirm our commitment to seeking an agreement with the EU for the family reunion of unaccompanied asylum-seeking children. We do not require the withdrawal agreement bill to achieve this.

“We have a proud record of helping vulnerable children in this country, including granting protection to 41,000 unaccompanied minors since 2010, and this will remain our priority.”