A Conservative peer has held a private meeting at parliament with the president of Uganda, who has faced condemnation for rights abuses and homophobia, prompting condemnation from Labour.

Dolar Popat, a businessman and Tory life peer, who is the government’s trade envoy for Uganda and Rwanda, was scheduled to hold a lunch on Friday at the House of Lords with Yoweri Museveni, who has led Uganda since 1988, officials in the upper house said.

Museveni, who is in the UK for the Illegal Wildlife Trade conference, was also due to have a private meeting with Timothy Clement-Jones, a Liberal Democrat peer, the officials said.

However, it was the event with Lord Popat that brought criticism from Labour, who said it seemed anomalous to let Museveni visit parliament on a day when neither the Lords or Commons were sitting, so he could not be challenged on his rights records by any ministers.

Theresa May and the foreign secretary, Jeremy Hunt, attended the wildlife conference but neither were scheduled to hold separate talks with Museveni.

The Ugandan president also met Prince William for talks at Buckingham Palace on Thursday.

Popat, who was born in Uganda and arrived in the UK as a teenager, is among 33 MPs and peers appointed by the Department for International Trade as envoys for certain countries or areas.

Uganda’s economy is forecast to grow in the coming years with the extraction of huge oil reserves discovered in the country in 2006.

Museveni is serving his fifth term as president, having changed the country’s laws to get past presidential term limits and age limits. While his supporters argue he has sought to bring stability to Uganda, he is accused of overseeing significant right abuses, particularly against critics.

The most recent Amnesty International report for Uganda said the rights to freedom of expression, association and assembly had been restricted, adding: “Journalists and others who criticised the president or his family were arrested, detained and harassed.”

In 2014, Museveni signed into law a bill toughening criminal penalties against homosexuality, with police continuing to harass LGBT people.

Labour’s shadow foreign office minister in the Lords, Ray Collins, said Popat’s meeting was unacceptable.

“The worsening situation in Uganda has seen local opposition leaders arrested, along with renewed threats to toughen laws that already discriminate against gay men and lesbians,” Lord Collins said.

“At the recent Commonwealth heads of government meeting, Theresa May said the United Kingdom had a ‘special responsibility’ to help change hearts and minds on anti-LGBT legislation. It beggars belief therefore, that the government is allowing President Museveni to visit our parliament without any minister set to take him to task on his human rights record.”

A spokesman for the Department for International Trade said Popat’s meeting with Museveni was “in his role as trade envoy to the country”.

The spokesman said: “It is not good enough to merely criticise other countries from the sidelines. Only by working with Uganda are we able to bring about the changes we would like to see in the country.

“As a government, we do not shy away from raising legitimate human rights concerns and encouraging Uganda to respect international law – we make this point very clearly in public and in private.”

• This article was amended on 15 October 2019. A previous version referred to the 2014 bill “toughening criminal penalties against homophobia”. This has been corrected.