The Turnbull government’s plan to drug test welfare recipients will “stigmatise” the poorest members of society, according to the United Nations special rapporteur on extreme poverty and human rights.

Philip Alston has written to the Australian government ahead of the Senate considering the welfare reform bill on Wednesday, questioning why “well-to-do [people] who spend far more on drugs” are not also subjected to the measure.

The fate of the bill is unclear because, although it is opposed by Labor and the Greens, the Nick Xenophon Team has criticised the two-year drug-testing trial but not reached a final position.

“The Australian government is conflating social protection and drug enforcement policies in a way that is counterproductive, unless the main goal is to stigmatise social security recipients,” Alston said in a statement.

“If the real goal is to reduce the use of illegal drugs, why start with the poorest members of society?”

“Will there also be a policy designed to drug test and crack down on the well-to-do who spend far more on drugs, and receive all sorts of tax deductions, social security payments and other government benefits?”

The UN special rapporteur argued the measure was “simply bad maths” as a cost-cutting measure because drug tests will cost between $500 and $900 per individual.

“By implying that all welfare recipients are potential criminals, the government is throwing away significant sums of taxpayer money on testing and controlling the very few who actually have a drug problem,” Alston said.

The social services minister, Christian Porter, responded that Alston’s criticism “completely misses the key focus ... which is to identify those who have drug issues which impede their ability to secure employment and move off welfare”.

“Drug use is a clear barrier to employment and this measure is designed to help overcome this barrier because it focuses on assisting them with appropriate treatment,” he said.

Porter said jobseekers were to be drug tested because it is a “condition of their [welfare] payment” to help them move into employment, unlike the comparison to “financial assistance provided to other taxpayers”.

The welfare reform bill contains the government plan to institute a two-year trial of drug-testing 5,000 welfare recipients as well a new demerit point system for docking welfare payments for failure to meet mutual obligations, unveiled in the May budget.

Drug testing will occur in three trial sites in south-western Sydney, Logan in Queensland and Mandurah in Western Australia.

The measure has been criticised by leading mental health expert and Australian of the Year, Patrick McGorry, and doctors from the Royal Australasian College of Physicians and the Australian Medical Association.

The bill also removes drug and alcohol dependency as an excuse for failing to meet mutual obligations, such as work for the dole or attending interviews with employment service providers.

Alston said the bill adds new conditions for receiving welfare that “undermine social protection, impose onerous conditions on claimants, and cast [welfare recipients] as often criminal, lazy or fraudulent”.

In September a government-dominated Senate committee recommended the bill be passed, claiming it will help welfare recipients deal with drug and alcohol addiction by tying their income support to their participation in drug counselling or rehabilitation.

A first failed test will see a welfare recipient placed on income management, via a basics card, and a second will see them referred for treatment. If they fail to engage in treatment, they will face sanctions including a possible loss of income support.

Alston’s letter argues the measure is “coercive” and “involuntary treatment for alcohol and other drug addictions is not effective”.

It warned that denying benefits to drug-dependent people could result in increases in poverty, homelessness and crime, and also lead to higher health and social costs.

Labor’s shadow human services minister, Linda Burney, and Greens welfare spokeswoman, Rachel Siewert, both seized on the report and called on the Senate to reject the bill.

Burney said it was clear from domestic and international experts the drug-testing measure would have “devastating impacts on vulnerable Australians with a drug dependency”.

Alston’s letter also notes concern about cuts to single-parent payments since 2005, including the cut made by the Gillard Labor government in 2012 and the welfare savings in the Turnbull government’s childcare bill which passed in March.

Alson noted the only justifications offered for the measures were “budget improvements” and “cost saving”.

“The aim of budget savings in itself cannot justify retrogressive measures on the rights to social security and to an adequate standard of living,” he warned.