Australia’s vast new facial-matching system would be prone to errors against people of colour, experts have warned.

Federal and state governments agreed last year to develop a system known as “the capability”, a powerful database that pools biometric information gleaned from driver’s licences, passports, visas and other sources.

The system will be used to identify unknown persons in near real-time and is aimed at helping “to identify suspects or victims of terrorist or other criminal activity, and help to protect Australians from identity crime”. The system has raised significant concern among academics, lawyers, privacy experts and human rights groups, who fear it will encroach on privacy rights, be used for mass general surveillance, and have a “profound chilling effect” on public protest and dissent.

Experts have also warned that facial matching is prone to error, particularly against people of colour. Australia’s system is partly based on a model employed by the US Federal Bureau of Investigation (FBI). An investigation of the FBI’s system by the US full house committee on oversight and government last year found it was most prone to error against African Americans.

Facial image matching system risks 'chilling effect' on freedoms, rights groups say Read more

“[Facial recognition technology] has accuracy deficiencies, misidentifying female and African American individuals at a higher rate,” the committee found. “Human verification is often insufficient as a backup and can allow for racial bias.”

Monique Mann, a director of the Australian Privacy Foundation and a lecturer at the faculty of law at the Queensland University of Technology, said studies strongly suggested such systems resulted in racial bias.

A separate FBI co-authored study found facial recognition was least accurate for African Americans, and a study by Georgetown University, titled the Perpetual Line-up, found that “face recognition may be least accurate for those it is most likely to affect: African Americans”.

“Their conclusion is that it’s less accurate in the identification of African-Americans compared to Caucasian individuals, despite police saying their systems are super objective, they don’t see race,” Mann told Guardian Australia.

“It gives us this false veil of objectivity of being racially neutral, when in actual fact it’s not.”

Prof Liz Campbell of Monash University, an expert on forensic and biometric databases, said the algorithms underpinning facial recognition systems often reflected the biases of the societies in which they were developed. In Britain and Australia, she said, this meant facial recognition was good at identifying white men.

“What seems to be the case is the algorithms are good at identifying the people who look like the creators of the algorithm,” she said. “What that means in the British and Australian context is it’s good at identifying white men.”

“It’s not that there’s an inherent bias in the algorithm, it’s just the people on who the algorithms are being tested is too narrow.”

Accuracy has also been a concern more generally. A pilot study in Wales found facial matching erroneously identified innocent people in 91% of cases.

The system requires legislation to be passed at a federal and state level. The federal bill is yet to be passed but some states, including NSW, are pressing ahead with their enabling legislation regardless. NSW Labor has serious concerns about the privacy implications, which they say are not being heeded by government.

Dutton's facial recognition scheme could target jaywalkers, lawyer warns Read more

“Labor is concerned about the privacy implications of ‘the capability’,” the shadow attorney general, Paul Lynch, said. “When this was debated in the assembly, I said that Labor was cautious about the proposal and would look carefully at what was done when it was implemented. As I said in the debate, I think this government has been quite disinterested in privacy issues.”

NSW Greens MLC David Shoebridge said error rates were likely to be higher for people from non-English speaking backgrounds. He said there was no clear process of appeal for people wrongly identified.

“You’re not even told that the facial verification service has been checked,” Shoebridge said. “There’s potentially extraordinarily large impacts upon you if this system gets the facial recognition wrong.

“What are we doing handing over our data to a scheme that hasn’t even be finalised?”