Police in Canada are building a predictive policing system that will analyze social media posts, police records, and social services information to predict who might go missing, says a government report.

According to Defence Research and Development Canada (DRDC), an agency of the Department of National Defence, Saskatchewan is developing “predictive models” to allow police and other public safety authorities to identify common “risk factors” before individuals go missing, and to intervene before something happens. Risk factors can include a history of running away or violence in the home, among dozens of others.

A DRDC report published last month shows the Saskatchewan Police Predictive Analytics Lab (SPPAL)—a partnership between police, the provincial Ministry of Justice, and the University of Saskatchewan—is analyzing historical missing persons data with a focus on children in provincial care, habitual runaways, and missing Indigenous persons, and building tools to predict who might go missing. In the next phase of the project, SPPAL will add social service and social media data.

The report doesn’t specify what kind of predictive insights authorities expect to glean about individuals from social media posts, but police already use social media to monitor people and events for signs of crime, or in the case of missing persons investigations, to discern when a person went missing. For example, police in Ontario made a missing woman’s case a priority after noticing that her usual patterns of social media activity had ceased.

The DRDC report states that municipal police services in Saskatchewan as well as the Ministry of Social Services Child and Family programs and regional RCMP have agreed in principle to share information with SPPAL. In Saskatchewan, more than 70 percent of children in provincial care are Indigenous, and over 100 long-term missing persons cases haven’t been solved.

Tamir Israel, a lawyer with the Canadian Internet Policy and Public Interest Clinic (CIPPIC), told Motherboard that using predictive models to inform decisions on child welfare interventions is concerning.

“We know that predictive models are far from infallible in real-life settings, and that there will be false positives,” Israel said in an email. “The consequences of an intervention based on a false positive can be very serious.”

Israel said that the risk of false positives increases when predictive models use data of “questionable fidelity” such as social media posts. He pointed out that the high number of missing Indigenous women and children in Canada makes them and other marginalized groups especially vulnerable to flaws or biases concealed in predictive models.

“We have already seen cases where predictive models had deep racial biases,” Israel said. He explained that while a model may be predictively valid across all communities, it could ignore cultural differences that lead to “distorted outcomes” when the same model is applied specifically to minority groups.

Ronald Kruzeniski, Saskatchewan’s Information and Privacy Commissioner, told Motherboard in an email that his office advises police “not to collect or use social media data” because of concerns about accuracy.

Kruzeniski cited the difficulty in knowing the true identity of a person behind a social media account and the relevance of old posts as reasons why social media data shouldn’t be used.

Motherboard reached out to Saskatchewan’s Advocate for Children and Youth for comment on SPPAL but did not receive a response.

Dr. Keira Stockdale, a psychologist with the Saskatoon Police Service (SPS) who authored the DRDC report,said the work done by SPPAL is done in accordance with legal requirements and “governed by [the] highest ethical and professional standards to proactively protect privacy and promote public safety.”

Stockdale believes the tools SPPAL is developing can be applied to a number of community safety problems, such as developing coordinated responses to the “illicit use” of opioids.

SPPAL is an outgrowth of a intervention-based approach to policing called the Hub model that partners cops with social workers and schools to identify and intervene with people believed to be at risk of becoming criminals or victims. In Saskatchewan and Ontario, information about people assessed for intervention by Hubs is entered into a Risk-driven Tracking Database (RTD) to store and analyze the data.

In February, a Motherboard investigation found that minors aged 12 to 17 were the most prevalent age group found in the Ontario RTD in 2017 and that children as young as six have been subject to Hub interventions. The Hub model of policing was developed in Saskatchewan before being exported to Ontario and across Canada. Stockdale said that the tools developed by SPPAL could be used by Hubs when assessing people for intervention.

The DRDC report notes that the work of SPPAL is intended to help build a data analytics solution to “support public safety partners and social services agencies across Canada.”

Israel noted that legislation recently introduced in Ontario could pave the way for predictive policing to flourish in that province.

Israel said the Comprehensive Ontario Police Services Act, which received royal assent last month, empowers the Minister of Community Safety and Correctional Services (MCSCS) to “collect high volumes of information from various policing agencies throughout Ontario” with little oversight from the province’s Information and Privacy Commissioner.

While the law does not require predictive policing, it opens the door to “broad-based adoption” of the practice and puts incentives in place without including necessary safeguards for personal information, Israel said.

Israel said the DRDC report also suggests that police could use SPPAL’s predictive models to triage missing persons cases and prioritize certain cases over others—for example, a possible kidnapping victim versus a habitual runaway.

“Law enforcement will be called on to rely on the outcome of the predictive model” when deciding how seriously to take a missing person case, Israel said. “But these predictive models are often opaque in their operation, relying on factors that the police officers themselves cannot assess or second-guess.”