This story was co-published with ProPublica.

Ariella Russcol specializes in drama at the Frank Sinatra School of the Arts in Queens, New York, and the senior’s performance on this April afternoon didn’t disappoint. While the library is normally the quietest room in the school, her ear-piercing screams sounded more like a horror movie than study hall. But they weren’t enough to set off a small microphone in the ceiling that was supposed to detect aggression.

A few days later, at the Staples Pathways Academy in Westport, Connecticut, junior Sami D’Anna inadvertently triggered the same device with a less spooky sound—a coughing fit from a lingering chest cold. As she hacked and rasped, a message popped up on its web interface: “StressedVoice detected.”

“There we go,” D’Anna said with amusement, looking at the screen. “There’s my coughs.”

The students were helping ProPublica test an aggression detector that’s used in hundreds of schools, health care facilities, banks, stores and prisons worldwide, including more than 100 in the US. Sound Intelligence, the Dutch company that makes the software for the device, plans to open an office this year in Chicago, where its chief executive will be based.

California-based Louroe Electronics, which has loaded the software on its microphones since 2015, advertises the devices in school safety magazines and at law enforcement conventions, and said it has between 100 and 1,000 customers for them. Louroe’s marketing materials say the detection software enables security officers to “engage antagonistic individuals immediately, resolving the conflict before it turns into physical violence.”

In the wake of the shooting at a Parkland, Florida, high school and other massacres, US schools are increasingly receptive to such pitches. Congress approved more than $25 million for school security improvements last year, and one analyst expects new technology could augment the $2.7 billion market for education security products. Besides Sound Intelligence, South Korea–based Hanwha Techwin, formerly part of Samsung, makes a similar “scream detection” product that’s been installed in American schools. UK-based Audio Analytic used to sell its aggression- and gunshot-detection software to customers in Europe and the US, including Cisco Systems’ professional security division. However, an Audio Analytic spokesman told ProPublica that it has since changed its business model and stopped selling the aggression detector software.

Adrienne Grunwald; Animation by Elena Lacey; Getty Images

By deploying surveillance technology in public spaces like hallways and cafeterias, device makers and school officials hope to anticipate and prevent everything from mass shootings to underage smoking. Sound Intelligence also markets add-on packages to recognize the sounds of gunshots, car alarms and broken glass, while Hauppauge, New York-based Soter Technologies develops sensors that determine if students are vaping in the school bathroom. The Lockport school district in upstate New York is planning a facial-recognition system to identify intruders on campus.

Yet ProPublica’s analysis, as well as the experiences of some US schools and hospitals that have used Sound Intelligence's aggression detector, suggest that it can be less than reliable. At the heart of the device is what the company calls a machine learning algorithm. Our research found that it tends to equate aggression with rough, strained noises in a relatively high pitch, like D’Anna’s coughing. A 1994 YouTube clip of abrasive-sounding comedian Gilbert Gottfried (“Is it hot in here or am I crazy?”) set off the detector, which analyzes sound but doesn’t take words or meaning into account. Although a Louroe spokesman said the detector doesn’t intrude on student privacy because it only captures sound patterns deemed aggressive, its microphones allow administrators to record, replay and store those snippets of conversation indefinitely.

“It’s not clear it’s solving the right problem. And it’s not clear it’s solving it with the right tools,” said Suresh Venkatasubramanian, a University of Utah computer science professor who studies how replacing humans with artificial intelligence affects decision-making in society.