Those of us who don’t confront the potential wide diffusion of on-officer body-worn cameras with excitement and hopefulness have already grown accustomed to some pat responses from advocates. Certain to be among them is the citation of a study from Rialto, California, that has made national news.

As the New York Times reported:

The Rialto study began in February 2012 and will run until this July [2013]. The results from the first 12 months are striking. Even with only half of the 54 uniformed patrol officers wearing cameras at any given time, the department over all had an 88 percent decline in the number of complaints filed against officers, compared with the 12 months before the study, to 3 from 24. Rialto’s police officers also used force nearly 60 percent less often — in 25 instances, compared with 61. When force was used, it was twice as likely to have been applied by the officers who weren’t wearing cameras during that shift, the study found.

After completion of the study, Britain’s The Guardian relayed the same statistics and reported “Rialto’s randomised controlled study has seized attention because it offers scientific – and encouraging – findings.” Civilian police monitoring groups, like Police The Police, touted the new technology with an internet meme that circulated widely on social media.

New York district court judge cited the Rialto study when she found unconstitutional the New York Police Department policy of “stop-and-frisk.” It seemed anyone concerned with policing and the use of force by officers was paying attention to Rialto. “I think we’ve opened some eyes in the law enforcement world. We’ve shown the potential,” study author, Tony Farrar, told The Guardian. “It’s catching on.”

Farrar says the study was conducted to measure the deterrence effect on use-of-force incidence. He explains,

The cameras were hypothesized to increase police officers’ self-consciousness and, therefore, increase their compliance to rules of conduct, especially concerning use of force. The findings suggest a more than 50 percent reduction in the total number of incidents of use of force compared to control-conditions, and nearly 10 times less citizens’ complaints than in the 12 months prior to the experiment.

The study claims officers randomly selected by shift to wear cameras ended up using force half as frequently as those without the cameras.

The report concludes with an important limitation most news reports and advocates for on-officer cameras ignore. Farrar encouraged caution for those basing policy recommendations on the study, explaining, “leaders cannot rule out the possibility that the cameras have (also) modified the behavior of those who interacted with the police. Members of the public with whom the officers communicated were also aware of being videotaped and, therefore, were likely to be cognizant that they ought to act cooperatively.” Because Farrar’s main research claim was that knowledge of being monitored changes behavior, he presumed this would apply symmetrically to officers and the civilians they stopped and subsequently notified they were being recorded. As such, this complicates simple, causative claims circulated in media reports.

Most news reports are clear in noting the Rialto study author, Tony Farrar, is also the Chief of Police for the Rialto police departent. However, few explain this as a potential conflict of interest or a methodological problem. Farrar’s study was conducted as part of his criminology Masters research at Cambridge University. He was, importantly, simultaneously was the head of the agency and managed both complaint review processes and those officers involved in the study.

Since Farrar’s hypothesis was that behaviors change when people know they are being carefully monitored, not only the cameras, but also the organizational authority of the individual designing and implementing the experimental study were variables.

Farrar’s research report and media articles relaying findings to public audiences fail to acknowledge Farrar’s role beyond that of an observer. He presumed a “view from nowhere” that necessarily troubles any research project. This is especially troubling, however, where the researcher is also a superior in a command-style bureaucracy. His position was hardly that of a neutral observer. Why, out of dozens of media reports, this is not raised as a problem is worth questioning. (I asked the NYT reporter if he holds investments in Taser, manufacturer of the AXON Flex wearable camera system. Obviously offended by the question, he nonetheless replied he does not.)

Farrar was doubly committed to his hypothesis proving true, both as a researcher and as a publicly accountable police administrator. But in the Rialto case, this is especially punctuated. The city of Rialto was ready to disband the police department and turn law enforcement functions over to the county sheriff. Farrar was hired as a reformer in a last ditch effort to save the department and the jobs of all the officers and staff who worked there. A department troubled by a history of questionable use of force—so questionable that it posed an existential threat to it—had to demonstrate reduced use of force and other complaints or face elimination.

Farrar had a chance to demonstrate his capability as a police administrator—not only to local policy-makers, but to his criminology research committee examining him for a graduate degree.

Research on the efficacy of policy aimed at reducing use of force is often equivocal or tentative. Researchers do mostly agree that if policy is effective, it is because of the buy-in of managers, administrators, and officers. So we should expect that something would change when the head of a department is deeply committed to a policy designed to reforming use of force among its officers. However, the cameras are getting all the credit in this study.