By of the

The FBI's crime reporting program is considered the final word on crime trends in the United States, but the agency rarely audits police agencies providing the information and when it does its reviews are too cursory to identify deep flaws.

In each of the past five years, FBI auditors have reviewed crime statistics at less than 1% of the roughly 17,000 departments that report data, a Journal Sentinel examination of FBI records has found. In all, they've audited as many as 652 police agencies during that time, or less than 4% of the total.

And a Journal Sentinel survey of police departments in the 30 largest U.S. cities found that nearly two-thirds have not been audited in the past five years.

Of those, six departments - including Oklahoma City, Philadelphia and Seattle - have never been reviewed by the FBI since the auditing program began 15 years ago.

That lack of scrutiny allows cases of undercounting of crimes, such as in Milwaukee where thousands of violent assaults were not included in the crime rate since 2006, to go unnoticed and gives the public a false sense of the true level of crime, criminal justice experts said.

"It would be more candid to not do any (audits)," said Eli Silverman, professor emeritus at John Jay College of Criminal Justice. "This way, at least you're not offering any pretense of checking on the validity of the stats. If you are going to do that little, then why do it? You either do it systematically, or you don't do it at all."

He called the audit process a "useful fig leaf."

The FBI this year did its first-ever audit of the Milwaukee Police Department, even though it's the largest law enforcement agency in the state, generating about one-quarter of FBI index crimes in Wisconsin. That just-released audit, conducted at the request of Police Chief Edward Flynn, examined 60 incidents - a number experts say is too small to draw conclusions from.

"If they only pull a (small sample), the likelihood that they will find assaults that were downgraded is very low," said James Alan Fox, criminology professor at Northeastern University, of the FBI review. "Their ability to identify systematic misclassification is limited by the total volume of cases they check."

Indeed, the audit failed to detect the scope of problems already identified.

A Journal Sentinel investigation in May found more than 500 serious assaults - including stabbings and beatings - over a recent three-year period had been misreported as minor crimes. Another 800 assaults followed the same pattern. Those findings came from a partial review that compared Milwaukee police crime data with nearly 60,000 cases referred to prosecutors from 2009 to early 2012.

After the Journal Sentinel story, the Police Department launched its own targeted review of more than 34,000 cases, and in June released an initial audit report that showed police underreported more than 5,300 aggravated assaults since 2006.

FBI crime data is often cited by police chiefs and elected officials to give residents a measure of safety in their communities. In Milwaukee, Flynn and Mayor Tom Barrett have touted four straight years of declining crime.

But the information receives little outside scrutiny and is susceptible to manipulation by local police departments, Silverman said of the overall system.

"(Crime data) is a tool that politicians and police leaders use, yet the system is so incentivized to cast a favorable light and there is very little checks and balances to make sure it's accurate," he said.

Faulty crime data has far greater implications than just numbers on a spreadsheet, said John Eterno, director of the graduate criminal justice program at Molloy College in Long Island, N.Y. For example, police departments use the statistics to develop crime-fighting strategies and make hiring decisions.

"If you're saying your crime numbers are really low, but meanwhile they are really high and you aren't hiring cops, you are going to exacerbate the problem of criminality," Eterno said. "You need to have the correct numbers to do the right thing."

Eterno and Silverman have studied crime reporting practices at the New York City Police Department and other law enforcement agencies. Their research has identified widespread problems with performance management systems, most notably CompStat, which is a tool used by Milwaukee police and hundreds of other police departments across the country.

CompStat is used to hold police commanders accountable for crime trends in their districts - they must answer to the police chief and high-ranking officials at monthly meetings about performance metrics such as arrests, traffic stops and crime figures.

Eterno said performance management systems like this can unintentionally provide motivation for police supervisors to downgrade crime.

Flynn and other officials have stressed that reporting problems in Milwaukee are due to human and computer error and not the result of any manipulation of data.

In 1929 the FBI created the voluntary Uniform Crime Reporting Program to make sure reporting is consistent across the country.

It wasn't until nearly seven decades later, in 1997, that the FBI began conducting audits of local police agencies to be sure the numbers were accurate. Now, the FBI conducts an audit of police agencies in each state once every three years, usually between six and nine police departments.

FBI spokesman Stephen Fischer said the federal agency's audit is a voluntary review that carries no penalties and is conducted at the discretion of a state's crime reporting program. In Wisconsin, that is the Office of Justice Assistance. Fischer declined to provide numbers on staffing levels in the FBI crime audit unit or the average workload of auditors.

Meanwhile, there is scant oversight at the state level. Because of lack of staffing, the state Justice Assistance office conducts no audits of crime data, the agency spokeswoman said.

More than 400 law enforcement agencies in Wisconsin report data to the FBI. Before the FBI conducted a review of Wisconsin agencies in May, federal auditors had visited the state in 2009, 2006 and 2000.

For the FBI, the reality is that its audit team is short-staffed, which means it can only review a limited number of agencies and criminal incidents each year, said Dan Bibel, director of the crime reporting unit for the Massachusetts State Police.

"I don't want to say they are doing a bad job," said Bibel, former president of the Association of State Uniform Crime Reporting Programs. "They are doing the best they can with what they have."

Silverman noted the FBI acknowledges that crime data is self-reported by police departments, but said it can be misleading to the public because the federal agency collects and releases the data as part of its annual "Crime in the United States" report.

"Most people assume the data comes with a certain grain of authenticity because they are FBI statistics," he said. "But in reality they are not really FBI stats, but those of the actual police department."

Self-policing the data

Without more robust oversight by FBI auditors, the crime numbers are only as good as a law enforcement agency's efforts. Therefore, experts said, police departments must scour their own data to ensure accuracy.

When the original Journal Sentinel investigation in May identified hundreds of violent assaults that were misreported, Police Chief Edward Flynn said that - along with a dysfunctional computer system - sergeants and lieutenants failed to catch mistakes on FBI crime classifications when reviewing reports.

Several weeks later, Deputy Inspector William Jessup told a reporter that records clerks were not properly trained on FBI protocols and were routinely changing weapon codes to allow violent attacks to be incorrectly reported as minor assaults.

Flynn and other police officials have also said the department's roughly 1,300 police officers were never trained on FBI crime coding because they are focused on solving crimes and making arrests. Instead, officers were taught to accept the default FBI crime codes that pop up when they fill out incident reports on a computer.

In late June, Flynn ordered nearly 70 members from more than 20 police sections to attend two full days of introductory training on compliance with crime reporting procedures. The training is being held this week at State Fair Park in West Allis.

The Milwaukee Fire and Police Commission plans to hire an outside consultant at a cost of roughly $25,000 to audit the Police Department's computer system and its business practices. Meanwhile, Milwaukee police are also conducting a comprehensive internal review of crime data from other crime categories and said they will release the results in the fall.

In addition to conducting internal audits, another way police departments can achieve accurate crime reporting is through "integrity testing," Eterno said. An example of that: An undercover police officer pretends to be the victim of an aggravated assault and then auditors see if the incident was properly coded for FBI purposes.

"I think any criminologist would like to see a better system of auditing crime," Eterno said. "It's a voluntary system, so you are just counting on these local departments to do something. The local departments need to do far better audits than what most are doing."

Past reporting on crime data flaws

In May, a Journal Sentinel investigation found that more than 500 cases of aggravated assaults were misreported by Milwaukee police from 2009 to early 2012 as lesser crimes not counted in the city's violent crime rate. Another 800 cases followed the same pattern but couldn't be checked with available records.

In June, Milwaukee police told the newspaper that record clerks routinely changed FBI weapon codes in a way that allowed serious assaults to be underreported as minor crimes. This happened when the computer system flagged errors that made clear the incident was more serious.

A Police Department internal audit, prompted by the investigation, found that more than 5,300 aggravated assaults were misreported a lesser offenses since 2006. The 20% error rate is 10 times the national standard.

John Diedrich and Emily Eggleston of the Journal Sentinel staff contributed to this report.