Ed Gillespie, Republican candidate for governor of Virginia, has made the gang MS-13’s alleged role in rising violent crime in his state a major facet of his campaign, running ads attempting to connect the policy positions of Democratic candidate Ralph Northam, who had previously refused to pre-emptively ban sanctuary cities, to an increasing threat from the gang. (Northam has since changed his stance.) Gillespie has repeatedly asserted that there are over 2,000 MS-13 gang members in Fairfax County, Virginia, and that the group’s membership is growing. But there’s a major problem with this claim: Gang membership and levels of gang violence are virtually impossible to quantify with any certainty, in Virginia or anywhere else.

A Washington Post fact-check of Gillespie’s claim found that the 2,000 gang members he referred to may be a number pulled from a Fairfax County government presentation on gang violence, which reported 2,000 total gang members and associates linked to 80 gangs in the county. The Washington Post gave this claim “Two Pinocchios,” concluding that “the plain fact is that no one knows how many MS-13 gang members are in Fairfax County.”

Indeed, attempting to quantify gang membership and violence in America is fraught with problems. The main one, according to Meena Harris, the director of the National Gang Center, is conceptual. According to Harris, “There’s no universal definition of gang, and the debate still continues over what constitutes a gang and a gang member.”

Who gets counted as a gang member — and whether those counts include people who are not identified as members of gangs but who are associated with a gang — can vary from state to state, department to department, and even officer to officer. “What’s accurate and what’s not accurate really depends on the level of training for the police officer,” Harris said.

The way agencies report and store gang membership information in databases can cause additional problems. A 2016 audit of the California gang database CalGang, for example, found that “agencies have failed to ensure that CalGang records are added, removed and shared in a way that maintains the accuracy of the system and safeguards individuals’ rights.” In another case earlier this year, problems associated with gang designations led the police department in Portland, Oregon, to stop identifying people as gang members.

Methodological differences in how gangs are counted from state to state and agency to agency means that drawing conclusions about trends in either gang membership or gang crime is exceedingly difficult.

And it’s not a new problem. A 1975 study by the late Walter Miller, an anthropologist and expert on youth gangs, is still relevant given the considerable challenges cities continue to face in quantifying gang membership. Miller examined gangs in six big cities and found that estimates of the combined gang population in those cities varied from 28,450 to 81,500, depending on which organization did the counting.

Gang violence is also difficult to quantify. Starting in 1996, the National Youth Gang Survey, an annual survey of over 2,000 law-enforcement agencies conducted by the National Gang Center, provided estimates of both national gang membership and gang violence, but it was discontinued in 2012, largely due to funding reasons, according to Harris. That leaves the FBI’s Uniform Crime Report as the only national annual source of data on gang murders, per Harris.

The FBI report provides information on the circumstances surrounding murders, including gang killings. The data shows that since 2000, these murders have accounted for a relatively stable percentage — between 5 and 7 percent — of all murders for which supplementary data has been reported to the FBI.

The problem with the FBI’s gang count, however, is that it is likely far too low. The National Youth Gang Survey, for example, estimates that roughly 13 percent of all homicides committed from 2007 to 2012 were related to gang violence, which is a far higher number than the one derived from the FBI report. That discrepancy is largely due to the way the FBI compiles data on the circumstances of murders: Its estimates are based on data reported by law-enforcement agencies. Determining gang involvement, therefore, is up to the individual police departments, and most either underreport gang killings or do not report them at all.

For most cities, formal reporting on the role of gangs in homicides is spotty at best. New Orleans, for example, reported zero gang killings to the FBI in 2015 even though a report by the city found that gang members were involved in 49 murders that year (30 percent of the city total). Similarly, none of Baltimore’s 2016 murders were reported to the FBI as a gang killing, even though Baltimore Police Commissioner Kevin Davis has highlighted gangs as a driver of violence there.

In total, of the 90 agencies reporting supplementary information on 25 or more murders to the FBI in 2016, 61 of them (68 percent) reported zero or one gang killing. At the same time, the majority of 2016 gang murders covered in the FBI crime report for that year came from just two cities: Chicago reported 347 gang killings, accounting for 48 percent of the 722 homicides in that city for which supplemental data was available, and Los Angeles similarly blamed gang-related incidents for 47 percent of the 2016 murders for which circumstance data was reported (181 out of 388). Together, those two cities accounted for 58 percent of gang murders reported to the FBI last year.

And then there’s the question of focus. Law-enforcement agencies that do measure increased gang membership or violence may simply be paying more attention. The number of law-enforcement agencies that have dedicated gang units increased notably between 2003 and 2011, according to the National Gang Center.

Charles West, the architect of the New Orleans anti-gang program, summed up the problem in an interview in 2014: “We were always told that we didn’t have a gang problem. But we had gangs of significant size, and people just weren’t talking about it.”

Since increases in reported gang violence may simply reflect increased awareness of gangs, more resources being dedicated to countering gangs, or better training of officers to identify gang members, any definitive comments about gang membership, gang violence or how those two things have changed over time should be taken with a very large grain of salt.