Behavioral studies of specific populations are extremely challenging things, as recruiting a set of participants that represent a true cross-section of the larger population is never easy. The mere willingness to participate in a study involves a necessary degree of self-selection within this population, which can seriously complicate behavioral studies—after all, it's entirely possible that a willingness to take surveys is the product of one of the behavioral traits under examination. That's why, even though a small survey can produce results with a high statistical confidence, it still may represent nothing more than a robust result within a non-representative group. All of that should serve as a precautionary background on a new survey of Wikipedia participants.

The short publication that describes these results is entitled, "Personality Characteristics of Wikipedia Members," and the news isn't especially good. Those contributing online feel that their true personality comes across better in the virtual environment; they also scored lower on a measure of agreeableness, and aren't especially open to new ideas.

It's easy to rationalize the first of these findings: if someone is more comfortable online, they're more likely to contribute to an online community. The other two, however, require a bit more in the way of mental gymnastics, given Wikipedia's relative newness and its focus on building a consensus about what constitutes relevant, unbiased information. In one press report on the findings, an author suggests that Wikipedia might provide a way for these individuals to compensate for real world inadequacies.

Credit to whoever did that reporting for noting the number of Wikipedians who were surveyed to produce these results: a grand total of 69, all from Israel. For some unfathomable reason, however, the reporter neglected to consider just how staggeringly unrepresentative those 69 are of Wikipedia at large. Wikipedia's own survey of its contributors racked up roughly 12,000 original contributors (and another 32,000 sporadic contributors). Given that there were responses from about 400 Israelis, that survey probably included about 175 Israeli contributors. And Israel appears to constitute a small corner of the Wikiverse, as there were roughly 20,000 responses each from the US and Germany; another five countries provided over 5,000 survey responses.

So, the 69 individuals surveyed appear to represent little more than the Israeli contributors that were willing to complete a survey. There's reason to be a bit skeptical about the survey, as well. The evaluation of personality traits appears to have used a standard series of questions that have been validated by others in the field. But the questions about where one best expresses their real identity were derived from a "Real Me Questionnaire" that consisted of only four questions.

There's nothing wrong with the study per se; it's a reasonable first attempt at addressing a recent social phenomenon that hasn't been extensively explored. But the survey size and geographic limits indicate that it should be interpreted very cautiously and not viewed as a generalized portrait of Wikipedia contributors. Unfortunately, the authors of the paper have done just that, both in the paper itself and in the interview material.

Of course, the paper itself probably would have only reached an audience of people trained to identify the limitations of research if it weren't for the press coverage. The fact that the author of that coverage was clearly aware of the issue (at least to the degree that he typed the numbers) but didn't bother to register its importance is pretty discouraging.

CyberPsychology & Behavior, 2009. DOI: 10.1089/cpb.2007.0225

Listing image by Image from Wikimedia Commons; Oscar the Grouch character