Article content continued

Prof. Ruths is no Luddite, offering knee-jerk skepticism of the new digital frontier. For years this field has been his main area of research, studying human behaviour based on data collected online, so this is intended to be an “instructive critique from someone in the field,” and he critiques his own work as much as others.

He is confident social media deserves all this academic enthusiasm. He thinks there is “incontrovertible evidence that there is strong social signature in social media data that we can use to gain insight.”

“But there’s a big difference between saying it’s in there, and we are able to get it out,” he said.

In the interview, and in the paper, he identified a few reasons researchers have got away with errors of interpretation that, in many cases, are “very basic” and well known.

One is ensuring a representative sample, a problem that is sometimes, but not always, solved by ever greater numbers. Another is that few studies try to “disentangle the human from the platform,” to distinguish the user’s motives from what the media are enabling and encouraging him to do.

Another is that data can be distorted by processes not designed primarily for research. Google, for example, stores only the search terms used after auto-completion, not the text the user actually typed. Another is simply that many social media are largely populated by non-human robots, which mimic the behaviour of real people.

Even the cultural preference in academia for “positive results” can conceal the prevalence of null findings, the authors write.

“The biases and issues highlighted above will not affect all research in the same way,” the authors write. “[But] they share in common the need for increased awareness of what is actually being analyzed when working with social media data.”

National Post

• Email: jbrean@nationalpost.com | Twitter: JosephBrean