In a recent post about Digg, I wrote: "it’s obvious to see that Digg users are growing in power. And while this could be a force for good, it also means that an unsupported claim on a blog could quickly be blown out of proportion." Unfortunately, this is exactly what happened to the O'Reilly contributor Steve Mallett, who was accused of stealing Digg's CSS by an anonymous blogspot blogger. This O'Reilly post explains more:

Steve Mallett, O'Reilly Network editor and blogger, was very publicly accused, via a Digg story, of stealing Digg's CSS pages. The story was voted up rapidly and made the homepage, acquiring thousands of diggs (thumbs-up) from the Digg community along the way. There was only one problem: Steve didn't steal Digg's CSS pages.

The real story has now made its way onto the Digg frontpage, but the issue has led some bloggers to (incorrectly) criticize the "wisdom of crowds" philosophy.

Greg Linden responds by saying "Take a majority vote from people who don't know the answer, and you're not going to get the right answer. Summing collective ignorance isn't going to create wisdom." Greg's conclusion may sound logical, but it's flawed. Nat from O'Reilly makes the same mistake, saying: "This is a classic Web 2.0 problem: it's hard to aggregate the wisdom of the crowd without aggregating their madness as well." But the problem is not that the wisdom of crowds doesn't deliver good answers (in fact it does) - the problem is that Digg is not a true example of a wisdom of crowds system. Let me explain...

For the wisdom of crowds to work, every individual must work independently. For example, if I ask 1000 people to guess the number of jellybeans in a jar and then average the results, I'll get a fairly accurate answer. However, if I allow the individuals to view the guesses of others before they vote, they may decide to give an answer which is similar to those given by other members of the group. Hence, it is more likely that the average answer will be inaccurate. The reason that Digg creates a mob mentality is that users can see how other members have acted before they vote. Seeing that a hundred users have approved a story already makes you think that there must be some validity to it. Instead of errors cancelling out, they are amplified. However, if you couldn't see those diggs, you wouldn't be so sure. Richard MacManus refers to the problem as the groupthink at Digg, which is a good way to describe it.

You could argue that del.icio.us is more true to the wisdom of crowds than digg - since most people bookmark pages independently and for their own benefit (rather than copying the bookmarks of others), groupthink is less likely. But unfortunately del.icio.us is subject to all kinds of external factors that could lead to inaccuracies.

I'm not a social scientist, so I expect this post has a few inaccuracies of its own. Feel free to correct them in the comments - just make sure you don't read the other comments first! :)

PS. I should probably add that I'm a fan of Digg - I'm just pointing out that its problems are not a reflection on the wisdom of crowds theory.

[More from Steve Mallett, Venture Chronicles and others]

————

UPDATE: I've added more comments on this over at Greg Linden's blog. I hope he doesn't mind if I reprint them here:

Pete Cashmore said...

Greg,

This isn't really true. In fact, ignorant people can make good collective decisions because their errors cancel out. The problem isn't that the wisdom of crowds doesn't work, it's that Digg doesn't reflect the wisdom of crowds as described by James Surowiecki. More here:

Digg and the So-Called Wisdom of Mobs

6:43 AM Greg Linden said...

Thanks for that link, Pete. Wisdom of the mob. Heh, heh.

I'm essentially trying to make the same point as you, though you may have made it more clearly than I. Majority vote may extract the wisdom of the crowd, or it may only extract the groupthink of the mob. Hiding the votes before people have voted might help a little, but I think there has to be a better, more reliable ways to extract the wisdom from the crowd.

I think what you want to do is attempt to identify experts in the crowd, people with the necessary information to make the decision, and weight their opinions more heavily. Slashdot discussions, Amazon customer reviews, many sites have early attempts at this, but obviously a lot more work needs to be done.

7:47 AM Pete Cashmore said...

Greg,

You've missed the point. The *whole point* of the wisdom of crowds is that the collective wisdom of a group can be *as good or better* than an expert or team of experts. For instance, instead of hiring consultants ("experts" by another name), companies should set up internal prediction markets so their employees can vote on issues relevant to the company. Picking out "experts" is the last thing that Digg should do - this may actually lead to a less accurate/truthful result. Instead, they need to remove the groupthink from the system.

One last point: is it actually a good thing for Digg if the stories are accurate and truthful? If the community cannot see how others are voting (necessary to prevent groupthink), then the community aspect becomes weaker. So while these measures might limit groupthink, it seems to me that it's actually a shared sense of identity that holds the Digg community together. To put it another way, Digg is more likely to survive precisely because of the inherent groupthink.

More at wikipedia.

Greg Linden said...

Pete, I think we're agreeing here.

I shouldn't have used the word "expert". That sounds like I'm referring to one person in the crowd. I meant making some effort to isolate the people in the crowd with an informed opinion.

Internal prediction markets work for exactly this reason. If you go to the Iowa Electronic Market and plop down cash, you likely have a good reason to be asserting that X is true. There is a cost to making a bet, so only people with informed opinions make bets.

Great point on how groupthink might help Digg succeed. There certainly are news outlets that have benefited from creating a strong community with groupthink.

Pete Cashmore said...

Greg,

Understood. So perhaps "editor" is a better word than "expert". Better still: "trusted people". I'm not opposed to subscribing to "trusted people", rather than topics. In fact, if you subscribe to a member of del.icio.us you generally get better results than subscribing to a tag - what's more, it's spam free.

But there's a tendency (risk?) that we're just trying to reintroduce the old-media hierarchies into this new "democratic" system. If there are to be editors, it needs to be the case that anyone can be an editor, and users can subscribe to whichever editor they choose (which might be what you're suggesting). If there are a limited number of editors, then you re-introduce the problems that new-media is trying to solve (ie. news is dictated from above by a select number of people). In fact, the word "editor" is so strongly associated with hierarchies that I'd prefer to avoid it altogether.

Like I say, I'm not suggesting that Digg should necessarily change - it may indeed be the ideal system if the aim is to foster a close community. But if you wanted to create a new system that was less prone to groupthink AND avoid the introduction of traditional hierarchies, you might consider some of these ideas.

PS. As you probably know, monetized prediction markets are more accurate, but setting one up means tackling all kinds of legal issues - it's something I've been considering for a while. I spoke a bit about this in my post on Smarkets.

10:41 AM