Several weeks ago, Digg made online headlines when it banned several dozen of its users, some of which were among its most highly ranked contributors (including Reg “Zaibatsu” Saddler, who by some metrics was the #3 Digg user of all time). On Digg’s blog, the company explained that the bannings were a result of script abuse, although some observers found this explanation to be unconvincing.

Shortly afterwards, Digg announced that it had raised $28.7 million in a new round of venture funding. And in the last few days, Digg jettisoned more members of its community with another fresh round of bannings, again including many high profile users. A separate Digg blog post patiently re-iterated Digg’s justifications for these actions.

A palpable level of online controversy has ensued as banned users have cried foul, while other users have applauded Digg’s actions. For anyone who is actively involved in online community-building, whether through a blog, a website, or a social network, examining the events of the past few weeks quickly reveals several important lessons about how Digg has handled the establishment and growth of its community. But perhaps more importantly, the way that Digg has gutted user morale and shown itself willing to cast off those that have been so crucial to its success also reveals the limits of its core crowdsourcing model.

Building a Flawed System





Kevin Rose demos Digg in 2004



Before we can get into all of that, let’s take a moment and revisit the past. Recall that Digg launched in December of 2004. At the time, Digg was regarded as a direct competitor to Slashdot due to its focus on tech stories. In a 2006 interview with ZDNet, Digg founder Rose proclaimed the site’s raison d’etre, saying:

[Digg] was just a side project of mine and something that we wanted to try out by giving power and control back to the masses. Typically with technology news sites a handful of editors choose which stories are relevant and which they believe the audience would like to read about. This was the first time that anyone experimented with allowing the general mass audience to decide what they believed to be the most important topic of the day.

At a time when the idea of Web 2.0 was just beginning to take hold and millennials were chafing under the constraints of receiving their news through the mainstream media, Digg offered the promise of news that was determined for the people, by the people. The fundamental assumption behind the site was similar to that of any crowdsourcing model: The wisdom of the masses will provide news stories that are more relevant, interesting, or informative than anything chosen by a select few.

Unfortunately, things haven’t quite worked out that way.

In order to understand why, it’s important to know the difference between Digg’s website (i.e. the nuts and bolts of how it functions) vs. Digg’s business (i.e. as a potentially money-making Silicon Valley startup). Both of these elements play important but distinct roles that led to the unmaking of Digg’s purported mission.

The Website

For those who are unfamiliar, Digg’s basic website functionality was as follows: Users found links they wanted to share from across the Internet and submitted them to Digg, where they were automatically entered into an “Upcoming Section.” Users could vote or “digg” up stories if they enjoyed them and if they received enough diggs, based on Digg’s proprietary algorithm, the story would be promoted to the front page. In theory, the Upcoming Section could be a thriving area of the site, where stories were discussed and voted upon, but as the site grew larger and the rate of submissions accelerated, it quickly became useless to check this section, due to the sheer volume of information (As of October 2008, roughly 10,000 to 15,000 stories are submitted to Digg per day. That is about one story every 7-10 seconds).

But Digg also had other features as well. Specifically, it allowed you to 1) Have your own profile page, and 2) Follow people by making them your “Friend.” Initially, profile pages displayed user stats, such as the number of stories submitted and the percentage of those stories promoted to front page (later, they would also display personal information such as photos and links to other social networking sites). Making someone a “Friend” allowed users to follow that person’s actions, including stories submitted and dugg.

Users quickly realized that one way to get diggs for their submitted stories was to make someone your Friend and consistently digg that person’s stories. Reciprocal diggs would usually follow. Using this process to find interesting stories, and to find online friends who would digg your own stories, was considerably easier than wading through the morass of thousands of stories submitted daily in the Upcoming Section.

As a result of all of these elements, the system favored stories submitted by users who had a combination of the following attributes:

• The time and the inclination to find interesting stories • The time to digg their friend’s stories • The will to both digg and submit stories consistently over a long period of time

Users who had these attributes naturally saw a large percentage of their stories get promoted to the front page. This was what led to the concept of “top users” or “power users,” a notion that was, in some ways, antithetical to the idea of democracy (and that’s completely leaving out the fact that the site needs editors to supplement the efforts of its users).

Why should a really fascinating or interesting story have a higher chance of getting promoted to the front page if it was submitted by a top user than a newcomer? Indeed, today still, a story submitted by a top user will instantly garner a significant number of diggs from followers, often regardless of the story’s quality. While Digg’s algorithm was theoretically supposed to correct for this, page after page of front page stories submitted by the same few individuals demonstrated that Digg was not doing its stated job very well. At one point, Digg’s top 100 users were responsible for over 50% of Digg’s front page stories.

This was Digg’s fundamental problem, although it didn’t have to be: It was an attribute shared by many other Web 2.0 companies. Only a small fraction of the total number of users contributed substantially to the site. According to recent Quantcast statistics, 1% of the Digg’s users are contributing 32% of the site’s visits. But this is no different than a company like Wikipedia, which clocks in billions of pageviews every month, yet whose articles and infrastructure are mostly maintained by a few thousand enthusiasts. While Wikipedia and similar sites have found ways to recognize and reward their active participants, Digg, as we’ll see, has constantly tried to shake the image that it is controlled by a select few.

The Business

From the business end, Digg has acted in ways to consistently grow its page views and expand its audience in the three and half years since its founding. About a year ago, Digg changed its interface to make friends’ stories significantly less accessible and to require more clickthroughs, a move that was widely decried by power users and which inadvertently led to the script situation that it is currently caught up in.

In a stunning analysis by ReadWriteWeb, the site also began dramatically expanding the variety of its front-page topics, focusing less and less on technology as the years went on (in other words, while the proportion of tech stories to all stories submitted remained roughly the same, the proportion of tech stories promoted to the front page went down dramatically). The implications of this were more troubling: Digg was actively manipulating the distribution of front page stories. Combined, these moves showed that Digg was becoming ever-more conscious of its need to appease investors as time went on. Despite a lucrative ad deal with Microsoft in 2007, social networks like Digg remained (and still remain) notoriously difficult to create a business model around.

Digg has also struggled to contain the influence of its top users throughout its existence. Far from acknowledging their contributions, Digg has constantly denied their key role its growth. On an episode of Diggnation, Rose even claimed that he didn’t know who the top users were, saying, “I don’t pay attention to who might be considered a top Digger, or who’s not.” In early 2007, Digg stopped hosting a public list recognizing users that were getting the most front page stories. According to Digg, this move was to prevent them from being harassed, although the list has since moved elsewhere with few complaints of any such trouble.

More recently, Digg has shown its independence by banning dozens of high-activity users, allegedly due to script use. Some of the scripts employed actually allowed users to digg their friend’s stories more easily, giving Digg pre-2008 levels of functionality. Digg’s recent actions are objectionable and suspect for a number of reasons. First of all, several of the banned users have complained there was no warning given and that the decision was irreversible. But most importantly, while many relative new users were swept up in the mix, top users such as “CosmicDebris” and several others were also eliminated.

These users have helped Digg to deliver dozens of millions of page views and clicks to sites across the Internet. It is not an exaggeration to say that they have helped to make Digg the popular site that it is today. For Digg to cast off these users like so much dead weight, rather than work with them constructively, is an indictment of Digg’s recklessness with its devoted community. It has also brought to light the tension between Digg and those users that have climbed their way to the top by learning Digg’s system. As a result, several other top users have expressed fear that their accounts, which they have poured hundreds of hours of time into, will be next on the chopping block.

Recent events have given hints about Digg’s upcoming trajectory. This past summer, Digg rolled out its Recommendation Engine, allowing users a new way to find interesting stories and taking some focus and dependency off of its top users. In July, Google allegedly and mysteriously walked away from a deal to buy Digg, amidst longstanding rumors that the cofounders have been intent on a sale. And earlier this month, Digg recently reported a new round of venture funding and declared it would be expanding worldwide. Put all of these together and you get the picture of an organization that is trying to establish it is not dependent on the efforts of a few dedicated and unpaid volunteers (i.e. its top users) to keep its website interesting and thriving.

But what has it sacrificed in its attempt to prove this?

Lessons Learned

The way that Digg has handled its growth reveals important lessons, both for competing social news sites and for social media companies looking to build thriving communities:

True Democratization of News is Difficult – Rose and his crew are undoubtedly a talented group of programmers, but even their Digg algorithm has had compensating for the flaws inherent in Digg’s system. In the years following its creation, Digg became less a democracy and more a republic, with a select few users responsible for the majority of front page stories. The Web is still struggling to come up with a news model that can efficiently crowdsource its editorial process, although sites that automate the process (e.g. Techmeme) or sites that rely on editors (e.g. Fark, Slashdot) are at least more transparent with their advantages and failings.

Recognition is a Key Motivator – Social networks typically have a tangible way for users to track their notoriety. MySpace has “Friends,” Youtube has “Number of Times Viewed,” and Twitter has “Followers.” Digg has “Stories Made Popular.” Top users often pointed to this number with pride, a reminder of the thrill of seeing one’s submission spread to thousands of eager readers. Digg, however, has done nothing to acknowledge their contributions and with its recent bannings, it has indicated it doesn’t believe it needs them at all.

While Digg’s growth may not be adversely affected by the accounts gone missing, it seems that when Time named “You” the Person of the Year in 2006, they were actually on to something more meta than originally thought: People like being recognized for contributions and the potential for Internet fame that may follow. Social networks that have gone on to insanely high valuations or become profit-making ventures have recognized this fundamental fact of Web 2.0. Digg has not.

Communities Require Nurturing – The way that Digg has treated it users has not been with the committed touch of a benign leader, but of a dictator that assumes its actions (or lack thereof) will be consistently met with the assent of its followers. Its town halls have been little more than PR exercises, and user-requested features like the Recommendation Engine have taken years to roll out, while others (e.g. forums) have yet to be implemented at all.

A comparison between Digg and Mixx quickly shows that while the former’s traffic still trounces the latter’s, Mixx’s founders and employees have been open about how they are fostering the growth of their website. Anecdotal comparisons between, say, “The Drill Down” podcast (which covers tech news as well as social news sites like Digg) and Mixx’s unofficial podcast, “Social Blend,” starkly contrast how each company has dealt with its fanbase. While Digg and its policies are frequently an object of criticism on “The Drill Down,” “Social Blend’s” contributors typically have nothing but kind things to say about their digital overlords.

Conclusions

Kevin Rose recently claimed that his primary goal in the most recent round of bannings was to enforce the TOS and make the site easier to use. According to Rose, the bannings are only a response to people trying to subvert the system:

What happens is that some people try and get around [Digg’s algorithm], and they try different ways of doing this. One of these methods is called script digging…and they go and they will digg their friend’s stories, in order to help propagate those stories on the recommendation engine, or to help promote them to the front page. They use automated methods…sometimes it’s very apparent…We have ways of telling how they do it. And so what’s going on is that these people are abusing the system, they are using extra resources and when they throw that many diggs at the system at once it sets our database slaves a little bit off kilter and they will have to readjust everything. We’re talking about, not damaging, but definitely causing a spike in usage and slowing down the site for other users, especially when multiple people are using it, or hundreds of people, whatever it may be. And so what we do is we always give people a second chance. We’ll stop them from doing this, we’ll put limitations on it. Oftentimes, and I hate to say it, but these people are trying to game the system… There are users that are on the site that will come out on their blog and swear that we have blocked them and that Digg is turning on their users...I can assure you, there’s no reason for us to do that. The thing we want more than anything else is a healthy, accurate community. We don’t want people to game the system, we want it to be fair to everyone. So, we’ve started banning people. If people are abusing us multiple times, I don’t care if they’re the number 1 digger or the number 5 millionth digger. We’ll ban them, because it’s B.S. It doesn’t matter how popular you are. I don’t care which user it is, if they are gaming the system, they are going to get banned. And we’ll be able to tell, we’ll be able to detect that.

If Digg’s management responds to this article they will undoubtedly re-iterate this and say that they are simply trying to protect the community (e.g. by enforcing the TOS equally). For the moment, let’s put aside whether or not it’s plausible that a few hundred people using the alleged scripts can significantly affect Digg’s formidable servers. What I have tried to establish is that Digg is giving you an incomplete story. To prove to its investors that its democratization model is functional, Digg has decided that it has a vested interest in getting rid of its top users. In its misguided efforts, it has destroyed the community that it sought to create.

While antagonizing and banning its most active contributors may be expedient, Digg will only ensure a new group of followers will similarly rise up over time, forcing the exact same measures down the line. This will continue until Digg solves the fundamental issue with its site: That those that are most committed will always have the most control. Digg may continue to grow in traffic and size, but with every controversial move like the recent banning measures, it will irritate a continually growing contingent of its most ardent supporters. Unless Digg can find a way to embrace its fans rather than eliminate them, it will find its brand promise as the leader of news democratization fading fast.

[Disclosure: I am Digg user whose account has not been banned…yet.]

David Chen is a writer/blogger/podcaster based in Boston, MA. You can e-mail him at davechensemail (at) gmail (dot) com, or follow him on Twitter.