The “encyclopedia anyone can edit” gets a bad rap for reliability, vandalism, and a toxic editor environment, but have you ever thought why these problems happen? Most Wikipedia criticism revolves around well-publicized vandalism or hoaxes, while many journalists (who should know better) gush about how Wikipedia is the new model for politics/ society/ technology/ raising llamas or whatever. There is very little discussion, even in the tech community, about Wikipedia’s software and policy choices.

As a former-Wikipedia editor and the founder of Newslines, the only direct crowdsourced competitor to Wikipedia’s news and biography-based pages, I have a unique insight into Wikipedia and what it will take to fix it (hint: it can’t be fixed). Feel free to leave comments.

1. It’s not an encyclopedia

Let’s get something straight first: Wikipedia is not just an encyclopedia, it’s a breaking news source, a news archive, a medical reference, a movie and music guide and a biography site, to name a few of its functions. The problem is that the site’s one-size-fits-all software does not work effectively for all those different functions. A biography site not only has a different market than an encyclopedia, but has different data, editing and presentation needs. This is a particularly dangerous problem in the medical field where Wikipedia has many articles that are dangerously imprecise, or wrong. This “mission creep” is because the encyclopedia part is done. There’s really not that much more to say about dinosaurs, so the site’s editors scramble, like elfs on meth, to add the latest news, Pokemon characters, porn stars, and other non-encyclopedic information, irrespective of whether that content is suitable or not.

There are many better sites than Wikipedia for this non-encyclopedic information, but they are all squeezed out by Wikipedia’s size, and its domination of Google’s search results (an unholy alliance I will go into in a future post). The end result is that readers across the web get poorer quality information than they should.

2. But it looks like an encyclopedia

Skeumorphism is when real-life interfaces are replicated in technology. Until recently, Apple’s calendar app looked like a real-life diary, complete with leather stitching (apparently based on the leather in Steve Jobs’ Gulfstream jet). Apple dropped this look because, while it provides some users comfort, it limits the diary’s functionality to think of it as though it were paper. The beauty of the web is that it isn’t restricted to paper and the data can be re-imagined to better suit the new media. Form follows function.

Wikipedia is what happens when a real-life encyclopedia is replicated on the web — it continues to act like a book, even though the web gives so many more possibilities than paper. Wikipedia does have two main features that distinguish it from a book, but both are implemented poorly. The first is that web pages have more space than paper pages, but that has resulted in incredibly long text-heavy articles, as well as raging debates between “inclusionists” (“Let’s make Wikipedia the sum of all human knowledge!”) and “deletionists” (It’s an encyclopedia, dammit!”). The other web feature that was a delight in Wikipedia’s early days was interlinking between articles, but Wikipedia’s links are implemented poorly (see below). There are very few Web 2.0 features. Social features are primitive: readers cannot make an easy list of their favorites, nor can they share to Facebook or Twitter with a single click. It’s as if the past ten years of progress had never happened.

3. And real-world Encyclopedias have pages

A great deal of the problems of Wikipedia are due to the site being organized around page-based articles. There are the obvious display problems: Wikipedia pages look like a dull grey textbook. Almost every page is an over-long stream of text with hardly any video or even images. Pages about movies don’t even have trailers. This has big implications for kids brought up using visual media like Instagram and YouTube who are not interested in getting information about their favorite movies, musicians and celebrities from a textbook. Check Taylor Swift – the page has thousands of words, but not a single video or music clip. As the quote goes: Writing about music is like dancing about architecture.

4. Readers can’t filter, sort or cross-reference pages

Because each Wikipedia page is structured as an article, there is no way to sort or filter the information on the page. What if I wanted to see Taylor Swift’s latest news first? Where is it on the page? What if I only want to see her video releases? In fact, her video releases are there, but in a separate table at the bottom of the page (which is inconsistent across all artists). By contrast, at Newslines, our writers add each news event as a separate post that can be sorted and filtered (using the filter icon at the top of the page) to create different views of the page. News events sorted by newest to oldest give a timeline of Swift’s latest news, while sorting from oldest to newest gives a biography view. We have now completed our project to allow filtering of the news events so that readers can quickly see an actors films, or awards, or wives, with a single button click. Wikipedia doesn’t let the reader control anything, just like a book.

5. Footnotes

There are 518 (!) footnotes in Taylor Swift’s page, taking up a huge amount of space at the bottom of the page. But the web doesn’t need footnotes because you can link directly to the source. I write about this at length in Wikipedia’s Broken Links. We solved this problem at Newslines by placing links that connect each newsline outside the post’s text, and creating links to sources on the verbs in the text, so they don’t conflict with proper names.

6. Page ownership

Because Wikipedia pages are structured as long-form articles, any editor on the entire system has veto power on any new information that is added. Let’s say I want to add new information to a page. Wikipedia will alert anyone who has the page on their watchlist, and the fighting will begin. It is much easier for an editor to click a button to revert an edit than it is to craft a new sentence, and recraft it multiple times until it is accepted. Wikipedia’s defenders will say that stops vandalism, but it also stops new or contrary information from updating pages. Once the main fighting is over, then the page calcifies, and updates become less frequent. What also has text that can’t be updated easily? A book.

7. Harassment

Expect a world of hurt if you want to get some text onto a Wikipedia page. Back in 2007 I tried to add a sentence on Richard Gere’s page about an ad Gere and Cindy Crawford placed in the Times of London saying their marriage was strong (they divorced a few months later). One editor removed my edit saying it was not “sensitive” to the subject. I found this absurd – Gere and Crawford had placed the ad in the newspaper themselves – they wanted the information to be known. I was forced into a process to find “consensus”. The editor pulled in his friends and outnumbered me. Weeks of debate followed, leading to me checking the Biographies for Living Persons policy, where I found the definition of “sensitivity” to be ambiguous and overly broad. When I suggested a minor change (a single comma). I was told in no uncertain terms to back off by the administrator who helped write the policy. She didn’t forget me. Several months later on an entirely unrelated page, and after the various editors on the page had actually reached a consensus, she swooped in and banned me. The information about Gere and Crawford is still not on Gere’s page.

8. Censorship & Bias

Does it matter is the information about Gere and Crawford’s marriage troubles is not on Wikipedia? Well, imagine my story multiplied by the millions of other Wikipedia pages and you will understand how much censorship and bias takes place on the site. A prime example is William Connolley, an English Global Warming researcher, who took control of Wikipedia’s global warming pages and moulded them to suit the agenda of the climate scientists he worked with. I tangled with Mr Connolly about the lack of any mention of the now discredited “Hockey Stick Graph” (which it turned out was deliberately removed by him), but was shot down time and time again. After thousands of edits deleting and massaging the pages, as well as deleting over 500 pages he did not agree with, he was exposed in the ClimateGate scandal and given a six-month ban on editing climate-related pages. Six months! How many people, including politicians, journalists and activists, were misinformed by his propaganda over the years? It should also be noted that the National Post articles criticizing Mr Connolly’s behavior are nowhere to be seen on his Wikipedia biography. The hive looks after its own.

9. Arbitrary rules, arbitrarily applied

During my time as an editor I encountered a great deal of Wikilawyering, a term used to describe being harassed by other editors who use Wikipedia’s extensive rulebook to beat you into submission. The number one thing I learned about Wikipedia is that there are few fixed rules. In fact one of the Five Pillars of Wikipedia is that “Wikipedia has no firm rules”. This naturally gives power to those who understand the rules, and to the administrators who apply them, the only real rule being that all rules can be broken if you are sufficiently connected. I was ostensibly banned for a “conflict of interest” (citing an article written in a magazine I published), yet many of the site’s administrators continue to contribute to articles despite blatant conflicts of interest, such as Mr Connolly.

Sock puppets (extra accounts made by users to hide their identity) are also a big no-no on Wikipedia, but I was specifically targeted by a sockpuppet user called “HeatedIssuePuppet,” and yet when I complained I was told that some users could have these extra anonymous accounts to keep their main account clean. Arbitrary rules allow wide discretion for admins to punish those who disagree with them while turning a blind eye to friends who break the rules. When someone says they want government to run like Wikipedia, you should run a mile. What they are saying is “My friends and I want to give the impression that there are rules, but actually we will apply them as we see fit, and we will exclude you with no recourse when you don’t agree with them”.

10. Gender and racial bias

Talking of exclusion leads us to Wikipedia’s infamous Gender Gap. Wikipedia editors are 90% young, educated, white males. Very few minorities and hardly any women. There have been many reasons put forward for why there are so few women: Women are not interested in adding knowledge, they are too busy, they have kids, they don’t understand technology, and other condescensions. All of these reasons are false. At Newslines we have 80% women and minority contributors. How can two crowdsourcing systems, making similar content have such difference in the people creating the content? There are two main reasons: we pay our writers and we have created systems where people can contribute without harassment.

When we first started I wanted to attract Wikipedia editors but got little interest. Wikipedia editors are interested in glory. They want to show off their knowledge (real or imagined) on one of the world’s biggest sites. They are not interested in small sites, even if they are being paid to post. By contrast, our writers were happy to make a dollar for writing each post. One of our writers told me she was saving for her Christmas fund. In fact, many of our writers have earned thousands of dollars. They see it as fun work for money, not free work for glory.

Wikipedia editors talking about gender bias is like lions at the watering hole wondering why the zebras are not thirsty. They blame women for not contributing, when they are the ones doing the exclusion. Wikipedia’s wiki-based software is perfect for young, educated, white men who want to fight each other to show who knows more. But the intensity of the fighting alienates other groups (as well as distorting the content to reflect young, white, male interests). This exclusion can never be fixed because the primary feature of wiki-editing is conflict.

11. The wiki is the problem

Every Wikipedia problem can be traced to the wiki software. The wiki software is responsible for the great success of the site, but also for its impending demise. The wiki system had great benefits on the upside — it allowed for fast content creation, but the conflict-driven editing has led to biased and incomplete content, lack of innovation, and many community problems. In 13 years the software has barely been updated to adapt to the needs of its users, or its readers.

It’s important to realize that Wikipedia was never designed from the ground up as a content creation system. Larry Sanger and Wales took off-the-shelf-wiki software and put it to work as an encyclopedia. Consider this exchange between Wales and the inventor of the wiki, Ward Cunningham, in Jan 2001.

My question, to this esteemed Wiki community, is this: Do you think that a Wiki could successfully generate a useful encyclopedia? — JimboWales Yes, but in the end it wouldn’t be an encyclopedia. It would be a wiki. — WardCunningham

What is the social cost of all this conflict? Many Wikipedia editors have slaved for months, if not years to try to get information added to the site. Time and emotional costs are never considered because there was a seemingly endless group of people willing to experience that pain to satisfy their ego. I suspect that creating documents using wiki software is one of the least efficient methods of content creation. When you have an infinite amount of monkeys anything is possible, but what happens when the monkeys would rather just fling poo?

12. Wikitopianism

None of these faults matter to the Wiki-believers. In their world the wiki software is perfect and that everything else — especially people — is responsible for Wikipedia’s problems. Gender bias – women’s fault. Toxic user environment – the fault of toxic editors (always other editors, not them of course). Too many rules – we need those rules so to control those pesky people. Paid editors – how dare they even add correct information! They are so caught up in Wikimania they don’t understand that it is the wiki itself that causes all of the site’s problems by creating a playing ground where only certain kinds of people survive, and where excessive rules are necessary to counter the flaws in the system.

This Wikitopianism infects much of Wikipedia’s community, with Jimmy Wales and the Wikimedia Foundation the worst offenders. They are completely blind to the failings of the site, but continue to boost it as though it is the solution to all mankind’s problems. But there is no excuse for outsiders to believe this. I have lost count of the number of articles that gush praise about the site by journalists who have never looked under the hood. It’s time to treat Wikipedia like any other software product in the marketplace. It has no magical right to exist beyond its usefulness just because it is Wikipedia. It’s been a great run, but it to survive in the future it must deliver better information than the alternatives. It can’t rely on a software model that has not been updated for 13 years.

Like Wikipedia

Wikipedia is in many ways like Communism. Even while people were lining up for a loaf of bread the Soviet Union’s leaders proclaimed their system to be the best in the world. Wikipedia is the Trabant of content, built by a group of party faithful swearing allegiance to a Dear Leader, using far too many resources to create an inferior product that is hopelessly outdated, underpowered and liable to collapse into a rusting heap at any moment.

Don’t think it will happen? Wikipedia is dying. The number of active administrators is falling. New users are not joining up. Pages are not being updated. Women and minorities shun the site. The Foundation that runs the site pulled in $50 million in donations last year, but can’t even organize the simplest software improvements. They have frittered away the money on useless projects that are hated by the community, such as the Visual Editor. However, the community itself is its own worst enemy and is resistant to change: What chance does the site have when the users don’t want change and the people at the top are incompetent and don’t care?

The 13th sin, apathy, is the greatest sin of them all.

There is a better way

Content doesn’t have to be built through conflict. It doesn’t have to exclude minorities and women, and it doesn’t have to have vandalism, errors, and inconsistencies. It doesn’t have to have the worst editing interface on the planet. Content creation should be fun. Over the past few months Newslines writers have added over 22,000 posts without a single edit war or controversy. Compare our pages with Wikipedia and you’ll see how we use the web as it should be used, not as a competitor to Encyclopedia Britannica, but purpose-built for the web with embedded YouTube and Soundcloud clips that help you understand the story of things.

Good software design reflects how people act, it doesn’t shoehorn them into inflexible and inhumane ways of working, even if the goal is noble. I hope you will follow us on our journey as we grow.