This week StackOverflow launched the public Beta of its new Documentation section. Since its opening, the Documentation system of SO, has attracted a lot of people and contributions, but there has been quite some controversy surrounding it as people have genuine and legitimate concerns about the state of the system, the quality of the provided examples and the reputation gains of such a system and the way this might affect the overall SO community and experience in the long term. I am actively participating in the public beta of Documentation with almost all badges earned already and over 60 actions listed in my profile at the time of writing this post and I think I have formed a reasonably complete opinion of its current state.

In general, StackOverflow’s Documentation feels like a natural extension of the Q&A website we all know and love and I hope it’s here to stay. Documentation offers something new and exciting to SO, something much needed by a lot of people: an easy way to begin. Many a new developer, myself included, have asked themselves How do I start coding in this language? How do I do this very simple thing? Usually Google would find a few good SO answers and hook you up like this, but a lot of the time that is not enough. Sometimes answers get outdated, people don’t update them and developers have a hard time at the beginning. That’s where Documentation comes in! Installation instructions, Hello World programs, simple syntax with examples, lots of the most basic topics that people will constantly search for. But it does not stop there. You can always find out how to do the more complex things with good examples and how to put it all together.

One of the main effects I anticipate to see due to Documentation is the community maturing. Over the past year or two, the quality of questions on SO has been going downhill, as people ask a lot of too specific questions without any practical use most of the time. Bad questions don’t always guarantee bad answers, but there is a certain link between the two. I think this whole problem stems from the fact that 90% of the important questions, the ones that you search for every day, have already been asked in the past and there is not a ton of things you might ask anymore that people will search for in the future. Sure, there are new frameworks popping up and a lot of updates happening, but the questions coming from those are a very small fraction of the questions that are asked daily. That’s where Documentation comes in! By having a large knowledge base available with consistent and up-to-date information in the form of Documentation, people will be discouraged to ask bad questions as they will have good enough examples to work from.

But there are sites like W3Schools or MDN or MSDN etc. that do the same, why do we need SO Documentation? I have seen people ask this a lot and their concerns are very much justified in my opinion. First off, these websites did the same thing before SO Documentation and they did it very successfully, focusing on their own tools and languages, always being up-to-date and having enough information for new developers and veterans alike. But they lacked character and the power of community moderation. Having people write examples for the things they know and work with, means more meaningful examples, more stuff that they felt they needed when they started out with a tool or a language or a certain topic. When you develop something, you have inside knowledge and you take some things for granted, but as a developer that has no inside knowledge of a tool or language or package, you sometimes need a different example or explanation than the one the original developer of said thing provides you with. There is potential for the community to fill that gap through Documentation, by adding the examples they feel like they need instead of the examples that the developers of their tools think they should be provided with.

This brings up a very valid concern about the Documentation system, the one regarding copy-pasting from the original documentation for any topic. This, however bad it could be, is unavoidable due to the nature of SO in general and Documentation in specific. A lot of the best answers on SO have been copied from the documentation that was provided on websites like the ones I mentioned above and that has always been the case. I am not blaming anyone for this, though. To learn and help others learn, you need to start somewhere and that is usually the most basic examples that you can work from. If what someone is asking is solved in a way very similar to said examples, there is no harm in using said examples and explaining them in more depth if possible. The Q&A format never required people to write answers from memory or without using any external resources and that is alright as long as people don’t blatantly copy paste whole pages without understanding the topic or providing any meaningful information about the question. This is also the case with Documentation and it will probably not change, especially as most of the stuff that people want to add to that system comes from the very simple questions that already have an answer in other websites. I’m not saying this is good in itself, but it is not new and, if it was acceptable in the Q&A format, it certainly is in Documentation as well.

It seems like I am praising Documentation a lot and trying to sugarcoat its shortcomings, but that is not the case. I see a lot of wrong with it as well, which could easily be ironed out during the public Beta and that could have also been ironed out before it, in the private Beta. There are three main problems I can identify with StackOverflow Documentation: weak moderation tools, lack of proper guidelines and the reputation system.

First off, let’s get the moderation tools out of the way. They are very very weak and have their own set of problems. Sure, when you have a small private Beta with high reputation users, it is easy to moderate the Documentation, but when you have millions of people contributing and some of them being inexperienced SO members, things can get quickly out of hand. Accepting a proposed change takes a couple of votes at most and anyone can vote as far as I can tell. Bad edits end up being accepted a lot of the time due to this.There is no review queue to ease people into the Documentation moderation. The idea of Documentation might be great, but the moderation tools lack the ease-of-use and will-to-participate factor that the ones in Q&A already had. Add to that the fact that there are no barriers for low reputation users and you get a big mess.

My suggestion is more of the same as what we used in Q&A: review queues, reputation barriers to vote in Documentation, the need of at least 5 people approving a change like we do with flags, a more free-form comment system in place of the suggested improvements system that we have etc. I would even go as far as to suggest some extra moderator features for people with the topic badges, so that someone with a gold badge could maybe lock a topic that has all the examples needed at some point and unlock it when the language specification changes and examples need to be worked upon again. More power to the community is a good thing and even more power to the elite of the community seems like a natural choice, provided that these people want and like to help, they don’t just do it for the reputation.

Then, there are the non-existent guidelines for editing and contributing to Documentation in general. There is a much linked page, the How do I ask a good question? that explains how to properly partake in the Q&A, but there is nothing similar for the Documentation. The SO team focused on explaining how to use the tools, but not how to use them good! And that matters a lot when dealing with a knowledge base that has the potential to be the most visited set of examples on the internet for virtually any kind of development information. There is incentive to contribute in the form of reputation, but there is no way to force good contributions by means of losing reputation for bad contributions (more on this later). If asking good questions and providing good answers was hard enough, what makes the SO team think that providing good examples for a topic is any different? Again, this is a transitional problem that probably wasn’t present in the private Beta and can be easily remedied by use of moderation tools, reputation changes and better guidelines that people can link to. Like before, the problem is that Documentation seems to forget what made SO great so far and doesn’t use the same tools.

Finally, the most discussed topic on Meta StackOverflow lately: reputation gains and the Documentation system. Just to get it out of the way, I am guilty of cheesing the system a little to get some reputation and I feel like this shouldn’t be possible, even in a Beta state. Now that this is all said and done, let’s talk about the issue at hand! Reputation has always been the incentive for SO users to partake and help others in the community environment. Sure, there are things like badges and privileges that are mostly honorary, but most of the time a nice solid number next to your name seems like incentive enough. When your main currency is reputation and the community values users strictly on that basis, any change to that system will produce a tremendous amount of feedback, positive or negative. Documentation has produced mostly negative feedback in terms of the reputation, as people have some very valid concerns about the dangers of the system and the effects it might have on the community.

When trivial edits can be accepted by anyone and anyone editing a topic can get reputation from upvotes, when there is no punishment for bad contributions and when you can get from simply and quickly deleting topics or examples that people have flagged unnecessary what you can get from writing great answers that take a lot of time, there is no wonder why people choose Documentation as a farming method for their reputation. Right now, the reputation system for Documentation is a mess and people that contribute must remember that some of the reputation they get from contributing in this Beta might go away later down the line. However, I am not one of those purists that say no reputation should be gained from Documentation or that it should be a separate system. If we want Documentation to be integrated, we have to embrace it, even if it is a quirky system with a lot of problems right now. What better way to embrace it than keeping it part of the main SO site and its systems? This point extends on what I said about the moderation tools as well, that for Documentation to feel at home, we need to give the community the same power it has over Q&A.

So how can reputation affect the SO community? Well, a lot of inexperienced users might be able to jump a lot of hurdles and cut the line to the higher leagues, meaning an influx of people with a lot of power and too little regard for the community. That is objectively bad. I have worked very hard in the past few months to get over the 1k reputation mark and I can tell you for a fact that I learned a lot from it both about myself and the community as a whole, as well as about being a good developer and having a code of ethics. Sure, there are people that did not learn a lesson like I did or others that got a lucky answer with 200 upvotes and haven’t contributed much apart from that, but still that is just a few people. Most people with 2k or 3k reputation are well-mannered, genuinely interested in helping and know how to play well with others in the context of SO. New users can sometimes be like that, but usually they need some time to adjust. If we give them a lot of reputation and allow them the privileges that they should get after a lot more time, they might wreak havoc on the community we so much love.

What can be done, then? That is for the SO team to decide, but the moderation tools I already suggested can be very useful to this effort. Other things include changes to the reputation system, privilege earning and the barriers people have to overcome to contribute. With a system in place like the spam/offensive flagging for the Q&A site only for bad edits and contributions, people will be discouraged to contribute bad content as there will be the potential of losing 100 reputation after 5 bad edits or something similar. That could remedy a lot of the problems with the system right off the bat. But more could be done, like adding a barrier to the Documentation for people wanting to contribute, say 200 reputation and maybe any kind of participation in a topic in the Q&A site that got them an upvote. That could be reasonable and save us some more trouble as a community. What else? Separate privileges! By tracking the two systems separately under the hood, SO can allow Q&A users to get their privileges for the Q&A section and Documentation users to get their privileges for the Documentation section. But on the outside, people will see a unified reputation counter that will make Documentation seem like it belongs to the SO ecosystem and isn’t just an afterthought.

A final consideration is the upvote system of the Documentation which is utterly broken as of now. Trivial edits can net someone hundreds of reputation in the long run, as the edited examples get upvoted. Limiting the amount of reputation per example to say 50 (10 upvotes) might remedy the problem. Even better, liming the daily reputation gain from Documentation to 50, will remedy the problem long-term. Making reputation come mainly from citations or something similar might help a bit, but I do not consider it a great idea. Basically, what I am saying is there are ways that I might have not considered, but the SO team must do something about all the concerns regarding the Documentation reputation inflation and they should do it fast in order for the problem to not get out of hand.

Summing it all up, I’ve got to say I like the idea behind the SO Documentation and I want to see it bloom and become the main way we learn about development. As it stands, it has its own set of problems like any new tool and needs for them be remedied before it gets out of public Beta. I trust the SO team and community will iron them out in the four weeks they are given until release and Documentation will live on to become a much loved tool for inexperienced developers and veterans alike. Until then, I will try to make the unified Documentation knowledge base dream a reality as best I can. See you all soon!