Someone recently wrote a rather narrow, uninsightful complaint about things that suck in open source development. The problems with that lengthy complaint, in short:

Software packaging appears broken sometimes simply because a given piece of software hasn’t gained enough popularity for anyone to bother. Keep in mind that with open source software you get to see (and use) software before it’s completely ready for prime time, pretty much by definition. Once it hits the “big time” (or even the “fairly small, but at least noticeable, time”), software in the open source world gets incorporated into the best software packaging systems known to man. Documentation, for all its problems in the open source world, is actually better on the whole than in the closed source world, from what I’ve seen. By far, the best documentation for any OS I’ve come across is FreeBSD’s. OpenBSD’s has a stunning reputation for completeness and usefulness as well. Debian’s manpage coverage is so extensive it boggles the mind. The books you can get for Linux in general are many, varied, and extensive. Contrast this with the well-known problems of MS Windows documentation, for instance. MacOS X has great documentation — for a closed source, proprietary OS. That’s not saying much, though, when contrasted with the completeness and extensiveness of FreeBSD’s documentation. SourceForge isn’t exactly a place for comparing the state of documentation and packaging of open source software with that of closed source software. That’s like going around to all the proprietary software vendors and checking to see what they’ve got in their project queues, measuring the quality of documentation and packaging for software that isn’t even guaranteed to get continued development funding (let alone release-worthy).

That’s not to say that there aren’t problems with packaging and documenting software in general. If “taw” referred to software in general, rather than specifically singling out open source software as the “bad apple” (and thus ignoring the fact that closed source software seems, on the whole, to fare worse), I’d have simply nodded my head in agreement. Something needs to be done about packaging (for distribution and installation) and documenting software as it’s developed.

I’ve already talked about ensuring you achieve good results with software deployment by ceasing to consider deployment procedures as separate from development. Since packaging/installation, aka “deployment procedure”, actually involves thinking about the structure of the software and writing code, it’s an easy thing to incorporate deployment development in your application development process. The two are a natural fit, which might help people to realize that they are, in fact, one — not two — after all. The deployment procedure for a piece of software is part of its interface. Learn that simple truth, and you should be able to put it all together. Voila, you’re set.

Documentation doesn’t seem to be as easy — not by a long shot. Documentation, as it is practiced, is a long, drawn-out process of duplicating significant parts of the source code of your software in English. Add to that the fact that you must also essentially create an after-action application use flowchart using the English language (rather than one of those old plastic flowchart stencils), and it starts looking like a severe pain in the butt that will never get finished properly.

The DRY principle applies to documentation as much as to actual software development. The principle, at its core, is simply a statement of the fact that when you repeat yourself, you introduce inconsistency. Bugs are unavoidable with duplication. You don’t duplicate code because when one part changes, the other part gets out of sync. One could even consider pretty much every single advance in programming practice in the last forty or fifty years to be an attempt at solving the problem of duplication. That’s the whole point of programming in the first place — automation, which saves you from duplicating effort.

It’s no wonder that documentation development, as essentially a plain-English duplication of software development, gets neglected — or ends up out of date and sometimes worse than useless. The only real solution to the problem, it seems, is to try to figure out how to eliminate duplication without making either programming or documentation suck. There have been some abortive attempts to achieve this, or something akin to it (see the invention of the eminent Dr. Knuth known as “Literate Programming”, COBOL’s syntax, and RDoc), but they haven’t tended to be really successful — whether its failure was in popularity or technical effectiveness — as a means of producing end-user documentation.

I’m not sure how to solve the problem, personally. It needs to be fairly universal in its applicability (it has to work just as well in Visual Studio, Emacs, and ed — vi too, of course, but that goes without saying since vi is the One True Editor), and it needs to truly eliminate the duplication problem. Finally, it needs to provide good end-user documentation. It’s a real problem.

Does anyone out there have any ideas for solutions?