Contributed by jcr on 2010-06-04 from the manly dept.

If you've ever opened up a raw man page in a text editor, somewhere in the back of your mind you heard the words of Arthur C. Clarke, "Any significantly advanced technology is indistinguishable from magic," or the words of Larry Niven, "Any sufficiently advanced magic is indistinguishable from technology." Either way, you knew you had a lot to learn.

The recent and upcoming mandoc (via mdocml) changes are significant improvements to how manual pages are handled in OpenBSD. This important work not only improves build times, but also improves rendering and flexibility. Kristaps Dzonsons and Ingo Schwarze (schwarze@) were kind enough to tell us about the ongoing work.

NOTE: If you find man page formatting bugs, please do not file a PR (Problem Report) with sendbug. You should report problems directly to Kristaps and Ingo, or better to the mdocml mailing lists: discuss...<AT>...mdocml.bsd.lv - high-level discussions and version announcements

tech...<AT>...mdocml.bsd.lv - low-level discussions mdocml is being used by OpenBSD, NetBSD, and DragonflyBSD as well as in the ports collection of FreeBSD, so reporting problems properly upstream helps everyone.



Also, you should check the todo list Ingo Schwarze (schwarze@) has put together so you don't report an already known issue:

Just in case you missed it, Kristaps gave a presentation on mdocml at AsiaBSDCon 2009; Deprecating groff for BSD manual display . You can find the paper and a video of the talk online. Also, you should get familiar with the magic by reading the relevant documentation, both in OpenBSD itself and in the mdocml suite.

A nice description of the mdocml suite is at mdocml.bsd.lv

mdocml is a suite of tools compiling -mdoc , the roff macro package of choice for BSD manual pages, and -man , the predominant historical package for UNIX manuals. The mission of mdocml is to deprecate groff, the GNU roff implementation, for displaying -mdoc pages whilst providing token support for -man. Why? groff amounts to over 5 MB of source code, most of which is C++ and all of which is GPL. It runs slowly, produces uncertain output, and varies in operation from system to system. mdocml strives to fix this (respectively small, C, ISC-licensed, fast and regular). The core of mdocml is composed of the libmdoc, libman, and libroff validating compiler libraries. All are simple, fast libraries operating on memory buffers, so they may be used for a variety of front-ends (terminal-based, CGI and so on). The front-end is mandoc, which formats manuals for display. The mdocml suite is a BSD.lv Project member.

I wrote mandoc, so I can provide some historical bits. mandoc exists because grohtml didn't let me change the colour of `.Sh' section headings. I wanted to HTML-format the manuals for mult and sysjail (cf. http://bsd.lv/) consistent with the style of the surrounding site and was upset when it didn't Just Work. So I wrote a little tool to consume my manuals in -mdoc and directly produce CSS and HTML, so mdocml is short for mdoc2html and CVS dates my first check-ins at 18 months ago. Anyway, 14 months ago mdocml branched into mdocterm and mdochtml, sometime before AsiaBSDCon-2009, where it was featured in a talk. Soon after it went into OpenBSD. mdocml begat mandoc, consolidating mdocterm and mdochtml with nroff-compatible arguments ( -Txx and -mxx ) around 12 months ago. Soon after, mandoc took on -man via libman also about 12 months ago. The only recent feature added was -Txhtml about 3 months ago. The rest is accommodating for roff's strangeisms, both in terms of input and output. Today, mandoc is in the base system of OpenBSD (linked to the build!), NetBSD, and Dragon Fly BSD as well as in ports for FreeBSD. mandoc's main code contributors are Ingo Schwarze (schwarze@) and Joerg Sonnenberger (NetBSD). Ingo has performed yeoman labour in fitting mandoc into OpenBSD's build process, submitting patch after patch to get it working properly with OpenBSD manuals, and making it byte-compatible with GNU troff. Joerg has also done considerable work to similar effect, and also contributed the compatibility layer that allows mandoc to work on most any Unix system. Ingo and Joerg are the downstream contacts for OpenBSD and NetBSD. Ulrich Sp�rlein is the downstream contact for FreeBSD and Sascha Wildner is the downstream for Dragon Fly BSD. Many others, most significantly Jason McIntyre (jmc@), have tested mandoc on all manner of manuals. I do ask that anybody with a manual please test with mandoc. And if it doesn't work, to cross-check mdoc.samples(7) and mdoc(7) to make sure that the manuals aren't broken before submitting a report. Lastly, if one's manuals are in -man format, for gods sakes re-write them in -mdoc format! (Search for " Fixing on a Standard Language for UNIX Manuals" for the scoop...)

From Kristaps Dzonsons:

When I made the mistake of filing an OpenBSD PR with sendbug on a man page formatting bug, Ingo was kind enough to send me the following.

In general, when groff copes, mandoc ought to cope, too. Except for very rare exceptions. We know there are still lots of small problems. It is not difficult to dig up more of these than we can fix quickly by just running automatic comparisons across the tree. The most helpful bug reports regarding non-fatal mandoc errors currently are those that report formatting errors with important consequences (e.g. layout completely garbled to the point that one cannot figure out the content any more)

report one problem at a time

say precisely what goes wrong, best with man(7)/mdoc(7) source code and groff and mandoc output But even those need not go to the OpenBSD bug tracker, or we will quickly put more nits there than people want to see in that place. Just send them to kristaps and me directly.

From Ingo Schwarze (schwarze@):