When you buy a Raspberry Pi, the $35 computer doesn't come with an operating system. Loading your operating system of choice onto an SD card and then booting the Pi turns out to be pretty easy. But where do Pi-compatible operating systems come from?

With the Raspberry Pi having just turned one year old, we decided to find out how Raspbian—the officially recommended Pi operating system—came into being. The project required 60-hour work weeks, a home-built cluster of ARM computers, and the rebuilding of 19,000 Linux software packages. And it was all accomplished by two volunteers.

Like the Raspberry Pi itself, an unexpected success story

Although there are numerous operating systems for the Pi, the Raspberry Pi Foundation recommends one for the general populace. When the Pi was born a year ago, the recommended operating system was a version of Red Hat's Fedora tailored to the computer's ARM processor. But within a few months, Fedora fell out of favor on the Pi and was replaced by Raspbian. It's a version of Debian painstakingly rebuilt for the Raspberry Pi by two volunteers named Mike Thompson and Peter Green.

It began because Thompson was ready for the next big thing. Thompson had been CTO and co-founder of Atomz, a hosted search service acquired by WebSideStory in 2005. Thompson got a "not unhealthy slice" of the $45 million or so that company sold for, and he eventually stopped working to spend a few years hanging out with his wife and kids.

A year ago, he was finally ready to get back into the technology field. Robotics is one of Thompson's primary interests, and the Pi looked like a great platform for that. It's cheap and small enough to be easily embedded into a variety of systems. But there wasn't an operating system fully optimized for the Pi's floating point unit, which is important in robotics projects and various other types of math-intensive applications.

"When I first learned of the Raspberry Pi, I was disappointed that none of the Linux distributions available for it would make use of the very fast floating point hardware that was present on the Pi," Thompson told me. "As a long-time Debian user, I was like, 'I'd rather see Debian [than Fedora]' and I wanted to see the floating point enabled because I have this longer-term interest in getting robotics working with these inexpensive boxes."

Debian had added floating point support for the ARMv7 processor, but not the ARMv6 processor used in the Pi. Debian "didn't see a product like the Raspberry Pi coming on the horizon. Even though ARMv6 in Pi has a pretty capable floating point unit, they didn't support it," Thompson said. Thus, "all the thousands or tens of thousands of software packages they built wouldn't support the Raspberry Pi."

Just as a graphics processing unit handles graphics operations very quickly, a "floating point unit performs all the math very quickly," Thompson said. "It's a peripheral that not every computer has, but when it does you really want to take advantage of it." The floating point unit is part of the PI's Broadcom BCM2835 system-on-a-chip.

If you don't take advantage of floating point capability in the hardware, a lot of mathematical operations must be performed in software, lengthening the amount of time it takes for the Pi to do useful work. This is important for robots, where complicated math operations are used for processing data from cameras and sensors, and for controlling motors with precision and speed, Thompson said. It's also important for multimedia processing, encoding music, physics simulations, or "pretty much anything that is numerically intensive."

A fruitful partnership

The path forward was clear to Thompson: rebuild Debian to run on the Raspberry Pi. This required shifting 19,000 packages from Debian to Raspbian—a monumental task.

Thompson didn't go it alone, though. He started a Raspberry Pi forum thread to talk to other people interested in bringing Debian to the Raspberry Pi. The thread caught the interest of Peter Green, a Debian developer and PhD student in the UK who went by the handle "plugwash" in the forums.

Green was a rare mix. He not only had the expertise to co-lead the project with Thompson, but he was also crazy enough to do it.

"I felt I was the only one of the people who were talking in the threads about the project that became Raspbian with enough Debian knowledge to make the rebuild a success," Green told me. "I'm sure there are other people within the Debian project who could have done it if they were interested and crazy enough, and there were many people in the Debian project who provided us with bits of help along the way."

When Thompson and Green got started, the Pi wasn't actually available yet. Even if it had been, it wouldn't be capable of rebuilding Debian in a reasonable amount of time. Thus, Thompson set up a cluster of eight Freescale iMX53 Quick Start Boards, each with 1GB of memory, a 1GHz ARMv7 processor, and (most importantly) SATA hard drives. One of the main reasons the Pi would be unsuitable for this type of work is the bottleneck introduced by USB storage, Thompson said. The Freescale boards were able to build Raspbian anywhere from four to 12 times faster than the Raspberry Pi would have.

Thompson spent nearly $3,000 on the cluster but recouped those costs through donations to the project. In addition to the single-board ARM systems, there is a separate Linux PC to serve as the repository for Raspbian builds. The repository server retrieves source packages from Debian repositories, schedules jobs for the Freescale systems, and collects the binary packages once the builds are complete. "The built packages are all staged in the repository and then synced with the external repository we maintain at www.raspbian.org where users pull their packages from," Thompson explained.

At first, Thompson used an ARM-based HP Media Vault MV5150 for the repository, but later upgraded to an Intel-based system when more horsepower was needed. Although each Freescale board has its own hard drive used for building packages, the main storage duties are handled by the repository server's 500GB drive, which is about two-thirds full now. Here's what the cluster looked like when he was first setting it up:

Thompson and Green weren't starting from scratch. Debian is already one of the most widely used Linux-based operating systems, obviously. Its ARMv7 port provided a solid foundation.

"We greatly leveraged the work done previously by the Debian Project to support floating point hardware on ARMv7 devices," Thompson said. "Other than the effort to actually build [19,000] software packages, 95 percent of the work done to port the software to support the Raspberry Pi was already done by Debian."

That's not to say the Thompson/Green work in bringing Debian to ARMv6 and its floating point unit was trivial, however. Green explained:

In Debian the compilers have default settings built into them. These default settings set things like the CPU family, the minimum CPU requirements, and the ABI [application binary interface] in use. Most packages leave those settings alone. We modified the compiler packages to reduce the default settings to ARMv6 For the majority of source packages rebuilding them with a compiler that uses our new defaults is enough to make them build ARMv6 code.

A lot of the original work was done manually, but Green eventually created auto-builder software to automate much of the process. Those auto-builders (which run in a chroot environment on top of Debian) are still running to this day, pulling updated packages from Debian repositories and automatically re-compiling them for Raspbian.

Those auto-builders also detect when there's a problem that prevents the package from being automatically rebuilt. Green explained in an e-mail:

There are several things that could cause a package to come out still containing ARMv7 code. We hacked together a script (using readelf) to check for ARMv7 code in packages so we could tell which packages... needed further attention to eliminate it. ARMv7 contamination can come from several sources. Static libraries [are one example]: these are taken from the build environment and incorporated into binaries. So if the static library contains ARMv7 code then it will go into the resulting binary. This was a big pain during the early days of the project as we had to use packages from Debian armhf [Debian's ARM floating point port] to break dependency cycles, but those packages could contain static libraries with ARMv7 code in and sometimes it took several attempts at rebuilding and installing different packages to figure out which library the contamination was coming from, and replace it with an uncontaminated version. However, this isn't really a problem anymore since having got past the bootstrapping problem we can nearly always build packages in a clean Raspbian environment.

Raspbian became available for download on April 20, 2012 in limited form, containing only about five percent of the Debian packages. "Enough to build a root filesystem that would boot to a command line," Thompson said.