Manjaro, Arch, and Debian

I’ve been running Debian-based distros for my first 8 years with Linux, but I decided to try Arch for my new Lenovo Yoga 2 Pro. Most recently, I was running Mint-Debian aka LMDE, but with this new hardware, I wanted to be more up to date with the latest software. For example, LMDE is currently using Gnome 3.4, whereas the latest is 3.10. LMDE also isn’t really rolling in the sense that Debian-Testing or Arch is, as new things only arrive about 3 times per year. Therefore, between the time to get into Debian, and the time to get the new batch of updates from LMDE means it can be many months after I’d read a news article about the new LibreOffice features before those bits would actually arrive on your computer. In fact, by then I might have forgotten why I was excited about their work in the new release.

I installed Arch but considered Manjaro also. The big benefit is a graphical installer:

Not everyone is comfortable in a command-line, but I believe everyone should be. Today, however, many people, especially those who came from Windows or the Mac, have not yet learned it, and so Manjaro is doing a valuable service to improve the initial out of box experience. There is plenty of time to learn more about how your computer works once it is installed and configured. It is surely possible to take the necessary steps, and the best practices in the wiki, and put them together into a friendly and beautiful installer.

I haven’t tried Manjaro so I can’t say how it would have worked for me. However, given the hardware problems I ran into such as needing to use rfkill, setting the acpi_backlight kernel command-line, needing to tweak the Synaptics configuration to make the mouse close to usable, etc., it would have been almost as difficult for me even with the work they are doing to lower the barrier to installation on this new hardware.

I had to know how to tweak GRUB, systemd, and other configuration files, so I may as well set it up. I hope Linux one day doesn’t require anything tricky to get it working out of the box, but that is not today.

Manjaro has taken the mature Ubuntu installer and ported it over to the Arch system. This is valuable to help new people getting into Linux, but assuming it works, the benefits over running Arch on a daily basis drop to near zero. In fact, running it is possibly less useful than Arch, a topic I’ll discuss later. Note that you can’t switch to the Arch repos after you install Manjaro because your computer might not boot.

Manjaro does provide benefits after installation, but they are of much less value. For example, they have added graphical notification of new updates:

Arch by default handles updates only on the command line, and has no GUI to prompt users when new updates are available. This is a nice but very optional feature as the alternative is just running the trivial command pacman -Syu.

Some things I disagree with about Manjaro. I don’t understand why they have their own repositories with a copy of all the packages when they are mostly focused on the problem of installation. Shipping a different installer entry point into the Arch world doesn’t require making a copy of everything. Manjaro might claim to gain benefits from having having separate repositories, but they are changing only a few packages so the improvements are tiny.

Arch is a bigger team, and so I trust them more to have the latest versions available with important security fixes, etc. Keeping up to date with the latest and best software is an ongoing and time-consuming process. It is the lifeblood of a rolling release. I don’t want to run something that isn’t a part of Arch in that important way. Manjaro should enable users running out of the Arch repository to not lock people in.

Manjaro takes released versions from Arch and runs additional tests on them. By doing this, they can claim to be more stable. However, if the underlying problem is that Arch is sometimes unstable, doesn’t that mean it just needs more people running the test builds? Why is waiting to start further testing a good idea?

Even consider the case that sometimes useful features don’t get enabled in the Arch packages. For example, if Manjaro wanted to offer x86 kernels with PAE support for machines with more than 4 gigabytes of RAM, they can setup their own respository with just that package. The cost to mirror and distribute a few packages is much smaller.

Or, people interested in doing the PAE work could just help out the Arch kernel developer who is probably very busy. It is easy to see problems as a reason to create something better and new. But a fork is social engineering and to be taken with care. I also don’t understand why Manjaro have their own wiki. It seems like it should just be a a few pages on the Arch wiki. The wiki is one of the best features of Arch, why would you start over rather than improve what is already there?

As a side note, Antergos is another (top-30) Arch-based distro with its own simple graphical installer but at the end points to the Arch repositories. For some reason, this team and community is less well-known than Manjaro. I don’t understand why so many small groups of people think they should make an OS. At Microsoft, a team of 4 developers would be maintaining a codec.

Mike Conlon has written papers about forks in software. In short, he said they should happen when people can’t get along. While we celebrate the diversity that comes from freedom, we shouldn’t always consider forks as a sign of something positive. Imagine living in a world that celebrated divorce instead of marriage. The benefits of community come with size and division of labor.

I don’t know much about the community of Arch. I have long respected Debian for its good people, generality, stance for freedom, sense of community, and vast institutional knowledge. The DPL election process itself encourages civil dialog about how things can improve. However, at some point with the kernel, systemd, Firefox, etc. the OS itself is just the tool to compile and distribute all the other components.

How Long Here?

I have run Arch for 6 weeks. I don’t know how long it will last, or whether I will find sufficient reason to leave. My review documented all the problems I ran into, but none are Arch’s fault. Put another way, I found no real problems in their package manager, compile and install scripts.

Some have left Arch and before I installed, I read reviews and articles to understand what the reasons were.

I discovered various different causes but I don’t think any of them will be a problem for me. One of common reasons was that there have been some transitions in the past that were disruptive for users: for example, the switch to systemd broke some systems, but there were others as well. Even if the users can fix the problems, the sum of the inconveniences of the years can leave a mark.

The switch to systemd was cited several times, and some thought it was bad idea being forced on them by a cabal. I personally believe it is important technology modernizing low-level usermode aspects of the OS. The kernel is a great piece of technology, but at the end of initialization, it starts one user-mode process and then waits for work. The kernel doesn’t even write its error messages to disk, just to a memory buffer. Such policy decisions as where to store the files and what format are left to usermode.

There are multiple free init systems out there, and so it is hard for groups to come to consensus about whether to switch and which is the right one. Arch because of its simpler social structures and smaller team, doesn’t have the open debate process like what Debian is currently going through. However, it doesn’t really matter how you came to that realization as long as it was the right result!

In most cases, Arch can provide multiple choices such as whether to run OpenBox, KDE, or Gnome. Supporting multiple init systems was a choice they did not provide, but that was also a good decision. Via the configuration files and command-line utilities, you can tweak a system in innumerable low-level ways. The idea that you gain any further useful customizability via multiple init systems is incorrect.

Arch was relatively late (mid-2011) in supporting package signing. (The benefit of this feature is that you can independently verify that a package came from Arch, no matter whether you download it from their servers, or a mirror, or any other place on the Internet.) Some left Arch, or threatened to, because it didn’t sign packages.

But at the same time, you need to be the type that wears a tinfoil hat to believe that anyone would secretly disrupt Arch or its mirrors. If the NSA wanted a secret hook, they’d put a gun to Linus’s head. They wouldn’t bother hacking Arch as it would take too long to accomplish anything. It was one of those features that is a good idea because it adds end to end security, but was only ever a theoretical problem for Arch users.

In any case, Arch now has signed packages. It was simply a lack of resources and more people complaining than programming. Arch is run by people who see what other distributions do and can learn from them. You don’t need a formal social organization to be able to learn from the good ideas of others.

Because Arch doesn’t have elections, release parties or yearly conferences, it doesn’t have as much a sense of community. Some people left Arch because they found too much elitism and rudeness. I experienced it personally already from one of the maintainers.

Upon filing a bug, I was told that it would have been easier for me to create the package I wanted than to keep discussing whether it even should be packaged. Of course, making your first package is much harder than installing it, just like making a cake is much harder than eating it. I told him that he should have just added the package like every other distro has, rather than keep coming up with invalid reasons not to, but that wasn’t convincing 😉

On Reddit, Arch users commented of my previous review that it was my fault for expecting it to work like Ubuntu, and that I deserved no “sympathy” for any issues I ran into. One fan of Arch even told me I shouldn’t be running it! I have been running Linux for 8 years and working with the command line since he was in diapers.

Of course, judging a community by individual emails and Reddit comments is scientifically invalid. Insanity in various forms is currently a pre-existing condition of society, and I don’t have reason to believe it is worse in Arch than Washington, DC or Los Angeles.

But the distros that have a mentoring process and such will find that it removes a distinction between users which are to be looked down upon, and contributors who are the masterminds. Every contributor was initially a clueless user and some distros even have a mentoring process.

Arch has a process for moving from user to contributor as well, but it is much more about demonstrating past contributions, and eventually given more upload permissions. It also has a user repository area (AUR) that requires no special rights to become a part of the system.

And in the AUR, they have a mechanism which allows people to vote on packages, and the most popular can be included in the main repository. These feedback loops are great forcing functions to make sure the developers listen to users. Bugs can also be voted on, to help developers prioritize and make the system more democratic. Even though Arch is missing some of more refined social structures, it has its own.

I’ve also read about how some people left Arch because they had multiple breakages over time and lost trust. However, there are wiki pages that explain how to handle the situation and best practices for system stability. Almost always the problem is a mis-configured system, or a bug in the upstream code. I have two kernels installed, the mainline (3.12.3) and the LTS (3.10.22) version. So if a kernel upgrade breaks, I can always boot to a known-good one. Because Arch doesn’t have the resources of the big vendors to debug kernel problems, it can be nice to have an extra as backup.

That doesn’t solve many other ways the system can break before it finishes booting, but it is nice to have. It is a fact of life that it is very easy to break an OS. Death is the norm in this world. I’ve read stories of people who accidentally configured their computer to sleep in response to the system start event.

The community of Arch is large enough that users run into all kinds of problems that more slowly moving distros would never see. Of course, Arch isn’t large enough to debug every problem but they have the resources to fix the bugs that “everyone” runs into very quickly. There is safety in numbers.

Many of the hardest transitions in Arch are in the past. I expect it to be smooth sailing now. I can think of little ways to improve it, but given that it is running all the latest software, I have no reason to install anything else right now.

Arch is under the radar compared to Ubuntu, Fedora, Mint, Debian, Suse, etc. It gets less than 1% of the news that those others get. It doesn’t have benevolent dictators making hardware deals and memorable pronouncements. It doesn’t have billion-dollar companies supporting them. Somehow, they’ve managed to survive.

The free software community is still surprising to me. 50% of computer users know of Linux. 10% of them know Ubuntu. 10% of them know Debian. 10% of them run Arch. 1% of them contribute to Arch. There are many good distributions and working on them. When you consider it is only on 1% of desktops, it is astounding how many solid choices there already are. Arch is the secret anarchic distro that works perfectly.

