Progress on kernel module signing in Fedora 18 [Sep. 18th, 2012|03:11 pm] jwboyer

There was a brief module signing session held at the 2012 Kernel Summit/Linux Plumbers conference a few weeks ago. As usual, LWN has great coverage of it so head over there for the details. As a result of this, module signing is going to get into the upstream kernel one way or another. That is great news. The bad news is that the implementation isn't going to be what we've been carrying in Fedora 18 thus far (and RHEL for quite some time). I spent some time last week reworking how the Fedora kernel handles modules signing as a result, so I thought I'd describe it briefly (or not so briefly) here.



David Howells has been leading the way for Red Hat on module signing for some time now. We've been using his patchset that embeds a signature in an ELF note in F18 since we branched and up to the F18 Alpha kernel. That has worked well for us and allowed to build on top of it for our Secure Boot patches. Knowing that Rusty Russell (the modules maintainer) found this to be overly complicated, David has been working on a 'modsign-rusty' branch at the same time. This is extremely convenient for me. I just needed to pick up that branch instead of the one we've been using and call it good, right? Well, sort of.



The ELF note variant was created in part because it is the most robust against preserving the signature in the module. Utilities such as strip and eu-strip won't remove it, which allows quite a bit of post-processing of modules while still making them functional in a kernel that expects them to be signed. Rusty's preferred method of dealing with signatures is to just stick them at the end of the module binary. That does indeed make the code much simpler, but it's very easy for that signature to get stripped off during post-processing. The "solution" to that is to process the modules all you want, then sign them. Or you can sign and install them unprocessed, then process them into new copies and sign those. Repeat for various combinations thereof.



In a distro kernel, we don't really want to ship multiple copies of the modules. Our kernels are already bloated enough. So in order to make this work, we need to sign the modules after post-processing. David's modsign-rusty branch did this by building the modules, generating the signature, stripping it maximally while keeping it loadable, and then signing that. So e.g. hfs.ko would be a stripped, signed module. He also left the unsigned, unstripped copy as hfs.ko.unsigned. Then 'make modules_install' would install the signed modules and you'd be all set. That works well for people building their own kernels by hand, but it doesn't work well for a Fedora distro kernel. Why? One word: debuginfo.



Our kernel builds have some craziness going on that generates appropriate debuginfo sub-packages for each kernel variant (PAE, debug, PAEdebug, etc). It's really no different than how other packages in Fedora work. Binaries are generated, the RPM find-debuginfo script runs through them and strips out the debug information and puts it into a .debug file, and creates appropriate ELF notes to point to them correctly so GDB can find the debug symbols. Except it does this on the installed binaries (modules) after the %install section in the RPM spec file. That means that 'make modules_install' has already been run by the time RPM gets around to creating the debuginfo stuff. With the modsign-rusty branch, that essentially meant that find-debuginfo was running on already stripped modules which led to two things. The first was useless debuginfo RPMs and the second was useless modules because it stripped the signature off anyway.



So the easy solution was to just disable debuginfo, right? Well, yes. But sometimes the easy solution isn't the best (though I could rant about debuginfo for a good long while). Instead, I had to figure out how to make appended module signatures and debuginfo play nice together. After trying a few "magic bullet" solutions that didn't pan out, I sat down and basically replaced the patch that did the actual module signing in the modsign-rusty branch with something else. It creates a 'make modules_sign' Kbuild make target that is intended to be called after 'make modules_install' is. That is, it operates on the installed module tree. An invocation looks something like this:



make ARCH=x86_64 V=1 INSTALL_MOD_PATH=/home/jwboyer/rpmbuild/B UILDROOT/kernel-3.6.0-0.rc6.git0.4.fc18.x 86_64 modules_sign KERNELRELEASE=3.6.0-0.rc6.git0.4.fc18.x8 6_64



That will go through and create a signed copy of any module built and installed in INSTALL_MOD_PATH. There is nothing particularly clever about this.



Now, I know you're asking yourself "How does that help at all? You said find-debuginfo runs after the %install section." Indeed it does. Except we don't even install the modules in %install anyway, we do that in %build because of our variants. The kernel.spec file is ... convoluted. Read it if you want a lesson in how complicated you can make RPM some time.



Anyway, I figured if find-debuginfo can run after %install then I can just let it run and then call the new modules_sign target after that. Turns out that was pretty easy to do. The kernel.spec file already overloaded the RPM macro responsible for debuginfo generation, so I just added on to it a bit. Essentially, we let find-debuginfo go ahead and finish it's processing of the files and the modules_sign invocation signs those processed modules and appends the signature. We get properly signed modules, working debuginfo, and ponies. Yay. OK, no ponies, but they poop a lot and are generally useless anyway.



There are some tweaks required because of our kernel variants handling and the modules-extra subpackage, but I'm going to gloss over those. The basic method works as described above. It is a slight departure from other Kbuild make targets in that none of them modify things outside of the kernel build tree to my knowledge. However, there are good reasons it does this as already described.



Is it fool-proof? No. A user (or utility) could still run strip on one of these installed signed modules and the signature would be removed. There isn't much we can do about that though, and in the Fedora case it won't be common for people to be mucking around with them at all.



So are we done? Also no! There are a few things remaining. The first is that I haven't actually committed that code yet. I sent the patches off to the Fedora kernel list and they'll sit there for a few days before I commit them. Also, while David may be somewhat clairvoyant with his modsign-rusty branch, it isn't exactly what the final code will look like. That is mostly because Rusty is leaving all of the signature verification stuff up to whoever wants to deal with that by providing a hook, and what that will look like isn't settled yet. There's talk of switching from PGP keys to X.509 certs, some discussion on how keys are handled, etc. We also need to adapt the code to import certs/hashes/keys from the UEFI DB variable for Secure Boot. However, what we have now is fairly close to the overall design of what should wind up in mainline at some point.



The main reason I did this before everything was completed upstream is because of third party repositories that provide their own kernel modules. I know, you are shocked that I care at all. See how nice I can be? Anyway, Fedora clearly isn't going to be signing out of tree modules. In a Secure Boot world, that means that those repositories are going to need to provide a public key somewhere that a user can import into the UEFI DB and then sign their modules. Since F18 is currently on target to support Secure Boot and people really like their kernel modules for some reason, I wanted to get Fedora as close as I could to what will wind up in upstream. That will give packagers time to look at how to integrate module signing into their build process before F18 ships, even if some further changes are needed. I'll continue to refine things as they develop upstream, but it's certainly better than shipping F18 with an implementation that nobody will be able to use.



Stay tuned for developments. If you have questions, let me know!