I wrote last month about the challenges of obtaining Linux symbols based on a build ID. I’m now pleased to say that getting Linux symbols based on a build ID is trivial on both Ubuntu and Fedora – things are looking up.

Here’s a recap of the posts on this topic to date:

Ubuntu

In my last post I described the solution that I created for Ubuntu – an enhanced Packages file that lists all the build IDs found in each package on Ubuntu’s site. This makes finding the URL for the package that you need trivial (grep), and then you can download and unpack the symbols (no installation required) from any version of Linux, as described in the Installation is optional section of my second post. I’ve updated my enhanced packages file and I’ll try to continue doing that every week or two. See the last post for details.

Here’s an example, for the build ID 04d4f3f1d378a3ba45195f8102e2145a0115714b. Beware that these steps lack any error checking and can easily go wrong, so understand what the steps do.

$ export buildid=04d4f3f1d378a3ba45195f8102e2145a0115714b

$ grep $buildid PackagesEnhanced BuildID: 04d4f3f1d378a3ba45195f8102e2145a0115714b /usr/lib/debug/usr/bin/zenity http://ddebs.ubuntu.com/pool/main/z/zenity/zenity-dbgsym_3.4.0-0ubuntu4_i386.ddeb

The download URL is the fourth field in the enhanced packages file so we can get all neck beardy and do a one-liner for the download:

$ wget $(grep $buildid PackagesEnhanced | cut -d ” ” -f 4) Saving to: `zenity-dbgsym_3.4.0-0ubuntu4_i386.ddeb’

Now it’s the familiar unpack dance. Adjust to your tastes.

$ ar -x zenity-dbgsym_3.4.0-0ubuntu4_i386.ddeb

$ rm -rf contents && mkdir contents && cd contents

$ tar -xf ../data.tar.*z

That’s it. We now have the debug information we were looking for:

$ readelf -n usr/lib/debug/usr/bin/zenity …

Build ID: 04d4f3f1d378a3ba45195f8102e2145a0115714b

Fedora

I reported last time that Fedora had a web server that would convert build IDs into package details, but that it was too far out of date to be useful. Well, it’s up to date now and they plan to keep it that way!

You can find everything you need to know about how to use it at https://darkserver.fedoraproject.org, but I’ll run through a sample anyway. Let’s start by setting buildid to the problematic build ID of the libc-2.16.so file I tried finding last time.

$ export buildid=cf7bdd994de74c7d4a0cff6a0293d96b64681e06

This command will grab a blob of data that describes the package containing that build ID:

Reading through the blob shows the package name as being glibc-debuginfo-2.16-28.fc18.i686 and another wget retrieves information about that package:

The package URL is easy to find in this data so we retrieve the package and, as described previously, unpack it:

$ wget http://koji.fedoraproject.org/packages/glibc/2.16/28.fc18/i686/glibc-deb-2.16-28.fc18.i686.rpm

$ rm -rf contents && mkdir contents && cd contents

$ rpm2cpio ../glibc-debuginfo-2.16-28.fc18.i686.rpm | cpio –idmv

$ readelf -n usr/lib/debug/lib/libc.so.6.debug Build ID: cf7bdd994de74c7d4a0cff6a0293d96b64681e06

Obviously a bit of scripting and parsing could automate this quite nicely, but I leave that as an exercise for the reader. Or take a look at darkclient, for all your darkserver automation needs.

I haven’t figured out how to use darkserver with a breakpad build ID (32 characters instead of 40) but I’ve discussed it with the darkserver developers and they sound interested in handling that case as well.

The darkserver developers are also planning to support Ubuntu at some point. So some day darkserver could provide one-stop shopping for finding symbols from many Linux distributions, thus fulfilling the original dream of build IDs!

Questions? Problems? Suggestions? Join the mailing list here:

Share your experiences here, or on the dark server mailing list.

In closing

Thanks to Kushal for making darkserver work.

Two distributions down. There can’t be very many more, can there…