ccache, the compiler cache, is a fantastic way to speed up build times for C and C++ code that I previously recommended. Recently, I was playing around with trying to get it to speed up my Linux kernel builds, but wasn’t seeing any benefit. Usually when this happens with ccache, there’s something non-deterministic about the builds that prevents cache hits.

Turns out someone asked this exact question on the ccache mailing list back in 2014, and a teammate from my Android days supposed a timestamp was the culprit. That, and this LKML post from the KBUILD maintainer in 2011 about determinism helped me find commit 87c94bfb8ad35 (“kbuild: override build timestamp & version”) that introduced manually overriding part of the version string that contains the build timestamp that can be seen from:

1 2 3 4 5 $ cat /proc/version Linux version 4.15.0-13-generic ( buildd@lgw01-amd64-028 ) ( gcc version 5.4.0 20160609 ( Ubuntu 5.4.0-6ubuntu1~16.04.9 )) #14~16.04.1-Ubuntu SMP Sat Mar 17 03:04:59 UTC 2018

With ccache, we can check the cache hit/miss stats with -s , clear the cache with -C , and clear the stats with -z . We can tell ccache explicitly which compiler to fall back to as its first argument (not strictly necessary). For KBUILD, we can swap our compiler by using CC= arg.

Let’s see what happens to our build time for subsequent builds with a hot cache:

No Cache

1 2 3 4 $ make clean $ time make -j4 ... make -j4 2008.93s user 231.69s system 346% cpu 10:47.07 total

Cold Cache

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 $ ccache -Cz Cleared cache Statistics cleared $ ccache -s cache directory /home/nick/.ccache primary config /home/nick/.ccache/ccache.conf secondary config ( readonly ) /etc/ccache.conf cache hit ( direct ) 0 cache hit ( preprocessed ) 0 cache miss 0 cache hit rate 0.00 % cleanups performed 0 files in cache 0 cache size 0.0 kB max cache size 5.0 GB $ make clean $ time KBUILD_BUILD_TIMESTAMP = '' make CC = "ccache gcc" -j4 ... KBUILD_BUILD_TIMESTAMP = '' make CC = "ccache gcc" -j4 2426.79s user 312.08s system 372% cpu 12:15.22 total $ ccache -s cache directory /home/nick/.ccache primary config /home/nick/.ccache/ccache.conf secondary config ( readonly ) /etc/ccache.conf cache hit ( direct ) 0 cache hit ( preprocessed ) 0 cache miss 3242 called for link 6 called for preprocessing 538 unsupported source language 66 no input file 108 files in cache 9720 cache size 432.6 MB max cache size 5.0 GB

Hot Cache

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 $ ccache -z Statistics cleared $ make clean $ time KBUILD_BUILD_TIMESTAMP = '' make CC = "ccache gcc" -j4 ... KBUILD_BUILD_TIMESTAMP = '' make CC = "ccache gcc" -j4 151.85s user 132.98s system 288% cpu 1:38.90 total $ ccache -s cache directory /home/nick/.ccache primary config /home/nick/.ccache/ccache.conf secondary config ( readonly ) /etc/ccache.conf cache hit ( direct ) 3232 cache hit ( preprocessed ) 7 cache miss 3 called for link 6 called for preprocessing 538 unsupported source language 66 no input file 108 files in cache 9734 cache size 432.8 MB max cache size 5.0 GB

The initial cold cache build will be slower than not using ccache at all, but it’s a one time cost that’s not significant relative to the savings. No caching took 647.07s, initial cold cache build took 735.22s (13.62% slower), and subsequent hot cache builds took 98.9s (6.54x faster). YMMV based on CPU and disk performance. Also, it’s not the most common workflow to do clean builds, but we do this for Linux kernel builds for Android/Pixel at work, and this helps me significantly for local development.

Now, if you really need that date string in there, you theoretically could put some garbage value in there (for the cache) long enough to save enough space for a date string, then patch your vmlinux binary after the fact. I don’t recommend that, but I would imagine that might look something like:

1 2 3 $ KBUILD_BUILD_TIMESTAMP = $( printf "y%0.s" $( seq 1 $( date | wc -c ))) $ make CC = "ccache gcc" vmlinux $ sed -i "s/$(KBUILD_BUILD_TIMESTAMP)/$(date)/g" vmlinux

Deterministic builds make build caching easier, but I’m not sure that build timestamp strings and build determinism are reconcileable.