Do you have dead code lying around? Of course you do. But you know from experience that even if some code looks dead, blindly removing it can cause real problems in production. What’s the solution? Tombstones.

We first heard of this idea by coming across this presentation by David Schnepper from Box. It’s a snappy five minute long video, and I highly recommend viewing it before reading the rest of this post.

This talk inspired us to implement it in Perl. Here’s how it works: every piece of code that you suspect is dead, you mark with a tombstone like this:

package Example; use Lokku::Tombstone qw(tombstone); sub example_possibly_dead_code { tombstone('2015-04-02', 'davidl'); # possibly dead code follows: ... }

The tombstone subroutine simply logs that it has been called, and that is all that it does. It attempts to be as fast as possible to avoid slowing production systems down if called in a tight loop. It has several safeguards, such as failing silently if the log file is getting too big.

Here’s the the tombstone subroutine, interspersed with explanatory text:

sub tombstone { my $yyyy_mm_dd_of_tombstone = shift; my $author = shift;

The two arguments passed to tombstone are just simple data about the tombstone: who added it and when. They come in useful to generate reports about the tombstone. We could have hooked into Git to pull this information out automatically instead, but I find quite helpful to have this information right there in the code in front of you.

make_path("$ENV{LOKKU_COMMON}/tombstone/vampires", { err => \my $ra_err }); if ($ra_err && @$ra_err) { for my $error (@$ra_err) { warn "Error making path $ENV{LOKKU_COMMON}/tombstone/vampires: $error"; } return; } my @now = gmtime(); my $filename = "$ENV{LOKKU_COMMON}/tombstone/vampires/vampires_" . strftime("%F", @now) . "_${USERNAME}_${yyyy_mm_dd_of_tombstone}_${author}.log";

tombstone logs to a local file named with the date, process username and tombstone ID. This will make reading and cleaning up the files later much simpler.

As you can see from the directory name, tombstoned code which is in fact not dead is referred to as a “vampire” :-)

Having a separate file per tombstone() call allows each tombstone to have a separate disk space quota, as I’ll explain now.

open my $fh, ">>:encoding(UTF-8)", $filename or do { warn "Could not open '$filename' $!"; return; }; my $size_in_bytes = tell($fh); if ($size_in_bytes == -1) { warn "tell failed: $!"; return; } elsif ($size_in_bytes > $SIZE_IN_BYTES_THRESHOLD) { return; }

It’s important we don’t accidentally fill up the disk, so each file has a maximum size as a simple disk quota mechanism. Excessive events simply get dropped - knowing whether a vampire was called 100 times or 1,000,000 times isn’t that important.

At any point, if an error occurs, the subroutine simply returns instead of throwing an exception to avoid breaking the functionality of the calling code.

flock($fh, LOCK_EX | LOCK_NB) or return; # non-blocking

Having several processes writing to a file concurrently can be tricky, so here we use a Linux file lock to avoid race conditions. We call it in non-blocking mode, so that if a process cannot grab the lock, the subroutine simply returns rather than hanging or throwing an exception.

# OK, using JSON::XS may slow this subroutine down a little bit, but it makes # parsing so much easier, so forgive me. state $JSON = JSON::XS->new; my $StackTrace = Devel::StackTrace->new(skip_frames => 1, no_args => 1); my $FirstFrame = $StackTrace->frame(0); my $SecondFrame = $StackTrace->frame(1) || $StackTrace->frame(0); my $rh_message = { gmtime => [@now], stack_trace => $StackTrace->as_string(), author => $author, yyyy_mm_dd_of_tombstone => $yyyy_mm_dd_of_tombstone, filename => $FirstFrame->filename, line => $FirstFrame->line, subroutine => $SecondFrame->subroutine, }; my $eval_rv = eval { my $string = $JSON->encode($rh_message); say $fh $string or warn "Could not print to '$filename': $!"; 1; }; if (! $eval_rv || $@) { warn $@; }

The log line we write to the file actually contains quite a bit of data. It contains the tombstone ID (date and author), a complete stack trace, the time it was called, and where it was called. All this data is very useful when looking at reports later: for instance, it’s good to know what the different stack traces are to be able to cleanly refactor or delete the code if we wanted to.

# close will unlock the file: close $fh or warn "Could not close file '$filename': $!"; return; }

And that’s the end of the subroutine!

We have a cronjob that regularly copies over all the log files to a central location, and another cronjob that deletes old log files. This code is very specific to our infrastrucure, so forgive us for not sharing it or releasing it on CPAN.

The final part is a web page that displays the results.

Since we’ve started using this technique, it has proved to be quite useful. We’ve deleted a ton of genuinely dead code, and we’ve managed to avoid deleting code that we thought was dead but wasn’t. I highly recommend it.

David Lowe