I read a lot of books, and these days quite a few of them are ebooks. I keep a list of books I’ve read, and I try to review a reasonable proportion of those when I get time. Jono Lange asked how I maintain this list, so I said I’d write it up.

First, books are in Calibre.

I keep the metadata about each book up to date there, and I write my reviews in the “Comments” section.

I then have a little script I run called calibre-dump-library . It looks like this:

1 2 3 4 5 6 7 #!/bin/bash calibredb export --all --dont-save-cover --dont-write-opf \ --formats = EPUB --to-dir = /home/aquarius/Documents/Books \ --template = "{authors}/{series:|(| }{series_index:0>2s| |) }{title}" python ~/bin/calibre-dump-comments.py > ~/Documents/Books/comments.json scp ~/Documents/Books/comments.json kryogenix.org:public_html/booklist/ python ~/bin/calibre-export-link-recent.py

This does four things:

Export the books from Calibre into an external folder

This is done in quite a careful way; my Documents/Books folder has all my books in it, organised by author. It’s synced elsewhere, and also available to my phone so I can fetch any book from my collection. The --template stuff is somewhat laboriously worked out so that The Name of the Wind by Patrick Rothfuss is in a folder named Patrick Rothfuss and is itself named (Kingkiller Chronicles 01) Name of the Wind, The.epub , so all the books in a series stay together and in order.

It uses calibredb , which is the command line tool to access, export, work with, and edit calibre’s database. Calibre does a pretty impressive job of making all its stuff available to external tools and scripts; you do not need to run the app itself to fiddle with the database. Here we’re using calibredb export to export a bunch of books to an external folder.

Export my reviews and some book metadata to a JSON file

I need the JSON file so I can drop it onto my website and build the booklist. So, this relatively trivial Python script runs calibredb list to dump the metadata contents out into a pipe-separated text list; we then parse that list in Python so that it’s sorted and any missing metadata is added, and then dump it as JSON.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 #!/usr/bin/env python import subprocess , json out = subprocess . check_output ([ "calibredb" , "list" , "-f" , "authors,title,comments,timestamp,series,series_index" , "--separator" , "|" , "-w" , "10000" ]) out = out . split ( "

" ) data = [] for line in out [ 1 :]: if line : ( bookid , authors , title , comments , timestamp , series , series_index , _ ) = [ x . strip () for x in line . split ( "|" )] if comments == "None" : comments = None if series != "None" and series_index : series_index = int ( float ( series_index )) anti_series_index = 999 - series_index else : series = None series_index = None anti_series_index = None if title : data . append ({ "authors" : authors , "title" : title , "comments" : comments , "timestamp" : timestamp , "bookid" : bookid , "series" : series , "series_index" : series_index , "ymd" : timestamp [: 10 ], "anti_series_index" : anti_series_index }) data . sort ( key = lambda k :( k [ "ymd" ], k [ "series" ], k [ "anti_series_index" ]), reverse = True ) print json . dumps ( data , indent = 2 )

Copy the JSON file to my website

The actual book list is a reasonably simple PHP script which reads the JSON file and displays the list of books, or displays a list of reviews if that’s what you picked. Not very complicated; if you’re desperate for the source, ask and I’ll send it over, but it’s really not hard.

Set up the recent list

It’s handy, when browsing my Books folder from my phone, to be able to see the books I added most recently to my list. So as a final step we create a folder called 000RECENT (so it is at the top of the list) and copy the ten most recently added books into it. This is also not complex stuff; walk through all the list, sort them in last-modified time order, pick the top 10, copy them.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 #!/usr/bin/env python import os , shutil RECENT = "/home/aquarius/Documents/Books/000RECENT" def getfiles ( path ): epubs = [] for root , dirs , files in os . walk ( path ): these = [ os . path . join ( root , f ) for f in files if f . endswith ( ".epub" ) and "000RECENT" not in root ] epubs += [( x , os . stat ( x ) . st_mtime ) for x in these ] return epubs files = getfiles ( "/home/aquarius/Documents/Books" ) files . sort ( cmp = lambda b , a : cmp ( a [ 1 ], b [ 1 ])) files = [ x [ 0 ] for x in files [: 9 ]] shutil . rmtree ( RECENT ) os . mkdir ( RECENT ) counter = 1 for source in files : target = os . path . join ( RECENT , str ( counter ) + ". " + source . split ( "/" )[ - 1 ]) shutil . copyfile ( source , target ) counter += 1 print "Recent files now are:" print "

" . join ([ x . split ( "/" )[ - 1 ] for x in files ])