Submitted by Tom Burton-West on October 4, 2013

When we first started working on large scale search we confronted the issue of whether to index pages or complete books as our fundamental unit of indexing.[i] We had some concerns about indexing on the page level. We knew we would need to scale to 10-20 million books and at an average of 300 pages per book that comes out to about 6 billion pages. At that time we did not think that Solr would scale to 6 billion pages.[ii] If we indexed by page, we also wanted to be able