Below are >13 case studies of difficult-to-find resources or citations, and how I went about locating them, demonstrating the various Internet search techniques described above and how to think about searches.

Missing Appendix: Anders Sandberg asked: Does anybody know where the online appendix to Nordhaus’ “Two Centuries of Productivity Growth in Computing” is hiding? I look up the title in Google Scholar; seeing a friendly psu.edu PDF link (CiteSeerx), I click. The paper says “The data used in this study are provided in a background spreadsheet available at http://www.econ.yale.edu/~nordhaus/Computers/Appendix.xls ”. Sadly, this is a lie. (Sandberg would of course have tried that.) I immediately check the URL in the IA—nothing. The IA didn’t catch it at all. Maybe the official published paper website has it? Nope, it references the same URL, and doesn’t provide a copy as an appendix or supplement. (What do we pay these publishers such enormous sums of money for, exactly?) So I back off to checking http://www.econ.yale.edu/~nordhaus/ , to check Nordhaus’s personal website for a newer link. The Yale personal website is empty and appears to’ve been replaced by a Google Sites personal page. It links nothing useful, so I check a more thorough index, Google, by searching site:sites.google.com/site/williamdnordhaus/ . Nothing there either (and it appears almost empty, so Nordhaus has allowed most of his stuff to be deleted and bitrot). I try a broader Google: nordhaus appendix.xls . This turns up some spreadsheets, but still nothing. Easier approaches having been exhausted, I return to the IA and I pull up all URLs archived for his original personal website: https://web.archive.org/web/*/http://www.econ.yale.edu/~nordhaus/* This pulls up way too many URLs to manually review, so I filter results for xls , which reduces to a more manageable 60 hits; reading through the hits, I spot http://www.econ.yale.edu:80/~nordhaus/homepage/documents/Appendix_Nordhaus_computation_update_121410.xlsx from 2014-10-10; this sounds right, albeit substantially later in time than expected (either 2010 or 2012, judging from the filename). Downloading it, opening it up and cross-referencing with the paper, it has the same spreadsheet ‘sheets’ as mentioned, like “Manual” or “Capital_Deep”, and seems to be either the original file in question or an updated version thereof (which may be even better). The spreadsheet metadata indicates it was created “04/09/2001, 23:20:43, ITS Academic Media & Technology”, and modified “12/22/2010, 02:40:20”, so it seems to be the latter—it’s the original spreadsheet Nordhaus created when he began work several years prior to the formal 2007 publication (6 years seems reasonable given all the delays in such a process), and then was updated 3 years afterwards. Close enough.

Misremembered Book: A Redditor asked: I was in a consignment type store once and picked up a book called “Eat fat, get thin”. Giving it a quick scan through, it was basically the same stuff as Atkins but this book was from the 50s or 60s. I wish I’d have bought it. I think I found a reference to it once online but it’s been drowned out since someone else released a book with the same name (and it wasn’t Barry Groves either). The easiest way to find a book given a corrupted title, a date range, and the information there are many similar titles drowning out a naive search engine query, is to skip to a specialized search engine with clean metadata (ie. a library database). Searching in WorldCat for 1950s–1970s, “Eat fat, get thin” turns up nothing relevant. This is unsurprising, as he was unlikely to’ve remembered the title exactly, and this title doesn’t quite sound right for the era anyway (a little too punchy and ungrammatical, and ‘thin’ wasn’t a desirable word back then compared to words like ‘slim’ or ‘sleek’ or ‘svelte’). People often oversimplify titles, so I dropped back to just “Eat fat”. This immediately turned up the book: Richard Mackarness’s 1958 Eat Fat and Grow Slim—note that it is almost the same title, with a comma serving as conjunction and ‘slim’ rather than the more contemporary ‘thin’, but just different enough to screw up an overly-literal search. With the same trick in mind, we could also have found it in a regular Google search query by adding additional terms to hint to Google that we want old books, not recent ones: both "Eat Fat" 1950s or "Eat Fat" 1960s would have turned it up in the first 5 search results. If we didn’t use quotes, the searches get harder because broader hits get pulled in. For example, Eat fat, get thin 1950s -Hyman excludes the recent book mentioned, but you still have to go down 15 hits before finding Mackarness, and Eat fat, get thin -Hyman requires going down 18 hits.

Missing Website: Bučar et al 2015, on the phenomenon of disappearing polymorphs quotes striking transcripts from a major example of a disappearing crystal, when ~1998 Abbott suddenly became unable to manufacture the anti-retroviral drug ritonavir (Norvir™) due to a rival (and less effective) crystal form spontaneously infecting all its plants, threatening many AIDS patients, but notes: The transcripts were originally published on the website42 of the International Association of Physicians in AIDS Care [IAPAC], but no longer appear there. A search using the quotes confirms that the originals have long since vanished from the open Internet, turning up only quotes of the quotations. Unfortunately, no URL is given. The Internet Archive has comprehensive mirrors of the IAPAC, but too many to easily search through. Using the filter feature, I keyword-searched for “ritonavir”, but while this turned up a number of pages from roughly the right time period, they do not mention it and none of the quotes appear. The key turned out to be to use the trademark name instead which pulls up many more pages, and after checking a few, the IAPAC turned out to have organized all the Norvir material into a single subdirectory with a convenient index.html ; the articles/transcripts, in turn, were indexed under the linked “Description of the Problem” index page. I then pulled the Norvir subdirectory with a ~/.gem/ruby/2.5.0/bin/wayback_machine_downloader wayback_machine_downloader 'http://www.iapac.org/norvir/' command and hosted a mirror to make it visible in Google.

Speech → Book: Nancy Lebovitz asked about a citation in a Roy Baumeister speech about sex differences: There’s an idea I’ve seen a number of times that 80% of women have had descendants, but only 40% of men. A little research tracked it back to this, but the speech doesn’t have a cite and I haven’t found a source. This could be solved by guessing that the formal citation is given in the book, and doing keyword search to find a similar passage. The second line of the speech says: For more information on this topic, read Dr. Baumeister’s book Is There Anything Good About Men? available in bookstores everywhere, including here. A search of Is There Anything Good About Men in Libgen turns up a copy. Download. What are we looking for? A reminder, the key lines in the speech are: …It’s not a trick question, and it’s not 50%. True, about half the people who ever lived were women, but that’s not the question. We’re asking about all the people who ever lived who have a descendant living today. Or, put another way, yes, every baby has both a mother and a father, but some of those parents had multiple children. Recent research using DNA analysis answered this question about two years ago. Today’s human population is descended from twice as many women as men. I think this difference is the single most under-appreciated fact about gender. To get that kind of difference, you had to have something like, throughout the entire history of the human race, maybe 80% of women but only 40% of men reproduced. We could search for various words or phrase from this passage which seem to be relatively unique; as it happens, I chose the rhetorical “50%” (but “80%”, “40%”, “underappreciated”, etc all would’ve worked with varying levels of efficiency since the speech is heavily based on the book), and thus jumped straight to chapter 4, “The Most Underappreciated Fact About Men”. (If these had not worked, we could have started searching for years, based on the quote “about two years ago”.) A glance tells us that Baumeister is discussing exactly this topic of reproductive differentials, so we read on and a few pages later, on page 63, we hit the jackpot: The correct answer has recently begun to emerge from DNA studies, notably those by Jason Wilder and his colleagues. They concluded that among the ancestors of today’s human population, women outnumbered men about two to one. Two to one! In percentage terms, then, humanity’s ancestors were about 67% female and 33% male. Who’s Wilder? A C-f for “Wilder” takes us to pg286, where we immediately read: …The DNA studies on how today’s human population is descended from twice as many women as men have been the most requested sources from my earlier talks on this. The work is by Jason Wilder and his colleagues. I list here some sources in the mass media, which may be more accessible to laypersons than the highly technical journal articles, but for the specialists I list those also. For a highly readable introduction, you can Google the article “Ancient Man Spread the Love Around,” which was published September, 20, 2004 and is still available (last I checked) online. There were plenty of other stories in the media at about this time, when the research findings first came out. In “Medical News Today,”, on the same date in 2004, a story under “Genes expose secrets of sex on the side” covered much the same material. If you want the original sources, read Wilder, J. A., Mobasher, Z., & Hammer, M. F. (2004). “Genetic evidence for unequal effective population sizes of human females and males”. Molecular Biology and Evolution, 21, 2047–2057. If that went down well, you might try Wilder, J. A., Kingan, S. B., Mobasher, Z., Pilkington, M. M., & Hammer, M. F. (2004). “Global patterns of human mitochondrial DNA and Y-chromosome structure are not influenced by higher migration rates of females versus males”. Nature Genetics, 36, 1122–1125. That one was over my head, I admit. A more readable source on these is Shriver, M. D. (2005), “Female migration rate might not be greater than male rate”. European Journal of Human Genetics, 13, 131–132. Shriver raises another intriguing hypothesis that could have contributed to the greater preponderance of females in our ancestors: Because couples mate such that the man is older, the generational intervals are smaller for females (i.e., baby’s age is closer to mother’s than to father’s). As for the 90% to 20% differential in other species, that I believe is standard information in biology, which I first heard in one of the lectures on testosterone by the late James Dabbs, whose book Heroes, Rogues, and Lovers remains an authoritative source on the topic. Wilder et al 2004, incidentally, fits well with Baumeister remarking in 2007 that the research was done 2 or so years ago. And of course you could’ve done the same thing using Google Books: search “Baumeister anything good about men” to get to the book, then search-within-the-book for “50%”, jump to page 53, read to page 63, do a second search-within-the-book for “Wilder” and the second hit of page 287 even luckily gives you the snippet: Sources and References 287 …If you want the original sources, read Wilder, J. A., Mobasher, Z., & Hammer, M. F. (2004). “Genetic evidence for unequal effective population sizes of human females and males”. Molecular Biology and Evolution…

Connotations a commenter who shall remain nameless wrote I challenge you to find an example of someone saying “this den of X” where X does not have a negative connotation. I found a positive connotation within 5s using my Google hotkey for "this den of " , and, curious about further ones, found additional uses of the phrase in regard to dealing with rattlesnakes in Google Books.

Rowling Quote On Death: Did J.K. Rowling say the Harry Potter books were about ‘death’? There are a lot of Rowling statements, but checking WP and opening up each interview links (under the theory that the key interviews are linked there) and searching for ‘death’ soon turns up a relevant quote from 2001: Death is an extremely important theme throughout all seven books. I would say possibly the most important theme. If you are writing about Evil, which I am, and if you are writing about someone who is essentially a psychopath, you have a duty to show the real evil of taking human life.

Crowley Quote: Scott Alexander posted a piece linking to an except titled “Crowley on Religious Experience”. The link was broken, but Alexander brought it up in the context of an earlier discussion where he also quoted Crowley; searching those quotes reveals that it must have been excerpts from Magick: Book 4

Finding The Right ‘SAGE’: Phil Goetz noted that an anti-aging conference named “SAGE” had become impossible to find in Google due to a LGBT aging conference also named SAGE. Regular searches would fail, but a combination of tricks worked: SAGE anti-aging conference combined with restricting Google search to 2003–2005 time-range turned up a citation to its website as the fourth hit, http://www.sagecrossroads.net (which has ironically since died).

UK Charity Financials: The Future of Humanity Institute (FHI) doesn’t clearly provide charity financial forms akin to the US Form 990s, making it hard to find out information about its budget or results. FHI doesn’t show up in the CC, NPC, or GuideStar, which are the first places to check for charity finances, so I went a little broader afield and tried a site search on the FHI website: budget site:fhi.ox.ac.uk . This immediately turned up FHI’s own documentation of its activities and budgets, such as the 2007 annual report; I used part of its title as a new Google search: future of humanity institute achievements report site:fhi.ox.ac.uk .

Nobel Lineage Research: John Maxwell referred to a forgotten study on high correlation between Nobelist professors & Nobelist grad students (almost entirely a selection effect, I would bet). I was able to refind it in 7 minutes. I wasted a few searches like factor predicting Nobel prize or Nobel prize graduate student in Google Scholar, until I search for Nobel laureate "graduate student" ; the second hit was a citation, which is a little unusual for Google Scholar and meant it was important, and it had the critical word mutual in it—simultaneous partners in Nobel work is somewhat rare, but temporally separated teams don’t work for prizes, and I suspected that it was exactly what I was looking for. Googling the title, I soon found a PDF like “Eminent Scientists’ Demotivation in School: A symptom of an incurable disease?”, Viau 2004 which confirmed it (and Viau 2004 is interesting in its own right as a contribution to the Conscientious vs IQ question). I then followed it to a useful paragraph: In a study conducted with 92 American winners of the Nobel Prize, Zuckerman (1977) discovered that 48 of them had worked as graduate students or assistants with professors who were themselves Nobel Prize award-winners. As pointed out by Zuckerman (1977), the fact that 11 Nobel prizewinners have had the great physicist Rutherford as a mentor is an example of just how significant a good mentor can be during one’s studies and training. It then appears that most eminent scientists did have people to stimulate them during their childhood and mentor(s) during their studies. But, what exactly is the nature of these people’s contribution. Zuckerman, H. (1977). Scientific Elite: Nobel Laureates in the United States. New York: Free Press. GS lists >900 citations of this book, so there may well be additional or followup studies covering the 40 years since. Or, also relevant is “Zuckerman, H. (1983). The scientific elite: Nobel laureates’ mutual influences. In R. S. Albert (Ed.), Genius and eminence (pp. 241–252). New York: Pergamon Press”, and “Zuckerman H. ‘Sociology of Nobel Prizes’, Scientific American 217 (5): 25& 1967.”

Too Narrow: A failure case study: The_Duck looked for but failed to find other uses of a famous Wittgenstein anecdote. His mistake was being too specific: Yes, clearly my Google-fu is lacking. I think I searched for phrases like “sun went around the Earth,” which fails because your quote has “sun went round the Earth.” As discussed in the search tips, when you’re formulating a search, you want to balance how many hits you get, aiming for a sweet spot of a few hundred high-quality hits to review—the broader your formulation, the more likely the hits will include your target (if it exists) but the more hits you’ll return. In The_Duck’s case, he used an overly-specific search, which would turn up only 2 hits at most; this should have been a hint to loosen the search, such as by dropping quotes or dropping keywords. In this case, my reasoning would go something like this, laid out explicitly: ‘“Wittgenstein” is almost guaranteed to be on the same page as any instance of this quote, since the quote is about Wittgenstein; LW, however, doesn’t discuss Wittgenstein much, so there won’t be many hits in the first place; to find this quote, I only need to narrow down those hits a little, and after “Wittgenstein”, the most fundamental core word to this quote is “Earth” or “sun”, so I’ll toss one of them in and… ah, there’s the quote!’ If I were searching the general Internet, my reasoning would go more like “‘Wittgenstein’ will be on, like, a million websites; I need to narrow that down a lot to hope to find it; so maybe ‘Wittgenstein’ and ‘Earth’ and ‘Sun’… nope, nothing on the first page, so toss in 'goes around' OR 'go around' —ah there it is!” (Actually, for the general Internet, just Wittgenstein earth sun turns up a first page mostly about this anecdote, several of which include all the details one could need.)

Dead URL: A link to a research article in a post by Morendil broke, he had not provided any formal citation data, and the original domain blocks all crawlers in its robots.txt so IA would not work. What to do? The simplest solution was to search a direct quote, turning up a Scribd mirror; Scribd is a parasite website, where people upload copies from elsewhere, which ought to make one wonder where the original came from. (It often shows up before the original in any search engine, because it automatically runs OCR on submissions, making them more visible to search engines.) With a copy of the journal issue to work with, you can easily find the official HP archives and download the original PDF. If that hadn’t worked, searching for the URL without /pg_2/ in it yields the full citation, and then that can be looked up normally. Finally, somewhat more dangerous would be trying to find the article just by author surname & year.

Description But No Citation: A 2013 Medical Daily on the effects of reading fiction omitted any link or citation to the research in question. But it is easy to find. The article says the authors are one Kaufman & Libby, and implies it was published in the last year. So: go to Google Scholar, punch in Kaufman Libby , limit to ‘Since 2012’; and the correct paper (“Changing beliefs and behavior through experience-taking”) is the first hit with fulltext available on the right-hand side as the text link “[PDF] from tiltfactor.org ” & many other domains.

Finding Followups: Is soy milk bad for you as one study suggests? Has anyone replicated it? This is easy to look into a little if you use the power of reverse citation search! Plug Brain aging and midlife tofu consumption into Google Scholar, one of the little links under the first hit points to “Cited by 176”; if you click on that, you can hit a checkbox for “Search within citing articles”; then you can search a query like experiment OR randomized OR blind which yields 121 results. The first result shows no negative effect and a trend to a benefit, the second is inaccessible, the second & third are reviews whose abstract suggests it would argue for benefits, and the fourth discusses sleep & mood benefits to soy diets. At least from a quick skim, this claim is not replicating, and I am dubious about it.