This week's Free exchange column discusses attempts to measure the consumer surplus generated by the internet (summary here). We are inviting experts in the field to comment on the piece and on related research. Our first contribution is from Yan Chen, a professor of information at the University's of Michigan's School of Information.

WITH the advent of the Web and search engines, online searching has become a common method for obtaining information. While Web searching is easy and convenient compared to going to a library, one might be concerned about the quality of the information obtained by those searches. Of the millions of search results, some may be inaccurate, or come from untrustworthy sources. While a searcher may save time by searching online, this time-saving may come at the expense of information quality.

My coauthors and I examine the time saved vs. quality tradeoff by comparing the processes and outcomes of Web searches in comparison with more traditional information searches using an academic library.

Our search queries and questions come from two different sources. First, we obtain a random sample of queries from a major search engine, which reflects the spectrum of information users seek online. As these queries might be biased towards searches more suitable for the Web, we also use a set of real reference questions users sent to librarians in the Internet Public Library (IPL). The mission of the IPL was to bridge the digital divide, i.e., to serve populations not proficient in information technology and lacking good local libraries. Thus, we expect that the IPL questions came from users who could not find answers from online searches. Compared to queries from the search engine, we expect the IPL questions to be less biased in favor of online searches. In sum, a combination of the two sources enables us to obtain a more balanced picture comparing Web and non-Web search outcomes.

As reported, for questions generated from search-engine queries, a Web search takes on average 7 minutes, whereas the corresponding offline search takes 22 minutes. For questions from the IPL, however, a Web search takes on average 9 minutes, whereas the offline search takes 19 minutes. Thus, search-engine questions lead to more time saved using the Web (15 minutes), whereas IPL questions generate a smaller margin in terms of time saved (10 minutes).

Next, we use trained raters to evaluate the quality of sources from the Web and non-Web searches. Perhaps the most surprising finding is that the overall Web source quality is not significantly different from that of non-Web sources when questions come from search-engine queries. Incidentally, the most frequently accessed URL by our searchers is the English Wikipedia. For IPL questions, however, non-Web sources are judged to be of significantly higher quality.

Therefore, while consumers save time without sacrificing quality by using Web search for most of their information needs, sometimes they obtain higher quality information from offline sources. Is the quality differential worth the extra time for offline searching? One hopes that the gap will eventually disappear once library contents are fully digitised and searchable online.