When the Freedom of Information Act was enacted in 1966, it was envisioned as a tool for journalists to facilitate government oversight and accountability. Although the FOIA is still generally thought of in this way, inextricably linked to the news media’s role as government watchdog, this view bears little resemblance to the reality of how FOIA is used today.

Nowadays, journalists account for only a small share of FOIA requests (just 7.6% by the estimate below). Since it was enacted, the FOIA’s user base has evolved to encompass a diverse ecosystem of organizations, entities, and individuals who use FOIA for a wide variety of reasons. Its users include lawyers, nonprofits, academic researchers, hospitals, political committees, hedge funds, government agencies, private individuals, and many others.

However, beyond knowing that these use cases exist, “[v]ery little is known about who uses the FOIA or how it is used.” The information needed for such a study is readily available in an agency’s FOIA logs. But analyzing those FOIA logs can be such an onerous task (more below), a comprehensive study on the use of FOIA has never been done.

Drawing on FOIA Mapper’s database of FOIA logs, here is my attempt at addressing these questions: who uses FOIA and why? The results are not comprehensive, but with a sample size of 229,000 requests from 85 agencies, it is the largest analysis on the use of FOIA to date.



Summary

Notes: Data for the chart comes from FOIA Mapper’s database of FOIA logs (searchable here). It includes only agencies who report both the name and the organization of the requester (85 out of 142 agencies in the database). The span of time covered by the data varies by agency, with an average time period of 3 years. The organizations were categorized manually, as described below.

“Businesses” includes all for-profit commercial entities, excluding law firms and news media. This definition is different from the “commercial use” fee designation, which also includes some organizations that are not strictly for-profit, such as labor unions and political action committees.

“Law firms” are commercial businesses, but due to their highly-specific use case (obtaining courtroom evidence) and the sheer size of the group, I thought they deserved to be broken out separately due.

“News media” also uses a narrower definition from the one used by government agencies. Here an organization is considered news media if its primary purpose is to make the information available to the public. This category includes independent blogs as well as major media outlets. But unlike the “news media” fee designation used by government agencies, it does not include research companies like Disclosure Insight, Inc., which has a blog on its website but uses the information it obtains through FOIA primarily for its paid research.

“Noncommercial” is a catchall category that includes four different groups: nonprofits, political committees, government agencies, and the offices of elected officials.

“Universities” includes both academic and non-academic entities within a university (e.g. investment office, university hospital, etc).

“Individual” requests are those that were not explicitly requested on behalf of an organization.

“Uncategorized” includes requests for which the category of the requester has not been determined. Scroll to the bottom for more information about how this data was compiled.

Aggregate breakdown

Here is the combined distribution for these 85 agencies.



*The time period of FOIA requests in the database differs by agency. To adjust for these time period differences, each agency’s FOIA request count has been normalized to one year. For example, the data includes 11 years of requests from the SEC. To prevent these requests from having a disproportionately large impact on the aggregate summary, each SEC request has been factored down by assigning it a weight of 1/11.

In total, there are about 300 different agencies in the federal government. Given how much the requester profile varies from agency-to-agency (as shown in the chart at the top), this distribution may not be representative of the federal government as a whole.

Although journalists are the most well-known users of FOIA, they are far from the biggest. Consistent with what other studies have found, the news media makes up only a small share of the government’s total FOIA requests — 7.6% by this estimate.

The biggest users of FOIA are commercial businesses. Including law firms, commercial businesses account for 55.7% of all requests in this sample.

Private individuals are the second biggest FOIA requester category with a 20.1% share.

The share for noncommercial organizations is 7.5%, roughly equal to the news media’s share.

Universities, the smallest group, account for 4.5% of all requests.

These groups are split out in more detail below. Shown in the charts are the most prolific FOIA requesters in each category.

Businesses

Five little-known financial research companies dominate



5 of the top 6 businesses on the list –those marked with an asterisk in the chart– all fit the same profile. They are little-known research companies specializing in financial data (see: International Business Research). Their FOIA requests were directed exclusively to the SEC and in extremely high volume.

As shown in the chart, these five companies account for 11% of all FOIA requests (1,121 of every 10,000). In raw terms, between 2006 and 2016 (the period of time covered by the SEC FOIA logs), these companies submitted a combined total of 71,863 FOIA requests to the SEC.

To put that number in perspective, according to the SEC’s annual FOIA reports, the agency received a total of 136,858 FOIA requests from all sources during that 11-year period. The 71,863 requests made by these five businesses account for just over half of that amount.

Here is what these requests translate to in tax dollars:

Total FOIA processing expenses incurred by the SEC during this period: $52 million .

Total processing fees recouped from requesters: $608,000.

How much of that expense is associated with these five businesses? The SEC does not report its expenses, nor what it collected in processing fees, on a per-request basis. So there is no way to know how much of the expense is attributable to these five requesters. But it’s safe to say it was not an insignificant amount.

To round out the top 10, three of the remaining five companies on the list are specialized research companies: FOI Services Inc., FOIA Group, and EMSI. The other two are in the Fortune 500: Merck (pharmaceuticals) and Thompson Reuters (research/media).

News media

The most frequent requesters are in general large, mainstream news outlets

It’s arguable whether Muckrock (a fellow Knight Foundation-backed FOIA outfit) should really be treated as a single business, and then whether that business should be categorized as “news media.” But accepting that as a premise, Muckrock is the news media’s biggest FOIA requester.

The Black Vault (another FOIA-related website with a conspiracy focus) and Greenwire (environment and energy news) are the only other organizations on the list that are not major media outlets.

The remaining seven companies are all among the biggest names in mainstream news. Of these, the Associated Press, winner of last year’s Pulitzer prize for its investigation of the fishing industry, filed the most requests by a substantial margin.

The chart below shows the 10 highest volume FOIA requesters if you restrict the list to companies that fit the traditional definition of news media.





Noncommercial organizations

Democratic political committees use FOIA more than their Republican counterparts. Nonprofits are a mixed bag.



The “noncommercial” category includes four separate groups: nonprofits, political committees, government agencies, and elected officials. For these last two groups, the list of most frequent requesters is not particularly meaningful, since no individual requester has submitted more than a handful of requests.

For the other two groups, political committees and nonprofits, the most active requesters are shown in the chart above.

Political committees

From the list of political committees, it’s clear the Democrats give FOIA a lot more focus than do the Republicans. The three official Democratic committees at the national level (the DNC, DSCC, and DCCC) filed 16x as many FOIA requests as their Republican counterparts. That figure does not include American Bridge 21st Century (another Democratic committee) or America Rising (the most active FOIA-requesting committee on the Republican side).

To add some perspective to these numbers, the request volume of the top political committees is comparable in size to that of the top news outlets. The #1 political committee, the DSCC, made 42 FOIA requests for every 10,000 in the sample (roughly 3,000 requests per year). For comparison, the #1 news outlet, The Associated Press, made 36 out of every 10,000 requests.

As for why political committees use FOIA, most of the requests from both sides relate to opposition research — digging for negative information to be used against a candidate from the opposing party. The names mentioned in the requests range from presidential election candidates down to those running for local office.

Nonprofits

The five most active nonprofits are a diverse group. It includes two investigative organizations dedicated to government accountability (Judicial Watch and Cause of Action), one labor union (The American Federation of Government Employees), an organization that advocates for animal rights (PETA), and an organization that litigates to defend civil liberties (ACLU).

Universities

GWU’s National Security Archive and U of I’s Illinois State Geological Survey

George Washington University makes far more requests than any university because it is home to the impressive National Security Archive, “the world’s largest nongovernmental collection” of declassified U.S. documents, a great resource.

The University of Illinois’ requests come primarily from the Illinois State Geological Survey, a geological and environmental research institute that is separate from the university’s academic departments

Search the FOIA logs

For more details about the FOIA requests described here, search the database of FOIA logs using the form below.

How this data was compiled / the challenges of working with FOIA logs

The data used in this analysis comes from FOIA Mapper’s database of FOIA logs — a document listing all FOIA requests that an agency has received. There are many ways they can be helpful when making FOIA requests, explained here in more detail. Making these FOIA logs more accessible is one of the primary aims of FOIA Mapper, namely, collecting them. digitizing them, and making them searchable online.

As alluded to at the top of the post, analyzing FOIA logs turns out to be more challenging that you might suspect. Here are the reasons why, and what makes an analysis like this one such an onerous process.

1. FOIA logs are not always easy to get

While all federal agencies are required to maintain FOIA logs, most do not post them online. In most cases, to get an agency’s FOIA log, you have to request it via Freedom of Information (i.e. “FOIAing the FOIAs”). Many agencies will not respond for months, and some will not respond at all.

To compile FOIA Mapper’s searchable FOIA log database, I submitted requests to close to 200 federal agencies. That was nearly a year ago, and I’m still waiting on documents from about one-third of them.

2. Not all FOIA logs include the identity of the requester

The information included in FOIA logs differs from agency-to-agency. And many do not contain the name of the organization making the request. In total, FOIA Mapper’s database of FOIA logs contains about 700k requests from 142 government agencies. Of those, only 85 agencies (229k total requests) include information about the requester’s organization.

3. They are often in non-machine-readable PDF format

FOIA logs do not typically arrive in clean, easy-to-use spreadsheets. More often, an agency will provide them as scanned PDFs, or sometimes even as paper printouts that have to be scanned manually, such as this one from the Department of Labor.

When you request data in Excel format, it should go without saying you want a file, not a 763-page Excel printout pic.twitter.com/RYGYxLHPF7 — FOIA Mapper (@foiamapper) October 5, 2016



Before data in a scanned PDF can be analyzed, it has to be converted into a machine-readable format using optical character recognition, which can be very time-consuming and also highly error prone.

4. Categorizing the requesters

The sample used in this analysis contains 229,000 requests. Assigning categories to that many organizations is not a simple task.

For this analysis, the organizations were classified using a combination of keyword / regular expression searches (e.g. organization names containing “university” or “college” are classified as universities) and manual inspection / manual entry. About 95% of the requests were categorized in this way (the remaining 5% make up the “unclassified” category), with an accuracy rate of about 95%.



Download the raw data used in the analysis — requester names, organizations, and categories.



Embed this graphic on your website

<div style="position: relative; margin: 0; width: 100%; height: 0; padding-bottom: 79.5%; overflow: hidden;"> <iframe frameborder="0" height="480" scrolling="no" src="https://blueshift.io/who-makes-foia-requests.html" style="position: absolute; width: 100%; height: 100%; left: 0; top: 0; overflow: hidden;" width="640"> </iframe></div>

Other studies examining the use of FOIA