Executive summary

This research is based on two sets of FOI Requests made to local authorities and an anonymous survey of FOI officers. The report aims to provide a clear illustration of the current scale and operational approach to FOI in local government. The key conclusions are:

Freedom of information is a significant legal instrument in the UK - but currently very little is currently understood about how councils cope with this sometimes significant workload. While central government Freedom of Information (FOI) statistics are well covered by the Cabinet Office series, there is no equivalent picture for local government. The goal of this research was to construct a more detailed picture of FOI at the local level, to understand how many requests were being sent, how much internal visibility exists over the status of requests, and calculate the contribution of requests made through WhatDoTheyKnow to the total number of requests made overall.

Foreword

This is why this report is so welcome. It gives us a first comprehensive picture of what's happening with FOI at the local level. It shows that requests numbers have indeed continued to grow and that local authority staff face a series of obstacles in dealing with requests. Above all, it underlines the importance of FOI as a tool of local democracy.

Since 2010, when our surveys ended, we have known little about what's happening. Councils are dealing with years of huge and deep cuts that have meant the loss of both resources and experienced staff. The government's recent revised Code around FOI has promised local government data on FOI requests, and a few have begun to publish them aleady. But aside from these changes, that may take time, local government and FOI is hidden from us.

The landscape of local government and FOI has also been a changing one. Our surveys of local government and FOI between 2005 and 2010 found that the requests numbers were very large and growing. In the space of five years, we found requests to local government jumped from 60,000 in 2005 to 197,000 in 2010. We also felt, based on our interviews and other anecdotal evidence, that this may have been a considerable underestimate. Despite this huge jump in numbers, we also found that local government learned to deal more effectively with requests, more than halving the estimated time spent on them.

Freedom of information is local. Although it is often national scandals that grab the headlines, most FOI use is at the local level. Our research at the Constitution Unit back in 2010 in the UK found that almost 80% of requests go to local government. This makes sense, given that most political participation and activity is at the level of town halls rather than Westminster. All this local use means we have to re-think and remember what FOI is for. It may be less about MPs' expenses and more about street lights, fixing roads and the micro-politics of our changing the nearby area. The real value of FOI, as one Scottish Information Commissioner put it, is to be found in the pages of local newspapers.

FOI statistics

Gathering information

This study focuses on the 418 local authorities in the UK (it does not include the large number of parish councils that are also subject to FOI), which have slightly differing powers and scope dependent upon the location and type of the council. For this report, mySociety sent a series of questions to local authorities in the UK grouped into two separate FOI requests. One group of questions related to FOI statistics held by the authority (specifically related to FOI Requests received in 2017, and the success and appeal rates of those requests). The second group of questions related to how FOI is administered internally, specifically concerning staffing, budget and software.

For each set a pilot was conducted, with sample FOI requests sent to 10 councils to test the validity/integrity of the questions, and to identify any amendments that needed to be made to the phrasing of the questions in order to achieve standardised results. This resulted in a clarification of one question, and the discovery that Scottish local authorities all publish their FOI statistics in a centralised portal (and so could be excluded from the request for FOI statistics). FOI Requests were then sent to the full cohort of councils and the results recorded.

Of the remaining 388 local authorities, 89% provided at least a partial response to the request for FOI statistics. Information for Scottish local authorities was retrieved from the Scottish Information Commissioner data portal. In several cases for Scottish local authorities these statistics included FOI Requests to the licensing board (which are treated by WhatDoTheyKnow as separate public bodies).

WhatDoTheyKnow was used to find how many FOI Requests were sent in 2017 via that platform. For Scottish authorities, the count from WhatDoTheyKnow was combined with a count for the relevant licensing board.

There were a small number of authorities that required clarification on the request made (either on the wording of a question, or in reverse, where the website listed as hosting the reply was unavailable). Some authorities refused the request but indicated that a reduced request would be viable. In some cases some statistics were detectable even in the event of a refusal (where for instance, the cost of querying X requests to determine information gave a value for the number of requests in the period).

A minority (6%) of non-Scottish councils referred to already available online releases of statistics. These were a mixture of spreadsheets reflecting date received and status of FOI requests, or pdf council reports detailing the overall statistics. While typically stored (with varying levels of visibility) on the council website, Leeds City Council released their FOI data in the shared open data repository Data Mill North. More councils releasing their statistics on the same portal (for instance https://data.gov.uk would make the information more transparent and easy to access.

The new FOI Code of Practice requires online disclosure of these statistics (and several councils made reference to plans to change their tracking systems to reflect this upcoming requirement), and this will represent a necessary change in practice for almost all councils. Individually, this will reduce the burden on each authority to spend time answering similar FOI Requests in the future, but without a centralised or standardized system of storing these statistics, this will make the job of externally gathering the information more time consuming.

Using the official FOI email address for the council, a survey (to be completed anonymously) was also distributed to FOI officers querying the structure of teams, and their perceptions of WhatDoTheyKnow. This received 129 responses.

Interpreting FOI statistics

There are several hurdles in working with statistics derived from FOI requests. These reflect internal tracking systems and so in turn reflect decisions made internally concerning the categorization and logging of requests within that authority, rather than representing a consistent schema for comparison.

What counted as a request being ‘refused' varied from council to council - in some cases it included those where the information wasn't held as well as those refused via an exemption. While asking these as separate questions would often have received two pieces of information (many councils did volunteer separate counts), other councils did not track to this level of detail. Many councils included in their response, warnings about the level of accuracy of the statistics provided (where for instance results were compiled from two different tracking systems) that, while suitable for aggregate analysis, make positive or negative judgements on particular authorities difficult to authoritatively construct. In some cases, two authorities (e.g. Christchurch and East Dorset District Councils) handled processes internally as one structure. Some councils included in their total a number of requests that were eventually referred onwards to other authorities, whereas others did not. Some councils recorded requests made under the Environmental Information Regulations separately to those made under the Freedom of Information Act and others combined the two. For consistency, the numbers in this analysis cover FOI and EIR requests.

Councils also differed in the time periods they used to record statistics. 61.5% recorded FOI Requests in calendar year, while 37.7% recorded in financial year (within this were councils who recorded by month or individually, and so could report in either). Two councils provided figures in other time periods as a result of changes in tracking systems during that period. Where it was unclear from the information provided, calendar year was assumed. This created issues in creating a figure for local government in ‘2017' as not all requests were over the same time period. Additionally, requests were not evenly distributed year on year, and so there is a difference in the average for those reported in calendar and financial year. As explored Appendix A, this effect remains (although is smaller) when accounting for other factors.

This is also reflected in requests made to WhatDoTheyKnow. For requests made through WDTK in the equivalent 2017 calendar and financial year, 38% more were made in the calendar year. As both counts are available for WDTK, this demonstrates the effect is in addition to other differences between the councils.

Type Count % Average FOI Requests Financial Year 141 37.7 987.56 Calendar Year 230 61.5 1123.26 Table: Table 1 Financial vs Calendar Year

Differences in the ‘success' rates of requests may also reflect differences in requests instead of differences in processing. One council might receive a large number of requests on an area with an exemption, while another may not. Similar requests may be joined in one area but submitted separately in another, resulting in one council returning ‘1 All Info Returned, 1 Refused On Exemption' while the other returns ‘1 Partial Info Returned'. Statistics on successes and failure of FOI requests (while illustrative when there are large differences) can only be said to be meaningful using a longitudinal study, in which the same public authority is examined periodically over a longer period of time.

This difficulty applies not only to statistics sourced through FOI, but also applies to the self-disclosed data in the Scottish Information Commission repository, which has a similar disclaimer to FOI gathered statistics, in that the numbers may represent different things for different authorities, and so have issues for comparative use.

How many FOI Requests were sent to local government?

In total

There are two issues in assembling a complete picture of how many FOI Requests were sent to local government in 2017: 1) Different time ranges used in disclosures (Calendar or Financial year) and, 2) Missing data points from FOI Requests that weren't returned or that didn't record this information.

These two issues were addressed by creating a model that adjusted for any systematic differences between different time ranges. This model was then used to fill in approximate values for the remaining councils. The methodology for this approach can be seen in Appendix A.

The result of this is an estimate of 468,780 FOI requests received in the calendar year 2017. There is a 95% confidence this value falls between 467,587 and 469,975 (range of 2,387).

Change over time

Until 2010, UCL's Constitution Unit conducted an annual survey of a sample of councils in England and created estimates for the overall number of requests made based on council type.

Year Number of Requests 2005 60,361 2006 72,361 2007 80,114 2008 118,569 2009 164,508 2010 197,737 Table: Table 2 Constitution Unit estimates of English Local Authority FOI numbers

However, the methodology for this survey was based on a small number of respondents and the report says that "the number of responding councils did not reach the level of a representative sample, this is not a scientifically reliable method of calculating the total number" (p. 51). This means that there is not data to conduct a reliable comparison across time. Interviews with councils conducted by the original researchers led to the impression that the 2010 figure might be an undercount.

Based on the data collected for this report, the estimate for English councils only, is 388,736 FOI Requests in 2017. This represents a 97% increase on the 2010 figure of 197,000 estimated by the Constitution Unit. In the same period FOI Requests sent to audited central government only increased by 6%. Whether this represents a historical under-count or a growth over time (10% a year), the current volume of FOI is nearly double the previously available estimate.

In evidence to the Independent Commission of Freedom of Information, Liverpool City Council said they had experienced an increase of 76% (1,217 to 2,139) between 2010 and 2014. This would represent an 15% annual increase. In their response to this study's FOI request, Liverpool had the third highest response - indicating the size of this increase is probably not typical, but that an average growth over time of 10% a year may not be unreasonable.

Differences in requests by council

There was a large amount of variation in the number of requests received by councils. This graph illustrates the distribution of requests by the number of councils. Where a council's request volume has been predicted, they are distributed as partial values over their probability curve.

Image: Chart 1 Distribution of requests received by councils predicted councils distributed over their probability.

How many FOI Requests came from WhatDoTheyKnow

In the 2017 Calendar year, 28,282 FOI Requests were made to local councils from WhatDoTheyKnow. Using the adjusted and predicted calendar year totals from above, WhatDoTheyKnow accounted for 6.02%[6.01,6.04] of local government FOI requests. Therefore 6% of FOI Requests made to local government can be identified as originating from WhatDoTheyKnow.

Council differences

95% of councils vary between having 2% and 13% of their requests sourced from WhatDoTheyKnow. More councils fall below 5% than above, with a minority of councils raising the average. The median council has 4.89% [4.80,5.00] of requests result from WhatDoTheyKnow.

Image: Chart 2 Distribution of WhatDoTheyKnow origin predicted councils distributed over their probability.

Success and timeliness of requests

Based on the partial returns described above, the following figures and charts demonstrate the different rates of success and completeness at different authorities. Percentages relate to the total number of councils with information for that area, rather than the total number of councils.

Completed inside statutory deadline

In local government, compliance rates with the statutory 20 days deadline for reply are high, with isolated exceptions.

8 (2%) councils reported 100% of replies fell inside the statutory deadline

councils reported 100% of replies fell inside the statutory deadline 260 (80%) reported upwards of 80% of replies fell inside the statutory deadline

reported upwards of 80% of replies fell inside the statutory deadline 7 (2%) reported compliance rates below 50%.

Image: Chart 3 Percentage of FOI Requests answered in statutory deadline by local authorities.

Information granted

While the Institute for Government has reported a growing trend of refusals in Central Government (approaching 40% withheld in 2017), the picture in Local Government is generally more positive.

5 (2%) councils reported 100% of requests were granted in full.

councils reported 100% of requests were granted in full. 106 (43%) reported upwards of 80% of requests were granted in full.

reported upwards of 80% of requests were granted in full. 19 (8%) reported that fewer than half of requests were granted in full.

Image: Chart 4 Percentage of FOI Requests granted by Council

Vexatious requests

There is a specific exception in the respective FOI acts that allows authorities not to comply with "vexatious" requests. This is interpreted by the ICO as allowing authorities to refuse "any requests which have the potential to cause a disproportionate or unjustified level of disruption, irritation or distress".

The proportion of vexatious requests are low, with a minority of councils regarding a request as vexious in 2017 (however, the volume of work involved in processing a vexatious request may be disproportionately higher compared to other requests).

101 (33%) councils had at least one request considered vexious, 61 (20%) had more than two.

councils had at least one request considered vexious, had more than two. 27 (9%) of councils had at least five vexatious requests.

of councils had at least five vexatious requests. Of councils that provided information concerning vexatious requests, 0.16% of requests were vexatious (with the true figure likely being lower).

Complaints

Requesters have the option of asking an authority for an internal review if they are unhappy with the decision reached, where the decision would initially be reviewed internally by the council, or following this, subsequently appealing to the Information Commissioner's Office (ICO). Complaints are generally uncommon, but most councils dealt with at least one in this time period.

305 (94%) councils had at least one internal review.

councils had at least one internal review. 217 (67%) councils had more than five.

councils had more than five. 208 (81%) councils had cases where the original decision was overturned.

councils had cases where the original decision was overturned. 183 (64%) councils had at least one complaint to the ICO.

councils had at least one complaint to the ICO. 44 (15% ) councils had more than five

) councils had more than five On average 1.4% requests were appealed to Internal Review.

requests were appealed to Internal Review. Of all tracked internal reviews, in 51-64% of cases the original decision was upheld in full.

of cases the original decision was upheld in full. Based on data from the ICO website, of 486 decision notices issued by the ICO related to local government in 2017, 350 (72%) contained complaints that were not upheld, 225 contained complaints that were upheld (46%), and 25 (5%) contained complaints that were partially upheld.