17. Quality and methodology

Revisions

Estimates for the most recent time periods are subject to revision due to the receipt of late and corrected responses to business surveys and revisions to seasonal adjustment factors, which are re-estimated every month. Estimates are subject to longer-run revisions, on an annual basis, resulting from reviews of the seasonal adjustment process. Estimates derived from the Labour Force Survey (a survey of households) are usually only revised once a year. Revisions to estimates derived from other sources are usually minor and are commented on in the statistical bulletin if this is not the case. Further information is available in the labour market statistics revisions policy.

One indication of the reliability of the main indicators in this statistical bulletin can be obtained by monitoring the size of revisions. Datasets EMP05, UNEM04 and JOBS06 record the size and pattern of revisions over the last five years. These indicators only report summary measures for revisions. The revised data itself may be subject to sampling or other sources of error. Our standard presentation is to show five years' worth of revisions (60 observations for a monthly series, 20 for a quarterly series).

Accuracy of the statistics: estimating and reporting uncertainty

Most of the figures in this statistical bulletin come from surveys of households or businesses. Surveys gather information from a sample rather than from the whole population. The sample is designed to allow for this, and to be as accurate as possible given practical limitations such as time and cost constraints, but results from sample surveys are always estimates, not precise figures. This means that they are subject to some uncertainty. This can have an impact on how changes in the estimates should be interpreted, especially for short-term comparisons.

There is a trade-off between sample size and sampling variability. As the number of people available in the sample gets smaller, the variability of the estimates that we can make from that sample size gets larger. What this means in practice is that estimates for small groups (for example, unemployed people aged from 16 to 17 years), which are based on quite small subsets of the Labour Force Survey sample, are less reliable and tend to be more volatile than estimates for larger aggregated groups (for example, the total number of unemployed people).

We can illustrate the level of uncertainty (also called “sampling variability”) around a survey estimate by defining a range around the estimate (known as a “confidence interval”) within which we think the real value that the survey is trying to measure lies. Confidence intervals are typically defined so that we can say we are 95% confident the true value lies within the range – in which case we refer to a “95% confidence interval”.

The number of unemployed people for August to October 2018 was estimated at 1,380,000, with a stated 95% confidence interval of plus or minus 73,000. This means that we are 95% confident that the true number of unemployed people was between 1,307,000 and 1,453,000. Again, the best estimate from the survey was that the number of unemployed people was 1,380,000.

As well as calculating precision measures around the numbers and rates obtained from the survey, we can also calculate them for changes in the numbers and rates. For example, for August to October 2018, the estimated change in the number of unemployed people since May to July 2018 was an increase of 20,000, with a 95% confidence interval of plus or minus 78,000. This means that we are 95% confident the actual change in unemployment was somewhere between an increase of 98,000 and a fall of 58,000, with the best estimate being an increase of 20,000. As the confidence interval for the change in unemployment (plus 98,000 to minus 58,000) includes zero, the estimated increase in unemployment of 20,000 is said to be “not statistically significant”.

In general, changes in the numbers (and especially the rates) reported in this statistical bulletin between three- month periods are small, and are not usually greater than the level that is explainable by sampling variability. In practice, this means that small, short-term movements in reported rates should be treated as indicative, and considered alongside medium-and long-term patterns in the series and corresponding movements in administrative sources, where available, to give a fuller picture.

Where to find data about uncertainty and reliability

Dataset A11 shows sampling variabilities for estimates derived from the Labour Force Survey.

Dataset JOBS07 shows sampling variabilities for estimates of workforce jobs.

The sampling variability of the three-month average vacancies level is around plus or minus 1.5% of that level.

Sampling variability information for average weekly earnings growth rates are available from the “Sampling Variability” worksheets within datasets EARN01 and EARN03.

Seasonal adjustment and uncertainty

Like many economic indicators, the labour market is affected by factors that tend to occur at around the same time every year; for example, school leavers entering the labour market in July and whether Easter falls in March or April. In order to compare movements other than annual changes in labour market statistics, such as since the previous quarter or since the previous month, the data are seasonally adjusted to remove the effects of seasonal factors and the arrangement of the calendar. All estimates discussed in this statistical bulletin are seasonally adjusted except where otherwise stated. While seasonal adjustment is essential to allow for robust comparisons through time, it is not possible to estimate uncertainty measures for the seasonally adjusted series.

Quality and Methodology Information reports

The Quality and Methodology Information reports contain important information on:

the strengths and limitations of the data and how it compares with related data

users and uses of the data

how the output was created

the quality of the output including the accuracy of the data

Labour Force Survey Quality and Methodology Information

Labour Force Survey performance and quality monitoring reports

Vacancy Survey Quality and Methodology Information

Workforce jobs Quality and Methodology Information

Average weekly earnings (AWE) Quality and Methodology Information

Labour disputes Quality and Methodology Information