Since the implementation of the checklist, as part of an overall national safety strategy, there has been a reduction in perioperative mortality.

There were 12 667 926 hospital admissions, of which 6 839 736 had a surgical procedure. Amongst the surgical cohort, the inpatient mortality rate in 2000 was 0·76 (95 per cent c.i. 0·68 to 0·84) per cent, and in 2014 it was 0·46 (0·42 to 0·50) per cent. The checklist was associated with a 36·6 (95 per cent c.i. –55·2 to –17·9) per cent relative reduction in mortality ( P < 0·001). Mortality rates before implementation were decreasing by 0·003 (95 per cent c.i. –0·017 to +0·012) per cent per year; annual decreases of 0·069 (–0·092 to –0·046) per cent were seen during, and 0·019 (–0·038 to +0·001) per cent after, implementation. No such improvement trends were seen in the non‐surgical cohort over this time frame.

This was a population cohort study. All admissions to any acute hospital in Scotland between 2000 and 2014 were included. Standardized differences were used to estimate the balance of demographics over time, after which interrupted time‐series (segmented regression) analyses were performed. Data were obtained from the Information Services Division, Scotland.

The WHO Surgical Safety Checklist has been implemented widely since its launch in 2008. It was introduced in Scotland as part of the Scottish Patient Safety Programme (SPSP) between 2008 and 2010, and is now integral to surgical practice. Its influence on outcomes, when analysed at a population level, remains unclear.

Introduction Surgery continues to be an important treatment for a wide variety of conditions with an estimated 312·9 million operations performed per year across the globe1, 2. Every surgical procedure has an associated risk of morbidity and death3. Multiple complex factors influence surgical outcome, with both technical and non‐technical components being key factors. Consequently, surgical outcomes are influenced by multiple team members, and the systems of care in which they work4, 5. The rates of adverse events vary across hospitals, regions and countries, with up to half due to provider or system‐wide shortcomings6-9. As a result, various measures to improve surgical team performance and, thus, mitigate against surgical complications or adverse events have been advocated10. The WHO Surgical Safety Checklist is one such measure that has been implemented internationally3. This checklist was launched in 2008 and has since become an integral part of the surgical process across the globe11-13. Its aim is to make surgical procedures safer, by ensuring adherence to established practices and creating a culture of communication and teamwork that supports patient safety. The checklist is used by the entire operative team at three key points during any intervention in which harm could ensue3, 14. The aim of its implementation in Scotland was to improve safety of surgical procedures3, thereby improving patient outcomes15 and mortality rates16. However, its impact on improvements in mortality rates after mandatory nationwide implementation has been questioned14, 17-19. The initial pilot study20 analysing the impact of checklist implementation in eight hospitals in eight separate countries noted a significant reduction in perioperative mortality rates and inpatient complications. This study had a trial format, however, and used data collated prospectively. Whether this reduction is replicated in the real‐world scenario outside the context of a trial has yet to be proven. Studies11, 12, 17, 21-24 performed in various healthcare facilities and regions have had mixed results regarding the influence of the checklist on outcome. Implementation of the checklist by mandate in Ontario, with limited training and support, demonstrated no significant reduction in death 3 months after hospitals certified compliance17. However, in a voluntary programme in South Carolina, hospitals completing a collaborative, unit‐based implementation protocol supported by educational programmes achieved a 22 per cent reduction in mortality rates25. The aim of the present study was to seek to understand whether similar improvements have been seen in hospitals across Scotland, where implementation of the checklist was mandated through a national collaborative programme to improve safety of hospital healthcare services. The Scottish Patient Safety Programme (SPSP) is a national initiative which aims to improve in‐hospital mortality rates in Scotland26. The SPSP was established in 2008 and had perioperative management, including implementation of the WHO surgical checklist, as one of its four key initial workstreams. This programme offered a systematic approach to improve patient safety across all hospitals in the country26. It used a multidisciplinary team approach to implement key drivers for change. By implementing the surgical checklist through the perioperative arm of SPSP, Scotland coordinated its introduction nationally, with regional and local support. This study aimed to determine whether implementation of the surgical checklist, through a national‐level improvement strategy combined with a unit‐based partnership, had an impact on population outcomes after surgery in Scotland.

Methods This was a population cohort study. The Information Services Division (ISD) of National Health Service Scotland collects data prospectively on all components of health service provision in the country. It uses a unique identifier which can track individual patient outcomes through time27. All admissions to an acute hospital in Scotland from 2000 to 2014 were included. Patients who were admitted to a psychiatric department, rehabilitation facility or long‐term care hospital were excluded from the study. Data were based on a continuous episode of treatment in hospital. Data were summarized and anonymized at source in ISD. Data were obtained on age, sex, deprivation, specialty, rates of return to theatre, operative urgency, cause of death and number of in‐hospital deaths. ISD follows the principles of the Data Protection Act 2018, UK28. This project was reviewed by a proportionate Research Ethical Review with the Integrated Research Application System and was approved (reference number 196391). It was an observational epidemiological study and, as such, no patient involvement was possible. Cohort definition and endpoints The inclusion criterion was any admission to an acute‐care hospital in Scotland. The surgical cohort was defined by any inpatient admission in which an operation (by OPCS‐4 code association with the admission) was done between 1 January 2000 and 31 December 2014. The non‐surgical cohort included all patients admitted to the same hospital settings in which no operation was undertaken. The primary endpoint was in‐hospital death. The secondary endpoint in the surgical cohort was return to theatre after procedures undertaken electively. Scottish Patient Safety Programme implementation of surgical checklist In brief, the development was based on the Institute for Healthcare Improvement collaborative model29. All local health boards were recruited to encourage clinicians, in a multidisciplinary team approach, to change the ethos around patient safety. SPSP implementations are driven nationally, but there are minor local variations to adapt to relevant specialties, context and regions. Implementation was introduced by use of the plan–do–study–act improvement cycle. Staff education or training to support the theatre leadership to implement change was provided, and biannual nationwide meetings were held to share effective approaches to checklist implementation. These were supported by continual data collection on adherence26. The checklist was established in Scotland between 2008 and 2010. By the end of 2010, its use was nearly uniform across the country30, 31. The time taken for adopting the checklist appeared to be 3 years (2008–2010), so the years until 2007 were defined as the preimplementation interval; 2008–2010 was defined as implementation; and 2011 onwards as the postimplementation interval. Statistical analysis Patient demographics in the three time intervals were described by use of proportions. Standardized differences were used to estimate whether these demographics were balanced over time (and thus unlikely to be confounders). Standardized differences of less than 10 per cent have been found to reflect well balanced demographics over time32. Interrupted time‐series (segmented regression) analyses were used to determine whether there were changes in the level and slope of the rates during the three intervals, and not a constant downward secular trend that continued over time33. The levels and slopes of the rates in the three time intervals were estimated with optimal weighted least squares34, 35, with a robust standard error to account for possible overdispersion due to clustering of outcomes within hospitals (even though hospital‐level data were not available, this robust standard error can account for overdispersion)36. All analyses were performed in SAS® version 9.4 (SAS Institute, Cary, North Carolina, USA).

Results In Scottish hospitals between 2000 and 2014, there were 12 667 926 admissions, of which 6 839 736 had a surgical procedure. Table 1 summarizes the demographic information for these surgical admissions. Over the preimplementation interval (2000–2001 to 2007–2008), during implementation (2008–2009 to 2010–2011) and over the postimplementation interval (2011–2012 to 2014–2015), the following co‐variables were found to be well balanced with mean absolute standardized differences of less than 10 per cent: sex (female: 0·8 per cent), older age (older than 80 years: 3·2 per cent), Scottish Index of Multiple Deprivation (SIMD) (SIMD 1: 3·0 per cent) and surgical specialty (trauma and orthopaedic surgery: 4·3 per cent). Admission type was found to have a mean standardized difference greater than 10 per cent across the three intervals (non‐elective admission: 10·3 per cent), indicating that this co‐variable could be a potential confounder. Table 1. Demographic information Standardized difference (%)§ Preimplementation* (n = 3 629 602) Implementation† (n = 1 384 425) Postimplementation‡ (n = 1 825 709) Preimplementation versus implementation Preimplementation versus postimplementation Implementation versus postimplementation Women 1 962 618 (54·1) 755 201 (54·5) 985 197 (54·0) –0·96 0·22 1·18 Age (years) 0–19 413 402 (11·4) 137 549 (9·9) 167 837 (9·2) 4·71 7·23 2·52 20–49 1 250 792 (34·5) 442 432 (32·0) 543 558 (29·8) 5·32 10·05 4·73 50–79 1 645 698 (45·3) 669 554 (48·4) 927 711 (50·8) –6·06 –10·97 –4·90 ≥ 80 319 710 (8·8) 134 890 (9·7) 186 602 (10·2) –3·22 –4·82 –1·59 Specialty General surgery 1 246 081 (34·3) 439 280 (31·7) 596 410 (32·7) 5·53 3·53 –2·01 Orthopaedics 553 391 (15·3) 242 323 (17·5) 321 701 (17·6) –6·10 –6·41 –0·31 Other 1 830 130 (50·4) 702 822 (50·8) 907 598 (49·7) –0·69 1·42 2·11 SIMD SIMD 1 882 743 (24·5) 316 824 (23·0) 409 965 (22·6) 3·45 4·50 1·05 SIMD 2 805 597 (22·3) 299 106 (21·7) 387 959 (21·3) 1·48 2·37 0·89 SIMD 3 713 275 (19·8) 281 949 (20·5) 370 022 (20·4) –1·74 –1·48 0·26 SIMD 4 632 085 (17·5) 253 796 (18·4) 342 368 (18·8) –2·36 –3·42 –1·07 SIMD 5 574 266 (15·9) 225 861 (16·4) 306 996 (16·9) –1·30 –2·64 –1·33 Missing 21 626 6889 8399 Non‐elective admission 856 453 (23·6) 259 751 (18·8) 317 758 (17·4) 11·85 15·38 3·53 Overall mortality trends in surgical cohort Fig. 1 shows the results of the time‐series analysis for overall mortality rate across the three intervals. In the preimplementation interval, the mortality rate had an absolute decrease of 0·003 (95 per cent c.i. –0·017 to +0·012) per cent per year; during implementation, the annual mortality rate decreased 0·069 (–0·092 to –0·046) per cent; and in the postimplementation interval, it decreased 0·019 (–0·038 to +0·001) per cent. The downward trend in mortality rate seen in this model during implementation was found to be significantly different from the trend in the preimplementation interval (P < 0·001). Although the mortality rate trend in the postimplementation interval was not found to be significantly different from that in the preimplementation interval (P = 0·153), the estimate for overall mortality rate in the preimplementation interval and that in the postimplementation interval were significantly different (P < 0·001). Among hospital admissions with an operation performed, the inpatient mortality rate in 2000 was 0·76 (95 per cent c.i. 0·68 to 0·84) per cent and in 2014 it was 0·46 (0·42 to 0·5) per cent; if the trend in the preimplementation interval had persisted, the 2014 mortality rate would have been 0·72 (0·59 to 0·85) per cent. Thus, the intervention was associated with reduced mortality rates, demonstrating a 36·6 (95 per cent c.i. –55·2 to –17·9) per cent relative reduction over the time studied (P < 0·001). Figure 1 Open in figure viewer PowerPoint Time‐series analysis for overall mortality rates across preimplementation, implementation and postimplementation intervals A time‐series analysis was also performed adjusting for admission type, which was found, on exploration of the demographic information, to be a potential confounder. The results of the adjusted model are shown in Fig. 2. Trends across the three time intervals were found to be similar in this model to those in the model for overall mortality rates shown in Fig. 1; this demonstrated that the intervention was associated with reduced mortality rates. Figure 2 Open in figure viewer PowerPoint Time‐series analysis for overall mortality rates adjusted for admission type across preimplementation, implementation and postimplementation intervals Return to theatre Fig. 3 shows the time‐series analysis for return‐to‐theatre rate after elective admission for surgery across the three time intervals. In the preimplementation interval, the return‐to‐theatre rate increased 0·002 (95 per cent c.i. 0·001 to 0·002) per cent per year; during implementation, the annual return‐to‐theatre rate decreased 0·003 (–0·005 to –0·002) per cent; and in the postimplementation interval, it decreased 0·002 (–0·002 to –0·001) per cent. The difference between the during‐implementation trend and the preimplementation trend was statistically significant (P < 0·001), as was that between the preimplementation trend and the postimplementation trend (P < 0·001). The differences in estimates for return‐to‐theatre rates preimplementation compared with postimplementation were statistically significant (P < 0·001). Figure 3 Open in figure viewer PowerPoint Time‐series analysis for return‐to‐theatre rates after admission for elective surgery across preimplementation, implementation and postimplementation intervals Overall mortality trends in non‐surgical cohort A time‐series analysis for the 5 828 190 patients in the non‐surgical cohort, with the same time frames for preimplementation, implementation and postimplementation, showed no improvement in mortality rates (P = 0·418).

Discussion This nationwide population‐based cohort study found a substantial reduction in mortality rates in patients undergoing surgical intervention in Scotland after the introduction and implementation of the WHO surgical checklist as part of a nationwide patient safety initiative. This reduction persisted when the data were controlled for urgency of admission, the only baseline demographic that altered during the time frame. No such trend in improvement was observed in the non‐surgical cohort. From these data, it can be inferred that the improvement seen in the surgical group was associated temporally with the implementation of the surgical checklist. The SPSP is a unique national programme that aims to change the healthcare culture to one that has patient safety at its forefront26. A key element has been the testing and application of evidence‐based interventions and quality‐improvement methodology. These implementations have been supported locally through co‐production, educational programmes and prospective data on adherence. Since its launch, the programme has contributed to a significant reduction in harm through relevant quality‐improvement strategies26, 37. After testing, review and feedback from health boards across Scotland, the surgical checklist was included as one of the ten Patient Safety Essentials to be implemented across all health boards in Scotland37. The surgical checklist was not a stand‐alone intervention. It was, however, the only Patient Safety Essential that targeted surgical patients specifically during the interval studied37. Thus, the addition of the checklist to the other parameters within the SPSP may have contributed to the improvement in results observed in the present study. The implementation of each of the SPSP interventions was mandated, but with emphasis placed on collaborative working with frontline clinicians and teams, together with local clinical leadership. The SPSP ensures that healthcare improvement implementations are adopted in a nationally coordinated approach and, as such, allows patient safety initiatives to be developed and rolled out regardless of hospital location, clinician experience or underlying knowledge. Neither checklist fidelity38, nor the effectiveness of the educational programmes39, has been assessed in this study. Furthermore, the surgical checklist in Scotland has predominantly involved only the clinical team, with variation noted regarding patient involvement, another area in which checklist quality could be improved40. Use of an observational approach at the population level reduces the risks of observation bias. The present data highlight real‐world improvements outwith the context of clinical trials or research centres. There are limitations, however, to both the data and the analysis. One of the key markers would have been to assess the rates of specific events, such as wrong‐site surgery. There was no access to these data in the present study; the rates of these events are so low anyway that identifying significant trends is challenging. Specific details on how each unit conducted training sessions or developed frameworks during the implementation stage were not available. However, all regions supported checklist implementation with additional structures and training of clinical staff in all theatre settings, tailored to suit each hospital and specialty. It is acknowledged that attributing causal links to the findings in population‐wide data set analysis is not possible. The data were obtained in a summarized manner and individual patient‐level data were not available, precluding multivariable analysis. Thus, standardized differences, reported previously in observational research32, were used to analyse changes in demographic characteristics over time and incorporated into bivariable analysis for the effect of urgent admission, the sole factor with a mean standardized difference greater than 10 per cent. The present findings of improved outcome are in agreement with several studies20, 21, 23, 24 looking specifically at the implementation of a surgical safety checklist, but are at odds with another17 analysing population‐based outcomes in the early phase after checklist initiation. The present study used a more longitudinal approach, allowing for bedding down as the checklist has become established as part of the workload culture of surgical theatre life in Scotland. This study provides further evidence that the success of checklist implementation is more pronounced when it is supported by a cohesive and wider approach to patient safety.

Acknowledgements R. Munro and R. Black of NHS National Services Scotland, UK, provided data intelligence. B. Robson helped review this paper; A. Longmate was involved in the preliminary design of this study; J. Ingram, J. Ferbrache and SPSP managers from health boards in Scotland provided some of the information on surgical checklist implementation practices at hospitals across NHS Scotland. Data, analytical methods and study materials used may be made available to other researchers on request. The lead author affirms that the manuscript is an honest, accurate and transparent account of the study being reported. No important aspects of the study have been omitted. Any discrepancies from the study as planned (and, if relevant, registered) have been explained. This study was not funded by any individual or group. The Research Governance Department of the University of Aberdeen sponsored this project and supported the application through ethical review and data management. Disclosure: A.A.G. has received royalties from multiple publishers for writing on improving healthcare, including through use of checklists: Objetiva, Sextante (Brazil); Profile Books Ltd (British Commonwealth); Cheers Publishing Company, Commonwealth Publishing Co (People's Republic of China); Jesenski & Turk, Mizaik Knjiga, Mozaik Knjiga (Croatia); Dokoran (Czech Republic); Lindhart og Rinhoft Forlag (Denmark); Pilgrim Group (Estonia); Editions Moyen‐Courrier, Fayard, Libraire Arthème Fayard, Moyen‐Courrier (France); Radarami (Georgia), S Fischer, Verlagsgruppe Random House (Germany); Crete University Press (Greece); Tericam Kindo (Hungary); Mehta Publishing House, Penguin Random House Books (India); Gramedia Pustaka Utama, Serambi Ilmu Semesta (Indonesia); Arjmand Press (Iran); Am Oved, Modan (Israel); Einaudi Editore (Italy); Misuzu Shobo, Shinyusha Co Ltd (Japan); Janis Roze Publishers (Latvia); Vaga (Lithuania); Mime Forlag (Norway); Magnum, Znak (Poland); Lua de Papel (Portugal); Codecs, Grup Media Litera, Litera, Streamland Ltd (Romania); Slovant Publishers (Slovakia); Mladinska Knjiga (Slovenia); Antoni Bosch Editor, Editorial Empuries, Galaxia Gutenberg (Spain); Asa Editore, Bookie Publishing House, Book21, Sosoh Publishing (South Korea); Volante (Sweden); Alpina, AST (Russia); Matichon, Openworlds (Thailand); Arbeiderspers, Nieuwezijds, Uitgeverij De Arbeiderspers, Uitgeverij Nieuwezijds (the Netherlands); Domingo, Koton Kitap (Turkey); Verlagsgruppe Vivat (Ukraine); CBS Television, Henry Holt, Houghton Mifflin, Harvard Business School Press, McGraw Hill, Pearson Publishing, Public Broadcasting Service, Picador USA (USA), First News‐Tn Viet Publishing Co, Suc Manh Ngoi But Co (Vietnam); and Harper Collins (World). The authors declare no other conflict of interest.