Assuring adequate nurse staffing in hospitals remains a contentious issue in healthcare management and national and local policy. While a number of studies conducted in the USA and internationally have found an association between inpatient nurse staffing and mortality, other adverse patient outcomes and longer lengths of stay,1–13 most investigators compare high and low staffed hospitals and use staffing averaged over months or a full year, or based on one-time reports by nurses of staffing on their last shift. While these studies suggest that low staffing increases adverse events and length of stay, they do not fully address the causal association of staffing and outcomes, leaving room to argue that other related factors rather than low staffing itself are the actual causal pathways for the associations observed of staffing and outcomes. More important, the high-low comparisons do not specifically capture or define the level of staffing that makes it difficult for nurses to deliver safe and reliable work nor do they directly address the organisation of nursing care and nursing teams in hospitals.

Additionally, the published research assessing staffing mix, the proportion of nursing and nursing support staff (licensed practical nurses (LPNs) and nurse’s aides) who are registered nurses (RNs) has been interpreted as reflecting efforts to substitute non-RNs to perform work that is the domain of RNs, including assessment and monitoring, patient surveillance, delivery of complex therapies, patient education and psychosocial support and preparation for self-care and discharge, among other activities.3 7 14–17 These studies of staff mix do not address the question of whether, when support staffing is low, RNs are asked to carry more of the work of the support staff, to the detriment of patient care.

In this study, we examine the association of RN and nursing support staffing on inpatient mortality using a data set that allows us to measure staffing for each unit and each shift, to identify shifts in which staffing is substantially lower than is typical and to measure for each patient the impact on risk of mortality of exposure to low staffed shifts. In an important extension of past work, we examined exposure to shifts with low RN staffing, low nursing support staffing and shifts for which RN and nursing support staffing were both low.

The patient sample was restricted to adult, non-surgical and non-psychiatric patients, without community acquired infections, with lengths of stay of three or more days, and admissions on unit-shifts for which staffing data were available (see figure 1 ). Our final sample was 78 303 adult medical admissions on 133 742 unit-shifts.

Using information from the hospital payroll system, we constructed for each unit and each 12 hours shift (day: 07:00 hours to 19:00 hours and night: 19:00 hours to 07:00 hours) the number of hours attributed to the unit for each staffing category (RN, LPN, nurse’s aide). LPN and nurse’s aides were combined into the category of nursing support staff. This decision was made after a review of staffing patterns found that LPNs were a small share of the nursing staff (less than 10% of the combined LPN-nurse’s aide hours) and that they were used in a limited manner in these hospitals. For each unit and shift, we estimated the census and hours of nursing per patient for the unit-shift. We excluded from our sample 6076 unit-shifts (from a total of 140 448) with RN staffing levels so low that nursing administrators in the system judged likely due to data error rather than actual staffing. Unit-shifts with patient-to-RN ratios higher than 4 for intensive care units (ICUs), 8 for medical and medical/surgical units and 6 for step-down units were excluded on this basis. Patients assigned to any of these unit-shifts were excluded from the sample. Using the system’s clinical data warehouse, we identified the unit on which each patient was hospitalised on each day of their admission. Data for unit assignment were not available for a substantial number of patients for their first day of admission, and we excluded all data on first day staffing from analysis.

This is a secondary data analysis. The study was conducted in a large urban US academic health system with three hospitals (two tertiary hospitals and one community hospital), with more than 2000 beds and over 100 000 patient discharges per year.

This study (R01NR010822) was designed by the research team and approved by the university and medical centre institutional review boards. Members of the research team jointly provided direction and oversight of the analysis, writing and submission of the manuscript for publication.

To assess the association between mortality and low RN and nursing support staffing, we conducted a survival analysis using Cox proportional hazards regression models with time from hospital admission as the time scale and in-hospital death as the outcome.

Regression models including these variables were used to estimate HRs and 95% CI. HRs were tested for significance using two-sided Wald tests. Schoenfeld residuals were examined to assess whether the proportionality assumption of the Cox model was met. Plots were consistent with proportionality. All statistical analyses were conducted using SAS, V.9.3.

There are three issues with using cumulative counts of exposure to shifts with low staffing in these analyses. First, to the extent that exposure early in an admission leads to an adverse event that extends the admission, and thus increases exposure to additional shifts with low staffing, exposure to shifts with low staffing may be endogenous. Second, to the extent mortality is associated with longer lengths of stay, there is a potential for reverse causality, with increased mortality risk associated with higher exposure to shifts with low staffing. Third, patients are likely to have higher nursing acuity earlier in an admission, making them more sensitive during this period to low staffing. In light of these concerns, we also modelled cumulative exposure to low staffed and high patient turnover shifts for the first days of an admission only, specifically the second to fifth day of each admission. The survival model uses this constrained count but models the hazard of mortality over the full admission. While the counts of cumulative exposure will be lower in this shorter period, we anticipate the association of exposure to low staffing or higher turnover to be more pronounced for this period.

To match a previous study using similar methods, 9 we constructed models that only included cumulative counts of shifts with low RN staffing or high patient turnover. We then constructed additional models that include cumulative counts of shifts with low RN staffing, low nursing support staffing, shifts with both low RN and low nursing support staffing and high turnover. Results are reported for the cumulative number of shifts with staffing 75% of or lower than the median.

Associations between mortality, nurse staffing and other variables were analysed using Cox proportional hazards regression models. Patient-level variables were included in the models to account for differences in mortality risk. The Cox models included cumulative counts of exposure to low-staffed and high-turnover shifts as time varying covariates.

Characteristics of patients, units and shifts were summarised with means and SD of continuously scaled variables and counts and percentages for categorical variables. For each unit, we calculated the proportion of shifts with low staffing and examined the distribution of these low staffed shifts across patients. We calculated the mean and SD of patient turnover and proportion of shifts with high-turnover.

To allow for seasonal variation in mortality, categorical variables for month were included in the model. To control for time trends in mortality, categorical variables for year were included.

Because demands on nursing staff increase as admissions, transfers or discharges increase, we constructed a measure of patient turnover for each shift equal to the sum of unit admissions, transfers and discharges, divided by start-of-shift census. This ratio was multiplied by 100 and divided by 2, so complete patient turnover would equal 100%. A shift was defined as high-turnover if turnover was greater than or equal to the mean plus 1 SD of the day-shift turnover for that unit, and a dummy variable for high-turnover was merged into the patient-unit-shift record.

As an alternative to constructing three mutually exclusive measures of exposure, we considered constructing an interaction term for shifts with both low RN staffing and low nursing support staffing. We would then use in the analysis variables the cumulated counts of shifts with any low RN staffing, any low nursing support staffing and a cumulative count of the interaction, that is, counts of when both staffing categories were low on a shift. This would be the standard approach to assessing an interaction in a linear model. We were concerned, however, that because we are using a hazard model in which cumulative counts are time varying covariates, the standard approach to implementing and interpreting an interaction might not be appropriate. To accommodate this concern, we opted for the construction of the three mutually exclusive measures of cumulative exposure.

Three zero-one low staffed variables was constructed for each unit-shift, one set to one when only RN staffing was low on the shift, the second set to one when only nurse support staffing was low, and the third set to one when both RN and nursing support staffing was low. For each patient, a cumulative count of exposure to shifts with only low RN staffing, only low nursing support staffing and both low RN and nursing support staffing were included in a survival analysis as time varying covariates.

We considered as cutoffs for low staffing 65%, 75%, 80% and 85% of the median staffing. The lower cutoffs increase the difference from median staffing considered low staffing and also reduce the number of shifts with low staffing, making the estimates less precise. This is a specific issue for shifts with both RN and nursing supporting (NS) staffing low, and particularly for analysis at the 65% threshold. As the threshold is set closer to the median, more shifts are included in the count of low staffing, but the shortfall of low staffing is smaller. To put this in context, at a threshold for low staffing of 80% of the median, 12% of the day shifts and 15% of the nightshifts met the criteria for low RN staffing. At a75% threshold, these are reduced to 10% and 9%, respectively. A cut-off at 75% of the median, used in our analysis, represents an appropriate tradeoff between a deeper measure of low staffing and adequate numbers of shifts with low staffing to analyse.

The academic medical centre whose data we use does not maintain a historic record of target staffing for each unit shift, a measure used in a prior study to measure low staffing as a substantial gap between target and actual staffing. 9 In this study, therefore, we use a measure of staffing substantially below typical for the unit and shift as a measure of low staffing. To assess whether unit-shift staffing was substantially lower than typical, we estimated the median hours per patient for RNs and nursing support staff for each unit on its day and night shifts and used this as a measure of typical staffing. We characterised a shift as low staffed for RNs or nursing support staff if the hours per patient for the shift was less than 75% of the median for each category of staff.

For the reasons discussed in the Methods section, we expect the estimated association of staffing and mortality to be larger in Model 4, which considers exposure only during the second to fifth days of admission. The estimated hazard for only low RN shifts increases to 1.048. The 95% CI widens to include the null of one (0.998 to 1.100), reflecting the greater imprecision due to the smaller number of low RN shifts in the shorter period. Despite a p value of 0.061, above the standard threshold of 0.05, because the change is consistent with expectations, we view these results as consistent with an association. The estimate for exposure to only low nursing support shifts does not change very much between Models 3 and 4 (1.032, 95% CI 1.008 to 1.057, p=0.01). The estimated hazard for exposure to shifts with both low RN and nursing support staffing increases substantially over the Model 3 estimate to 1.136 (95% CI 1.089 to 1.185, p<0.001), above the level associated with only low RN or nursing support staffing.

In Model 3, using cumulative counts from staffing over the entire stay, the HRs on exposure to shifts with only low RN staffing, only low nursing support staffing, and both low RN and nursing support staffing were all greater than one and statistically significant, with similar estimated hazard ratios for each of the three staffing variables (low RN: 1.027, 95% CI 1.002 to 1.053, p=0.035; low nursing support: 1.030, 95% CI 1.017 to 1.042, p<0.001; both low RN and nursing support: 1.025, 95% CI 1.008 to 1.043, p=0.004).

Four survival models are presented that include measures of patient variables, and the exposure to low RN staffing, low nursing support staff, both, and high turnover are presented in table 4 (full regression results are in online supplementary appendix 1 ). Paralleling prior studies, 9 Model 1 presents results for the measure of the cumulative number of low RN staffed shifts experienced. Mortality risk increased with exposure to more low staffed RN shifts (HR per low staffed shift, 1.023, 95% CI 1.011 to 1.035). When this exposure was restricted to the second to fifth days of admission (Model 2), the HR increased to 1.091 (95% CI 1.059 to 1.125).

Forty-five per cent of patients experienced no higher turnover days, with 32% experiencing one, 14% experiencing two and 9% three or more. When counts from the second to fifth days of admission were considered, 56% experienced no high turnover days and 32% one high turnover day.

Table 3 presents the number of shifts experienced by patients during the study time period with either low staffing or high-turnover. Looking over the entire admission, 70% of patients experienced no shifts with low RN staffing and 36% no shifts with low nursing support staffing. Eighty-nine per cent of patients experienced no shifts with both low RN and low nursing support staffing. Nearly 30% of patients had one or more shifts with only low RN staffing, 64% one or more shifts with only low nursing support staffing and 11% one or more shifts with both low RN and nursing support staffing. The proportion of patients experiencing no low staffed shifts were higher when exposure was limited to the counts from the second to fifth days of an admission, with the increase varying from three to seven percentage points ( table 3 ).

Mortality in this sample was 3.9% ( table 2 ). This mortality rate is reasonable for a hospitalised population in which surgical, paediatric, labour and delivery, psychiatric and rehabilitation patients are excluded. Mortality declined slightly over the 6-year period studied, from 4.2% in 2007 to 3.6% in 2012. These year-to-year changes are accommodated in our modelling by including year as a fixed effect.

Forty-seven per cent of patients were male ( table 2 ). The mean age was 62 years, with a range from 18 to 112. The average Charlson comorbidity score was 2.4, with a range from 0 to 21, demonstrating a wide range of comorbidity among this population but mostly at the low end of the range.

Ten per cent of day shifts and 9% of night shifts had RN staffing lower than 75% of the median staffing, with ICUs more likely to have low staffing ( table 1 ). Twenty-two per cent of day shifts and 23% of night shifts had nursing support staffing lower than 75% of the median staffing, again with ICUs more likely to have low staffing. On day shifts, 44% of shifts with low RN staffing had low nursing support staffing; 19% of the shifts without low RN staffing had low nursing support staffing. Twenty-one per cent of shifts with low nursing support staffing had low RN staffing. On night shifts, these percentages were 43%, 21% and 17%, respectively (analysis not shown).

Over the study period, in our analytic sample, there were 133 742 staffed unit-shifts; 43% were in medical or medical-surgical units, 41% in critical care and 16% in step-down units ( table 1 ). Patient turnover across these shifts averaged 33% but was highly variable (SD 16%). Eight per cent of shifts were categorised as high-turnover ( table 2 ).

Discussion

Measuring low staffing as below 75% of the median for a unit, we found low RN staffing on 10% of day shifts and 9% of night shifts, and low nursing support staffing from LPNs and nurse assistants on over 20% of day and night shifts. Units also had higher than usual turnover, defined as turnover one SD above mean turnover, on 8% of shifts. As a result, approximately 30% of admissions had one or more shifts with only low RN staffing, 64% one or more shifts of only low nurse support staffing, 11% one or more shifts on which both RN and nursing support staffing was low and over half one or more high turnover shifts.

Following the approach used by two recent papers examining shortfalls in RN staffing against unit standards,9 20 we found in Models 1 and 2 that each shift of low RN staffing was associated with an increased hazard of mortality, a 2.3% increase in the hazard when the count of low RN shifts was extended over the whole admission and 9.1% when the count was restricted to the second to fifth days of the admission. These results are consistent with those in the two previous papers. Unlike the results reported in the Needleman 2011 paper,9 we found no association of exposure to high patient turnover shifts and mortality.

We extended the prior research by examining the combined association of exposure to shifts with low RN staffing and low nursing support staffing. In both Models 3 and 4, we found an association with mortality of exposure to shifts with only low RN staffing, only low nursing support staffing and shifts where staffing of both RNs and nursing support staff was low. In Model 3, the estimates were comparable across low staffing shifts including where both RN and nursing support staff were low. In Model 4, the effects are larger than in the other models for low RN only staffing and the hazard substantially elevated for shifts with both low RN and nursing support staffing.

In a recent paper by Griffiths and colleagues, using a continuous measure of staffing below and above unit mean levels in a UK hospital, the researchers found both effects of low RN and low nursing support staffing on mortality.21 In that study, the authors did not find the interaction effect observed here as the elevated hazard observed in our Model 4.

This study and the Griffiths study examine nursing support staffing not as a substitute for RNs, as studies of skill mix do, but rather, they examine the impact of shortfalls in support staffing given established RN-nursing support staffing models. The findings of both of these studies raise questions regarding the basis of the association of nursing support staffing and mortality. We would propose two possible mechanisms that should be explored in further studies.

One is that nursing support staff, while not formally trained in patient assessment and monitoring, nonetheless contribute to these tasks as part of their contact with patients and through a developed ability to recognise patients who may need attention by others on the staff. When nursing support staff are less available, this contribution to the safety of patients is reduced.

A second possible mechanism involves a shift of the work of nursing support staff to RNs when nursing support staffing is low, reducing the RNs ability to carry out their other work, notably assessment and monitoring. This mechanism is suggested by the limited literature examining RN assumption of transferable work or work that is primarily the support staff’s, such as delivering and retrieving food trays, transporting patients, obtaining supplies and equipment and arranging transportation, and the role of low support staffing in increasing the work load of RNs.22 The impact of assumption of these necessary tasks and potential crowd out of other RN work can be linked to the growing literature on the role of missed care by RNs when staffing is low as a contributor to increased adverse events such as mortality.23–29

Future research should examine the mechanisms by which low nursing support staffing increases patient risk, whether it reflects the direct loss of nursing support staff work or the increased demands on RNs, and whether the risk when both RN and nursing support staffing are low is elevated, as reflected in our Model 4, or not, as observed in the Griffith study and our Model 3.

There are several limitations to this study. The measure of low staffing was based on typical unit experience and was not adjusted for shift-to-shift variations in patient acuity. In addition, 75% of median as a standard for low staffing was not independently assessed. The hospitals studied did not have staffing systems that present recommended staffing levels, and we cannot assess the sufficiency of staffing at the median for the period studied. Since the period examined, the institutions studied have modified their staffing.

A second limitation is that the study was conducted in only three hospitals associated with a large US academic medical centre. While the results are comparable to two other single institution studies9 20 and a study on hospital-acquired infections,30 replication in other institutions and with a broader set of outcomes is critical.

It has been suggested that one possible explanation for the association of low staffing and mortality is that staffing is set lower if patients are admitted to the unit for palliative rather than curative care. While we do not believe this explains the unit-specific day-to-day variations in staffing, we cannot rule this explanation out from our data.

The findings confirm what is known from the literature, that low RN staffing increases the risk for patients, and it also expands that conclusion. Our finding that the risk is intensified when support staffing for nurses is low, and the effect may be intensified when both RN and nursing support staffing are low is a new finding. Whatever the staffing model at an institution, our results suggest that shortfalls from typical or targeted staffing in both RN and nursing support staffing can have negative consequences for patients. This study should encourage hospital administrators and managers to strive for both adequate RN and nursing support staffing every shift to assure delivery of safe and reliable care. It also suggests that maintaining and reviewing data not only on average staffing but the frequency with which target staffing is not met may improve patient safety.