Trial Oversight

Details regarding the iCOMPARE trial have been reported previously.1,18 The institutional review board at the University of Pennsylvania approved the protocol (available with the full text of this article at NEJM.org) and served as the institutional review board of record for all participating programs that signed on to an institutional affiliation agreement. Twenty-three programs opted for a local review process. The institutional review board at Children’s Hospital of Philadelphia reviewed and approved the analysis of Medicare claims to test safety hypotheses.

Trial Design and Program Selection

A total of 63 programs underwent randomization in a 1:1 ratio to a group with standard duty-hour rules (following the 2011 ACGME duty-hour regulations with its 16-hour limit on intern shift length) or to a group with flexible duty hours, which allowed directors to extend work-hour limits beyond the 16-hour limit. (A summary of regulatory differences between the two duty-hour policy groups is provided in Table S1 in the Supplementary Appendix, available at NEJM.org.) In the flexible programs, directors selected services in which to implement the flexible rules and generally maintained flexible shifts in those services for the duration of the trial year.

We selected programs to meet sample-size requirements for the primary hypothesis. We included only programs with at least one affiliated hospital in both the upper half of resident-to-bed ratios and the upper three quartiles of patient volumes for 17 prespecified medical conditions (which included those chosen for their common treatment on internal-medicine services and their elevated mortality). Before randomization, at least one such hospital in each program had to be identified by the program director as being a hospital in which the director would implement flexible schedules if the program was randomized to the flexible-policy group. A total of 179 internal-medicine programs were approached, and 63 of their directors agreed to participate. The programs and the hospitals that were designated by program directors are listed in Table S2 in the Supplementary Appendix and constitute the trial populations. Complete Medicare data from all 63 programs were available for analysis.

Patient Population and Data

All outcomes were ascertained from Medicare claims to ensure uniform measurement across participating hospitals.14,19 Claims records were obtained from the Medicare Inpatient, Outpatient, Physician Part B, Home Health Agency, and Hospice files. We also used the Centers for Medicare and Medicaid Services Master Beneficiary Summary file for beneficiary demographic, vital status, and insurance information, and a validated date of death.20 We selected claims for patients who were 65.5 years of age or older and who were admitted with one of the 17 qualifying medical conditions to a hospital in the iCOMPARE trial during the pretrial year (July 1, 2014, to June 30, 2015) or the trial year (July 1, 2015, to June 30, 2016). If a patient had multiple qualifying admissions, we included the first qualifying admission during each of the pretrial and trial years. Only patients who were in Medicare fee-for-service for a period of at least 6 months before the index admission and at least 30 days after the index admission were included.

In October 2015, the International Classification of Diseases, 10th Revision (ICD-10) system was adopted by Medicare. All ICD-10 codes were recoded to those of the ninth revision of the ICD21 (Table S3 in the Supplementary Appendix).

Statistical Analysis

In the primary analysis, we tested the hypothesis that the change in 30-day all-cause mortality from the pretrial year to the trial year in the flexible programs would not be worse than that in the standard programs by more than 1 percentage point (noninferiority margin). A secondary analysis examined risk-adjusted mortality. We also report the results of five secondary analyses of additional safety measures: rates of readmission or death within 7 days and 30 days after discharge, in which in-hospital deaths were counted as readmissions on discharge day zero (death date) to avoid inappropriate credit for an early death in the readmission analysis; the rate of at least one patient safety indicator, according to AHRQ criteria (Table S4 in the Supplementary Appendix); payments made by Medicare (Section S1 in the Supplementary Appendix); and the rate of a prolonged length of hospital stay.14-17 A prolonged stay was defined as a condition-specific length of stay that exceeded the point in a hospitalization when rates of discharge typically begin to decline and was derived from lengths of stay at eligible hospitals not participating in the trial. For example, a prolonged stay for pneumonia was defined as a stay longer than 3 days (Table S5 in the Supplementary Appendix).

For each outcome, we report the average rate in each trial group in each year (trial year and pretrial year), the change from the trial year versus the pretrial year within each group, and the between-group difference in the change in the outcome from the trial year to the pretrial year (difference-in-difference analysis). The noninferiority margin of 1 percentage point was chosen for each outcome, which would indicate a difference-in-difference not exceeding 1 percentage point and provide evidence that the flexible policy did not adversely affect patient outcomes as compared with the standard policy (Section S2 in the Supplementary Appendix). We used the t-test to compare groups with respect to continuous outcomes and the chi-square test for dichotomous outcomes. Our noninferiority trial did not include a prespecified plan for multiple comparisons but instead defined only one primary outcome (unadjusted 30-day all-cause mortality at any location) and five secondary outcomes, with risk-adjusted results reported as additional secondary outcomes.

The risk-adjustment models controlled for the qualifying medical conditions and coexisting conditions, as defined by Elixhauser et al.,22,23 with some additional variables (Section S3 in the Supplementary Appendix). All covariates for patients were ascertained by means of a 6-month review of claims and present-on-admission logic as implemented in past studies.7,14,17 The risk models also included age categories, sex, race or ethnic group, transfer-in status, hospice status, admission through emergency department, and direct admission to the intensive care unit (Table S6 in the Supplementary Appendix). All risk models were developed with the use of data from patients at 154 hospitals that met the criteria for inclusion in iCOMPARE but that were not affiliated with a randomized residency program (Table S7 in the Supplementary Appendix). For risk adjustment, we performed regression modeling using PROC LOGISTIC24 software for binary outcomes and PROC ROBUSTREG25 software for continuous outcomes (SAS Institute). All analyses, unless specified, are based on an intention-to-treat approach.

Since flexible programs had the discretion to extend hours beyond the 2011 ACGME limits on any, all, or no services at their hospitals, we performed a subgroup analysis involving patients who were treated on services that were chosen to use flexible schedules and who were admitted for one of the 17 qualifying medical conditions. To identify these patients, we asked each program director to provide the dates when their attending physicians were supervising trainees on flexible services. We used the attending physicians’ National Provider Identification Numbers and the Medicare Inpatient (Part A) and Physician Part B claims to identify the subgroup of trial-year patients who were on a flexible service on the first day of hospitalization for their index admission. In this analysis, patients who were treated by the same attending physician on the same services (intensive care unit or medical floor) in the same hospital during the pretrial year provided the pretrial data in the flexible programs. For the standard programs, we used the subgroup of patients in standard programs from either year who had an attending physician with data for the same services in both the pretrial and trial years. To ensure stable estimates for this analysis, we required each program to have at least 100 patients eligible for analysis in each of the pretrial and trial years. Since neither the attending physicians nor their patients underwent randomization to flexible programs within a hospital, we also report the average risk of death at the time of admission among patients who were included in this focused analysis and analyze the risk-adjusted outcomes (Tables S8 and S9 in the Supplementary Appendix).