Author + information
- Received July 24, 2012
- Revision received March 12, 2013
- Accepted March 15, 2013
- Published online August 1, 2013.
- J. Matthew Brennan, MD, MPH∗,
- Jeptha P. Curtis, MD†,
- David Dai, PhD, MS∗,
- Susan Fitzgerald, MS, RN‡,
- Akshay K. Khandelwal, MD§,
- John A. Spertus, MD⋮,
- Sunil V. Rao, MD∗,
- Mandeep Singh, MD¶,
- Richard E. Shaw, MD#,
- Kalon K.L. Ho, MD, MSc∗∗,
- Ronald J. Krone, MD††,
- William S. Weintraub, MD‡‡,
- W. Douglas Weaver, MD§,
- Eric D. Peterson, MD, MPH∗∗ (, )
- National Cardiovascular Data Registry
- ∗Department of Medicine, Duke Clinical Research Institute, Durham, North Carolina
- †Department of Internal Medicine, Yale University School of Medicine, New Haven, Connecticut
- ‡American College of Cardiology Foundation, Washington, DC
- §Department of Medicine, Henry Ford Hospital, Detroit, Michigan
- ⋮Department of Medicine, Saint Luke's Mid America Heart Institute, Kansas City, Missouri
- ¶Department of Medicine, Mayo Clinic, Rochester, Minnesota
- #Department of Medicine, Sutter Pacific Heart Centers, San Francisco, California
- ∗∗Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
- ††Department of Medicine, Washington University School of Medicine, St. Louis, Missouri
- ‡‡Department of Medicine, Christiana Health Care System, Newark, Delaware
- ↵∗Reprint requests and correspondence:
Dr. Eric D. Peterson, Duke Clinical Research Institute, 2400 Pratt Street, Durham, North Carolina 27705.
Objectives This study sought to update and validate a contemporary model for inpatient mortality following percutaneous coronary intervention (PCI), including variables indicating high clinical risk.
Background Recently, new variables were added to the CathPCI Registry data collection form. This modification allowed us to better characterize the risk of death, including recent cardiac arrest and duration of cardiogenic shock.
Methods Data from 1,208,137 PCI procedures performed between July 2009 and June 2011 at 1,252 CathPCI Registry sites were used to develop both a “full” and pre-catheterization PCI in-hospital mortality risk model using logistic regression. To support prospective implementation, a simplified bedside risk score was derived from the pre-catheterization risk model. Model performance was assessed by discrimination and calibration metrics in a separate split sample.
Results In-hospital mortality was 1.4%, ranging from 0.2% among elective cases (45.1% of total cases) to 65.9% among patients with shock and recent cardiac arrest (0.2% of total cases). Cardiogenic shock and procedure urgency were the most predictive of inpatient mortality, whereas the presence of a chronic total occlusion, subacute stent thrombosis, and left main lesion location were significant angiographic predictors. The full, pre-catheterization, and bedside risk prediction models performed well in the overall validation sample (C-indexes 0.930, 0.928, 0.925, respectively) and among pre-specified patient subgroups. The model was well calibrated across the risk spectrum, although slightly overestimating risk in the highest risk patients.
Conclusions Clinical acuity is a strong predictor of PCI procedural mortality. With inclusion of variables that further characterize clinical stability, the updated CathPCI Registry mortality models remain well-calibrated across the spectrum of PCI risk.
- American College of Cardiology
- National Cardiovascular Data Registry CathPCI Registry
- percutaneous coronary intervention
- risk prediction
The CathPCI Registry mortality risk prediction models were developed to standardize assessment of percutaneous coronary intervention (PCI) outcomes in a discipline where patient characteristics are associated with more than a 100-fold variation in expected mortality (1,2). These models have multiple clinical applications, including use for patient and family counseling (3) and serving as a foundation for quality assessment and improvement initiatives (4,5). To satisfy these various applications, 3 mortality risk models have been previously developed, including a “full” model with angiographic details, a pre-catheterization model, and a simplified bedside risk score (2). Although each of these models has unique strengths, all 3 have been shown to have excellent performance in contemporary clinical practice (2).
Whereas the existing CathPCI Registry mortality models have many assets, they have been criticized for failing to accurately define risk among “extreme risk” patients (6), such as those with cardiogenic shock and those who have suffered cardiac arrest prior to PCI. This criticism has led to concerns about whether decision makers will adopt risk-averse patterns of patient care, especially as public outcome reporting and payment incentives for procedural outcomes are becoming increasingly common (7,8).
In response to these concerns, and to further define risk at the highest end of the spectrum, a series of new variables were included in the 2009 updated Version 4 CathPCI Registry data clarification form (DCF v4). These variables have recently been incorporated into the CathPCI Registry risk adjustment model that is currently used for site-level outcome reporting. In this analysis, we sought to: 1) assess the association between clinical instability and in-hospital mortality following PCI in contemporary practice; and 2) evaluate the performance of the updated CathPCI Registry DCF v4 mortality models across the spectrum of procedural risk.
The CathPCI Registry is an initiative of the American College of Cardiology Foundation and the Society for Cardiovascular Angiography and Interventions. Details of the registry and the development of its mortality risk prediction models have been reported previously (2). In the present analysis, patients were included if they underwent a PCI procedure at any of the 1,253 participating centers contributing data to the CathPCI Registry using DCF v4 between July 1, 2009, and June 30, 2011. Patients were excluded if they were transferred to another acute care center from the center performing the index procedure (8,619 patients were excluded, including all patients at 1 center). Only the first PCI procedure per admission was included. The resulting cohort was randomly allocated to either a model development (60%) or a model validation (40%) sample, with stratification according to the presence of ST-segment elevation myocardial infarction (STEMI) to ensure appropriate distribution of these high-risk patients. As the primary analytic center for the CathPCI Registry, the Duke Clinical Research Institute performed the analyses for this risk prediction model, had complete access to the data, and vouches for its integrity.
Beginning in July 2009, several updated and new variables were incorporated into the CathPCI Registry DCF v4, including those describing cardiogenic shock and cardiac arrest. In DCF v4, “cardiogenic shock” was redefined using a systolic blood pressure <90 mm Hg and/or cardiac index <2.2 l/min/m2 (vs. systolic blood pressure <80 mm Hg and/or cardiac index <1.8 l/min/m2 in DCF v3), and it was further classified as occurring within 24 h or at the start of the PCI procedure (vs. at any time since hospital admission in DCF v3). “Cardiac arrest,” a new variable in DCF v4, included pulseless clinical scenarios requiring cardiopulmonary resuscitation within 24 h of the PCI procedure. The definitions for other data elements are available on the NCDR (National Cardiovascular Data Registry) website (9).
To delineate pre-procedural clinical instability, a composite variable including cardiogenic shock and procedural status was developed for use in the DCF v4 mortality models, with the following 6 ordinal categories describing decreasing levels of procedural urgency: 1) sustained shock (occurring within 24 h and at the start of PCI) and salvage; 2) sustained shock alone or salvage alone; 3) transient shock (occurring within 24 h or at the start of PCI, but not both) without salvage; 4) emergent PCI without shock; 5) urgent PCI without shock; and 6) elective PCI.
In this cohort, missing data were rare (<5%) for all variables except glomerular filtration rate (GFR) (8%) and ejection fraction (EF) (29%) and were imputed as follows: 1) for variables pertaining to past medical history, pre-procedural intra-aortic balloon pump, presence of subacute stent thrombosis, and “highest risk” coronary lesion, missing data were imputed to “no”; 2) for body mass index (BMI), missing values were imputed to the sex-specific median; 3) for GFR, missing values were imputed to the sex-, prior renal failure-, and STEMI-specific median; and 4) for EF, missing data were imputed to the strata-specific median based on a history of congestive heart failure, prior myocardial infarction, pre-procedural cardiogenic shock, and the presence of STEMI. These imputation rules have been previously shown to yield results similar to those using multiple imputation methods (2).
Potential candidate variables were screened using both clinical judgment and knowledge gained from previous mortality models. Following a review of the univariate association between potential variables and in-hospital mortality, a multivariate logistic regression with backward selection was performed to identify variables for the final model (p < 0.05 for model inclusion). Spline functions were evaluated for all continuous covariates and were used in the modeling of EF (60%), BMI (30 kg/m2), age (70 years), and GFR (30 and 90 ml/min/1.73 m2).
As with the previous CathPCI Registry model development efforts, 3 models were developed, including: 1) a “full” model with all selected candidate variables; 2) a “pre-cath” model excluding angiographic data; and 3) a “limited” pre-cath risk prediction model including only variables with the strongest impact on the full model, based on Wald chi-square values. The full risk prediction model is currently used for institutional reporting of risk-adjusted outcomes, whereas the pre-cath and limited models have been developed to facilitate personalized patient consent and bedside risk approximation. Regression coefficients from the pre-cath model were converted to whole integers to create the simplified (“limited”) CathPCI Registry mortality risk prediction score (10). SAS statistical software (version 9.2, SAS Institute, Cary, North Carolina) was used for all calculations.
Comparison of old (v3 DCF) versus updated (v4 DCF) risk-adjustment models
The risk adjustment models presented here (v4 DCF) have important differences when compared with previously published models (v3 DCF). For each of the updated (v4 DCF) models, consistent variables have been recalibrated to fit the most contemporary data. Compared with the previous (v3 DCF) full model, the updated (v4 DCF) full model has substituted the “pre-procedural clinical stability” variables for those that previously described “PCI status.” Additionally, the new model version does not include stratification by “STEMI status” (STEMI vs. no STEMI) for GFR, BMI, lesion risk, or New York Heart Association class. Finally, indicators for recent cardiac arrest (<24 h), number of diseased vessels, and chronic total occlusions have been added to the updated (v4 DCF) models that are presented here. These changes have also been carried over to the pre-cath model.
For the simplified “bedside risk scoring system,” PCI status has been replaced with the composite “pre-procedural clinical stability” and the “cardiac arrest <24 h” variables (without stratification for STEMI status). Additionally, BMI, prior PCI, diabetes, and left ventricular ejection fraction have all been added to the simplified model.
Following development, each of the 3 models was applied to the validation sample. Model discrimination was assessed using a C-index, and model calibration was assessed by rank-ordering patients from lowest to highest predicted risk and comparing predicted versus observed event rates within risk strata. Although the Hosmer-Lemeshow goodness-of-fit test was expected to be significant in a sample of this size, results were provided for reference. Calibration and discrimination were further assessed within several important patient subgroups, including according to sex, age (≤70 years, >70 years), and presence of diabetes mellitus and STEMI. This model validation was performed as a part of the v4 model development process and is reported here to further qualify the validity of the current risk-adjustment model in contemporary practice.
Between July 2009 and June 2011, 1,249,547 PCI procedures were performed at 1,253 participating CathPCI Registry sites (Fig. 1). Following exclusions, 1,208,137 procedures from 1,252 sites remained, with an average age of 65 years and including 33% women, 36% with diabetes, and 40% with a prior PCI (Table 1). Pre-procedural and angiographic characteristics were well balanced across the cohorts used for mortality model development (n = 724,883) and validation (n = 483,254).
Pre-procedural clinical instability in contemporary practice
In total, 660,851 procedures (57%) were performed in nonelective clinical scenarios, including 37.4% urgent, 17.0% emergent, and 0.3% salvage cases. Percutaneous coronary intervention was performed within 24 h of a cardiac arrest in 1.8% (n = 21,746) of patients. Cardiogenic shock and salvage status were both relatively rare within 24 h prior to the PCI procedure, and the distribution of these high-risk indicators was similar across most PCI centers (median: 2.6% of PCI cases, interquartile range [IQR]: 1.7% to 3.7%).
Observed in-hospital mortality increased incrementally with increasing clinical acuity (Table 2). In the absence of cardiogenic shock, the risk of in-hospital mortality for elective, urgent, and emergent cases was 0.2%, 0.6%, and 2.3%, respectively. In the presence of transient shock but not salvage status, the risk of in-hospital mortality was 15.1%; with sustained shock or salvage, the risk was 33.8%; and, with sustained shock and salvage, the risk was 65.9%. After adjusting for patient demographics, comorbidities, and angiographic characteristics, clinical acuity retained the strongest association with in-hospital mortality (Table 3).
Updated CathPCI Registry mortality model performance
The full, pre-cath, and bedside risk adjustment models are provided in Tables 3 and 4. Compared with CathPCI Registry DCF v3 models, other variables new to the DCF v4 models included the treatment of chronic total occlusions (odds ratio [OR]: 1.59, 95% confidence interval [CI]: 1.42 to 1.78) and stent thrombosis (OR: 2.11, 95% CI: 1.71 to 2.59). Using the full CathPCI Registry DCF v4 model, the majority of patients (>95%) had a predicted mortality risk of <5%, and only a small minority of patients (<1.5%) had a predicted risk >20% (Fig. 2).
Each of the 3 models performed well in the overall validation sample (full model, C-index: 0.930; pre-cath model: 0.929; CathPCI Registry Risk Prediction Score: 0.925) and within each of the pre-specified patient subgroups (Table 5). The full model calibration plots are shown for low-risk (<5% predicted risk) and high-risk (>20% predicted risk) patients in Figure 3. The model was well calibrated throughout the range of risk, with slight underestimation of risk among those with a predicted risk of 20% to 50% and slight overestimation among the small minority of patients with a predicted risk >50%.
In contemporary practice, less than one-half of all PCI procedures (45%) are considered elective, and a large proportion of procedures (18%) are performed in the setting of clinical instability. Cardiogenic shock carries a significant incidence of in-hospital mortality (11), and early revascularization in this setting has been shown to improve both short- and long-term survival (12–14). In this analysis, we have shown that both the duration of shock and its association with cardiac arrest are important features in the estimation of predicted survival following PCI. Discrimination of risk within the rubric of “cardiogenic shock” is possible in the setting of a national cardiovascular procedural registry, and with the inclusion of more detailed data characterizing clinical acuity, accurate risk prediction is possible throughout the spectrum of procedural risk.
Although a minority of patients (<3%) present with either cardiogenic shock or salvage status, clinical acuity remains the strongest predictor of PCI procedural mortality in contemporary practice. As in the SHOCK (Should We Emergently Revascularize Occluded Coronaries for Cardiogenic Shock) trial (12), early mortality following PCI in the setting of cardiogenic shock remains high in contemporary practice. Nevertheless, despite the comparatively high mortality rates associated with cardiogenic shock, PCI in this setting is not a futile exercise. Among the sickest PCI patients with both sustained shock and a recent cardiac arrest, nearly 35% survived to hospital discharge. Without recent cardiac arrest, the probability of survival was considerably higher, with nearly 65% and 85% hospital survival among patients who underwent PCI in the setting of sustained or transient shock, respectively.
The probability of survival for patients with cardiogenic shock is higher in contemporary practice than was previously reported by the SHOCK trial (12); however, direct comparison of these results to those previously reported is challenging. Although the definition of cardiogenic shock is nearly identical between the SHOCK trial and that used for the current CathPCI Registry DCF, the SHOCK trial analysis did not stratify patients according to persistence of shock; rather, patients were randomized within 36 h of an index myocardial infarction and no more than 12 h after the diagnosis of shock (12). In that setting, 30-day survival among the revascularization cohort was 53.3%, which was considerably lower than that observed in contemporary practice—even among patients with sustained shock. Consistent with the SHOCK trial results, survival to hospital discharge among post–myocardial infarction cardiogenic shock or salvage patients treated in the CathPCI Registry between 1998 and 2002 was low (40.6%) (15). Nonetheless, the definition of cardiogenic shock in the previous American College of Cardiology-NCDR analysis was considerably more restrictive than in DCF v4 (systolic blood pressure <80 mm Hg, cardiac index <1.8 l/min/m2, and pulmonary capillary wedge pressure >20 mm Hg in the setting of clinical hypoperfusion, or the requirement for mechanical or pharmacologic support to maintain a systolic blood pressure >80 mm Hg or cardiac index >1.8 l/min/m2). Although the apparent contemporary improvements in the survival of patients with cardiogenic shock may be due to less restrictive data definitions or more restrictive patient selection, it is perhaps as likely that greater access to early revascularization and advances in both pharmacologic and mechanical treatment modalities are responsible for recent improvements in patient outcomes (11).
Despite a reasonable probability of survival following PCI for even the most acute patients, considerable trepidation persists among some in the interventional cardiology community that treatment of these patients may adversely affect publicly reported clinical outcomes. On the contrary, in this analysis, we have shown that the current CathPCI Registry mortality risk prediction models are well calibrated throughout the range of risk, with a slight overestimation of risk among the minority of patients with the highest predicted risk.
In addition to further stratification of clinical stability, the updated models have included an indicator for interventions involving chronic total occlusions, building on previous work that has suggested an increased procedural risk of mortality as compared with non–chronic total occlusion interventions—especially when the intervention is unsuccessful (16). Although the updated models perform well across hospitals, a quantitative appraisal of the actual impact of “extreme risk” cases on risk-adjusted hospital and provider outcomes remains the focus of ongoing research efforts.
Direct comparison of the current CathPCI Registry DCF v4 mortality models with previously published CathPCI Registry (DCF v3) models (2) is not possible. Even though calibration of the current models is similar to those previously developed (C-statistic: 0.930 vs. 0.924), the application of the 2 bedside risk prediction tools to a common clinical scenario may be instructive. In the case of a previously healthy patient 64 years of age who presents emergently to the cardiac catheterization lab with an anterior STEMI associated with transient cardiogenic shock and an EF of 45% without cardiac arrest, the patient's probability of in-hospital mortality following PCI is estimated at roughly 2% (43 points) using the current v4 CathPCI Registry model. By comparison, using the v3 CathPCI Registry model, this same patient would have had an estimated probability of in-hospital mortality of roughly 10% (49 points). As demonstrated here, although population-level risk discrimination is similar for each of these models, the difference in actual risk estimation is apparent among certain high-risk patient populations.
Risk prediction models have multiple clinical applications, including patient and family counseling, as a foundation for quality assessment and improvement. As an adjunct to patient counseling, the CathPCI Registry mortality models have been incorporated as the backbone of the recently presented Personalized Risk Information Services Manager, or PRISM, consent process (3). In addition, both the National Quality Forum (4) and the Leapfrog Group (5) have endorsed the use of the CathPCI Registry model of risk-adjusted in-hospital mortality following PCI as a quality standard; the measure is used by multiple payers, including Blue Cross Blue Shield, WellPoint, and UnitedHealthcare Services for both quality improvement and recognition-reward programs. Given their wide application, it is vital that these models are reflective of both: 1) the most current understanding of clinical characteristics associated with PCI-related mortality; and 2) the most contemporary PCI cohort available. The current CathPCI Registry mortality models are responsive to both of these needs.
Despite its expanding role as the largest clinical PCI registry in the United States, the CathPCI Registry under-represents smaller volume practices. Even though the NCDR has a Data Quality Program (which includes an assessment for reasonableness and completeness via the data quality report) and a data validation (audit) program (17), the data remain susceptible to collection and reporting biases and errors. Although variables indicating procedural acuity have strict data definitions, the accuracy of coding of these variables is particularly susceptible to center-level variation. In addition, even though the current version of the CathPCI Registry includes several important variables that are associated with a high-risk for in-hospital mortality, there may be other clinical characteristics (such as patient frailty) (18) that are also prognostically important. Such variables are difficult to capture in the context of a quality improvement registry, but may further refine risk prediction.
Despite apparent improvements in survival in the setting of high-risk PCI, peri-procedural clinical instability remains a powerful predictor of in-hospital mortality. With the inclusion of indicators for high-risk PCI, the updated CathPCI Registry DCF v4 mortality models perform well in both low- and high-risk PCI patient populations.
The authors would like to thank Erin LoFrese for her editorial contributions to this manuscript. Ms. LoFrese did not receive compensation for her assistance, except for her employment at the institution where this study was conducted.
This research was supported by the American College of Cardiology Foundation's National Cardiovascular Data Registry (NCDR). The views expressed in this manuscript represent those of the author(s), and do not necessarily represent the official views of the NCDR or its associated professional societies identified at www.ncdr.com. Dr. Curtis has received salary support from the American College of Cardiology and Centers for Medicare & Medicaid Services; he also owns stock in Medtronic. Dr. Khandelwal has received consulting fees from Terumo Medical Corp. Dr. Spertus has received consulting fees from the American College of Cardiology Foundation for the analysis of NCDR data. Dr. Weaver has served on the Data and Safety Monitoring Board of Boston Scientific. Dr. Peterson has received consulting fees from Janssen Pharmaceuticals, Boehringer Ingelheim, Pfizer, and Eli Lilly. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- body mass index
- confidence interval
- DCF v4
- Version 4 CathPCI Registry data clarification form
- ejection fraction
- glomerular filtration rate
- interquartile range
- odds ratio
- percutaneous coronary intervention
- ST-segment elevation myocardial infarction
- Received July 24, 2012.
- Revision received March 12, 2013.
- Accepted March 15, 2013.
- American College of Cardiology Foundation
- Califf R.M.,
- Peterson E.D.,
- Gibbons R.J.,
- et al.,
- for the American College of Cardiology and the American Heart Association
- Peterson E.D.,
- Dai D.,
- DeLong E.R.,
- et al.,
- for the NCDR Registry Participants
- ↵Spertus J, Gialde E, Chhatriwalla A, Gosch K, et al. Testing an evidence-based, individualized informed consent form to improve patients' experiences with PCI. Paper presented at: the American Heart Association 2011 Scientific Sessions; November 2011; Orlando, FL.
- ↵National Quality Forum. PCI mortality (risk-adjusted) [online forum]. May 9, 2007. Available at: http://www.qualityforum.org/MeasureDetails.aspx?actid=0&SubmissionId=1138#k=pci. Accessed November 7, 2011.
- ↵The Leapfrog Group. Website. Available at: http://www.leapfroggroup.org/home. Accessed November 7, 2011.
- Resnic F.S.,
- Normand S.L.,
- Piemonte T.C.,
- et al.
- Resnic F.S.,
- Welt F.G.
- ↵National Cardiovascular Data Registry. Coders' dictionary [pdf]. National Cardiovascular Data Registry web site. Available at: http://www.cardiologyinoregon.org/2012_NCDR_in_Oregon/CathPCI_v4_CodersDictionary_4.4.pdf. Accessed July 9, 2013.
- Reynolds H.R.,
- Hochman J.S.
- Hochman J.S.,
- Sleeper L.A.,
- Webb J.G.,
- et al.,
- for the SHOCK Investigators
- Urban P.,
- Stauffer J.C.,
- Bleed D.,
- et al.
- Suero J.A.,
- Marso S.P.,
- Jones P.G.,
- et al.
- Brindis R.G.,
- Fitzgerald S.,
- Anderson H.V.,
- Shaw R.E.,
- Weintraub W.S.,
- Williams J.F.
- Singh M.,
- Rihal C.S.,
- Lennon R.J.,
- Spertus J.A.,
- Nair K.S.,
- Roger V.L.