Author + information
- Received March 18, 2013
- Accepted April 1, 2013
- Published online July 1, 2013.
- Vasim Farooq, MBChB∗,
- Yvonne Vergouwe, PhD†,
- Philippe Généreux, MD‡,
- Christos V. Bourantas, MD, PhD∗,
- Tullio Palmerini, MD‡,
- Adriano Caixeta, MD‡,
- Hector M. Garcìa-Garcìa, MD, PhD∗,
- Roberto Diletti, MD∗,
- Marie-angèle Morel, BSc§,
- Thomas C. McAndrew, MS‡,
- Arie Pieter Kappetein, MD, PhD⋮,
- Marco Valgimigli, MD¶,
- Stephan Windecker, MD#,
- Keith D. Dawkins, MD∗∗,
- Ewout W. Steyerberg, PhD†,
- Patrick W. Serruys, MD, PhD∗∗ ( and )
- Gregg W. Stone, MD‡
- ∗Department of Interventional Cardiology, Erasmus University Medical Centre, Thoraxcenter, Rotterdam, the Netherlands
- †Department of Public Health, Erasmus University Medical Center, Rotterdam, the Netherlands
- ‡Columbia University Medical Center/New York-Presbyterian Hospital, Cardiovascular Research Foundation, New York, New York
- §Cardialysis BV, Rotterdam, the Netherlands
- ⋮Department of Cardiothoracic Surgery, Erasmus University Medical Centre, Thoraxcenter, Rotterdam, the Netherlands
- ¶Department of Interventional Cardiology, Cardiovascular Institute, University of Ferrara, Ferrara, Italy
- #Cardiology Department, Bern University Hospital, Bern, Switzerland
- ∗∗Boston Scientific Corporation, Natick, Massachusetts
- ↵∗Reprint requests and correspondence:
Dr. Patrick W. Serruys, Interventional Cardiology Department, Erasmus MC, 's-Gravendijkwal 230, 3015 CE Rotterdam, the Netherlands.
Objectives This study sought to validate the Logistic Clinical SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) score in patients with non–ST-segment elevation acute coronary syndromes (ACS), in order to further legitimize its clinical application.
Background The Logistic Clinical SYNTAX score allows for an individualized prediction of 1-year mortality in patients undergoing contemporary percutaneous coronary intervention. It is composed of a “Core” Model (anatomical SYNTAX score, age, creatinine clearance, and left ventricular ejection fraction), and “Extended” Model (composed of an additional 6 clinical variables), and has previously been cross validated in 7 contemporary stent trials (>6,000 patients).
Methods One-year all-cause death was analyzed in 2,627 patients undergoing percutaneous coronary intervention from the ACUITY (Acute Catheterization and Urgent Intervention Triage Strategy) trial. Mortality predictions from the Core and Extended Models were studied with respect to discrimination, that is, separation of those with and without 1-year all-cause death (assessed by the concordance [C] statistic), and calibration, that is, agreement between observed and predicted outcomes (assessed with validation plots). Decision curve analyses, which weight the harms (false positives) against benefits (true positives) of using a risk score to make mortality predictions, were undertaken to assess clinical usefulness.
Results In the ACUITY trial, the median SYNTAX score was 9.0 (interquartile range 5.0 to 16.0); approximately 40% of patients had 3-vessel disease, 29% diabetes, and 85% underwent drug-eluting stent implantation. Validation plots confirmed agreement between observed and predicted mortality. The Core and Extended Models demonstrated substantial improvements in the discriminative ability for 1-year all-cause death compared with the anatomical SYNTAX score in isolation (C-statistics: SYNTAX score: 0.64, 95% confidence interval [CI]: 0.56 to 0.71; Core Model: 0.74, 95% CI: 0.66 to 0.79; Extended Model: 0.77, 95% CI: 0.70 to 0.83). Decision curve analyses confirmed the increasing ability to correctly identify patients who would die at 1 year with the Extended Model versus the Core Model versus the anatomical SYNTAX score, over a wide range of thresholds for mortality risk predictions.
Conclusions Compared to the anatomical SYNTAX score alone, the Core and Extended Models of the Logistic Clinical SYNTAX score more accurately predicted individual 1-year mortality in patients presenting with non–ST-segment elevation acute coronary syndromes undergoing percutaneous coronary intervention. These findings support the clinical application of the Logistic Clinical SYNTAX score.
The SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) score is as an objective, anatomical-based tool to determine the complexity of coronary artery disease and guide decision making between cardiac surgery and percutaneous coronary intervention (PCI) (1–5). Since the SYNTAX trial, numerous validation studies have confirmed the SYNTAX score to be an independent predictor of long-term mortality in a broad range of patient types (6–8). The use of the SYNTAX score is now advocated in both the U.S. and European revascularization guidelines (9,10). In addition, the U.S. Food and Drug Administration mandates the SYNTAX score as entry criteria in ongoing contemporary stent and structural heart disease trials, investigating percutaneous left main coronary intervention (EXCEL [Evaluation of XIENCE PRIME Everolimus Eluting Stent System (EECSS) or XIENCE V EECSS Versus Coronary Artery Bypass Surgery for Effectiveness of Left Main Revascularizations] Clinical Trial; NCT01205776) (11)and transcatheter aortic valve implantation (SURTAVI [Safety and Efficacy Study of the Medtronic CoreValve System in the Treatment of Severe, Symptomatic Aortic Stenosis in Intermediate Risk Subjects Who Need Aortic Valve Replacement]; NCT01586910).
The lack of clinical variables, and the lack of a more individualized approach of the SYNTAX score—with patients categorized as “low,” “intermediate,” or “high” risk—has recently been shown to be potentially misleading. In a post hoc analysis of the SYNTAX trial (12), relatively higher- and lower-risk patients were identified in the low and high SYNTAX score groups, respectively. The Logistic Clinical SYNTAX score (13)was designed to overcome these limitations, by augmenting the anatomical SYNTAX score with clinical variables, and to individualize long-term (1-year) mortality predictions in patients undergoing contemporary PCI.
The purpose of this study was to validate the Logistic Clinical SYNTAX score in patients with non–ST-segment elevation acute coronary syndromes (ACS), in order to further establish its clinical utility.
The ACUITY (Acute Catheterization and Urgent Intervention Triage Strategy) trial design has previously been described (14). In brief, this was a multicenter, prospective, randomized trial of patients with moderate- and high-risk non–ST-segment elevation ACS, managed with an early invasive strategy. Coronary angiography was performed in all patients within 72 h of randomization to a specific anticoagulation regime undertaken during the intervention. The choice of either bare-metal (BMS) or drug-eluting stents (DES) was per operator discretion; approximately 85% of the PCI population underwent DES implantation. First-generation sirolimus-eluting or paclitaxel-eluting stents were exclusively used if DES was to be implanted. Dual antiplatelet therapy with aspirin and clopidogrel was strongly recommended for at least 1 year. All major adverse events were adjudicated by an independent clinical events committee blinded to treatment assignment. In line with a previous validation study of the anatomical SYNTAX score in the ACUITY trial (8), the subgroup of PCI patients in whom quantitative coronary angiography was performed in the formal angiographic substudy of the ACUITY trial (n = 2,627) was used for validation in the present study (15). All patients underwent anatomical SYNTAX score analyses by 3 interventional cardiologists, appropriately trained for SYNTAX score reading and blinded to the clinical outcomes (16).
Logistic Clinical SYNTAX score
The Logistic Clinical SYNTAX score has previously been developed and cross-validated (13)in a pooled population of 7 contemporary DES trials (n = 6,309) (2,17–23). It is composed of a Core Model (consisting of the anatomical SYNTAX score, age, creatinine clearance [CrCl], and left ventricular ejection fraction [LVEF]), and an Extended Model (consisting of an additional 6 clinical variables) to improve the accuracy of 1-year mortality predictions. During development and cross-validation (13), the Core Model was shown to substantially improve the predictive ability of the anatomical SYNTAX score, with a minor incremental benefit in improving mortality predictions for the Extended Model.
For validation purposes, all predictor values in the Core Model were present in >85% of patients. For the additional predictors of the Extended Model, all values were present in >90% of patients, except peripheral vascular disease (PVD) (not recorded) and body mass index (weight [100%, n = 2,627], height [36%, n = 945]). The definition of PVD was expanded to include previous transient ischemic attack or cerebrovascular accident, because these variables represented extracardiac arteriopathy. This modification to the variable was rerun in the original development and cross-validation population (13), with no change in predictive performance of the Logistic Clinical SYNTAX score (Fig. 1). Multiple imputation (5×) of missing values was undertaken using an imputation strategy that takes the correlation between all potential predictors into account. The method of chained equations with the Multivariate Imputation by Chained Equations (MICE) algorithm in R software (R Foundation for Statistical Computing, Vienna, Austria) was used (23,24).
Continuous variables are expressed as means ±SD or medians and interquartile ranges (IQR), as appropriate. Binary variables are expressed as counts and/or percentages. The possible nonlinearity of the continuous predictors of the Core Model with 1-year all-cause death in the ACUITY trial were assessed with restricted cubic spline functions. These are flexible functions that can accommodate curves in the form of the association to assess the assumption that patient characteristics are linearly related to the log odds of the outcome event (23,25). Statistical analyses were performed with R software (24)and SPSS (version 17.0, SPSS Inc., Chicago, Illinois).
Calibration, discrimination, and clinical usefulness
Calibration, discrimination, and clinical usefulness (23)were assessed for the Logistic Clinical SYNTAX score (Core and Extended Models) and anatomical SYNTAX score in the ACUITY trial. Calibration refers to the agreement between observed and predicted outcomes. The possible over- or underestimation of the predicted risks were graphically assessed with validation plots. Discrimination was studied with the concordance (C) index, which is identical to the area under the receiver-operating characteristic curve. The C-index estimates the probability that, of two randomly chosen patients, the patient with the more favorable prognostic score will outlive the patient with the less favorable prognostic score, and ranges from 0.5 (no discrimination) to a theoretical maximum of 1.
Clinical usefulness was assessed using decision curve analyses (26–29). These analyses estimate a “net benefit” for prediction models that provide individual risk estimates. Patients are classified as high or low risk at a chosen threshold value. The net benefit considers the benefit of the classification (patients correctly classified as dying within 1 year) and the harms (patients wrongly classified as dying within 1 year). The threshold value for classification is used in the decision-curve analysis to weigh correctly classified patients against wrongly classified patients. The references were that everyone was classified as high risk (i.e., died at 1 year), or that everyone was classified as low risk (i.e., alive at 1 year). The interpretation of the decision curve is that the model with the highest net benefit, at a particular threshold value, is the preferred model.
The incidence of all-cause mortality in the quantitative coronary angiography–PCI cohort of the ACUITY trial (n = 2,627) was 2.4% at 1 year (62 deaths). Baseline characteristics are shown in Table 1(30). The mean age was 60.7 ± 11.7 years; 33% were women; 62% of patients presented with non–ST-segment elevation myocardial infarction and the remainder with unstable angina. Approximately 85% of patients were implanted with DES, and 14% implanted with BMS. The median anatomical SYNTAX score was 9.0 (IQR: 5 to 16) with a maximum value of 59.5. Approximately 40% of the study population had 3-vessel disease, and 28.5% had diabetes mellitus.
The univariate associations of the variables in the Core Model (age, CrCl, LVEF, anatomical SYNTAX score) with 1-year mortality in the ACUITY trial are illustrated in Fig. 2.
Validation of the Logistic Clinical SYNTAX score
For the anatomical SYNTAX score, a good agreement was found between observed and predicted mortality outcomes (Fig. 3). For the Logistic Clinical SYNTAX score (Fig. 3), the Core Model demonstrated good agreement between observed and predicted mortality outcomes at the lower risks, with some underestimation at the higher risks (>5%). The Extended Model demonstrated good agreement between observed and predicted mortality outcomes across all recorded risk ranges.
The area under the receiver-operating characteristic curve demonstrated a substantially higher predictive accuracy of the Core and Extended Models for 1-year all-cause death, compared with the anatomical SYNTAX score in isolation (Fig. 4).
The net benefit of the Core and Extended Models, and the anatomical SYNTAX score, are shown on the y-axis on the decision curves (Fig. 5). The net benefit was highest for the Extended Model across all potential threshold values of 1-year mortality. This was followed by the Core Model, and then the anatomical SYNTAX score.
In this study, the Logistic Clinical SYNTAX score was validated in a large, prospective, randomized trial of patients with predominantly moderate- and high-risk non–ST-segment elevation ACS undergoing PCI (8). These findings indicate that the Logistic Clinical SYNTAX score is statistically robust, having previously been cross-validated (“internal-external” validation procedure ) during development in >6,000 patients in 7 contemporary stent trials (13). The current study provides further legitimacy toward the clinical application of the Logistic Clinical SYNTAX score. In addition, there are several notable findings: 1) In patients with non–ST-segment elevation ACS, both the Core (age, CrCl, LVEF, and anatomical SYNTAX score) and Extended Models (Core Model and additional 6 variables) substantially improved the predictive accuracy of 1-year mortality predictions, compared with the anatomical SYNTAX score alone. 2) The Core and Extended Models were both shown to discriminate well in the study population; the Core Model underestimated mortality predictions in patients at higher risk (>5%), which the Extended Model corrected for. 3) Decision-curve analyses, a method to assess clinical usefulness, confirmed the progressive improvement in 1-year mortality predictions over a wide range of thresholds, with the Extended Model versus the Core Model versus the anatomical SYNTAX score.
One of the main messages of the present study, and of the previous development and cross-validation study of the Logistic Clinical SYNTAX score (13), was that the Core Model was shown to contain most of the predictive information for 1-year mortality. The Extended Model, which incorporates 6 additional clinical variables to the Core Model, including diabetes and peripheral vascular disease, yielded only modest incremental improvement for 1-year mortality predictions. However, the Extended Model proved to be of additional clinical value in patients with non–ST-segment elevation ACS in improving the accuracy of higher risk predictions (>5%), as shown by the validation plots (Fig. 3). In addition, the decision-curve analyses showed that the Extended Model provided additional clinical value compared with the Core Model, by improving the accuracy of 1-year mortality predictions across all considered risk threshold values (Fig. 5).
Decision-curve analyses were recently proposed as an assessment of clinical usefulness and weigh the harms (false positives) against the benefits (true positives) of using a risk score to make treatment decisions (27,29). Comparatively, the C-index is the probability of the correct ordering of risks (i.e., of 2 randomly chosen patients, the patient with the lower prognostic score will outlive the patient with the higher prognostic score), and has previously been suggested to be insensitive in detecting the clinical value of a risk prediction score (23,29,32,33). Indeed, in the present study, only a minor increase in the C-index was observed, when comparing the Extended and Core Models (C-index: Extended Model: 0.77, Core Model: 0.74) (Fig. 4).
Nevertheless, as the Core Model retained a substantial superior ability over the anatomical SYNTAX score in predicting 1-year mortality, the Core Model can be used with reasonable accuracy, with the knowledge that the Extended Model would improve the mortality predictions, particularly in the higher-risk ranges.
The FREEDOM (Future Revascularization Evaluation in Patients With Diabetes Mellitus: Optimal Management of Multivessel Disease) trial has recently reported that diabetics with multivessel disease (predominantly 3- vessel disease, without left main involvement), to confer a mortality benefit in undergoing coronary artery bypass graft surgery compared with PCI with DES (34). The present study provides insights to the potential importance of diabetes as a risk factor for PCI. With the Logistic Clinical SYNTAX score, diabetes added only minor improvement to the predictive accuracy of the Core Model, as 1 of 6 other variables in the Extended Model. Essentially the Core Model captures most of the patient comorbidity, with variables such as CrCl, and is therefore likely to be a reflection of the presence of systemic and coronary atherosclerotic burden. Evidence to support this hypothesis comes from a population-level cohort study, demonstrating that the rate of myocardial infarction was substantially higher in nondiabetics with chronic kidney disease than in diabetics without chronic kidney disease (35). In addition, a recently published large meta-analysis demonstrated the importance of kidney disease as a predictor of clinical outcomes, including mortality, irrespective of the presence or absence of diabetes (36). Thus, the outcomes for patients with diabetes after PCI may be favorable if end-organ manifestations of diabetes are not yet present, such as chronic kidney disease (37).
The ACUITY trial population had relatively low anatomical SYNTAX scores. This is likely to be representative of real-world practice, because other contemporary, all-comer stent trials reported similar mean anatomical SYNTAX scores to those of the ACUITY trial (17,19–22). Despite this limitation, 3-vessel disease still represented approximately 40% of the study population in the ACUITY trial. Second, BMS were used in only 14% of the study population, whereas the Logistic Clinical SYNTAX score was developed and cross-validated in patients undergoing DES implantation (13). The validation process was rerun in the ACUITY trial with BMS patients excluded, and the predictive performance of the Logistic Clinical SYNTAX score did not change (data not shown). Third, because the present study assessed first-generation DES, we cannot exclude the possibility of improved mortality with newer generation DES. In the original development and cross-validation study of the Logistic Clinical SYNTAX score, composed of 7 contemporary stent trials and >6,000 patients, stent generation (first against newer generation) was not shown to have an impact on 1-year mortality (13). Conversely, newer generation DES have been associated with reductions in definite stent thrombosis and composite clinical outcomes; however, improvements in mortality have not been shown (38,39). In addition, we cannot exclude the possibility of newer generation antiplatelet therapy to have had an impact on clinical outcomes (40). Fourth, cardiogenic shock is an important subset of patients that cannot be assessed with the Logistic Clinical SYNTAX score, due to under-recruitment of these patient types in all-comer stent trials, predominantly due to the inability to gain appropriate informed consent (13,41). Lastly, we cannot exclude the possibility of the addition of a functional component to the calculation of the anatomical SYNTAX score to improve the predictive accuracy of the Logistic Clinical SYNTAX score (42).
Compared with the anatomical SYNTAX score alone, the Core and Extended Models of the Logistic Clinical SYNTAX score more accurately predict individual 1-year mortality in patients presenting with non–ST-segment elevation ACS undergoing PCI. These findings provide further legitimacy toward the clinical application of the Logistic Clinical SYNTAX score.
The authors thank all of the study participants whose work made this study possible.
For the logistic regression formula used to predict 1-year death, please see the online version of this paper.
Dr. Kappetein has reported that he has served on the Steering Committee of the SYNTAX trial. Dr. Valgimigli has reported that he has received research grants for lectures and advisory boards from Iroko, Eli Lilly, Medtronic, and honoraria for lectures and/or advisory boards from Cordis, Medtronic, Abbott, Eisai, Merck & Co., Inc., AstraZeneca, MedCo, and Terumo. Dr. Windecker has reported that he has received research grants (paid to his institution) from Abbott, Biosensors, Biotronik, Cordis, Boston Scientific, Medtronic, and St. Jude Medical. Dr. Dawkins has reported that he is a full-time employee of Boston Scientific and holds stock in Boston Scientific. Dr. Stone has reported that he has served as a consultant to Boston Scientific. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose. Drs. Farooq and Vergouwe contributed equally to this paper. David E. Kandzari, MD, has served as Guest Editor.
- Abbreviations and Acronyms
- acute coronary syndromes
- bare-metal stent(s)
- creatinine clearance
- drug-eluting stent(s)
- interquartile range(s)
- left ventricular ejection fraction
- percutaneous coronary intervention
- peripheral vascular disease
- Received March 18, 2013.
- Accepted April 1, 2013.
- American College of Cardiology Foundation
- ↵SYNTAX Working Group. SYNTAX score calculator. [Online tool]. May 19, 2009. Available at: http://www.syntaxscore.com. Accessed November 1, 2012.
- Kappetein A.P.,
- Feldman T.E.,
- Mack M.J.,
- et al.
- Garg S.,
- Sarno G.,
- Girasis C.,
- et al.
- Farooq V.,
- Brugaletta S.,
- Serruys P.W.
- Palmerini T.,
- Genereux P.,
- Caixeta A.,
- et al.
- Levine G.N.,
- Bates E.R.,
- Blankenship J.C.,
- et al.
- Wijns W.,
- Kolh P.,
- Danchin N.,
- et al.,
- Task Force on Myocardial Revascularization of the European Society of Cardiology (ESC) and the European Association for Cardio-Thoracic Surgery (EACTS)
- Farooq V.,
- Serruys P.W.,
- Stone G.W.,
- Virmani R.,
- Chieffo A.,
- Fajadet J.
- Farooq V.,
- Vergouwe Y.,
- Räber L.,
- et al.
- Aoki J.,
- Lansky A.J.,
- Mehran R.,
- et al.
- Genereux P.,
- Palmerini T.,
- Caixeta A.,
- et al.
- Valgimigli M.,
- Campo G.,
- Percoco G.,
- et al.,
- MULTISTRATEGY Investigators
- Steyerberg E.W.
- ↵R Development Core Team. A language and environment for statistical computing. 2006. Available at: http://www.R-project.org/. Accessed August 1, 2011.
- Harrell F.E.
- Vickers A.J.,
- Elkin E.B.
- Steyerberg E.W.,
- Vickers A.J.
- Cook N.R.
- Pepe M.S.,
- Janes H.,
- Gu J.W.
- Farooq V,
- van Klaveren D.,
- Steyerberg E.W.,
- et al.
- Bangalore S.,
- Kumar S.,
- Fusaro M.,
- et al.
- Alexopoulos D.,
- Galati A.,
- Xanthopoulou I.,
- et al.
- de Boer S.P.,
- Lenzen M.J.,
- Oemrawsingh R.M.,
- et al.
- Nam C.W.,
- Mangiacapra F.,
- Entjes R.,
- et al.,
- FAME Study Investigators