Author + information
- Received January 9, 2014
- Revision received May 8, 2014
- Accepted May 23, 2014
- Published online November 1, 2014.
- Mehdi H. Shishehbor, DO, MPH, PhD∗∗ (, )
- Sridhar Venkatachalam, MD∗,
- William A. Gray, MD†,
- Christopher Metzger, MD‡,
- Brajesh K. Lal, MD§,
- Lei Peng, MSc‖,
- Hend L. Omran, BA‖ and
- Eugene H. Blackstone, MD∗
- ∗Heart and Vascular Institute, Cleveland Clinic, Cleveland, Ohio
- †Department of Medicine, Center for Interventional Vascular Therapy, Columbia University, New York, New York
- ‡Wellmont CVA Heart Institute, Kingsport, Tennessee
- §Department of Surgery, University of Maryland School of Medicine, Baltimore, Maryland
- ‖Endovascular Clinical Science Division, Abbott Laboratories, Santa Clara, California
- ↵∗Reprint requests and correspondence:
Dr. Mehdi H. Shishehbor, Department of Cardiovascular Medicine, Desk J3-05, Heart and Vascular Institute, Cleveland Clinic, 9500 Euclid Avenue, Cleveland, Ohio, 44195.
Objectives This study sought to examine operator experience measured by time-related variables on outcomes with protected carotid artery stenting (CAS).
Background Studies on experience have focused on operator and institutional CAS volumes alone in the absence of a better metric.
Methods Using the CHOICE (Carotid Stenting for High Surgical-Risk Patients; Evaluating Outcomes Through the Collection of Clinical Evidence) multicenter prospective data from October 1, 2006 to June 1, 2012, 5,841 evaluable subjects were identified. Operator experience within this study was assessed using 5 variables for each operator: 1) baseline CAS volume; 2) time from first CAS to each subsequent CAS; 3) time between each CAS; 4) CAS volume in the institution; and 5) medical specialty (cardiology, surgery, or radiology/neurology). Institutional experience was determined by CAS volume within the study. Embolic protection device dwell time was used to assess technical performance, and 30-day death, stroke, or myocardial infarction composed the clinical outcome. Hierarchical logistic regression and linear mixed models were used.
Results Cardiologists (p < 0.001) along with operators with longer time interval from first CAS (p < 0.001) had reduced embolic protection device dwell times (technical performance). Increased time interval between CAS was the only independent predictor of 30-day death, stroke, or myocardial infarction (adjusted odds ratio: 1.05, 95% confidence interval: 1.02 to 1.09, p = 0.005). Prolonged embolic protection device dwell time was associated with 30-day death, stroke, or myocardial infarction (adjusted odds ratio: 1.08; 95% confidence interval: 1.01 to 1.17; p = 0.03).
Conclusions The time interval between CAS procedures, specialty assignment, and time from first CAS are important measures of operator experience that may significantly affect technical performance and clinical outcome.
Previous surveys have demonstrated a close association between carotid artery stenting (CAS) experience as represented by operator or institution volume and the occurrence of periprocedural adverse events (1–3). In fact, in the absence of a better measure, multispecialty consensus documents recommend a minimum number of CAS procedures to achieve competence, and this has also been a requirement for randomized trials (4–6). However, this simplistic and broad approach may be inadequate (7–9).
In an attempt to improve our understanding of operator-related factors on technical performance for CAS, we hypothesized that the embolic protection device (EPD) dwell time previously shown to be predictive of periprocedural stroke risk (10,11) may potentially serve as an acceptable metric. We expanded on the previous measures of experience (operator and institutional volume) by including time-dependent variables such as time elapsed since first procedure (as a representation of duration of time the operator has been practicing CAS) and time interval between procedures within the study. Additionally, real-time CAS volume for each operator was considered. The aim of this analysis accordingly was to test the effect of several operator and institution experience characteristics on EPD dwell time (technical performance) and 30-day death, stroke, or myocardial infarction (DSMI; clinical outcome) using the large prospective Acculink/Accunet subgroup of the CHOICE (Carotid Stenting for High Surgical-Risk Patients; Evaluating Outcomes Through the Collection of Clinical Evidence) clinical study.
Study design and patient selection
CHOICE was a prospective, single-arm, adjudicated, multicenter, post-market study designed to examine outcomes of CAS using Abbott Vascular (Santa Clara, California) carotid stents (Acculink and Xact) and embolic protection systems (Accunet, Emboshield, Emboshield Nav6). The study was initiated in 2006 and completed in 2012 at 366 sites in the United States and included 17,925 evaluable patients (96% of the total enrolled population) treated by 913 operators. Evaluable subjects were defined as patients who completed their 30-day follow-up visit or experienced DSMI within 30 days following the procedure. The current analysis pertains to the subgroup of all evaluable patients (N = 5,841) from the CHOICE clinical data who were exclusively treated with the Acculink carotid stent in conjunction with the Accunet distal EPD and for whom the EPD dwell time was available. Because operator learning curve and post-procedural outcomes may vary across devices, this analysis focused on a single stent, Acculink, and a single EPD, Accunet, which have the longest history in the CAS practice in the United States.
To be enrolled, a patient had to be considered to benefit from carotid revascularization but be at high surgical risk for carotid endarterectomy and be able to provide an informed consent for CAS. Carotid disease requiring treatment was defined as an ultrasound or angiographic stenosis of the common or internal carotid artery of ≥50% for symptomatic or ≥80% for asymptomatic patients. Symptomatic status was determined based on ipsilateral transient ischemic attack (including amaurosis fugax) or stroke within 180 days before the study procedure. There were no exclusion criteria for this study. The operators represented a broad range of medical specialties grouped under 3 main categories: cardiology, surgery, and radiology/neurology. The CHOICE study mandated institutional review board approval and oversight, adjudication of neurological events, and annual reporting of study progress to the Food and Drug Administration.
Patient demographics, comorbidities, previous treatment for carotid stenosis, aortic arch type, presence of aortic arch disease, target lesion characteristics, and procedural data were obtained prospectively and recorded using electronic case report forms. The baseline characteristics are summarized in Tables 1 and 2⇓. The patients (N = 5,841) were treated with Acculink/Accunet by 597 operators at 248 institutions. Overall, 14% of patients were symptomatic at the time of CAS, and 22% were octogenarians (Table 1). Patient characteristics differed across the 3 clinician medical specialties (Table 1). Cardiologists had a higher proportion of patients with concomitant coronary artery disease (71%) and congestive heart failure (26%). Alternatively, surgeons and radiologists/neurologists had more patients with previous interventions (42%) or carotid endarterectomy (40%) to the target lesion and symptomatic carotid disease (28%), respectively (Table 1). No clinically meaningful differences in vessel characteristics were noted among the 3 groups (Table 2).
Operator and institution experience characteristics
Operator experience characteristics were assessed using 5 variables for each operator within this analysis: 1) baseline operator CAS volume (self-reported total number of CAS procedures performed as primary operator before the operator enrolled his or her first patient in CHOICE); 2) time from first CAS to each subsequent CAS procedure for each operator; 3) time between each CAS procedure for the same operator; 4) operator volume (number of CAS procedures for each operator within the study); and 5) operator medical specialty. For each operator, the time from first CAS to each subsequent CAS procedure was calculated by sorting the procedure dates for each operator within the analysis. Similarly, the time between each CAS procedure within the analysis was calculated as the time elapsed since the previous procedure for the same operator. Institution experience characteristic was assessed using 1 variable – total CAS volume for each institution within the study (institution volume).
Per Table 1, cardiologists performed the majority of procedures (68%), followed by surgeons (23%) and radiologists/neurologists (9%). There was a wide variation in operator and institutional experience variables as shown in Table 3.
Technical performance and clinical outcome
The EPD dwell time, defined as the time between EPD deployment and recovery, was used to measure technical performance. Clinical outcome was assessed using a composite of post-procedure 30-day DSMI. In addition, 30-day death and stroke (DS) was used as an outcome for the adjusted analysis. All strokes and suspected strokes were adjudicated by an independent Clinical Events Adjudication Committee (Harvard Clinical Research Institute, Boston, Massachusetts) or a sponsor-internal adjudication committee. As per protocol, patients were assessed by a NIHSS (National Institutes of Health Stroke Scale) certified medical professional (nonoperator) or by an independent neurologist (i.e., nonoperator) at 3 pre-specified intervals: 1) within 14 days before CAS; 2) within 24 h post-procedure; and 3) at 30 days. Stroke was classified as major if the follow-up NIHSS assessment was >4 points from the pre-procedure score. In the absence of a 30-day NIHSS, a stroke was adjudicated as major if significant symptoms persisted beyond discharge. All other strokes were categorized as minor. Death and MI were site reported if they occurred anytime through the 30-day follow-up assessment. The protocol definition of MI was based on the development of new pathological Q waves in ≥2 leads (Q-wave MI) or elevation of creatine kinase levels to >3× the upper limit of normal accompanied by elevated creatine kinase-myocardial band level greater than the upper limit of normal (non–Q-wave MI). The protocol did not require routine testing with electrocardiogram or cardiac enzyme measurements after the procedure. For the purpose of endpoint analyses, patients who had an additional CAS procedure ≥31 days after the previous CAS procedure were consented again and counted as a new patient. For patients who were treated with bilateral CAS procedures within an interval of ≤30 days, endpoints were assessed with respect to each procedure, and baseline demographic and comorbidity data from the first procedure were used in the analysis.
The analysis population (N = 5,841) was described overall and by clinician medical specialty, consisting of cardiology (n = 3,969), surgery (n = 1,331), and radiology/neurology (n = 541). Baseline demographics, comorbidities, lesion characteristics, and procedural data for the entire cohort and by clinician medical specialty are presented using descriptive summary statistics. Among the 5 operator variables, 2 (baseline operator volume and operator medical specialty) were defined at the operator level. The remaining 3 operator variables (time from first CAS to each subsequent CAS procedure, time between each CAS procedure, and operator CAS volume in the analysis) were defined at the patient level because the values of these variables vary for each patient. Similarly, institution volume was also a patient-level variable based on the definition.
The impact of operator and institution variables on EPD dwell time and 30-day DSMI were assessed using multivariable regression analyses while adjusting for important patient baseline variables. Of the 6 operator and institution variables, baseline operator CAS volume was available in 5,240 (of 5,841) patients, and the analysis therefore was limited to this population. The steps involved in the regression modeling process were as follows. First, patient baseline variables were examined with respect to distribution and amount of missing values. The EPD dwell time was not normally distributed; therefore, the log transformation of the EPD dwell time was used in the analysis. There was a small percentage of missing values in patient baseline variables (<2% in 24 variables, <4% in 7 variables, and <9% in 3 variables). To improve statistical efficiency and potentially avoid bias, missing patient baseline values were imputed using the fully conditional specification method based on predictive mean matching (12). This was verified by sensitivity analyses using the Markov chain Monte Carlo method, which assumes multivariate normality (13). Furthermore, multiple (n = 10) imputed datasets were created to reflect the uncertainties on the imputed values (12). Second, important baseline patient, lesion, and procedural variables (Tables 1 and 2) were identified using stepwise regression analyses (linear regression for dwell time and logistic regression for 30-day DSMI). In addition, bootstrap bagging methods with an inclusion frequency of 60% were used (14,15). Third, to incorporate the hierarchical nature of the data, statistical methods that address the within-operator clustering were employed. A linear mixed model was used for EPD dwell time, and a hierarchical logistic regression was used for 30-day DSMI. Both models included 6 operator and institution variables and adjusted for important patient baseline variables. The hierarchical regression analyses were performed on each of the 10 imputed datasets, and the respective results for EPD dwell time and DSMI were combined to generate valid inferences using the method by Rubin (13). All statistical analyses were performed using SAS software (version 9.2, SAS Institute, Cary, North Carolina) and R software (R Foundation, Vienna, Austria).
Experience and EPD dwell time
Among the 3 clinician medical specialties, cardiologists had the lowest mean EPD dwell time (p < 0.001) (Figure 1A). A pattern of lower EPD dwell time was also noted with increasing levels of operator and institutional experience (Figures 1B to 1F). In hierarchical multivariable linear regression analysis, operator experience, as measured by a prolonged time interval from first CAS to each subsequent CAS for the operator (p < 0.001) and cardiology specialty (vs. surgery or radiology/neurology, p < 0.001) were independent predictors of lower EPD dwell time (Table 4, Figures 2A and 2B). Moreover, a shorter time interval between each CAS procedure for the operator showed a trend toward lower EPD dwell time (p = 0.06). The presence of heavy calcification at the target lesion site, target lesion length, and pre- and post-procedure lesion stenosis were other significant predictors of prolonged EPD dwell time (Table 4).
Experience and DSMI
At 30-day post-procedure follow-up, DSMI and DS occurred in 4.1% and 3.6%, respectively, in a mixed population of symptomatic (14%) and asymptomatic (86%) patients with carotid stenosis. Occurrence of any stroke, major stroke, and fatal stroke was 3.2%, 0.8%, and 0%, respectively. In unadjusted analysis, cardiologists and surgeons had lower 30-day DSMI than did radiologists/neurologists (Figure 3A). For the other operator and institution variables, DSMI rates were similar across the different experience levels (Figures 3B to 3F). In the adjusted hierarchical multivariable logistic regression analysis (Table 5), operator experience, as measured by a prolonged time interval between each CAS procedure for the operator, was a significant predictor of DSMI (adjusted odds ratio: 1.05, 95% confidence interval: 1.02 to 1.09, p = 0.005) (Figure 4A). However, there was no association between medical specialty and DSMI (Figure 4B). Similar associations were also noted for the DS outcome (Table 5). Patient-related factors that were also significant predictors of DSMI included age, symptomatic carotid disease, history of coronary artery disease, aortic arch disease, and chronic kidney disease.
EPD dwell time and DSMI
The mean EPD dwell time for the entire analysis population (N = 5,841) was 13.5 ± 8.1 min. In unadjusted analysis, a reduced EPD dwell time was associated with better DSMI rates (Figure 5A). In a hierarchical multivariable logistic regression analysis, after adjusting for baseline patient variables and operator experience variables, prolonged EPD dwell time was associated with a higher 30-day DSMI rate (adjusted odds ratio: 1.08; 95% confidence interval: 1.01 to 1.17; p = 0.03) (Table 5, Figure 5B). The findings were again similar when tested for the 30-day DS outcome (Table 5).
In this large, multicenter, prospective analysis to evaluate outcomes with CAS using distal embolic protection (Acculink/Accunet only), our results demonstrate the following principal findings: 1) operator experience characterized by time from first CAS, time to each subsequent CAS, and time interval between CAS are closely associated with technical performance as measured by EPD dwell time; 2) longer time interval between CAS procedures is an independent predictor of increased 30-day DSMI; and 3) decreasing EPD dwell time as a measure for technical proficiency and patient selection is associated with improved clinical outcome (30-day DSMI). These findings suggest the possible need to reconsider our approach in defining operator experience that has thus far solely relied on baseline operator and institution CAS volumes.
With CAS, learning curves can be constructed in 2 broad domains: technical performance (EPD dwell time) and clinical outcome (30-day DSMI). Whether technical proficiency translates into better clinical outcome has not been previously studied, although earlier reports suggest an association between EPD dwell time and periprocedural stroke risk (10,11). In our adjusted analysis using the large amount of CHOICE Acculink/Accunet data, we demonstrate that a shorter EPD dwell time (technical proficiency) is independently associated with reduced 30-day DSMI. EPD dwell time is an important endpoint because it is dependent on both operator- and patient-related factors. The predictors of prolonged EPD dwell time (heavy calcification at the target lesion site, history of hypercholesterolemia, target lesion length, pre- and post-procedure lesion stenosis) suggest the importance of proper patient selection. Furthermore, it highlights technical proficiency in resolving complications during the procedure such as hemodynamic disturbances or no-reflow due to distal embolization.
This analysis of the CHOICE post-market study is well suited to provide deeper insights into the role of operator experience on outcomes. Unlike previous studies such as CAPTURE and CAPTURE 2 (Carotid RX Acculink/Accunet Post-Approval/Market Trials to Uncover Unanticipated or Rare Events), CHOICE enrolled patients at more institutions across the United States (1,10). Operators had a wide range of experience with CAS at baseline as measured by self-reported operator volume and also represented a variety of medical specialties. The hospital settings were diverse and included community, private, and teaching centers across the United States. Furthermore, stroke outcomes were adjudicated according to a standardized definition. These features combined with the statistical power provided by the large sample size (>5,000) are extremely important to address the effect of operator experience on outcomes.
Based on available evidence, the association between operator experience and CAS outcomes follows the pattern previously described for coronary angioplasty—the inverse “volume-outcome relationship” (16,17). In CAPTURE 2 (n = 5,297), Gray et al. (2) found an inverse relationship between CAS operator volume and 30-day DS. A similar relationship was noted with hospital CAS volume. The findings were consistent despite stratification by 2 medical specialties, interventional cardiology and vascular surgery. However, 82% of the operators and 66% of the institutions observed no events; therefore, their conclusions were based on the few operators and institutions with events. Moreover, the study by Gray et al. (2) was limited to the asymptomatic nonoctogenarian patient population. In another large CAS study using Medicare beneficiaries ≥65 years (n = 24,701), Nallamothu et al. (3) found an inverse relationship between annual operator volume and 30-day mortality. Despite the large sample size, the study was limited by inclusion of CAS without EPD use, lack of data on stroke or MI, and the absence of lesion or procedural characteristics in the adjusted analyses.
In the present analysis, we attempted to improve on the available evidence to date by introducing 2 time-related experience variables that were not evaluated previously but could potentially affect outcome. Time from first CAS to each subsequent CAS procedure for the operator is important for 2 reasons. First, operators are constantly improving their skills with every vascular intervention they perform even outside of carotids. A longer time from first enrollment would enable us to incorporate this collateral experience. Second, it may facilitate accounting for technological advancements that could potentially affect operator outcomes. The other time-related variable, namely the time between each CAS procedure, is especially meaningful in the context of low- or medium-volume operators or during the operator’s early learning curve. Contrary to other reports, our multivariable analysis failed to show a significant association between CAS volumes (operator or institution) and 30-day DSMI. This is possible because operators and institutions with higher CAS volumes also tend to treat more complex patients such as those with symptoms and octogenarians. Also, some operators may not have enrolled all the patients on whom they performed CAS during the study period. The time between each CAS procedure for the operator was found to be a significant predictor of DSMI, implying the need to quantify operator CAS volume over shorter intervals (procedures per week or month) rather than per year or as total numbers. Given the strong association of EPD dwell time with DSMI and of interval between CAS procedures and EPD dwell time, it may be possible to establish a common goal that would reflect more dynamic best practices for CAS operators that reflect ongoing expertise in addition to expertise and training at baseline alone.
First, CHOICE is essentially a descriptive study designed to investigate post-marketing 30-day DSMI outcomes and is neither randomized nor controlled to study varying levels of operator expertise. However, all events were prospectively adjudicated. Second, the operator volumes, institution volumes, and time between each CAS procedure were not inclusive of CAS procedures outside of the study, which likely resulted in underestimation of true experience. Although causal inferences cannot be drawn, the overall findings are useful in improving our understanding of time-dependent measures of experience. Third, EPD dwell time may also be influenced by arch anatomy and technical difficulty with crossing lesions. Nevertheless, given the association with 30-day DSMI, careful patient selection based on anatomy and lesion characteristics is paramount regardless of operator experience. Because we did not study proximal EPD, our findings cannot be extrapolated to this population. Lastly, data on medication use, particularly statins, were not available; this may have confounded our results.
In this large analysis from a multicenter, prospective study on the use of CAS with embolic protection, operator experience, represented by time-related variables, was closely associated with technical performance and 30-day DSMI. The time interval between CAS procedures and the time from first CAS to subsequent CAS are important measures of operator experience. Technical performance based on EPD dwell time is a significant predictor for 30-day DSMI. As with any hypothesis-generating research, future prospective studies will be needed to validate our findings.
The CHOICE trial was sponsored and funded by Abbott Vascular. Dr. Shishehbor is a consultant and educator for Abbott Vascular, Medtronic, Covidien, Spectranetics, and Bayer but has waived all personal compensation. Dr. Gray has served on the Advisory Board of Abbott Vascular; has received research support from Abbott Vascular; and has served on the Steering Committee of the CHOICE trial. Dr. Metzger has served as a consultant for Abbott Vascular, Cordis, and Medtronic; and has an agreement for performing “hands-on” carotid courses for Abbott Vascular, but he has received no stocks, interests, grants, or research support. Ms. Peng is a full-time, stock-holding employee of Abbott Vascular. Ms. Omran is a full-time employee of Abbott Vascular. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- carotid artery stenting
- death and stroke
- death, stroke, or myocardial infarction
- embolic protection device
- myocardial infarction
- National Institutes of Health Stroke Scale
- Received January 9, 2014.
- Revision received May 8, 2014.
- Accepted May 23, 2014.
- American College of Cardiology Foundation
- Gray W.A.,
- Yadav J.S.,
- Verta P.,
- et al.,
- for the CAPTURE Trial Collaborators
- Gray W.A.,
- Rosenfield K.A.,
- Jaff M.R.,
- et al.,
- for the CAPTURE 2 Investigators and Executive Committee
- Rosenfield K.,
- Babb J.D.,
- Cates C.U.,
- et al.
- Ahmadi R.,
- Willfort A.,
- Lang W.,
- et al.
- Chaturvedi S.,
- Matsumura J.S.,
- Gray W.,
- et al.,
- for the CAPTURE 2 Investigators and Executive Committee
- Stef van Buuren KG.-O.
- Rubin D.B.
- Breiman L.
- Efron B.,
- Tibshirani R.J.
- Hlatky M.A.,
- Dudley R.A.