Author + information
- Received August 22, 2017
- Revision received October 24, 2017
- Accepted October 26, 2017
- Published online February 19, 2018.
- Devraj Sukul, MD, MSca,
- Deepak L. Bhatt, MD, MPHb,
- Milan Seth, MSa,
- Pearl Zakroysky, MPHc,
- Daniel Wojdyla, MScc,
- John S. Rumsfeld, MD, PhDd,
- Tracy Wang, MD, MHS, MScc,
- Sunil V. Rao, MDc and
- Hitinder S. Gurm, MDa,e,∗ ()
- aDepartment of Internal Medicine, Division of Cardiovascular Medicine, University of Michigan, Ann Arbor, Michigan
- bBrigham and Women’s Hospital Heart & Vascular Center, Harvard Medical School, Boston, Massachusetts
- cDuke Clinical Research Institute, Durham, North Carolina
- dDepartment of Internal Medicine, Division of Cardiology, University of Colorado School of Medicine, Denver, Colorado
- eDivision of Cardiovascular Medicine, VA Ann Arbor Healthcare System, Ann Arbor, Michigan
- ↵∗Address for correspondence:
Dr. Hitinder S. Gurm, 2A 394, 1500 East Medical Center Drive, University of Michigan Cardiovascular Center, Ann Arbor, Michigan 48109-5853.
Objectives This study sought to compare the appropriate use and outcomes of percutaneous coronary intervention (PCI) between top-ranked and nonranked hospitals.
Background The U.S. News & World Report “Best Hospitals” rankings are an influential consumer-directed publication of hospital quality, and are commonly used in promotional campaigns by hospital systems.
Methods Hospitals in the National Cardiovascular Data Registry CathPCI registry between July 1, 2014, and June 30, 2015, were classified as top-ranked if they were included in the 2015 U.S. News & World Report 50 best “Cardiology and Heart Surgery” hospitals. The remaining were classified as nonranked. We compared in-hospital mortality, post-procedural bleeding, post-procedural acute kidney injury, and the proportion of appropriate PCI procedures between top-ranked and nonranked hospitals.
Results A total of 509,153 PCIs at 654 hospitals were included, of which 55,550 (10.9%) were performed at 44 top-ranked hospitals. After adjusting for patient case mix, PCIs performed at top-ranked hospitals had similar odds of in-hospital mortality (adjusted odds ratio [aOR]: 0.96; 95% confidence interval [CI]: 0.83 to 1.12; p = 0.64), acute kidney injury (aOR: 1.10; 95% CI: 0.98 to 1.22; p = 0.099), and bleeding (aOR: 1.15; 95% CI: 0.99 to 1.31; p = 0.052). Top-ranked hospitals had a slightly lower proportion of appropriate PCI compared with nonranked hospitals (89.2% vs. 92.8%; OR: 0.56; 95% CI: 0.45 to 0.69; p < 0.001).
Conclusions PCI performed at top-ranked hospitals was not associated with superior outcomes compared with PCI at nonranked hospitals. The inclusion of metrics based on clinical data may be important for hospital quality rankings.
Over the past 25 years, numerous consumer-directed rankings of hospital quality have been developed in an effort to increase transparency and accountability in health care. The U.S. News & World Report “Best Hospitals” is arguably one of the most widely read and influential hospital profiling systems in the United States (1).
One of the goals of public reporting and hospital profiling initiatives is to guide consumers in selecting where they should receive health care when a choice exists. Previous research evaluating the validity of the “Best Hospitals” rankings for cardiovascular care found that, in general, top-ranked hospitals performed better than nonranked hospitals with respect to outcomes such as mortality for conditions such as myocardial infarction or heart failure that frequently require urgent or emergent care (2–6). These previously described differences in cardiovascular quality of care may have diminished in recent years due to regional and national quality improvement initiatives (7–9).
Many of these quality improvement initiatives have focused on percutaneous coronary intervention (PCI), one of the most common cardiovascular procedures performed in the United States (10–12); however, significant variation in outcomes and appropriate use of PCI persist (13–15). Because these data are not consistently publically reported, patients often turn to consumer-directed hospital ratings to assess hospital quality, yet it is unclear if these ratings are reflective of PCI quality as assessed by clinical outcomes data.
Therefore, using a large national clinical registry of PCIs performed in U.S. hospitals, we compared clinical outcomes and appropriate use of PCI between top-ranked and nonranked hospitals according to the 2015 U.S. News & World Report 50 best “Cardiology and Heart Surgery” rankings.
We performed a retrospective analysis on data from the National Cardiovascular Data Registry CathPCI Registry, an initiative by the American College of Cardiology and the Society for Cardiovascular Angiography and Interventions. A more complete description of the registry has been described previously (10). All PCIs performed between July 1, 2014, and June 30, 2015, at 1,545 hospitals were initially included.
We excluded hospitals that performed <400 PCIs during the study period as guidelines recommend that urgent or elective PCI should be performed at high-volume centers (16). We also excluded hospitals that performed <25 elective PCIs to account for potential miscoding. After exclusions, the study sample was divided into PCIs that occurred at top-ranked and nonranked hospitals. Top-ranked hospitals were defined as hospitals included in the 50 best “Cardiology and Heart Surgery” programs per the 2015 U.S. News & World Report “Best Hospitals” publication (17). The remaining hospitals were considered nonranked. A brief description of the methodology used for the 2015 U.S. News & World Report “Best Hospitals” publication is detailed in the Online Appendix.
The primary clinical outcomes included in-hospital mortality, post-procedural bleeding, and post-procedural acute kidney injury (AKI) during the incident hospitalization when PCI was performed (outcome definitions are described in the Online Appendix). These outcomes were determined for all patients except those that were transferred to another facility after PCI. We also examined the proportion of appropriate PCI based on the 2012 Appropriate Use Criteria for Coronary Revascularization (18). Each PCI was first classified as mappable or nonmappable to the Appropriate Use Criteria, and then each mappable PCI was classified as appropriate, uncertain, or inappropriate.
To assess the relationship between a hospital’s ranked status (top ranked vs. nonranked) and in-hospital outcomes, we developed hierarchical logistic regression models for each outcome as the dependent variable with random intercepts for each hospital. The covariate of interest was the hospital’s ranked status. Adjusted odds ratios (aORs) and 95% confidence intervals (CIs) are provided as measures of effect size. Covariates used to adjust for patient risk were derived from previously published and validated risk models for mortality (19), bleeding (20), and AKI (21).
The proportion of appropriate PCI was calculated by dividing the number of appropriate PCIs by the number of mappable PCIs. To compare appropriate use of PCI between top-ranked and nonranked hospitals, we used weighted linear regression with the hospital as the unit of observation. The modeled dependent variable was the logit-transformed proportion of appropriate PCI. The covariate of interest was ranked status with model weights reflecting PCI volumes at each site. The logit-transformed proportion of appropriate PCI was estimated as: log [proportion of appropriate PCIs/(1 − proportion of appropriate PCIs)]. A similar method was used to compare the proportion of inappropriate PCI between top-ranked and nonranked hospitals.
Scatter plots are provided to demonstrate the distributions of hospital appropriateness and risk-adjusted mortality rates (RAMRs) between top-ranked and nonranked hospitals. To assess for high- or low-performing outliers, we used a funnel plot with additive adjustment of 95% and 98% control limits as described by Ieva and Paganoni (22). A detailed description of how hospital mortality ratios and RAMRs were calculated can be found in the Online Appendix.
Last, we used Spearman’s rank correlation to assess the correlation between adjusted in-hospital outcome rates and the published individual rankings of the top-ranked hospitals.
All analyses were performed on the primary cohort that included all PCIs as well as a subgroup of PCI patients who were not transferred in from another hospital or emergency department; did not experience cardiogenic shock or cardiac arrest before or at the time of PCI; and did not present with acute coronary syndrome (ACS) including ST-segment elevation myocardial infarction, non–ST-segment elevation myocardial infarction, or unstable angina. This subgroup, hereinafter referred to as elective non-ACS PCI, consisted of patients who potentially had the opportunity to research and select the specific hospital where they wanted to undergo PCI.
To examine whether the inclusion of low-volume hospitals affected the primary findings, we conducted a sensitivity analysis evaluating in-hospital outcomes in which we included hospitals performing <400 PCIs during the study period.
A p value <0.05 was considered statistically significant. Standardized differences are also provided as a measure of baseline imbalance that is independent of sample size (23,24). In cases with large sample sizes, p values may be less informative as small differences may be statistically significant in these cases (25). All analyses were performed by the Duke Clinical Research Institute using SAS version 9.4 (SAS Institute, Cary, North Carolina).
A total of 691,572 PCIs were performed at 1,545 hospitals between July 1, 2014, and June 30, 2015. Of those, 891 hospitals performed <400 PCIs and 13 hospitals performed <25 elective PCIs. After exclusions, the final cohort comprised 509,153 PCIs performed at 654 hospitals.
Forty-nine of the 50 top-ranked “Cardiology and Heart Surgery” hospitals participated in the National Cardiovascular Data Registry CathPCI Registry. Of these, 5 performed <400 PCIs during the study period and were excluded, leaving 44 top-ranked hospitals included in the analysis. Top-ranked hospitals had greater bed capacity, performed a larger volume of PCI procedures annually, and were more likely to be university-affiliated and located in urban settings compared with nonranked hospitals (Online Table 1).
Of the 509,153 PCIs included in the study cohort, 55,550 (10.9%) PCIs were performed at top-ranked hospitals and 453,603 (89.1%) were performed at nonranked hospitals. There were significant differences in baseline characteristics of patients undergoing PCI at top-ranked hospitals versus nonranked hospitals (Table 1). After adjusting for case mix, there were no significant differences in the adjusted odds of in-hospital mortality (aOR: 0.96; 95% CI: 0.83 to 1.12; p = 0.64), AKI (aOR: 1.10; 95% CI: 0.98 to 1.22; p = 0.099), or bleeding (adjusted OR: 1.15; 95% CI: 0.99 to 1.31; p = 0.052) at top-ranked versus nonranked hospitals (Figure 1A).
There was a significant difference in the distribution of appropriate, uncertain, and inappropriate PCI between top-ranked and nonranked hospitals (p < 0.001) (Table 2). Top-ranked hospitals had a lower proportion of appropriate PCI (89.2%) compared with nonranked hospitals (92.8%). Moreover, top-ranked status was associated with significantly lower odds of hospital-level appropriate PCI (OR: 0.56; 95% CI: 0.45 to 0.69; p < 0.001) and higher odds of inappropriate PCI (OR: 1.87; 95% CI: 1.47 to 2.38; p < 0.001) compared with nonranked status.
The distribution of hospital-level appropriateness and RAMRs are demonstrated by the scatter plot shown in Figure 2 and Online Figure 1. The median (interquartile range [IQR]) hospital-level proportion of appropriate PCI was 93.7% (IQR: 89.5% to 96.3%) and median RAMR was 1.7% (IQR: 1.3% to 2.2%). Most hospitals were distributed around the intersection of the median of these 2 outcomes.
Among nonranked hospitals, there were 26 (4.3%) hospitals with mortality ratios above the 95% control band and 13 (2.0%) hospitals above the 98% control band (Figure 3). Conversely, there was 1 (2.3%) top-ranked hospital above the 95% control band. None of the top-ranked hospitals were outside of the 98% control band. Only 1 hospital had a mortality ratio below the lower 95% control band, potentially identifying a high-performing outlier hospital. This hospital was nonranked.
Elective non-ACS PCI
A subset of 79,682 PCIs at 653 hospitals comprised the subgroup of elective non-ACS PCI. Among these PCIs, 13,416 (16.8%) were performed at 44 top-ranked hospitals. Baseline characteristics of patients who underwent PCI at top-ranked and nonranked hospitals can be found in Online Table 2. There were no significant differences in the adjusted odds of mortality (aOR: 1.37; 95% CI: 0.81 to 2.33; p = 0.24), AKI (aOR: 1.17; 95% CI: 0.96 to 1.44; p = 0.12), or bleeding (aOR: 1.27; 95% CI: 0.99 to 1.63; p = 0.06) between top-ranked and nonranked hospitals (Figure 1B).
Among elective non-ACS PCIs, there was a significant difference in the distribution of appropriate, uncertain, and inappropriate PCI between top-ranked and nonranked hospitals (p < 0.001) (Table 2). Top-ranked hospitals had similar proportions of appropriate PCI (58.4%) compared with nonranked hospitals (58.1%), and a higher proportion of inappropriate PCI (12.1% vs. 10.9%). After accounting for hospital volume, there were no significant differences in the odds of performing appropriate PCI (OR: 0.98; 95% CI: 0.84 to 1.19; p = 0.98) or inappropriate PCI (OR: 1.16; 95% CI: 0.93 to 1.44; p = 0.20) at top-ranked versus nonranked hospitals.
There was little variation in hospital-level RAMR for elective PCI as the rates of mortality were low. There were no in-hospital deaths among patients undergoing elective non-ACS PCI at 29 (65.9%) top-ranked and 537 (88.2%) nonranked hospitals (Online Table 3).
Correlation between ranking and adjusted outcomes among the 50 best hospitals
Within the 44 best “Cardiology and Heart Surgery” hospitals there were no significant correlations between a hospital’s specific ranking and adjusted rates of each in-hospital outcome (Online Table 4).
In a sensitivity analysis, we added hospitals that performed <400 PCIs to the top-ranked and nonranked cohorts. This resulted in a total of 56,947 PCIs performed at 49 top-ranked hospitals and 604,651 PCIs performed at 1,251 nonranked hospitals. Among elective non-ACS PCIs, there were 13,737 PCIs performed at 49 top-ranked hospitals and 87,143 PCIs performed at 1,244 nonranked hospitals. Results from the sensitivity analysis were generally consistent with the primary findings (Online Table 5).
Using the 2015 U.S. News & World Report 50 best “Cardiology and Heart Surgery” rankings to identify top-ranked and nonranked hospitals in a contemporary national clinical registry of PCI procedures, we evaluated the association between a hospital’s ranked status and post-procedural outcomes and appropriate use of PCI. This study has 4 principal findings. First, evaluating ranked hospitals as a group, we found no evidence to suggest that PCI at nonranked hospitals resulted in inferior clinical outcomes compared with top-ranked hospitals. This finding was consistent among the subgroup of elective PCIs. Second, when evaluated individually with respect to hospital-level mortality ratios, no top-ranked hospital was found to be a high-performing outlier demonstrating exceptional performance. However, there were also no top-ranked hospitals that were low-performing outliers, whereas a small proportion (2.0%) of nonranked hospitals had mortality ratios lying above the 98% control band. Third, elective PCI performed for non-ACS presentations generally appears safe across the spectrum of studied hospitals, with 86.7% of hospitals reporting no in-hospital deaths over the 1-year study period. Last, there was no correlation between adjusted outcomes and the specific ranking of the 44 top-ranked hospitals. This suggests that PCI quality based on clinical outcomes cannot be reliably derived from the specific rankings of the best “Cardiology and Heart Surgery” hospitals.
Although no study has specifically evaluated the validity of these rankings in the context of PCI care, our findings differ from previous studies that, in general, demonstrated superior outcomes at top-ranked hospitals (2–5). The current study differs from previous research in several important ways that may potentially explain these differences and provide context to our findings.
First, by using clinical data instead of administrative data, we could: 1) identify complications from comorbidities more accurately; and 2) use more robust risk-adjustment methods (26,27). In addition, the appropriate use of PCI cannot be reliably derived from administrative data. Finally, clinical data in this registry are subjected to quality control processes evaluating the completeness, consistency, and accuracy of data (28).
Second, our study included all-comers undergoing PCI, whereas prior work has focused solely on the Medicare population (2–5). By using Medicare data, the same data source used by U.S News & World Report to assess outcomes, previous studies may have been more likely to arrive at similar conclusions as the “Best Hospitals” rankings. These rankings may also have limited generalizability for procedures such as PCI, where a substantial proportion of patients are below 65 years of age. For instance, in our study, 46% of the patients were under 65 years of age and 41% were primarily covered by a non-Medicare insurer.
Third, in accordance with national guidelines recommending that elective or urgent PCI be performed at high-volume centers, we excluded hospitals performing <400 procedures during the study period (16). By doing so, we might have diminished the effect of the rankings if comparisons were made to a cohort including nonranked low volume hospitals. However, in a sensitivity analysis including low-volume hospitals, we found no evidence to suggest inferior outcomes at nonranked hospitals. Our findings should reassure patients that there are many centers with adequate PCI volumes performing high-quality PCI.
Last, the inclusion of an appropriateness measure is a novel dimension of the current study. Over the last decade, there has been increasing emphasis placed on performing PCI for evidence-based indications as illustrated by the development of Appropriate Use Criteria for Coronary Revascularization (18,29) and the declining rates of inappropriate PCI (14). We believe that PCI appropriateness is an important indicator of quality, serving as a measure of physician decision-making when faced with treating the vast array of coronary artery disease presentations. Recently, Bradley et al. (30) demonstrated that hospital-level appropriateness was not related to clinical outcomes, implying that it measures a distinct domain of hospital quality. Although the rates of appropriate PCI in the entire study cohort was high, PCI at nonranked hospitals was associated with significantly higher odds of appropriate PCI compared with top-ranked hospitals.
As one of the most popular and influential hospital profiling initiatives, U.S. News & World Report continually re-evaluates and updates their hospital profiling methodology in a transparent manner. We believe that the use of clinical data in conjunction with administrative data may overcome many of the limitations of solely using Medicare claims.
This study should be interpreted in the context of specific limitations. First, the 2015 U.S. News & World Report “Best Hospitals” rankings incorporate a complex methodology attempting to evaluate multiple aspects of hospital quality including domains of structure, process, and outcomes (a brief description of this methodology is detailed in the Online Appendix). For this study, we focused on PCI appropriateness and outcomes, which are important, albeit narrow aspects of cardiovascular care. Second, the principle outcome evaluated by U.S. News & World Report rankings was 30-day risk-adjusted mortality, whereas we were only able to evaluate in-hospital outcomes. Therefore, it is possible that patients might have experienced adverse outcomes after discharge that were not captured in our analysis. Third, our definition of elective non-ACS PCI was different from the definition of nonacute PCI used in prior studies (14,31). Our definition was more restrictive allowing us to identify cases where patients may have had a choice of where to receive PCI. In addition, while the CathPCI registry includes a broad array of demographic and clinical variables for risk adjustment, unmeasured factors might have affected case mix adjustment between hospitals. Last, participation in the CathPCI registry is voluntary and not all hospitals that perform PCI participate in the registry. Indeed, 1 of the top 50 hospitals did not participate in the CathPCI registry during the study period. These hospitals may be different than hospitals not participating in the registry; therefore, our findings may not be generalizable to all PCI-capable hospitals. Nevertheless, this study evaluated a substantial proportion of top-ranked and nonranked hospitals performing a large number of PCIs throughout the United States.
PCI performed at top-ranked hospitals was not associated with superior outcomes compared with PCI at nonranked hospitals. These findings should reassure patients that safe and appropriate PCI is being performed across the nation in hospitals participating in the CathPCI registry and meeting minimum recommended hospital volume targets regardless of ranking by a popular consumer-directed hospital ratings system. Moreover, the rankings failed to identify hospitals providing the highest levels of quality PCI care, suggesting that incorporation of clinical measures of PCI appropriateness and outcomes should be considered in the formulation of such rating systems. Moving forward, much like the initiatives by the American College of Cardiology and Society of Thoracic Surgeons, we believe it is incumbent on hospitals, clinical registries, and national professional organizations to assist in public reporting efforts aimed at improving transparency, accountability, and quality in health care.
WHAT IS KNOWN? The U.S. News & World Report “Best Hospitals” is arguably one of the most widely read and influential hospital profiling systems in the United States.
WHAT IS NEW? Although the majority of PCIs in the United States are performed at nonranked hospitals, we found no significant differences in the odds of important in-hospital outcomes after PCI when compared with top-ranked hospitals according to the 2015 U.S. News & World Report “Best Hospitals” publication for “Cardiology and Heart Surgery.” These findings should reassure patients that safe and appropriate PCI is being performed across the country.
WHAT IS NEXT? Future hospital quality profiling initiatives should consider including clinical measures of PCI appropriateness and outcomes when formulating hospital ratings.
Dr. Sukul was supported by a National Institutes of Health T32 postdoctoral research training grant (T32-HL007853). Dr. Bhatt has served on the advisory board for Cardax, Elsevier Practice Update Cardiology, Medscape Cardiology, and Regado Biosciences; has served on the Board of Directors of the Boston VA Research Institute and Society of Cardiovascular Patient Care; has served as the chair of American Heart Association Quality Oversight Committee; has served on the data monitoring committees of the Cleveland Clinic, Duke Clinical Research Institute, Harvard Clinical Research Institute, Mayo Clinic, and Population Health Research Institute; has received honoraria from the American College of Cardiology (Senior Associate Editor, Clinical Trials and News, ACC.org), Belvoir Publications (Editor in Chief, Harvard Heart Letter), Duke Clinical Research Institute (clinical trial steering committees), Harvard Clinical Research Institute (clinical trial steering committee), HMP Communications (Editor in Chief, Journal of Invasive Cardiology), J Am Coll Cardiol (Guest Editor; Associate Editor), Population Health Research Institute (clinical trial steering committee), Slack Publications (Chief Medical Editor, Cardiology Today Intervention), Society of Cardiovascular Patient Care (Secretary/Treasurer), and WebMD (CME steering committees); has served as the Deputy Editor for Clinical Cardiology; has served as the chair of the NCDR-ACTION Registry Steering Committee and VA CART Research and Publications Committee; has received research funding from Amarin, Amgen, AstraZeneca, Bristol-Myers Squibb, Chiesi, Eisai, Ethicon, Forest Laboratories, Ironwood, Ischemix, Lilly, Medtronic, Pfizer, Roche, Sanofi, and The Medicines Company; has received royalties from Elsevier (Editor, Cardiovascular Intervention: A Companion to Braunwald Heart Disease); has served as a site co-investigator for Biotronik, Boston Scientific, and St. Jude Medical (now Abbott); has served as a trustee of American College of Cardiology; and has performed unfunded research for FlowCo, PLx Pharma, and Takeda. Dr. Rumsfeld is Chief Innovation Officer for the American College of Cardiology. Dr. Wang has received institutional research grant support to the Duke Clinical Research Institute from AstraZeneca, Boston Scientific, Bristol-Myers Squibb, Daiichi Sankyo, Eli Lilly, Gilead Sciences, Pfizer, and Regeneron Pharmaceuticals; and received consulting or honoraria from Gilead, Merck, and Sanofi. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Abbreviations and Acronyms
- acute coronary syndrome(s)
- acute kidney injury
- adjusted odds ratio
- confidence interval
- interquartile range
- percutaneous coronary intervention
- risk-adjusted mortality rate
- Received August 22, 2017.
- Revision received October 24, 2017.
- Accepted October 26, 2017.
- 2018 American College of Cardiology Foundation
- ↵About U.S. News & World Report. Available at: http://www.usnews.com/info/features/about-usnews. Accessed December 29, 2016.
- Mulvey G.K.,
- Wang Y.,
- Lin Z.,
- et al.
- Williams S.C.,
- Koss R.G.,
- Morton D.J.,
- Loeb J.M.
- Peterson E.D.,
- Roe M.T.,
- Rumsfeld J.S.,
- et al.
- Brindis R.G.,
- Fitzgerald S.,
- Anderson H.V.,
- Shaw R.E.,
- Weintraub W.S.,
- Williams J.F.
- Nissen S.E.,
- Brush J.E. Jr..,
- Krumholz H.M.
- Share D.A.,
- Campbell D.A.,
- Birkmeyer N.,
- et al.
- Bernheim S.M.,
- Grady J.N.,
- Lin Z.,
- et al.
- Yeh R.W.,
- Rosenfield K.,
- Zelevinsky K.,
- et al.
- Levine G.N.,
- Bates E.R.,
- Blankenship J.C.,
- et al.
- ↵(2014) U.S. News & World Report's Best Hospitals 2015 Edition (U.S News & World Report, Washington, DC).
- Patel M.R.,
- Dehmer G.J.,
- Hirshfeld J.W.,
- Smith P.K.,
- Spertus J.A.
- Peterson E.D.,
- Dai D.,
- DeLong E.R.,
- et al.
- Rao S.V.,
- McCoy L.A.,
- Spertus J.A.,
- et al.
- Tsai T.T.,
- Patel U.D.,
- Chang T.I.,
- et al.
- Ieva F.,
- Paganoni A.M.
- Tijssen J.G.,
- Kolm P.
- Messenger J.C.,
- Ho K.K.,
- Young C.H.,
- et al.
- Patel M.R.,
- Dehmer G.J.,
- Hirshfeld J.W.,
- et al.
- Bradley S.M.,
- Chan P.S.,
- Spertus J.A.,
- et al.