Author + information
- Edward L. Hannan, PhD, MS, MS∗ ()
- Department of Health Policy, Management, and Behavior, University at Albany School of Public Health, Rensselaer, New York
- ↵∗Reprint requests and correspondence:
Dr. Edward Hannan, Department of Health Policy, Management, and Behavior, University at Albany School of Public Health, One University Place, Rensselaer, New York 12144.
There is a large body of literature on the dangers of public reporting of health outcomes, and 1 of the primary dangers cited is the avoidance of high-risk cases (1–4). It should be noted that some reports on the avoidance of high-risk cases may be inaccurate or overstated (5,6). Also, when high-risk patients do not undergo invasive treatment, it may be the appropriate decision, or they may soon undergo treatment by a provider that is more experienced and competent. Nevertheless, withholding of appropriate treatment is a serious concern in the era of public reporting that should be carefully examined and addressed. This includes an investigation of the ability of statistical models to accurately predict the risk for seriously ill patients and an assessment of the detrimental impact on providers that treat high-risk patients.
In this issue of JACC: Cardiovascular Interventions, Sherwood et al. (7) use 2010 data from the CathPCI registry to examine the calibration of high- and low-risk percutaneous coronary intervention (PCI) patients, to compare ratings of hospitals based on the average level of risk of their patients, to calculate observed/expected in-hospital mortality ratios, and to assess the impact of combining all high-risk patients over a 2-year period into a single year on hospital risk-adjusted mortality (RAM) ratings. Findings were that the National Cardiovascular Data Registry (NCDR) model used to risk adjust outcomes slightly overpredicted risk for high-risk cases, hospitals treating the highest-risk patients on average had better RAMs than hospitals with lower-risk cases, and combining all high-risk patients over a 2-year period into a single year didn’t negatively impact hospitals’ RAM ratings.
Sherwood et al. (7) are to be congratulated for undertaking this important work and for developing methods to examine the impact on providers of treating the highest-risk patients. It is notable that they defined “high-risk” in different ways, all of which led to the same conclusion (that there is no adverse impact on ratings of providers who treat high-risk patients), and this is a confirmation of their findings. However, there are reasons why many PCI providers who currently avoid high-risk procedures may not be convinced to change in a climate of public dissemination. First, the fact that sites that treat more high-risk cases have better RAM ratings is not necessarily a result of performing high-risk cases. It could also be due to the fact that they are better quality hospitals, both for high- and low-risk cases. This competing hypothesis could be examined by comparing their performance with other hospitals separately for high- and low-risk cases. I realize that this was theoretically done by combining high-risk cases into a single year, but comparing hospitals across high- and low-risk cases is a more clear-cut and understandable way to test the competing hypothesis.
Second, with regard to the concentrated risk-year analysis, it is comforting to see that the observed/expected ratios and number of outliers are similar to what they are for the overall analysis, but it would be of more interest to compare individual hospital ratings with the 2 sets of analyses. What is of most concern to hospitals and physicians is how their own rating is impacted by high-risk cases.
Perhaps my skepticism of the ability of the Sherwood et al. (7) findings to change current practice is a result of my own inability to be persuasive in this regard. In 2006, the New York State Department of Health convened “town hall” meetings for the purpose of educating physicians and hospitals about the cardiac registries, risk adjustment, and among other things, the impact of high-risk cases on hospital and cardiologist ratings. At that time, the primary complaint about inclusion of high-risk cases in public reporting centered around the reporting of shock patients. The method I used to make the argument that ratings are unaffected by the inclusion of shock patients was to develop separate statistical models with and without shock patients included, and then to compare hospital and physician RAM ratings, and outlier status for the 2 models. The findings for both PCI and coronary artery bypass graft (CABG) surgery were that more hospitals and more physicians had better, rather than worse, observed/expected ratios for shock cases than for other cases, and that the outliers for PCI and CABG were identical when shock cases were taken out of the databases and new statistical models were created.
These arguments were not effective in quelling the concerns expressed by hospitals and physicians, and in response to continued concerns about the detrimental impact of including shock patients in the PCI and CABG surgery registries, the New York State Department of Health decided to eliminate shock patients from public reporting as of January 2006. It should be noted that shock is defined more restrictively in New York, and is more similar to what is regarded as refractory shock in other databases (and will be referred to as refractory shock in New York as of 2015). Shock is defined in New York’s registries as acute hypotension (systolic blood pressure <80 mm Hg) or low cardiac output (<2.0 l/min/m2) despite pharmacological or mechanical support. It can be coded if the patient has ongoing resuscitation or has a ventricular assist device.
Table 1 demonstrates the impact on the number of patients with shock undergoing PCI in New York between 2005 and 2011.
As indicated, the number of patients rose substantially in the first year of the exclusion (from 83 to 133) and has been rising almost steadily since then. In 2012, the number was more than double of what it was in 2005 when shock cases were publicly reported. Although there are many possible reasons for this increase, including improved emergency medical service policies and more transfers and direct transports to PCI hospitals, these data do suggest that the policy of excluding shock from public reporting has enabled more shock patients to undergo PCI, which is the best treatment for most of them. The fact that the mortality rate has risen for these patients is arguably an indication that the additional patients in subsequent years are a higher-risk group of patients on average than the patients undergoing PCI before the exclusion.
Recently, a group of cardiologists in New York requested that the Department of Health expand the criteria for excluding shock patients so that patients with acute hypotension or low cardiac index that do not have either pharmacological or mechanical support would also be excluded. The Department of Health consulted its Cardiac Advisory Committee (CAC), consisting of cardiac surgeons, cardiologists, and other clinical experts within and outside of New York, and the CAC advised against expanding the exclusion.
Since 2011, New York has also excluded a subset of patients with anoxic encephalopathy from public reporting in its PCI registry. The criteria include PCI done for acute myocardial infarction, documented cardiac arrest before arrival at the catheterization laboratory, coma following the cardiac arrest, no in-lab death, persistent severe hypoxic encephalopathy present at the time of death or the decision to withhold or withdraw care, and medical record documentation of a post-PCI consultation by neurology or critical care documenting the presence and severity of anoxic/hypoxic encephalopathy. There have also been inquiries to the Department of Health to expand this definition, but on the advice of its CAC, the Department has chosen not to do so.
The decision to exclude high-risk patients from public reporting is a difficult one, particularly for a procedure like PCI for which short-term mortality is so low with high-risk patients excluded. For example, in New York in 2011, the mortality rate for PCI patients without shock, hemodynamic instability, or an acute myocardial infarction in the 24 h before the procedure was 0.65%. When high-risk patients are excluded, the number of patients that are publicly reported may not decrease much, but the number of deaths could decrease substantially, and this makes it difficult to meaningfully compare hospitals (8).
In my view, if risk factors are to be considered for exclusion in public reporting, the following are 3 conditions that should be present.
1. The risk factor should have a severe adverse outcome (mortality for most reports) rate. This is because it is high-risk cases that are perceived to be detrimental to a provider’s ratings (although this is not necessarily true).
2. The risk factor should be relatively rare. This is important because if a risk factor has a high mortality rate and a low prevalence, a death may be difficult to recover from without the opportunity to have many similar high-risk cases in the reporting period. For example, if a shock patient dies, the decrement to the hospital for that 1 case is roughly 1.0 (death) − 0.50 (probability of death) = 0.5. To make up for that decrement with elective patients having an average mortality rate of 1% (0.01), it would take 50 straight cases without a short-term death.
3. The risk factor should be such that the risk for any given patient may be difficult to estimate accurately. For instance, it could be argued that the probability of mortality for a given shock patient as defined in the preceding text is difficult without knowing the patient’s precise systolic blood pressure or cardiac index at various times before undergoing PCI.
In conclusion, the efforts of Sherwood et al. (7) to clarify the impact on provider ratings of performing high-risk PCI cases should be applauded. Government agencies and other rating organizations, health policy experts, and analysts need to be armed with this type of information when making decisions about crafting public reports that rate providers, and this information should be shared and discussed with providers that are being rated.
↵∗ Editorials published in JACC: Cardiovascular Interventions reflect the views of the authors and do not necessarily represent the views of JACC: Cardiovascular Interventions or the American College of Cardiology.
Dr. Hannan has reported that he has no relationships relevant to the contents of this paper to disclose.
- 2015 American College of Cardiology Foundation
- Resnic F.S.,
- Welt F.G.
- Moscucci M.,
- Eagle K.A.,
- Share D.,
- et al.
- McCabe J.M.,
- Joynt K.E.,
- Welt F.G.
- Hannan E.L.,
- Cozzens K.,
- King S.B. III.,
- Walford G.,
- Shah N.R.
- Sherwood M.W.,
- Brennan J.M.,
- Ho K.K.,
- et al.
- ↵Percutaneous Coronary Interventions (PCI) in New York State, 2009-2011. New York State Department of Health, 2014. Available at: http://www.health.ny.gov/statistics/diseases/cardiovascular/docs/pci_2009-2011.pdf. Accessed August 1, 2014.