Author + information
- Spencer B. King III, MD, MACC, Editor-in-Chief, JACC: Cardiovascular Interventions∗ ()
- ↵∗Address correspondence to:
Dr. Spencer B. King III, Saint Joseph’s Heart and Vascular Institute, 5665 Peachtree Dunwoody Road NE, Atlanta, Georgia 30342.
I was recently interviewed for a research project about what type of clinical trials get performed and which ones get published. The discussion was to be specifically about “real-world experience” or “real-world evidence.” I bristled at the term that I have grown to dislike very much, and I told the interviewer as much. The interviewer redefined the subject so that I would stop ranting about the term and changed it to “any studies that are not randomized trials.” The purpose of this survey of editors seems to be to understand the significance of nonrandomized studies, the strengths and weaknesses of nonrandomized studies, whether authors and reviewers are given specific guidelines for preparing and evaluating such studies, and, not so subtly implied, whether there is editorial bias against publishing nonrandomized studies.
The strengths of randomized trials are well known and are primarily the elimination of significant baseline variables, either recognized or nonrecognized. Without randomization, unrecognized confounding variables have to be assumed and accounted for. Does this mean nonrandomized studies are lacking validity? Clearly not, nor can we assume that all randomized trials are valid. They can, despite randomization, suffer from poor design and methods, inadequate power or from studying highly refined subgroups because exclusions result in a residual population that does not reflect the patients to which the conclusions are applied. Some questions cannot be answered without randomized trials, but some randomized trials cannot answer the questions.
After I got the interviewer away from the “real world,” I discussed the kind of nonrandomized clinical studies that we see in interventional cardiology. They range from single-center retrospective observations to well-planned prospective registries with structured follow-up and often independent outcomes adjudication. Not infrequently, we see large registries addressing issues that have evidence already generated by randomized trials. What value do they have? The randomized trials may be targeted to a selected population but a large registry that is more inclusive can inform the answer if it is in the same direction as the randomized study. Therapies that change can be evaluated by nonrandomized registries collecting sequential data before and after a change. Post-approval surveillance of new devices can provide critical information regarding rare occurrences that went undetected in randomized trials.
I wondered about some of the questions I was asked. Do we need instructions to authors about what makes a valuable nonrandomized study? Do we need instructions to reviewers? The same general guidelines for priority apply for randomized as well as nonrandomized studies. Is it new? Is it true? And, is it relevant? Novelty (new) probably applies to both types of studies in the same way. Validity (true) depends on the ability to understand the strengths and weaknesses of the study design and execution, and they differ between the types of comparisons. Clinical importance (relevance) sometimes is judged on how results apply to the broad population or to specific subsets, differences that may be more appropriate to randomized or nonrandomized studies.
This interview about the value of nonrandomized studies caused me to reflect on what we are doing at JACC: Cardiovascular Interventions. We accept more studies that can be classified as nonrandomized than randomized so we obviously see value in these. I am not sure that author or reviewer instructions are necessary, but there is a great deal written about structuring observation studies, registries, and population studies. Major strengths of prospective registries are the size and inclusiveness, as well as clear definitions and organized follow-up. A lot of excellent papers have come from registries that have some of these features such as the New York State Registry, the NCDR (National Cardiovascular Data Registry), the SCAAR (Swedish Registry), and the National Health System of the United Kingdom. These study large population groups, but they may not allow as complete an evaluation as prospective targeted registries with mandated clinical follow-up would. The PARTNER (Placement of Aortic Transcatheter Valve Trial) registry and the French TAVI (Transcatheter Aortic Valve Implantation) registry and others come to mind.
Will nonrandomized studies play an increasing role in generating evidence? Most think so. Large randomized trials are getting very expensive. Collection of massive amounts of data for other purposes is becoming much cheaper. Therefore, it is attractive to try to generate evidence from data already being collected (i.e., administrative data). There are several problems using billing data to answer clinical questions. The data from billing codes is notoriously incomplete, and although it may serve a purpose for avoiding billing errors, it is far from avoiding clinical errors. Another approach to performing trials at lower cost was the SAFE PCI-For Women (Study of Access Site for Enhancing PCI for Women) trial (1). They used NCDR baseline data to identify and enroll subjects for a randomized trial.
Innovative approaches to research will clearly evolve and cost containment will be a major consideration. Some questions will be answerable only by randomization whereas others will provide answers through well-performed nonrandomized studies. Neither of these approaches are “outside the real world,” and neither apply perfectly to the “real patient.” Let’s drop “real world” and describe our studies more precisely.
- American College of Cardiology Foundation
- Rao S.V.,
- Hess C.N.,
- Barham B.,
- et al.