Author + information
- Mitchell W. Krucoff, MD∗ ( and )
- W. Schuyler Jones, MD
- Division of Cardiology, Duke University Medical Center and Duke Clinical Research Institute, Durham, North Carolina
- ↵∗Address for correspondence:
Dr. Mitchell W. Krucoff, Division of Cardiology, Duke University Medical Center and Duke Clinical Research Institute, 2400 Pratt Street, Durham, North Carolina 27710.
- National Evaluation System for Health Technologies
- objective performance goal
- peripheral vascular intervention
The growth of peripheral vascular intervention (PVI) (1) includes the responsibility to advance better, safer devices and procedures. Regulatory approval of innovative technology is based on statutory requirements for “reasonable assurance” of safety and effectiveness for defined indications and populations of use. Once approved, clinicians may use devices for “off-label” indications or populations in real-world practice. Best-practice guidelines developed by expert evaluations are intended to guide clinicians on behalf of all patients treated, on- or off-label.
Randomized controlled trials (RCTs) represent the most rigorous evidence for both device approval and practice guidelines but are expensive and often exclude the diversity of real-world patients. In PVI, the diversity of devices, patients, clinical presentations, operator expertise, and adjunctive therapies compounds both regulatory and clinical questions beyond what can be addressed with RCTs. Data rigorously comparing PVI tools are limited (2).
In this issue of JACC: Cardiovascular Interventions, Shammas et al. (3) report findings from 103 patients undergoing planned femoropopliteal atherectomy at 10 centers in the single arm WISE LE study. The study evaluated the effectiveness of the WIRION Embolic Protection System (EPS) for noninferiority in 30-day major adverse events compared with 2 studies using an EPS approved for PVI (SpiderFX, Medtronic, Minneapolis, Minnesota). Weighted major adverse event averages were used to create a performance goal (PG) as comparator (4,5).
Single-arm studies with objective performance criteria (OPC) or PG comparators may support labeling decisions in PVI (6). The 2017 U.S. Food and Drug Administration (FDA) guidance on real-world evidence (7) states, “An OPC is usually developed when device technology has sufficiently matured and can be based … on information pooled from all available studies on a particular kind of device … a performance goal (PG) refers to a numerical value that is considered sufficient for use in the evaluation of an investigational device regarding a safety and/or effectiveness endpoint. But, generally … the data used to generate a PG are not considered as robust as those used to develop an OPC.”
Data from the WISE LE study illustrate both the strengths and the limitations of a single-arm study with a PG comparator. Patients in WISE LE and the PG had similar baseline characteristics and medical comorbidities. Lesion length was significantly longer in WISE LE than DEFINITIVE Ca++ (5). In WISE LE, 10 different atherectomy devices and 15 interventional guidewires were used at the operator’s discretion. High procedural success rates and very low periprocedural complications suggest that WISE LE reflects a small group of highly experienced operators. The absence of information about patients’ clinical histories, indications for procedures, and anatomic burden and severity of disease in addition to the above could introduce bias in ways not accounted for in the noninferiority analysis.
The WIRION EPS filter is approved by the FDA for embolic protection with carotid stenting. The WISE LE versus PG study is poised to provide FDA evidence to consider expanding the device label to PVI. With a clear mechanism of action (a filter that catches debris), reasonable assurance of noninferiority to an approved (SpiderFX) EPS powered for 92 patients in a per protocol analysis versus a 19.26% major adverse event rate at 30 days constitutes a significantly “less burdensome” study design than an RCT. There is no question, on one hand, that randomization could have mitigated unintended bias. On the other hand, if an RCT were cost prohibitive, clinicians could use the WIRION EPS for PVI off-label, without any evidence beyond personal judgment. So on one hand, Shammas et al. (3) should be congratulated for contributing data of supporting the expansion of the PVI operator’s toolkit on behalf of patients with peripheral artery disease, even as, on the other hand, clinical trial purists may debate whether unintended bias plays a role in these results.
The Zen koan asks, “What is the sound of one hand clapping?” It is an unanswerable riddle until the narrow-minded perspective of the student is supplanted by more enlightened perception. Traditionally, PVI device trials fragment evidentiary efforts into stand-alone reports of either RCT or single-arm designs that, like “one hand clapping,” lead to more questions to ponder than answers to guide regulatory or clinical practice decisions.
In 2016, the FDA launched the National Evaluation System for Health Technologies (NEST) (8), a transformational shift from historically perceived “either/or” studies to more “enlightened” emphasis on complementary evidence accrual. “Federal regulatory frameworks governing medical products are designed to: (1) provide evidence that a product benefits patients when used as intended and should be available despite accompanying risks; and (2) ensure timely access to needed therapies and diagnostics. Historically, policymakers and product developers have viewed these objectives as being in tension. However, ensuring safety, expediting patient access, and enabling innovation can be complementary goals within a regulatory framework for medical devices” (8).
NEST shifts the emphasis to efforts to support a continuum of evidence accrual that both reduces study costs and enhances data quality and consistency. NEST leverages existing sources of real-world evidence in “a strategic approach to linking and using clinically based data sources, such as registries, electronic health records (EHRs), and claims data” to “potentially reduce the burdens of obtaining appropriate evidence” to provide “a more comprehensive and accurate framework … for assessing the risks and benefits of devices” (8) for both RCTs and single-arm designs.
To make RCTs and single-arm studies in PVI more informative and complementary to each other, there are 2 basic steps: 1) developing consistent data structure and definitions that constitute a core of information used for all PVI studies; and 2) implementing that data structure into electronic systems that can be “dual-purposed” to meet the needs of both real-world clinical practice and clinical research documentation. The 2015 Peripheral Academic Research Consortium report of consensus definitions of clinical syndromes, patient and anatomic descriptors, and procedural and clinical outcomes for PVI devices (9) is one example. A second is the Registry Assessment for Peripheral Interventional Devices project (10), a NEST program whose deliverables include 1) a standardized “minimum core” dataset for PVI studies with informatics mapping to electronic health records and registries; and 2) creation of OPC for femoropopliteal PVI.
Patients with peripheral artery disease have many unmet needs that innovative PVI devices may address, but benefit/risk evidence is challenged by fragmented studies of heterogeneous patients, presentations, anatomy, procedures, operator experience, and outcomes, compounded by the historical variability of nomenclature and definitions. Implementing standardized structure through the transformational NEST approach supports all designs and complementary data across studies, promoting ongoing accrual of knowledge for PVI devices and outcomes.
Shammas et al. (3) have likely opened the door to a tool for embolic protection in the neck to join the tool kit for PVI. Future incorporation of standardized, structured data for PVI studies in the NEST era, the gaps between RCTs and single-arm studies, the quality of PG and OPC comparators, and the differences between, on one hand, the quality and cost of research efforts and, on the other hand, the real-world scope of PVI operators and outcomes, will converge to complement one another. From historical views of one hand or the other, the 21st-century NEST devices model may bring 2 hands together as 1 source of evidence—a thunderclap long overdue.
↵∗ Editorials published in JACC: Cardiovascular Interventions reflect the views of the authors and do not necessarily represent the views of JACC: Cardiovascular Interventions or the American College of Cardiology.
Both authors have reported that they have no relationships relevant to the contents of this paper to disclose.
- Subherwal S.,
- Patel M.R.,
- Chiswell K.,
- et al.
- Shammas N.W.,
- Pucillo A.,
- Jenkins J.S.,
- et al.
- McKinsey J.F.,
- Zeller T.,
- Rocha-Singh K.J.,
- Jaff M.R.,
- Garcia L.A.,
- for the DEFINITIVE LE Investigators
- ↵U.S. Food and Drug Administration. Use of real-world evidence to support regulatory decision-making for medical devices 1 guidance for industry and Food and Drug Administration staff. Available at: https://www.fda.gov/ucm/groups/fdagov-public/@fdagov-meddev-gen/documents/document/ucm513027.pdf. Accessed June 7, 2018.
- Shuren J.,
- Califf R.M.
- Patel M.R.,
- Conte M.S.,
- Cutlip D.E.,
- et al.
- Jones W.S.,
- Krucoff M.W.,
- Morales P.,
- et al.