In a previous post, I wrote about the enormous attention and scorn that had been cast upon a trial known as ORBITA.
The ORBITA trial was a small randomized trial, whose results questioned the usefulness of an interventional procedure -- specifically, angioplasty -- in relieving the symptoms of angina in stable patients with coronary artery disease.
The results of the trial threatened the livelihoods of thousands of interventional cardiologists. As expected, they criticized the trial with a vengeance.
What were their criticisms? The trial was small. The trial enrolled a selected (and therefore biased) population. Their conclusion: the results of the trial were not relevant. Their message: never change practice based on such a small trial.
Many cardiologists whose salary depends on interventional procedures have put the ORBITA trial into the dustbin of history.
Given this experience, it is really interesting to see what just recently happened.
Last week, another small trial -- known as CASTLE-AF -- was published. It evaluated the effects of another interventional procedure -- catheter ablation -- in the treatment of atrial fibrillation in patients with chronic heart failure.
The trial reported an apparent resounding success. According to the investigators, lives were saved, and treated patients really improved. If the trial results were replicable, thousands of patients would now need an expensive interventional procedure.
The internet exploded in joy. Many electrophysiologists (who would perform the expensive procedure) examined the trial and apparently found no serious flaws. Some had severely criticized earlier trials with drugs for heart failure that had also reported that lives were saved. But the results of those earlier trials would not have been of particular financial benefit to the electrophysiologists.
So was the CASTLE-AF trial really that definitive?
First, the CASTLE-AF trial was about the same size as ORBITA, and it enrolled a highly selected population. Only 10% of the patients screened for the study were actually enrolled.
Second, the trial was significantly underpowered to test its hypothesis, because it was terminated well before its prespecified targets had been reached. The trial fell short of its planned enrollment targets by 32% -- a huge miss.
Third, randomized patients as well as events following randomization were excluded from the analysis. This is not the right procedure for an intention-to-treat analysis.
Fourth, a significant proportion of the patients were lost to follow-up. Incredibly, the number of patients lost to follow-up was more than 25% of the total number of events. That is unacceptable. Furthermore, more patients were lost to follow-up in the intervention group than in the control group. These are telltale signs that the results of the trial are really fragile.
Fifth and most importantly, the primary analysis was based on just a handful of events. And it is already known that -- in patients with chronic heart failure -- trials that report striking results based on a small number of events are typically not reproduced when they are repeated in a more rigorous manner.
These issues are what people should be talking about.
Did the electrophysiologists talk about any of these issues? Did they complain about the high cost of treatment?
Nope.
Cynics love to talk about the fact that conclusions based on early studies are often reversed by later trials. (They call it "medical reversal.") Did these cynics point out the possibility that the results of this trial could be reversed?
Nope.
How about the interventionists who proposed that the ORBITA trial be ignored because it was small? Did they say that the CASTLE-AF trial also be ignored?
Nope.
Instead, there has been unrestrained celebration. This makes perfect sense. If the findings are replicable, this trial represents a major financial boon to electrophysiologists and their cardiology practices. They will reap rich rewards for doing these procedures.
Does anyone really think that the CASTLE-AF trial provides a definitive answer on the utility of catheter ablation in the patients who were studied?
If you are able to find people who believe the trial should change practice, you should also ask them what they do for a living.
As an electrophysiologist once said in a tweet, there is a simple take-home message: "COI=key."
Disclosures
Packer recently consulted for Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Cardiorentis, Daiichi Sankyo, Gilead, Novo Nordisk, Relypsa, Sanofi, Takeda, and ZS Pharma. He chairs the EMPEROR Executive Committee for trials of empagliflozin for the treatment of heart failure. He was previously the co-PI of the PARADIGM-HF trial and serves on the Steering Committee of the PARAGON-HF trial, but has no financial relationship with Novartis.