Two Modest RCT Proposals: Parachutes and COVID-19
The Science Wars Part VII
"I am assured by a very knowing American of my acquaintance in London; that a young healthy child, well nursed, is, at a year old, a most delicious, nourishing, and wholesome food; whether stewed, roasted, baked or boiled, and I make no doubt, that it will equally serve in a fricassee, or ragout." -Jonathan Swift, A Modest Proposal
I have previously written about the nuances of randomized control trials (RCTs) here and here, but there is an extremely well known "modest proposal" that demonstrates that there are clear ethical hurdles that relegate RCTs to a reasonable subset of potential trial models. That subset should be further reduced according to the needed power level, and held to greater scrutiny under those circumstances, which would ultimately be relatively rare. But that's a discussion for another article.
Source: Hulton/Getty via Smith and Pell, 2003
The Modest Proposal for the Parachute RCT
In 2003, Cambridge Professor Gordon Smith and colleague Jill Pell of the Glasgow Department of Public Health published a now infamous meta-analysis of RCTs on parachutes in the BMJ (Smith and Pell, 2003). The tongue-in-cheek paper is a good and humorous read that spans a few pages. An excerpt:
Definition of outcomes
The major outcomes studied were death or major trauma, defined as an injury severity score greater than 15.6
Meta-analysis
Our statistical approach was to assess outcomes in parachute and control groups by odds ratios and quantified the precision of estimates by 95% confidence intervals. We chose the Mantel-Haenszel test to assess heterogeneity, and sensitivity and subgroup analyses and fixed effects weighted regression techniques to explore causes of heterogeneity. We selected a funnel plot to assess publication bias visually and Egger's and Begg's tests to test it quantitatively. Stata software, version 7.0, was the tool for all statistical analyses.
Aaaaaaand…the results:
Results
Our search strategy did not find any randomised controlled trials of the parachute.
Because of course it did not. Because science is not about testing the set of all hypotheses rigorously using a strict and formalized set of methodologies. Like everything else in the universe, there is an economics to science. There is also human judgment—hopefully of the discerning kind—at every corner of the scientific process from the moment a hypothesis is imagined. And yes, a lot of that is common sense. If that's too vague a concept, try human decency. We do not run RCTs to find out the ranges of safe temperatures for the preparation of babies as a food.
Already, even without getting into the math (though I do plan to in the future), we see the strictness of the Cochrane Hierarchy of Evidence crumble. Of course, you shouldn't even need math to show that it will fail that test, too, because none was ever presented in its establishment. It should never have been touted as more than a general, but fallible guide. Nor should the research community that worships at its footsteps have been shielded in a bubble from the rigorous mathematics and statistics communities that immediately began mocking it as a strict guide, and never stopped [until right up when the mathematics community went Woke just a couple of years ago, but that's its own extremely sad story and probably sign of the end times].
Source: (Aslam, Georgiev, Mehta & Kumar, 2012) CC BY-NC-SA 3.0
Back to parachute RCTs. This story isn't over…
More recently, a real parachute RCT was conducted—for a two foot drop. The results were again published in the BMJ in a "Christmas edition" (Yeh et al, 2018). The study concluded that parachutes show no benefit in the prevention of death.
The Results
23 people participated (out of 92 screened).
There was no difference in the primary outcome. Mortality and major traumatic injury occured in 0% of both groups, both at 5 minutes and 30 days.
The average height of the jump was 0.6 meters with an average velocity of 0 Km/hr. 0% of the parachutes actually deployed.
According to the inclusion-exclusion criteria, the result does not apply to those suspected of mental illness. It may very well be that parachutes do prevent injury or death to schizophrenics, the chronically depressed, or physicians who believe that "quality of evidence" is "well enough defined to tell statisticians that they don't understand their subject if they're not following it",
but then only where they decide their personal experience trumps textbook methods.
(Yes, I extended him my usual invitation to a recorded conversation on the topic. No, he did not dare to reply.)
In recent years (quite recent, in fact), the medical community has pushed back at the parachute story, as with a study (Hayes et al, 2018) designed by Vinay Prasad.
Leave it to a crack team of scientists to conclude that most medical practices are not fabrics that allow for safe control during otherwise deadly circumstances. The more interesting question is whether or not we can leave the "fabric" part out of that conclusion.
On the other hand, the Hayes paper does nothing to demonstrate that most medical practices fall into the aforementioned subset of exclusions. Literally all it did was provide a count of the number of papers in medical literature that were defensive about the topic.
A Modest Proposal for an RCT in the COVID-19 Era
Recall the Massachusetts data project that seems to show that the number of COVID-19 deaths in MA is closer to 8,000 than the CDC's claimed 20,000.
The 20,000 number was around 4,000 more than the total number of death certificates that mentioned COVID-19 in any way, shape, or form. But it is clear that many of the remaining 16,000ish deaths were not all due to COVID-19. Heck, we can't even tell whether many of those who died even had COVID-19 symptoms at all.
🐭Nodeling 🌸 @Nodeling1
@WAPFLondon @Jikkyleaks @ONS @EduEngineer I've posted this before, but it's always worth posting again for a laugh... https://t.co/16EhqBji8dI was recently thinking about the "died with" vs. "died from" COVID-19 mortality confusion, and another mystery came to mind: how the experiment COVID-19 injectable products supposed reduce non-COVID-19 deaths (dramatically) to the point that it is implied that among 20-44 year olds who take it, there is substantially less mortality due to accidents, suicides, and homicides.
So, a modest proposal has come to mind: Let us propose an RCT…
Each time somebody at the FDA or CDC age 20-44, or a child of an employee of one of those agencies in that age range, tests positive for SARS-CoV-2, we randomly assign that person to one of two arms:
The Control: We inject them with a saline solution marked "Comirnazi"
The Treatment Arm: We inject them with a dose of one of the experimental gene therapy products. Then we put them all in a room together with a wrench, some rope, a lead pipe, a knife, a revolver, and a candlestick, and tell them they can't leave until everyone left alive gets a clue.
The primary endpoint is how many of them die from non-COVID illness.
And now, your moment of zen. A video by Joel Smalley:
I am inspired to start making my own videos now.
Disclaimer: Don't try any of this at home. Any of it.
Ha ha! I had no idea that last line was coming! Thank you!!