A Letter to the Editor-in-Chief of Clinical Infectious Diseases
The Chloroquine Wars Part XLII
I would like to take this moment to encourage readers to take the step I have taken and write a Letter to the Editor of Clinical Infectious Diseases. The email of Editor-in-Chief Robert T. Schooley, MD can be found here. In order to raise the standard of published science, it is important that he hear from those who have concerns about the veracity of scientific claims made in his journal. I encourage those of you with scientific and statistics backgrounds to do as I have done and point out some of the many flaws I did or did not identify in my article about the paper.
My own letter is below.
I recently read the meta-analysis of ivermectin efficacy written by first time lead author Yuani Roman, MD, MPH, and her colleagues. Having read thousands of papers, researched the background information cited in many of them, and often re-modeled or checked results for statistical accuracy, this stands out as the single sloppiest paper I have ever encountered published in a medical journal or junior high newspaper. The errors are not simply numerous, but of such variety that it brings to mind a video I came across of animals that express themselves by throwing feces. While that may make for an interesting study in primate behavior, it seems inappropriately classified as evidence regarding the efficacy of ivermectin in prevention or treatment of SARS-CoV-2/COVID-19.
After spending a number of hours with Roman's paper that I wish I had back, and those from which it draws data, I came up with a list of problems too numerous to include in a substack email of defined information limits, so I was forced to cull some paragraphs and ultimately stopped short of enumerating them all. Perhaps you could be so kind as to compile a more complete list as others write in. Though perhaps that isn't necessary as the authors got most every detail wrong, while penning an inappropriate conclusion statement that fails to match the most important (mortality) results of the paper, the true results after correcting the many errors in the paper, nor takes into account starkly positive research regarding ivermectin's efficacy as a prophylactic.
Further, I would recommend severing ties with such a research group that mishandles so much information that they seem at best entirely incompetent at handling data. The lead author, Roman, was a coauthor with the lead author of a meta-analysis on hydroxychloroquine efficacy published on the heels of the Surgisphere paper, including Mehta's analysis (get it?) as the majority of the data points in the results. It seems extremely difficult to imagine anyone who works with data reading that paper with a straight face. A little laughter is good for the soul, of course.
It would do the community a great service to raise the reputation of Clinical Infectious Diseases above the fray of such journals as are now being reclassified as the funny pages.
With Much Bewildered Amusement,
Mathew Crawford, Administrator of the Ultimate Intergalactic Arbitration Council of Statistics and All Other Matters
Here's an interesting assessment of health research in general. The link is a review of a blog post at bmj.com:
Hi Matthew, very much enjoyed your irreverent letter to Dr Schooley and your previous demolition of Roman et al.
Whilst Roman et al has been the subject of much specific criticism, I've only come across one specific critique of the Bryant et al meta-analysis. (Clearly, there's been plenty of broad/vague criticism which seems to have consisted of recitation of the "small studies, low-quality studies" mantra!)
The one specific critique is by Gideon Meyerowitz-Katz
If you can find a moment, would you be able to give your opinion as to his critique. I'm not statistically competent to pass judgment and so would very much appreciate the thoughts of someone who is. Thanks!