Defining Away Vaccine Safety Signals Part 2

The Chloroquine Wars Part XLIX

A lot of discussion has been had in many circles since my article on the CDC's vaccine safety signal reporting I published Monday. While I would not declare myself anything like a pharmacovigilance expert, I continued to read and learn more myself, and after viewing comments, I would like to offer additional discussion and analysis. Later in this article, I will give a second, more common sense explanation of the PRR safety signal used to generate the CDC's internal reports that should help many readers understand the problem.

But first, let's start with the mail bag. A number of critiques were offered. There was a striking difference between the numerous constructive conversations I had with some scientists and researchers (which I have hopefully absorbed well, adding support for this second article), and those who seemed to want to say, "You're wrong and here is why." Let's take a look at the most coherent snapshots of the latter.

"There are other statistics for measuring safety."

I think you meant to send this to the CDC.

"OK, this feels like anti-vaxx nonsense wrapped up in fancy clothes. I think they are misrepresenting how it is used; I'd refer to something like https://pubmed.ncbi.nlm.nih.gov/11828828/ instead; has over 1,000 citations."

The game of labeling people who believe that the Precautionary Principle is important and take it as a duty to put effort into its embodiment as "anti-vaxx" (dismissible immediately as "nonsense") has run its course and become increasingly transparent. Would it make more sense for me to adopt a policy of either, "Okay, you can inject me with anything you want," or, "Keep that needle away from me," as binary absolute operating procedures?

I do appreciate the inclusion of a link worth reading, though the counting of citations like medals on a general's jacket makes me suspicious that this was a 30-second PubMed search for the sake of appearing expert.

The linked paper is indeed an examination from 2001 of the use of the CDC's exact reporting signal for COVID-19 vaccines. But hardly looks like a counter to my critique since the PRR function was applied to 15 newly-marketed drugs of no particular relationship that would result in correlating AEs. A total of 481 signals were found. It is noteworthy that the result stands in striking contrast to the several experimental vaccines that have generated orders of magnitude more AE reports while apparently appearing "safe and effective" to the CDC.

However, the scientific literature has not been kind to the use of PRR over the years.

• In 2016, Wisniewski et al (including authors from Astra-Zeneca, Pfizer, and other pharmaceutical corporations) examined good signal detection practices and stated, "calculation of PRRs...should not replace nor delay the performance of formal epidemiological studies." Maybe the CDC should get to work on a risk report at long last?! The study goes on to say such signal reports, "are not easily interpretable in terms of clinical impact." H/T David Wiseman, PhD

• In 2017, Hauben and Maignen walked through the history of disproportionality analysis, including signal functions such as PRR, and found substantial evidence of what has been termed "masking" by the honest statisticians in the pharmacovigilance community. Masking takes place when---you guessed it---strong associations exist between AEs recorded from multiple interventions (sort of like two nearly identical gene therapy vaccines, ahem). So it appears that in my article that I rediscovered an already named principle, and the literature agrees with my assessment. H/T Daniel Jacob Bilar

• In 2018, Viswam et al found PRR to have the lowest sensitivity among various methods for AE signal detection.

I read a few more papers today, and overall it certainly looks like the pharmacovigilance community has my back in the meat of my critique.

"The PRR, on the other hand, is a tool for comparing the relative frequency of a particular side effect to the relative frequency of that side effect in other drugs. As such, it is fully intended to be invariant with respect to the total number of AEs."

Like the first comment from the mail bag, I think this person (a mathematician I'm told) thought they were disagreeing with me. But they immediately continue...

"Another sneaky misunderstanding is that the author implies that each COVID vaccine is being compared to the other COVID vaccines. While one could certainly do that calculation, the FDA has a large AE database for vaccines, and safety calculations are normally done to the universe of approved vaccines."

"Sneaky misunderstanding" may be the most confused phrase I read among the critiques, and it comes across as projection.

The second sentence is wrong, but has a point. The CDC likely would have stated if they were comparing the COVID-19 vaccines against "the universe" of vaccines. What they said, specifically (if vaguely) is this:

CDC will apply appropriate comparator vaccines (e.g., adjuvanted vaccines like Shingrix and/or Fluad for adjuvanted COVID-19 vaccines) and adjust for severity and age distributions where applicable. 

Understand two points:

1. The CDC has refused repeated attempts at communication with the group I work with. That would be the kind of communication in which we might learn the specifics.

2. COVID-19 vaccines (over 400,000) are an enormous proportion of all VAERS reports (around 700,000 prior to COVID-19 vaccine rollout) and will likely outstrip all other reports combined by the time those handling report submissions catch up with the backlog.

3. The median age of a VAERS report historically is just 3, while COVID-19 vaccine reports in VAERS skew heavily toward the elderly. After adjustment for age distribution, COVID-19 vaccines will be almost entirely compared with other COVID-19 vaccines for the purpose of PRR signal analysis, and chi-squared statistics.

I may have jumped off the academic track without a PhD in mathematics, but I'm aware of how much work I've done to understand the problem, and how much I haven't.

"This complaint falls flat. This is the criterion to suggest a signal based on PRR. Further on in the document, Section 2.5, they note that there are several sources by which a potential signal can be detected:

FDA empirical Bayesian data mining, through CDC PRR data mining, and through descriptive analysis."

Yes, there is a very limited list in a statement that reads like, "maybe we'll do something aside from read the empty signal analysis reports, and maybe we won't." It is telling that the CDC had to be prodded by some people from the state of Washington to assess myocarditis signals after failing to learn from Israel which had mass vaccinated ahead of the rest of the world. Were they ignoring analysis of the signals, or was there just an empty signal report they could toss in the trash?

I've done my share of Bayesian data mining. My students would tell you that I emphasize the importance of conditional analysis as the single most valuable real world application of anything that I teach. Around a half dozen articles in this series walk through an application of Bayes' Theorem. And I can brag that the editor of Mathematics Magazine told me that my class on the topic was the single best he'd ever seen. When I trade, I crush the results of every hedge fund in the known universe (really), and I make no bones about the fact that nearly everything interesting that I do in making that happen is the result of clever Bayesian analysis.

And yet, here I am at a total loss for what in the heck I would possible do to apply conditional statistics with the reported VAERS numbers that would tell me anything more than, "HOLY COW THESE ARE THE LARGEST REPORT NUMBERS ANYWHERE IN THE WHOLE DATABASE BY A WIDE MARGIN EVEN THOUGH THE VIDEO INSTRUCTING THE PROCESS OF SUBMISSION WAS TAKEN DOWN AND WE MIGHT SHOULD BE TAKING A CLOSER LOOK AT ALL THIS AND NOBODY IS BOTHERING TO STUDY TRUE-TO-REPORTED RATIOS OR ESTABLISH A RISK REPORT!" 

But it would have been nice if the supposed statistician who left this comment walked through something like an example. They continue,

"Based on what I can tell, the author is misunderstanding here. The CDC document does not say that they are using a Chi-square test, but a Chi-square statistic. The statistic can be used even if the assumptions of the chi-square test are violated. It would be, at that point, just a measure of the consistency or lack of consistency of the table. It's like calculating a mean, you can do that even if you don't assume a Normal distribution and use a t-test."

It is unclear what the supposed statistician thinks I am misunderstanding, nor does this read the way I would expect a statistician to have written it (*cough*). The point I made, and will reiterate, is that the VAERS reports for these vaccines retain enough self-similarity that the chi-square[d] statistic will be muted. Feel free to run the numbers to show me otherwise, but to what numbers geek is this not obvious? The "statistician" continues,

"PRR deals with relative rates, because otherwise we would conflate "dangerous" with "common".

This means the scale-invariance the author complains about is a good thing.

The PRR signal detection method is just one of several methods, which means the scale-invariance the author complains about is not hiding potential problems."

The literature disagrees. Otherwise, these are statements without warrants after I gave a pretty clear example of how a large number of deaths would not trip the signal. What more do you want?

Believe it not, I left out the most nonsensical of the negative feedback. But I am happy to have learned a lot over the past 54 or so hours.

You know what is interesting about every single critique I received?

1. They were sent anonymously through other people. (cowards)

2. Not one of them put numbers in a spreadsheet (as I did) to show why I was wrong, which would have taken less time than the 10- and 12-paragraph TL;DR compilations of motivated reasoning, ad hominems, and childish credential waving.

3. While each one of them responded with an air of casual expertise, none of them knew the pharmacovigilance literature well enough to know that the masking I observed was an already recognized phenomenon (not that I did, but I'm not trying to convince anyone that my argument requires a credential).

4. Not one of them expressed the concern with which I ended my article---over the misclassification of vaccine deaths as COVID-19 deaths. Can you even pretend not to be a partisan in the discussion when you skip over the most shocking part of the story?

If this is the best the critics can do, I feel substantially more secure in my current position. But again, I'm not an expert, so I'll keep my mind open. But it would be nice if the CDC returned an email.

Okay, now it's my turn:

Warning: If you are a mathematician or statistician and slough this issue off as not meaningful, and while looking as obtuse as these commenters, you are going to further erode an already rapidly waning confidence in our scientific and medical institutions.

At some point, the pitchforks do come out. And they function just fine in the face of cyber attacks and power grid failures.

I've worked with many highly credential mathematicians, actuaries, scientists, and other smart people in labs, offices, and on various projects. These include people of impeccable integrity. Some of them, in the service of science and education, sent me additional articles to read on the topic at hand. Those I have worked with also narcissistic sociopaths---maybe even a few of the Kunlangeta. The reason I stopped pursuing research (or any academic path at all) while I was in college is that I was offered money to write fake papers. You can't fool me because I've seen too much and know what kind of horse shit you eat for breakfast.

I also know a lot of people---some extremely smart, and some of more average intelligence---outside of those enclaves. Some of them don't need to read half the books you do in order to understand a thing. And the people outside of those enclaves understand the credentialed New Mandarin class of the West better than it understands them. The clothes are thinner than the emperor may think. If you don't understand that articles like this analyzing (to varying degrees of precision, certainly) the orchestration of societal progression by oligarchs through the Best Slave Pageant we call an educational system that promotes trained narcissistic sociopaths alongside the talented-but-terminally-unaware are a dime a dozen, you're living in a bubble that is about to collapse. 

The war drums are beating, you're too comfortably smug to survive, and I've done everything I can to warn you or encourage you onto a different path.

And if you can't smell the hint, you suffer from something worse than COVID. If the drums don't sound like tick-tick-tick, you're tone deaf.

If you believe that you can rug pull the Precautionary Principle that has ruled medicine for decades like the illusionists you've aspired to become (when you found your overdefined educational paths lacking opportunity to add value), you should think again. If you hold a position of academic influence, but fail to voice a concern over corporations and medical establishments pushing experimental new technologies at even the world's children while looking lackadaisically past primum non noctum, you may find yourself held accountable.

Now, back to our regularly scheduled program…

A Second Look at the CDC's Signal Analysis

Since my last article was a bit mathy for some readers, I thought I'd take a moment to explain the PRR signal function using a simple analogy. The PRR is a ratio of two similar quantities, each of which is

The fraction of the number of one type of adverse event out of the total number of adverse events ascribed to a vaccine.

To make the point, I added two rows for Vaccine 3 on a tab in my spreadsheet. Row 12 shows the computation of a/(a + b) for each AE while Row 13 shows the computation of c/(c+d) for each AE.

Row 12 shows six fractions that add up to 1. They are the result of dividing the numbers immediately above them by the total number of AEs (244) just to the right of them in column I.

Row 13 shows six fractions that add up to 1. They are the result of dividing the numbers of the same AE for all the other vaccines by their total (which is 1619 minus 244).

Row 14 contains the PRRs, which are the ratios obtained by dividing each number on Row 12 (in red) by each number in Row 13 (in blue).

Each PRR is a ratio of ratios. The ratios in each Row 12 and Row 13 might be likened to heights at points on two essentially identical plucked guitar strings. The PRRs tell us how in sync are the waves. When the waves are relatively in sync, the PRRs are closer to 1, and generate no safety signal.

And that's the whole point: these vaccines establish a similar method for inducing an immunizing antibody response (using the spike protein). So, the number of AEs can grow extremely large, clearly indicating dangers including death and paralysis, without the PRRs ever signaling a safety concern.

I hope that helps.