Defining Away Vaccine Safety Signals

The Chloroquine Wars Part XLVIII

"We demand rigidly defined areas of doubt and uncertainty." -Douglas Adams, The Hitchhiker's Guide to the Galaxy

What if it turns out that vaccines are killing and crippling millions of people around the world, but that those harmed are just well enough spread out that almost nobody saw sufficient signals to build an intuition about the problem? And what if the agency most responsible for examining safety signals defines their algorithm using a nonsensical mathematical formula that hides nearly all serious problems?

Last night I tried to go to sleep early in order to shift my exercise routine to a morning schedule. But I'd received a late phone call from a friend whom I work with researching vaccine safety/danger, just as I was brushing my teeth. He remains anonymous for now due to fear of job loss or reprisal. I let the phone call go, but I couldn't just let it go. I needed to know why he would call me that late, so after spending an hour in bed not sleeping, I checked my email. I understood immediately.

---

Definitions

To a mathematician, everything depends on definitions. Whatever we state mathematically, the definitions of the terms we use should be traceable back to the axioms of the field we're working in. Mathematics is an artfully woven tapestry of axiomatic structure, lit brightly by definitions. Often, we create new definitions for the task at hand, but make sure that these relate clearly to the pyramid of definitions that come before it---generally as standard as possible to language common within the field. This becomes both a habit and also a part of the self-reinforcing social structure of the mathematics community. We talk definitions that sound boring to untrained ears, but we recognize that definitions are our palettes. We port definitions to applied fields, and we create new ones to use where needed. No mathematician or statistician with a soul forgets the well from which they draw energy and meaning.

Credit: Sharon Apted

What we don't do (after having any bad habits distilled away over the course of maturity into the role) is lackadaisically establish definitions entirely ill-suited to a purpose. So, when I read what my friend found, it startled me. So much for getting to bed early.

---

Adverse Events and Safety Signals

During the past few months, many people (including myself) have learned for the first time about the Vaccine Adverse Event Reporting System (VAERS) where health care workers and patients can submit adverse events (AEs) suffered post-vaccination. While the VAERS database is understood to record an often small subset of the AEs, the information can still be used for purposes of establishing safety signals. A new influenza vaccine can be compared to other influenza vaccines, for instance. If the old ones were safe enough, and the reported AEs of a new one are in line or better than for past vaccines, then the risk-benefit analysis for the new vaccine (assuming sufficient efficacy) either remains the same or improves.

As with a great deal of health care regulation during the declared pandemic, changes were made to the VAERS system and also to safety signal analysis leading up to the experimental mass vaccination program officially targeting COVID-19. Without much fanfare, the CDC published a document on January 29, 2021 entitled Vaccine Adverse Event Reporting System (VAERS) Standard Operating Procedures for COVID-19. There is a lot to talk about in this document, but let us focus on Section 2.2, which begins on page 14. Here, the CDC states that, "A series of tables will be generated using the VAERS automated data," and that these, "will be refreshed daily for internal use," but "not for public release". One might wonder why the CDC would not want additional outside eyeballs on such data---particularly since it took them two full months to figure out that myocarditis was an issue with the vaccines despite Israel warning about it two full months before the CDCs scheduled, delayed, and finally held meeting in late June. Maybe the CDC should hire somebody to read the pertinent news?

We get to section 2.3, and this is where things get really crazy. This is where signals (for assessing safety/danger of the vaccines) get defined. Subsection 2.3.1 begins (emphasis mine),

CDC will perform PRR data mining on a weekly basis or as needed. PRRs compare the proportion of a specific AE following a specific vaccine versus the proportion of the same AE following receipt of another vaccine (see equation below Table 4). A safety signal is defined as a PRR of at least 2, chi-squared statistic of at least 4, and 3 or more cases of the AE following receipt of the specific vaccine of interest.  

Only a real dork would emphasize the word 'and', right? A logic dork, mind you, but we'll get to that...

First, note that PRR is the proportional reporting ratio, and these PRR numbers are the outputs of a function defined by the CDC based on four variables (which they list in a table as capital letters, then apply in a function as lower-case letters, which always makes me a little uncomfortable as I rarely see such sloppy transition from definitions to application, and somehow they always seem to come from government documentation where I worry about ass covering and plausible deniability).

Look at the numerator of this formula. The variables a and b are specific to each vaccine. Now, consider what would happen if an extremely dangerous vaccine were introduced that resulted in 20 times as many AEs of all types as all the other vaccines to which it gets compared.

The PRR remains invariant in the scaling of adverse events! 

This means that one vaccine that kills and cripples 20 or 50 or 1,000 times as much as a very safe vaccine will show the same PRR (mild adjustments for variables c and d notwithstanding), and no safety signal will be identified by the CDC. By design. In my circle of discussion, I've found out (to my dismay) that this definition of PRR has also been used for other vaccines. Just during the COVID-19 vaccination campaign, important safety signals have come and gone without notice, such as the MedDRA term "death" showed up as a signal in dispersion analysis in February, but no longer does due to the rising quantity of so many other AEs, which forces structural mean-reversion of the PPR function (toward 1) by inflating the denominator. In other words, the number of an AE, a, is normalized to the total number of adverse events (not the number vaccinated, or doses), the ratio of which is then normalized again to the aggregated totals (fraction c over c + d) from other vaccines.

It is not necessary to scale all of the AEs to hide a signal with this function. The numerator-fraction of the PRR changes only slightly when a handful of AEs scale up, even for extreme values of the scalar, k. Here is an example where AEs of various kinds are reported in numeracy (55, 7, 6, 11) for one vaccine, and 20 or 1,000 times as many of the first two AEs are reported for another:

Since the numerator does not change all that much as the AE numeracy scales, and the denominator remains unchanged, the potential for a signal changes very little. And if ratios among AEs change little between vaccines (like for an AE that is the result of the presence of the spike protein) due to correlation, the denominators will change in a manner that is highly similar to the proportional changes in the numerators! And this should be intuitively clear to most mathematicians given the similarities of the numerator and denominator functions, along with the fact that each is a piece of a whole:

Here is an example from a spreadsheet I quickly put together to see PRR numbers for plausible data sets of AEs:

Where we do see PRRs stray far from 1 (a sort of baseline) is for the safest vaccine (the first up top), or among the AEs that are least likely to be causally associated with the actual vaccines (which I labeled as "not spike protein related" since the spike protein seems to be the variable that distinguishes the current vaccines from traditional ones). Note that even if I take a cell such as Vaccine 3's AE 3 (which could be Bell's palsy or heart failure or death) and plug in some enormous number like 1500, there is still no safety signal as per CDC definitions:

And when we list out dozens and dozens of AEs, and bring closer the numbers of each AE for each vaccine (as is mostly the case), the PRR function becomes even less sensitive to individual changes of scale per AE. Certainly there are conditions that result in safety signals, but these are far at the extremes for the AEs that we most need to understand.

Do you kinda get the sense that the PRR function is designed to hide signals of unsafe vaccines, not to identify them?

---

Maybe this is a matter of incompetence?

Hanlon's razor: never attribute to malice that which is adequately explained by stupidity.

There is no doubt that governments are jam packed with incompetent people in important roles (often scattered around a handful of brilliant ones who unfortunately lend their credibility). Perhaps that is why somebody like Richard Feynman was called in to help figure out what went wrong with the Challenger shuttle disaster. So, I'll take this time to note that the person who called me last night has also been emailing the CDC for weeks, either with no response or an obtuse brush off. Maybe I'll have the time to write about that in another article.

However, Hanlon's razor is one of the most oversubscribed principles of human interaction. A firm belief in its truth is an invitation for con artistry. Wherever there are concentrations of money or power, we should expect to find the fully manifest game theory of hawks and doves. 

At some point, when the potential for conflicts of interest are high and the point of failure is fundamental to the task of those doing the job, incompetence should no longer be differentiated from criminal intent. Mathematicians and statisticians worthy of the title do not miss the kind of scale invariance or mean-reversion we see embedded in the PRR---particularly not when working in a dedicated group on a serious problem. There is a pride among geeks in identifying subtle mathematical or logical flaws in a system, and this is not subtle at all. Given that even psychologists have a need to pay attention to scale invariance, this is not an easily excused mistake. In fact, a computational check of the PRR function against past data likely would have clued in a middling programmer without the fundamental mathematical training. And we're not done, yet…

Let's look back at the definition of a safety signal (emphasis mine):

A safety signal is defined as a PRR of at least 2, chi-squared statistic of at least 4, and 3 or more cases of the AE following receipt of the specific vaccine of interest.  

Not only does the PRR need to get out of line for a safety signal to be generated, the use of 'and' instead of 'or' means that other additional criteria must also be satisfied before the CDC self-reports a safety signal! And the chi-squared criterion isn't helping the situation. Those unfamiliar with statistics will not recognize the problem, but when the different vaccines all result in similar problems (like myocarditis or blood clots, as has been noted), the chi-squared statistics will remain muted. In fact, chi-squared statistics are not even supposed to be used on data that is likely to be correlated when causal. Presumption of a negative test result is not a reasonable test standard. It's as if nobody at all thought about the possibility that the spike protein being an underlying thread among the vaccines could possibly be important. 

So, when you define away the problems on paper, they just cease to exist?

Calling this a safety system is decidedly unsafe. Even worse---given that numerous academics, including statisticians, reviewed this document, it is hard to believe that the scale invariance embedded in the definition of PRR, or the logic that includes meeting multiple criteria at the same time, went unnoticed. It certainly appears that the CDC's goal was to establish an illusion of safety, and a reason to ignore the true signs of danger. At best, they unfathomably hired nobody whose job it was to remain independently-minded who could clue them in?

If my understanding of this situation is correct, the mass vaccination program should be immediately halted until the safety data is gathered, cleaned, and examined. We cannot tolerate a misleading statement of "vaccines are safe and effective" in the face of regulatory agencies defining away the responsibility of performing the risk analysis needed to verify safety. The CDC leadership should be immediately replaced and investigated, and independent analysts should reformulate the task of tracking vaccine safety results.

I really hope I've misinterpreted something. I stumble over ideas and make mistakes just like everybody else. Unlike the CDC, I'd like for you to share my numbers widely and invite critique. By all means, forward this article to everyone you know. Maybe one of them can explain my error and set me straight. So...please forward this to everyone you know. 

Everyone.

I'd rather be a little embarrassed in the eyes of many if I'm wrong than to watch even one more person get sick or die and perhaps become another false COVID statistic.