What does the Flaxman controversy say about the U.S. CDC?
While steelmanning whether children 6 months through 4 years should get the Pfizer vaccine or not, a controversy was highlighted over the Flaxman et al. (2022) paper. This paper was used prominently by the U.S. CDC during its vaccine approval process (U.S. CDC, 2022).
The controversy started after “a mother who reviews this data on my personal time” reported an error about the reported death rates on Twitter to the study’s main author, Dr. Flaxman (Kelley, 2022; Kelley, 2022b). Dr. Flaxman admitted to the error on Twitter (Flaxman, 2022). The study was subsequently fixed and re-published, and significantly changed the results: “We have fixed an error” (Flaxman et al., 2022c).
One might think, “that’s unfortunate but that’s just science in action”. The paper was used by the CDC prominently, but one might think, “that’s just an unfortunate mistake in a fluid situation and the vaccine would have been approved anyway due to other evidence”.
However, some facts about the controversy raise some questions:
It was a paper about the United States that was authored by a team mostly out of the United Kingdom. The team made basic mistakes mixing data from incompatible databases which is something a U.S. team accustomed to such data would not do, particularly the CDC itself which publishes the raw data.
The paper originally mixed cumulative COVID-19 deaths and annualized deaths for other causes. No study was used to justify this (Flaxman et al., 2022d), and it was removed in the final version. However, the CDC knowingly mixed annualized and cumulative deaths as it noted on its slides (U.S. CDC, 2022b; U.S. CDC, 2022c).
The paper was a pre-print meaning it was not yet peer-reviewed. This is not an inherent problem but suggests extra caution.
Here’s the steelman defending the CDC:
The CDC are under extreme pressure in an emergency situation with limited resources so they depended on an external team of scientists and didn’t have the time to investigate it in detail due to competing demands for time. The lead author, Dr. Flaxman, has co-authored other research on COVID-19 epidemiology in the United States so he was being trusted (Monod et al., 2021).
Despite the cumulative statistic ultimately being removed from the study, cumulative mortality is a plausible way to compare COVID-19 deaths to other deaths because lockdowns, masks, and social distancing might have reduced the death rate that otherwise would have occurred. The use of the cumulative statistics were clearly noted on the slides.
Even if COVID-19 ranks lower than flu & pneumonia, there are still a significant number of COVID-19 deaths for this age group and the basic point stands that childhood deaths are a significant justification for approving the vaccine as the flu vaccine is similarly approved for this age range (U.S. CDC, 2022e).
Here’s the skeptic steelman:
Instead of analyzing their own data which is not particularly complicated (U.S. CDC, 2022d), the CDC used the analysis of a non-peer reviewed study by a team from the United Kingdom who were so unfamiliar with the data that they made a basic and significant mistake that, when fixed, largely changed the story the CDC presented at the vaccine approval meeting. The fixed study shows half the rates as seen in the CDC slides, ranked COVID-19 7th instead of 4th/5th, and ranked flu & pneumonia as worse mortality than COVID-19 (Flaxman et al., 2022b).
The CDC knowingly used an uncommon comparison of death rates (cumulative vs. annualized), later removed from the study, that had purely speculative assumptions which were not even mentioned. The original study reported annualized COVID-19 statistics but the CDC consciously chose to use the cumulative statistics. The presentations also emphasized that COVID-19 was the top infectious cause of death which is no longer true in the study results.
The CDC could have presented mortality much more simply, gotten the point across that COVID-19 kills lots of children, and the vaccine would have still likely been approved, so what was the point of such gratuitous exaggerations? At best, this is a sign of general incompetence; at worst, motivated reasoning and vaccine zealotry. In either case, this makes one wonder about the CDC’s other data and interpretations.
We reached out with questions to Dr. Flaxman, Dr. Daley, Dr. Fleming-Dutra, and Dr. Oliver but have not heard back for days. If they respond, we’ll update this post.
Thanks for reading Steelman Anything! Subscribe for free to receive new posts and support my work.