Deming's Philosophy and Vaccine Development

The 2021 recipients of Lasker~DeBakey Clinical Research Award, biochemist Katalin Karikó and immunologist Drew Weissman, have been widely and deservedly heralded for their groundbreaking contributions to mRNA research. Shapiro and Losick (2021) wrote in Cell that their “sustained efforts in adapting mRNA as a platform for producing therapeutic proteins in host cells are what enabled fast development of highly effective vaccines against the pandemic-causing SARS-CoV-2.” Many recipients of the Lasker award have gone on to win the Nobel Prize, and one might expect Karikó and Weissman to follow this path.*

There is no Nobel Prize for statistics, however.** If there were, the statisticians whose research made it possible for two mRNA vaccines to be rigorously tested and approved — less than a year after the genetic sequence of SARS-CoV-2 was published — would certainly be candidates for this award. The Phase 3 clinical trials for these vaccines were able to be conducted rapidly, while adhering to all research protocols, because of the groundbreaking statistical work in vaccine trial design that has been done over the past 70 years.

Dr. Ivan Chan, who has contributed greatly to this research, recently gave the Deming Lecture at the Joint Statistical Meetings on the topic “Deming Spirit in Action: Statistical Innovation, Quality, and Leadership in Vaccine Development.” You can watch the lecture, a wonderful resource for learning about some of the statistical research that has been done in the field of vaccines, at the American Statistical Association website at https://ww2.amstat.org/meetings/jsm/2021/webcasts/. I wrote earlier this year about how Deming’s methods of quality improvement could be applied to vaccine distribution; Dr. Chan discusses their application to vaccine development and testing.

The talk has three parts. Dr. Chan assumes that the listener is familiar with terms such as “confidence interval,” “Type 1 error,” “power,” “odds ratio,” and “control group.” If you have encountered these terms before, you should be able to grasp the main ideas (if not all the technical details) from his talk.

1. In the first 20 minutes, Dr. Chan discusses the basics of vaccine development and testing (for more background, see the primer on statistical vaccine trials by Mehrotra, 2006). He reviews the statistical lessons from the Salk polio vaccine experiments in the 1950s, particularly the importance of randomizing vaccine trial participants to get either the vaccine or placebo and ensuring that the trial is double-blinded (that is, neither the study participant nor the person administering the treatment knows whether the participant is getting the vaccine or the placebo).

2. In Part 2, covering about the next 25 minutes of the talk, Dr. Chan talks about some of his own research in vaccine trial innovations: on determining the sample size needed for a clinical trial to be able to establish vaccine efficacy and safety and on using adaptive designs (where data that accrues during the clinical trial can be used to modify the design while the study is ongoing). He also discusses challenges in evaluating correlates of protection (biomarkers that can be used to predict the level of protection an individual has against a pathogen), with examples from his research on determining antibody levels that correspond to protection against chickenpox and shingles.

   Dr. Chan references a few terms during this more technical part of the talk that I’ll explain here. DSMB stands for “Data Safety Monitoring Board,” an independent group that provides oversight and monitoring of the clinical trial.*** The “Prentice criteria,” proposed by biostatistician Ross Prentice in 1989, are a set of four criteria that a biomarker should meet to be able to serve as a surrogate endpoint in a clinical trial (in the vaccine setting, these boil down to having a biomarker that is related to the vaccine and also to protection from infection or disease). You can read about the details of the four phases of clinical trials here; phase 3 is the large-scale randomized trial that compares the vaccine with a placebo.

3. Part 3, starting a little after minute 45, addresses statistical aspects of the COVID vaccine trials. Dr. Chan outlines how statisticians, building on decades of previous research, were able to design phase 3 trials that were able to evaluate the vaccines’ efficacy and safety within a few months. One aspect that helped, of course, was having practically unlimited levels of funding. A second contributor to the rapid conclusion of trials was the alacrity with which people volunteered to participate. Many clinical trials take years to complete because it takes a long time to enroll enough people for the randomized study to be able to produce statistics with the desired precision. That was not a problem in most of the COVID vaccine trials, where people were lining up to participate. But the third reason these trials could be run so quickly is that the statistical methods, developed over decades of research, were already in place for determining the necessary sample sizes, randomization protocols, interim stopping rules, and analyses.

Scientists have been justifiably lauded for their role in developing COVID vaccines. Without the statistical innovations that have been developed for vaccine trials, however, it might have taken years to run the phase 3 trials that established that these vaccines are safe and effective across diverse populations. Deming (1986, p. 20) wrote: “Hopes without a method to achieve them will remain mere hopes.” The fact that vaccines for COVID could be made available to the public a little more than one year after the virus was sequenced, saving millions of lives around the globe, is a testament to the power of statistical research and methods.

Copyright (c) 2021 Sharon L. Lohr

Footnotes and References

*Not this year, though. All of the 2021 Nobel science awards were awarded to men. Only 10 women have received a Nobel prize in Physics or Chemistry (10 women but 11 awards, since Marie Curie received the award for Physics in 1903 and for Chemistry in 1911) in the history of the award; 12 women have received a Nobel prize for Medicine.

**Statistics wasn’t really considered a separate discipline when Alfred Nobel died in 1895 (the first academic statistics department was established at University College London in 1911), and so was not included among the original set of awards. However, the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel, sometimes called the Nobel Prize for Economics, started in 1969. Some of the Sveriges Riksbank prizewinners have been recognized for statistical research.

***If you want to learn about DSMBs and clinical trials in a fictional setting, you might want to check out the mystery novel Data Games: A Techno Thriller by statistician Herbert Weisberg. The novel takes place in 2026, after the COVID epidemic has subsided (thanks in large part to the vaccines), and concerns the development of a new cancer-fighting drug — and the efforts of malefactors to discredit it. The mystery turns on issues of biomarkers, surrogate endpoints, and machine learning models, and all of these are explained through the action of the novel. As the protagonist’s wife says on page 46 of the novel, “Who knew being married to a statistician could be so exciting?”

References

Baden, Lindsey R., El Sahly, Hana M., Essink, B., et al. (2021). Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. New England Journal of Medicine, 384, 403-416. Technical information on the Moderna vaccine trial is in the Supplementary Appendix.

Chan, Ivan S. F. and Bohidar, Norman R. (1998). Exact Power and Sample Size for Vaccine Efficacy Studies. Communications in Statistics - Theory and Methods, 27, 305-322.

Deming, W. Edwards (1986). Out of the Crisis. Cambridge, MA: Massachusetts Institute of Technology Center for Advanced Engineering Study.

Mehrotra, D. V. (2006). Vaccine Clinical Trials - A Statistical Primer. Journal of Biopharmaceutical Statistics, 16, 403-416.

Polack, Fernando P., Thomas, Stephen J., Kitchin, Nicholas et al. (2020). Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine, 383, 2603-2615. Technical information on the Pfizer/BioNTech vaccine trial is in the Supplementary Appendix.

Prentice, Ross L. (1989). Surrogate endpoints in clinical trials: Definition and operational criteria. Statistics in Medicine, 8(4), 431-440.

Shapiro, Lucy and Losick, Richard (2021). Delivering the message: How a novel technology enabled the rapid development of effective vaccines. Cell, 184, 1-4.