Vaccine testing and the trolley problem

Posted on by

As you know, I am not ready to proclaim a vaccine a success based on what I see as a small number of cases in the sample population. A lot of you think I am wrong about that, and I hope I am.

But my intuition is still that there is something unsatisfying about the testing protocol.

The actual protocol seems to be to give the vaccine to a large sample and wait for people to get exposed to the virus as they go about their normal business. My inclination is to deliberately expose a smaller sample to the virus and see how well the vaccine works.

One argument against deliberate exposure is that the response of people to deliberate exposure may not be the same as their response to normal-business exposure.

But in favor of deliberate exposure:

–you can clearly test how well the vaccine does in a specific populations, such as people with and without obesity.
–you can clearly test how well the vaccine does at two levels of viral load.
–you can get results quickly, rather than wait for months for people in the sample population to become exposed the the virus
–you could identify the main contacts of people to whom the virus is exposed and make sure that the vaccine reduces spreading.

Whether you deliberately expose 100 people or wait for 100 cases to emerge, that is still 100 cases either way. It seems to me that deliberate exposure is equivalent to throwing the switch in the trolley problem. Those 100 cases are 100 cases either way, it’s just that the experimenter didn’t specifically choose them.

I think that the case for deliberate exposure as a testing protocol ends up being pretty strong. What am I missing?

UPDATE: a reader points me to a story about a proposed challenge trial.

Challenge trials are controversial because of the risks involved with infecting patients with a potentially lethal virus

But again I ask, what is the difference between infecting a group of people and waiting for a group of people to become infected?

28 thoughts on “Vaccine testing and the trolley problem”

  1. I don’t think you are missing anything at all about voluntary challenge trials being a good idea, Arnold. Especially since so many people seem eager to expose themselves to the virus without the benefit of any vaccine.

    That’s an entirely separate question from how good we should feel about the initial vaccine results already in which is what many commenters are disagreeing with you about.

  2. One argument against deliberate exposure is that the response of people to deliberate exposure may not be the same as their response to normal-business exposure.
    One could call this an important argument. Suppose the experimenters used direct exposure by spraying the virus directly into the nose of the subject and the vaccine failed. Would the developers of the vaccine argue “No one gets exposed like that.” If the vaccine succeeds does the other side make the same argument?

    A more acceptable direct exposure experiment would include several (many?) methods of exposure. Depending on how they conduct the experiment, (for example, staged with some methods followed by other methods) the experiment could have high costs and/or lengthy periods and still leave open the objection that the set of exposure methods didn’t include enough methods.

    In the end, a direct exposure experiment, even if it took as long and had the same costs as the current method, with enough variation in exposure methods, viral doses, and population characteristics would yield more confidence in the experiment results.

  3. The difference is that in a passive exposure trial, if any volunteer gets sick and dies, nobody will have to feel guilty about it because that was just in the course of things. Apparently, many WEIRD people have gotten awfully thin-skinned about such guilt feelings. Also, passive exposure trials don’t remind people of (or rather don’t give some people to outflank other people in holiness by reminding them of) Dr. Mengele and Unit 731.

    • Apparently, many WEIRD people have gotten awfully thin-skinned about such guilt feelings.

      You got that right.

      • The thing is, WEIRD people’s refusing to conduct challenge trials leads (very plausibly, has already led) to orders of magnitude more deaths than any reasonable challenge trial could ever have resulted in, but since these deaths are “just in the course of things” too, they are easy to ignore and not feel guilty about.

  4. Arnold, I wonder if I should be happy that you are not serving on the FDA vaccine advisory committee. What additional evidence (to what you have now) would you require to decide on the vaccine at the December 8-10 meetings? What do you think other members know that you don’t know, and vice-versa?

    • I’ve seen no indication that Arnold thinks the EUA should be delayed or rejected.

      If anything following his advise would have sped all this up several months.

      • Months ago many things could have been different. I’m asking what to do right now. I’m not suggesting that Arnold wants to delay the vaccine until there is relevant and reliable evidence on its effectiveness, or that he wants to apply his own criterion on what evidence is enough to prove effectiveness. But he has speculated about its effectiveness and I don’t understand whether his speculations make any difference to the committee’s imminent decision.

        My point is about what Arnold’d like to know right now to decide and to what extent it may differ from what the committee’s members know. I’m not questioning preferences (indeed, I assume that Arnold and the members wanted a good vaccine on March 15 or earlier). I’m trying to understand Arnold and the members’ beliefs and the possibility of different opinions on the evidence.

    • Arnold, I wonder if I should be happy that you are not serving on the FDA vaccine advisory committee. What additional evidence (to what you have now) would you require to decide on the vaccine at the December 8-10 meetings? What do you think other members know that you don’t know, and vice-versa?

      I would take the libertarian point of view, which means never rejecting a drug on efficacy grounds. At most, a libertarian would reject a drug on safety grounds. There is plenty of data to demonstrate the safety of the vaccines. We can let people judge/test the efficacy for themselves.

      A tougher question is whether to go out to the public and say that everyone should (must?) take the vaccine as soon as supplies are available. If I am really confident that the vaccine works well enough for that policy to eradicate the virus, then yes. To make that call, I personally would want to see more evidence. Maybe by December 10 we will have it–the virus has been spreading so fast that the placebo sample group may have had more than 100 additional cases by then. But I doubt that the FDA will be making the “everyone should take it” call on that date in any case.

  5. Challenge trials have been discussed extensively elsewhere. Although their usefulness from a scientific/speed standpoint are very attractive, the driver in their use is mainly political, moral, and value-based. Many scientists have been extremely opposed to them on those grounds.

    It’s similar to the discussion around the decision to send people to war, even if they are willing. Many are aware of the risks, but sign up anyway on moral and values grounds. Many signed up for challenge trials here, but the appropriate debate on the political, moral, and values grounds was never done. It was just rejected out-of-hand.

  6. Arnold, I have been following your recent posts about vaccine testing with interest. I am also skeptical about what the much publicized tests really amount to. But, an important detail about vaccines is that you are potentially harming an otherwise perfectly sane individual. Therefore, it is even more important to prove that the vaccine is safe than in the case of drugs that cure a disease that the participants of the test already have. If the vaccine is proven safe, its effectiveness will be tested when it is administered to the general population.

    Thus, I am not sure that reducing the size of the testing sample or accelerating the process provides any benefit. The worst scenario is not that a particular vaccine is not very effective but that it has adverse side effects. This is specially important in the case of Covid because the risk of the disease is not too high for a large part of the population.

    On this crisis a lot of bad decisions have already been taken. Mass vaccination of a harmful vaccine would be a disaster.

    • Isn’t safety the easiest thing to prove. A lot of people have gotten these vaccines. No serious adverse effects so far. Even if it proves to be a placebo on effectiveness, we are just looking for an excuse to declare victory on this fake pandemic anyway. All the vaccine really needs to do is not kill us.

      • To be clear, I don’t think this is a fake pandemic per se, but a much exaggerated one. I do agree with you that a placebo vaccine could be a great excuse to move on.
        That is why I think that the key is to prove safety not effectiveness.
        But proving safety requires many people tested and enough time to develop side effects.

  7. At this stage, with some efficacy proved. I don’t think there is much objection to doing challenge trials. You can certainly test if participants have an immune response, and you can also screen out the most risky volunteers.

    The main issue I would see is that to fully control the experiment, you should give some participants the placebo and also infect them. I think most people would baulk at that, so you will need some way to ensure each batch of virus you are dosing the participants with is viable. I.e. you would need to infect cell culture, or minks or something.

  8. It’s not throwing the switch in the trolley problem. It’s the exact opposite— that is, it is comparable to harvesting the organs of one healthy person to save five. We can’t treat people as a means to an end. That said— If people are willing to volunteer because they want to contribute and understand the risks, I am for it.

  9. Challenge trials have been conducted for other vaccines. RSV, for instance. Certainly it’s a very useful tool in the development arsenal. But the OP misunderstands the nature of the calculus.

    “But again I ask, what is the difference between infecting a group of people and waiting for a group of people to become infected?”

    Challenge studies *create* new cases that would not have existed absent the trial. In a field study the cases you are “waiting for” in a field study would have happened anyway. A challenge study increases the total number of cases, a field study does not. If you do a challenge study, it’s reasonable to think you have increased the total number of cases by: trial population * (1 – attack rate in that population).

  10. I agree that challenge trials are worth carefully considering, but I don’t think your framework is the correct way of thinking about what the trade-off is. All else equal, challenge trials create cases of the disease that may never have otherwise occurred. All else equal, passive trials do not, because those who become infected in the placebo arm would have been infected regardless of whether they were in the trial. So there is a real moral distinction between the two.

    The real trade-off that gets missed is the time to vaccine availability, which then can reduce prevalence for everyone. If a challenge trials lets you get the targeted level of infection to test effectiveness earlier, you can start distributing the vaccine earlier. Thus a challenge trial that results in 1000 deliberate infections and leads to effectiveness data six months earlier might prevent 1,000,000 infections in the background population.

  11. The cost is that challenge studies create new cases: Not everyone exposed experimentally would have been exposed in the absence of the experiment. The question is whether the data obtained is worth that cost. In the case of this pandemic, it’s not clear what the practical benefit is. It won’t speed up manufacture of the vaccine, because they started that on a risk basis. It won’t speed up approval, because that was waiting on the safety results anyway. In some hypothetical alternate disease scenario, it might speed things up because the disease isn’t spreading fast enough to create the cases you need, but sadly we’re not in that world. So the economics question is what practical value would this experimental data have, that would be worth inflicting enough cases to acquire it. (I additionally worry about the externality of telling medical researchers to not worry about ethics, but the cost/benefit test needs to be dealth with first.)

    • NO! It isn’t “telling medical researchers to not worry about ethics”. It’s telling ethics people to get real and realize life has trade-offs. To take cognizance of both the seen and the unseen.

      • It seems clearly in conflict with the traditional “do no harm” — deliberately making some patients sick. Breaking with that might be worth it, but that’s the ethical issue for a lot of people in medicine. (And history implies it is surprisingly dangerous to do so.) But from a seen and unseen perspective, what’s actually the benefit from doing such a study? We have a whole bunch of observational studies, and also animal models, so most of the things we might learn from a human challenge study are mostly known. There might be tighter error bars, but I’d like to see an example of something we might expect to learn from such a study that would actually be worth any money.

        • There is a difference between “do no harm” and “do no direct harm”. That’s what one of the trolley problems is about. The trolley will run over and kill 5 people unless you switch the track, in which case it will kill one person. Even though you will indirectly kill 5 people if you don’t switch, lots of people won’t do it. They will feel guilty if their action causes one death but not if their inaction causes 5 deaths.

          • The important distinction is that you are acting on the trolley vs acting on a person. That is, you wouldn’t harvest the organs of the one to save the five but you would switch the trolley.

          • What’s the difference between acting on a person with a trolley vs acting on a person with a scalpel?

          • In the first you can say, “Not my problem.” And if someone says, “they’re just as dead”, you can stick your fingers in your ears and say, “NA NA NA NA NA NA I can’t hear you.”

            Okay, that wasn’t very charitable.

Comments are closed.