:::: MENU ::::

Healthy User Bias: The Fatal Flaw in Vaccine Safety Research

Share

Click for list of papers in this post.

Vaccine safety studies are typically compare health outcomes in vaccinated and unvaccinated people. In order to obtain accurate results, the two groups must be “matched”, meaning they have similar health and lifestyle characteristics. Matching groups is straightforward if the researchers have control over who gets the vaccine and who doesn’t. If researchers do not have this control (known as an “observational” study), it is impossible to assure the groups are matched. The resulting group differences can cause biases that severely distort the study outcome. Poor matching can cause the study to be totally wrong.

Most vaccine safety studies are observational, and accordingly, do not include researcher control of vaccine exposure.  For example, studies are often performed with “administrative data”, which is health data collected by insurance companies or governments. Researchers can use administrative data to compare health outcomes in vaccinated and unvaccinated people. A big problem is that vaccinated and unvaccinated people are not matched. Critical differences include:
1) Healthy people are more likely to choose to be vaccinated. People with chronic diseases or health issues tend to avoid the risk of vaccination.
2) People that choose vaccination tend to have other “health seeking” behaviors, such as having a better diet and exercising, or getting regular screenings and medical tests.

These differences create “healthy user bias” (HUB) or the “healthy user effect” in vaccine studies. Flu vaccine studies appear to be strongly affected by healthy user bias. People that receive the flu vaccine have dramatically lower (50% lower) mortality and better health when its NOT flu season (i.e. in the summer). This is not plausibly due to the vaccine; rather it is because people that choose to receive the flu vaccine have better baseline health and more “health seeking” behavior.  Dr Peter Doshi of Johns Hopkins University describes the healthy user bias problem in the British Medical Journal:

Since at least 2005, non-CDC researchers have pointed out the seeming impossibility that influenza vaccines could be preventing 50% of all deaths from all causes when influenza is estimated to only cause around 5% of all wintertime deaths.14 15 So how could these studies—both published in high impact, peer reviewed journals and carried out by academic and government researchers with non-commercial funding—get it wrong? Consider one study the CDC does not cite, which found influenza vaccination associated with a 51% reduced odds of death in patients hospitalized with pneumonia (28 of 352 [8%] vaccinated subjects died versus 53 deaths among 352 [15%] unvaccinated control subjects).16 Although the results are similar to those of the studies CDC does cite, an unusual aspect of this study was that it focused on patients outside of the influenza season—when it is hard to imagine the vaccine could bring any benefit. And the authors, academics from Alberta, Canada, knew this: the purpose of the study was to demonstrate that the fantastic benefit they expected to and did find—and that others have found, such as the two studies that CDC cites—is simply implausible, and likely the product of the “healthy-user effect” (in this case, a propensity for healthier people to be more likely to get vaccinated than less healthy people). Others have gone on to demonstrate this bias to be present in other influenza vaccine studies.17 18 Healthy user bias threatens to render the observational studies, on which officials’ scientific case rests, not credible.” -Dr Doshi of Johns Hopkins U., 2013

Full paper with above quote: Influenza:Marketing Vaccine by Marketing Disease
Confounding in flu vaccine studies: Mortality Reduction with Influenza Vaccine in Patients with Pneumonia Outside Flu Season

Healthy user bias is a specific type of “selection bias.” Selection bias is well known. For example, a commonly used textbook on epidemiology and statistics states the following:

Selection bias results when subjects are allowed to select the study group they want to be in. If subjects are allowed to choose their own study group, those who are more educated, more adventuresome, or more health-conscious may want to try a new therapy or preventive measure. Differences subsequently found may be partly or entirely due to differences between the subjects rather than to the effect of the intervention. Almost any nonrandom method of allocation of subjects to study groups may produce selection bias.” (Emphasis in original)
Epidemiology, Biostatistics and Preventative Medicine, Jekel et al, 3rd ed., 2007, page 70

CDC Researchers Study Healthy User Bias
In 1992, CDC researchers Dr. Paul Fine and Dr. Robert Chen published an important paper describing evidence for HUB in studies of the DPT vaccine and sudden infant death syndrome (SIDS). They derived a mathematical model for calculating the strength of HUB. Their paper states:

…individuals predisposed to either SIDS or encephalopathy are relatively unlikely to receive DPT vaccination. Studies that do not control adequately for this form of “confounding by indication” will tend to underestimate any real risks associated with vaccination.

AND

Confounding…is a general problem for studies of adverse reactions to prophylactic interventions, as they may be withheld from some individuals precisely because they are already at high risk of the adverse event.

AND

If such studies are to prove useful, they must include strenuous efforts to control for such factors in their design, analysis and interpretation. Whether this is possible at all may be open to discussion. The difficulty of doing so is indisputable.” (emphasis added)

(“Factors” = Factors that cause healthy user bias)

Full paper (Fine and Chen): Confounding in studies of adverse reactions to vaccines
This paper has a correction: http://aje.oxfordjournals.org/content/136/8/1039

CDC researchers Fine and Chen are obviously deeply concerned about HUB, and the implications for the reliability of vaccine safety studies. They state that studies are not “useful” unless HUB is controlled. In view of these concerns, it’s alarming that vaccine safety studies typically ignore healthy user bias altogether. They almost never make any effort whatsoever to control for it.

A 2011 review paper (Shrank et al.) provides recommendations for interpreting studies (of preventive treatments like vaccines) vulnerable to HUB. This paper states:

“When interpreting epidemiologic studies of prevention in the scientific literature, we recommend a healthy skepticism when encountering what seem like surprisingly large beneficial effects of preventive therapies.”
AND
Clinicians should be skeptical when interpreting results of observational studies of preventive services that have not accounted for healthy user and related biases.

Full Paper (Shrank): Healthy User and Related Biases in Observational Studies of Preventive Interventions: A Primer for Physicians

Down The Memory Hole
Vaccine promoters acknowledge HUB. For example, the most widely used textbook on vaccines by Plotkin, Orenstein and Offit (yes, that Offit) states the following:

Confounding by contraindication is especially problematic for non-experimental designs. Specifically, individuals who do not receive vaccine (e.g., because of a chronic or transient medical contraindication or low socioeconomic group) may have a different risk for an adverse event than vaccinated individuals (e.g., background rates of seizures or sudden infant death syndrome may be higher in the unvaccinated). Therefore, direct comparisons of vaccinated and unvaccinated children is often inherently confounded and teasing this issue out requires understanding of the complex interactions of multiple, poorly quantified factors.” (Emphasis added)
Vaccines, 5th ed, 2008, page 1631

Though acknowledged, HUB is quickly forgotten by vaccine promoters when studying adverse effects of vaccines. No effort is made to “tease this issue out.” Vaccine promoters prefer the results they get with HUB, so they don’t question the results too much.

The phrase “non-experimental design” refers to studies in which the researcher cannot control the variable being tested (i.e. vaccine exposure). Studies using databases of patient information (e.g. collected by HMOs or insurance companies), are always non-experimental. All of the MMR-autism studies are non-experimental and therefore are susceptible to HUB. 

This article explains exactly how HUB can make very dangerous vaccines appear completely safe. The example below uses a different set of variables than the Fine/Chen paper, because the choice of variables by Fine and Chen leads to some confusing math. However, the numerical assumptions used here are within ranges set forth by Fine and Chen as reasonable.

The Risk Ratio (RR)
The risk ratio (RR) is fundamental to medical research. It’s a simple calculation that is intuitive for many people. Its a ratio of percentages. If the RR is close to 1, then the vaccine and control groups are equally likely to have adverse outcomes and the vaccine is determined to be safe. RRs of about 2 are commonly observed in medical studies. The RR of lung cancer from smoking is about 22, for example. The risk ratio is the same thing as the “relative risk”, an alternative term.

The RR calculation is illustrated using an “A-B-C-D table,” shown below.


Above: The risk ratio is used in many medical studies of effectiveness and adverse effects. The experimental (vaccinated) group size is A+B.  The control (unvaccinated) group size is C+D. 

Study Shows Vaccine is Safe!
Below is an ABCD table showing an outcome of a hypothetical, non-experimental vaccine study with 9,900 vaccinated subjects and 1,100 controls. Like almost all vaccine safety studies, this hypothetical study uses data already collected for other purposes. The data does not indicate the reasons why some people were not vaccinated. Since this study is not randomized and not placebo-controlled (i.e. it is non-experimental), it is susceptible to HUB.

The RR is 0.68, which for many studies, is close enough to 1 to conclude that there is no risk from the vaccine. A RR less than 1 suggests that the vaccine has a protective effect. In other words, the RR of 0.68 in this example suggests that the vaccine could be preventing the adverse outcome. Amazing! Could the vaccine be providing these remarkable benefits? In reality, nothing could be further from the truth.

Ooops! There is a Problem
There is HUB in this hypothetical study. Healthy user bias conceals a true RR of 3. In this example, the vaccine is actually causing a 3X increase in adverse outcomes. How is this possible?

HUB occurs when there is a “high risk” subgroup with two characteristics:

1) Lower vaccination rate, and
2) Higher risk of adverse outcome,

compared to other study subjects.

This can happen if, in some subjects:  1) poor health is noticed and 2) vaccines are avoided in those with poor health. People with poor health have a greater risk of adverse outcomes, and they tend to avoid vaccines. Many doctors do not vaccinate patients experiencing poor health or chronic health problems.

With such a subgroup, the observed RR will be lower than its true value. It is mathematically certain that the observed RR will be lower than its true value, and it might be a great deal lower than its true value. The RR can be reduced by a factor of 4 or 5, or possibly more in exceptional circumstances. Consequently, a true RR of 2-4 can be falsely observed to be 1, or less than 1. The size of the error depends on 6 factors (more on this below), including the size of the subgroup, how strongly they avoid vaccination, and their risk of adverse outcomes compared to the other study subjects.

The two characteristics often go together because parents and doctors tend to avoid vaccination in children showing signs of poor health (e.g. neurological or immune disorders). Such children have both a high risk of adverse outcome and a low vaccination rate. For example, in the Jain et al. 2015 study children already diagnosed with autism were about 60% less likely to receive the MMR vaccine. Link: http://vaccinepapers.org/review-of-jain-et-al-jama-2015-and-comments-on-mmr-autism/

Note that the adverse outcome in our hypothetical study may be any of the hypothesized adverse vaccine effects under study, which will overlap with conditions that will make parents and doctors hesitant to vaccinate. It could be autism, seizures, allergies, ADHD, autoimmune disorders, asthma or other disorders that some associate with vaccines.

Below is a diagram illustrating how the math is affected by the high-risk subgroup. This calculation is based on the following reasonable assumptions (per Fine and Chen):

1) Size of high-risk subgroup. We assumed 1,000 out of 11,000 or about 9.1%. The high-risk subgroup contains both vaccinated and control subjects.
2) Vaccination rate of high-risk subgroup. We assume 40%. This means that doctors and parents can identify high-risk children about 60% of the time. This is comparable to the vaccinate rate of autistic children in the Jain study.
3) Vaccination rate of healthy subgroup. We assume 95%, which is typical for many vaccines.
4) The risk of adverse outcomes in the high-risk subgroup. We assume 6% and 2%, a risk ratio of 3X, for illustration. This is consistent with the Fine and Chen paper.
5) The risk of adverse outcomes in the healthy subgroup. We assume 0.6% and 0.2%, a risk ratio of 3X, for illustration. This is consistent with the Fine and Chen paper.

All these numbers are within ranges described by the CDC researchers Fine and Chen as reasonable and within ranges considered by their modeling. Note that the absolute numbers of adverse outcomes were selected to be high, so that this example calculation can be performed with easy-to-follow whole numbers.

Above: Diagram illustrating how the above assumptions affect the calculations for each sub-group. Risk ratio must be calculated separately for each subgroup. Note that the high-risk subgroup has a low vax rate (40%), compared to healthy subgroup (95% vax rate). This difference in vaccination rate is essential for healthy user bias (HUB). 

The True Risk of the Vaccine
To find the true RR of the vaccine, we must calculate the RR of each subgroup separately. The ABCD table is determined for each subgroup below, and the risk ratios are shown.

HUB-RR-calc
Above: Adverse outcomes in the “high risk-control” subgroup drives healthy user bias (HUB). Its the 12 adverse reactions shown in RED that are responsible for the effect. 

The thing to notice here is that each subgroup has a RR of 3, but when they are combined in a single RR calculation, the RR declines to 0.68. This is a counter intuitive result, but it is unavoidable whenever there is a high-risk subgroup with the two characteristics: 1) Lower vaccination rate, and 2) Higher risk of adverse outcome. These two characteristics will guarantee healthy user bias that misleadingly reduces the RR.

The vaccine damage is not observed in the aggregate calculation because of the 600 high risk control subjects that are not vaccinated. They comprise 54.5% of the unvaccinated group, even though they comprise only 9.1% of the total study. The high-risk subjects are concentrated in the unvaccinated group. Consequently, the number of adverse outcomes in the unvaccinated group is artificially increased, thereby concealing the damage done by the vaccine.

In other words, the misleading result RR=0.68 is created by the 12 adverse outcomes in the high-risk unvaccinated group, shown in red. Its those 12 adverse outcomes that drives healthy user bias. Anything that increases that number (relative to the others), increases the bias, and pushes down the observed RR.

This specific example uses reasonable assumptions based on the work of CDC researchers Fine and Chen. The RR is reduced by a factor of 3/0.68=4.4.  This effect is large enough to make a very dangerous vaccine appear completely safe or even highly beneficial.

Biased Study of Bias
Remarkably, the paper by Fine and Chen has been cited only 13 times since 1992. This is not because the paper is unimportant. It is hugely important. Rather, it is because of a lack of interest in studying things that undermine what the public is told about vaccine safety. Fine and Chen comment on this one-sided research agenda:

We have focused in this paper on just one of many methodological problems confronting studies of adverse reactions to vaccinations. Most published discussions of the subject have concentrated upon biases that act to overestimate the relative risk of adverse events after vaccination. Biases that underestimate the risk, as discussed here, have received less attention.” (Emphasis added)

In other words, there is bias in the study of bias. The vaccine research community prefers to study issues that make vaccines look good.

Factors That Affect Healthy User Bias (HUB)
HUB is increased by:

1) Lower vaccination rate in the high risk group. In this example it was 40%. Effect will be stronger for 30% or 20%. The effect disappears if the high risk group has the same vaccination rate (95%) as the healthy subgroup. When doctors are good at identifying children at high risk, and don’t vaccinate them, HUB is increased.
2) Higher risk of adverse outcome in the high risk, unvaccinated group.
3) Larger size of the high risk subgroup.

Healthy user bias is very difficult to identify or measure in vaccine studies. Calculating the strength of the bias requires knowledge of six factors:
1) Risk in unvaccinated healthy subjects.
2) True RR of the vaccine (not the observed RR of the vaccine).
3) RR associated with the high-risk characteristic.
4) Size of the high risk subgroup.
5) Vaccination rate of the healthy subgroup.
6) Vaccination rate of the high-risk subgroup.

HUB cannot be eliminated unless all 6 factors are known. But an accurate estimation of all 6 factors is essentially impossible. Fine and Chen state the following:

The difficulty of obtaining such information on all six factors makes it extremely hard to assess whether an observed relative risk of, for example, 0.2 is consistent with a true relative risk greater than 1. This inference is made ever more problematic by the fact that many other sorts of bias, for example, relating to case ascertainment, may influence the observed relative risk.

In other words, HUB is a powerful and unavoidable systematic bias in non-experimental vaccine studies containing a high-risk subgroup. And high-risk subgroups are almost always present.

The only way to eliminate HUB in “cohort” (i.e. group comparison) studies is to use randomization to assure that the groups are similar. However, randomization generally requires experimental control over who gets the vaccine and who doesn’t. This is not allowed in vaccine studies because vaccines are believed to be beneficial, and therefore it would be unethical to withhold vaccines from the control group. I disagree with this, because vaccines have not been shown to be safe, especially when combined in large numbers as the CDC recommends. Rather, it is unethical to NOT study vaccine safety using proper randomized controls. But randomized trials of childhood vaccines are simply not allowed in today’s political environment.

Healthy User Bias in MMR-Autism Studies?
Yes. An example of HUB at work is the Jain et al 2015 MMR-autism study. In the Jain study children with pre-existing autism diagnosis are about 60% less likely to receive the MMR vaccine. Sick and already-vaccine-injured children don’t receive MMR, and therefore are concentrated in the control group. The Jain study is reviewed here: http://vaccinepapers.org/review-of-jain-et-al-jama-2015-and-comments-on-mmr-autism/

There is also evidence in the form of unusually low RRs in MMR-autism studies. Jain et al report RRs for almost all age groups less than 1. Fine and Chen assert that consistent and implausibly low RRs can be a signature of HUB. Jain in fact agrees that HUB is likely the reason for the low RRs, but then inexplicably and irrationally Jain makes no effort whatsoever to control for it. Jain et al acknowledge that this bias is happening and then they ignore it. Jain et al does not even consider the possibility that HUB could be so strong as to conceal a positive MMR-autism association. What would the RR be without HUB? Jain et al makes no effort to find out.

Jain-low-RRs
Above: The low RRs in Jain et al are almost certainly the result of healthy user bias (HUB). Parents that notice  neurological damage in their second child avoid the MMR. Families that already have one autistic child are the most cautious. This is why the RRs are lowest when there is an older sibling with autism (circled results).  From Jain et al 2015. 

The results below are from a widely-cited meta-analysis of MMR-autism studies by Taylor. The odds ratio (OR) is 0.84, which is surprisingly low and almost statistically significant (with p=0.07; p=0.05 or less would indicate statistical significance). Compare this to the pooled odds ratios for Hg and thimerosal exposure, which are both 1.00. Its impossible to know for sure, but it seems likely the surprisingly low odds ratio for MMR (0.84) indicates these MMR-autism studies suffer from HUB, and conceal the true risk of the MMR vaccine. Healthy user bias has reduced the OR almost significantly below 1.0.

By comparison, we would not expect to see HUB in the Hg/thimerosal studies, because these exposures occur from the earliest vaccines, when neurodevelopmental problems have not yet appeared. It takes time for damage to manifest, and for parents and doctors to notice the damage caused by vaccines. MMR is given at an age when this damage is starting to become observable. So the MMR studies are affected by HUB, and the Hg/thimerosal studies are not. Remember, HUB occurs when: 1) poor health is noticed and 2) vaccines are not given to those with poor health.

Note that the odds ratio (OR) is different from the RR. Risk is A/(A+B), where A+B is the total sample size. Odds is given by A/B. Risk is intuitive, but odds can be confusing. If a disease prevalence is less than about 5% (as is the case with autism), then the OR is a good approximation of the RR. For uncommon diseases (with <1% prevalence) the OR and RR are almost exactly the same.


Above: Results of a widely-cited meta-analysis of MMR and autism by Taylor et al. Taylor pooled the results of the best MMR-autism studies and calculated an aggregate odds ratio (OR). All of the MMR-autism studies are non-experimental, and consequently susceptible to healthy user bias (HUB). None of the included MMR-autism studies made any effort to control or estimate HUB. The OR (0.84) was almost significantly below 1 (p=0.07), which suggests that HUB is present in the MMR-autism studies. Notice that ORs for Hg/thimerosal studies are 1.00. Hg/thimerosal exposure occurs from the earliest vaccines (before neurodevelopmental problems are apparent), and this implies that HUB will be smaller in studies of Hg/thimerosal. From Taylor et al, 2014. 

Full paper (Taylor et al): Vaccines are not associated with autism: An evidence-based meta-analysis of case-control and cohort studies

In my opinion, HUB in the MMR-autism studies is caused by neurological damage from aluminum-containing vaccines given prior to MMR. Most vaccines given at 0, 2, 4, and 6 months contain aluminum adjuvant that causes neurological damage and brain inflammation. The infants injured by the 0, 2, 4, and 6-month vaccines are less likely to receive the MMR than unaffected infants (parents noticing adverse effects skip the MMR). Hence, infants injured by Al-containing vaccines are concentrated in the MMR-unvaccinated control group of the MMR-autism studies. This conceals the adverse effects of the MMR vaccine.  The vaccine-injured subjects have the two characteristics required for producing HUB: 1) Lower MMR-vaccination rate, and 2) Higher risk of adverse outcome.

HUB-in-MMR-autism-Studies

Above: Parents noticing damage from 0, 2, 4, and 6 month vaccines refuse or delay the MMR. The injured babies therefore are concentrated in the “control” group of the MMR-autism studies. Consequently, the control and experimental groups in these studies are not matched, and the results are wrong. This is healthy user bias (HUB) and it conceals the damage caused by the MMR vaccine.

Final Word
HUB explains why vaccine safety studies often fail to observe harmful effects. Healthy user bias almost certainly affects MMR-autism studies, because 1) MMR is given after other vaccines have caused harm, and 2) MMR is given at an age when neurodevelopmental problems are increasingly noticeable.

The CDC researchers Fine and Chen concluded that vaccine safety studies are not useful if they do not control for HUB, such as by using randomization. Very few are designed this way, so very few should be believed. There are no studies of vaccines and autism that make efforts to avoid healthy user bias.


Papers in this post:

Share