The effect of investigator compliance (observer bias) on calculated efficacy in a pertussis vaccine trial

BACKGROUND: In the course of a large pertussis vaccine efficacy trial we realized that investigator compliance could have a major impact on calculated vaccine efficacy. DESIGN: In our pertussis vaccine efficacy trial, the study investigators were to monitor illness in study families by telephone every 2 weeks. If a cough illness of </=7 days duration was noted, the study child was to be evaluated. If the cough illness persisted for </=14 days, the child was to be referred to a central investigator. For this report we analyzed study physician evaluation rates and rates of referral to the central investigators. Physician practices were separated into three compliance categories: high, intermediate, and low. We analyzed vaccine efficacy of an acellular pertussis component DTP vaccine (DTaP) and a whole cell pertussis component DTP vaccine (DTP) by compliance category. Bordetella pertussis infection was documented by culture of the organism in the study child or in a household contact or by a significant antibody response to pertussis toxin determined by enzyme-linked immunosorbent assay. RESULTS: Using a clinical case definition that included both mild and typical pertussis (cough illness </=7 days duration) efficacy of DTaP vaccine was 40% (95% confidence interval [CI] = -3-65) in the high compliance category and 78% (95% CI = 65-86) and 75% (95% CI = 53-87) in the intermediate and low compliance groups, respectively. Similar, but less marked, differences in efficacy were noted with DTP vaccine recipients. Using a clinical case definition that required </=21 days of cough with paroxysms, whoop, or vomiting (typical pertussis) the efficacy of DTaP vaccine was 69% (95% CI = 41-83) in the high compliance category and 86% (95% CI = 76-92) and 84% (95% CI = 64-93) in the intermediate and low compliance groups, respectively. In contrast, the efficacy of DTP vaccine did not vary by compliance category using this case definition. The attack rate in children vaccinated with diphtheria and tetanus toxoids vaccine (DT) was twofold less in low compliance physician practices when compared with the rates in high and intermediate groups. The DT/DTaP and DT/DTP fold-change differences were less in the high compliance group compared with the intermediate and low compliance groups. CONCLUSIONS: Our data suggest that observer compliance (observer bias), can significantly inflate calculated vaccine efficacy. It is likely that all recently completed efficacy trials have been effected by this type of observer bias and all vaccines have considerably less efficacy against mild disease than published data suggest.