Does Glyphosate cause non-Hodgkin lymphoma (NHL)? Observing that different studies in herbicide application workers give very different results, the authors of a recent[1] meta-analysis have proposed a new approach. By choosing those results which correspond with the highest exposure in each study, and ignoring all the rest, it might be possible to detect a causal association, if there is one.
It being self-evident that the meta-risk ratio by itself would have no meaning, unless the high exposures were very similar, the main task is to show that the meta-risk ratio is statistically significant. Significance would suggest that a causal association was possible, even if no-one could tell how strong the association was.
Significance testing in the biological sciences centres on whether or not the 95% confidence interval (95% CI) includes 1.0. This is a convention, not universally agreed with.
In the recent paper the meta-risk ratio was 1.41 (95% CI = 1.13 to 1.75). The 1.41 part was not proven to be a meaningful number (exposures ranged from a minimum of 2 days to a minimum of 105 days) but the 95% CI values are both > 1.0 and so the result can be called statistically significant.
Given that this is a novel approach, some testing of whether it was subject to differential bias would be expected. The biases in people who were exposed to an agent for 100 days before being diagnosed with NHL would probably be different from those who could remember just two days or could only remember that it had happened once or more. If the confidence interval is to be thought meaningful, then differential bias would need to be sought, corrected for, or shown to be non-significant. It was very interesting therefore to note that the weight given to the low bias studies was proportionately low. While mathematically legitimate, the authors of the meta-analysis had found a way to underweight the two high quality studies, essentially by dividing these studies in smaller, less powerful, parts. Had they also accidentally included different amounts of bias? If so, would this be sufficient to account for the reported significant finding?
In fact the authors made no systematic analysis of this. The approach was not justified in that way. The two high quality studies could be tested in this way, the others could not.
The per-person precision of the two high quality studies was very similar and considerably lower than in the low quality studies. This should have been commented upon. It wasn’t.
A more obvious meta-risk calculation would have been to combine the results of “ever/never” exposures. If ever being exposed was truly a risk factor for NHL then a significant association could support that interpretation. Once again the value of the risk ratio would be of doubtful meaning but would at least be the result of comparing like with like. An ever/never analysis was presented, but used values taken from an earlier stage of the most powerful study, the AHS[2]. This was shown as a meta-risk ratio of 1.30 (1.03 to 1.64). This would count as statistically significant, but only just. Out of interest we[3] independently calculated the ever/never value they would have obtained if the full powered version of AHS had been included. This time the meta-risk ratio was 1.15 (0.90 to 1.47) which would be described as non-significant.
Summary
In an unproven approach to meta-analysis it is possible to play down the effect of high quality but negative findings and give weight to low quality positive findings and, at the same time appear to be following a logical, if robotic, method. The published result of doing this, while having no quantitative meaning, could be indicative of non-zero risk, but only if there was also evidence that the novel approach was not simply a result of amplified bias. This was not demonstrated, and yet there was evidence of such problems in the raw data.
A more traditional analysis finds a non-significant association between glyphosate use and NHL incidence.
The published mathematical work does not inform a causation analysis or a liability risk analysis. However, by prompting a more interpretable analysis of the same data, the case for a non-association is supported. The best evidence, the AHS, is clearly opposed to there being a causal association.
[1] https://doi.org/10.1016/j.mrrev.2019.02.001
[2] J Natl Cancer Inst (2018) Vol.110(5): djx233. G Andreotti et al. Glyphosate Use and Cancer Incidence in the Agricultural Health Study.
[3] A discussion of the recently published meta-analysis is available from Re:Liability (Oxford) Ltd. www.reliabilityoxford.co.uk
