Carey Gillam, who is the research director for the anti-GMO group USRTK, which is working closely with the glyphosate plaintiffs, seized the moment to hit the panic button in her article in the UK The Guardian, warning, “Weedkiller ‘raises risk of non-Hodgkin lymphoma by 41%’.” Many media outlets and environmental activist groups also uncritically reported this figure as a serious challenge to the Environmental Protection Agency’s (EPA) conclusion that glyphosate is probably not carcinogenic.
The source of 41 percent statistic was a controversial February 2019 meta-analysis published in the journal Mutation Research. The article, a reanalysis of the most recent epidemiological data on glyphosate and cancer, suggested there was “a compelling link” between exposure to glyphosate-based herbicides and non-Hodgkin’s lymphoma (NHL), a cancer thousands of plaintiffs suing Bayer blame on their use of Roundup.
The paper attracted harsh criticism from many experts (See my GLP piece 41% glyphosate-cancer increase claim under fire: Did authors of new meta-study deliberately manipulate data or just botch their analysis?), who challenged the authors’ methodology and the conclusions drawn from the analysis.
This criticism went unanswered until last week when senior author of the paper Lianne Sheppard, a Professor at University of Washington’s Departments of Environmental & Occupational Health Sciences, and Biostatistics, attempted a response in Forbes. Her column, Glyphosate Science is Nuanced. Arguments about it on the Internet? Not so much, published under the heading Perspectives from the cutting edge of science, is a strange document.
Its timing is particularly odd, coming out almost a year to the day after publication of the paper. Given the elapsed time, one might have expected her to acknowledge certain facts and developments that have occurred in the past 12 months. But we got nothing of the sort.
Sheppard began by telling readers how her participation in the EPA’s Scientific Advisory Panel 2016 assessment of glyphosate dragged her into the depths of “politically charged science.” She was “shocked to see that the EPA’s approach to distilling the scientific information obfuscated the evidence.” And she charged the EPA with misinterpreting the animal carcinogenicity studies on glyphosate. Her disagreement with the EPA’s approach is what impelled her and two other Scientific Advisory Panel members to embark on their now controversial meta-analysis.
Since I was the principal object of Sheppard’s displeasure, I will briefly recount my criticism of her paper, then bring in the crucial facts and developments that, inexplicably, Sheppard ignored in her Forbes piece.
Flaws in the meta-analysis
In what follows, I will refer to the meta-analysis, using the name of the first author, as the “Zhang paper.” I read the paper soon after it came out and posted a critique on February 18, 2019 on Forbes.com, where I was a contributor at the time.
Reading the introduction, I noticed how the authors described the “controversy” surrounding glyphosate, suggesting that the two sides in the debate were somehow equivalent. In fact, the two sides are not comparable. Over a dozen international and national health agencies have reviewed the safety of glyphosate, some repeatedly, and have found it unlikely to cause cancer and safe when used according to its label. This misrepresentation of the background set off my baloney detection sensors (the term is Carl Sagan’s).
Next, I noticed the authors seemed inclined to subtly denigrate the results of the only cohort study included in their meta-analysis: the Agricultural Health Study (AHS), a prospective cohort study of 54,000 farmers. They appeared to feel there must be something wrong with the analysis by the National Cancer Institute, because it failed to find any association between exposure to glyphosate and any of more than 20 malignancies, including solid cancers and lymphopoietic cancers such as non-Hodgkin’s lymphoma (NHL)—the subject of the Zhang meta-analysis.
In fact, the AHS is, by far, the highest-quality study we have on the topic. Zhang and colleagues showed no comparably skeptical inclination toward the case-control studies they analyzed, which are far inferior to the AHS. The first problem with the meta-analysis I pointed out was that, to have validity, the studies that were combined must be of comparable quality and use compatible definitions of exposure. Otherwise, one is comparing apples to oranges.
The five other studies included in the meta-analysis were case-control studies. Case-control studies, unlike cohort studies, are subject to recall bias, because cases with a serious disease may think back over their exposures differently from controls, and this may lead to a spurious association. Moreover, unlike the AHS, the five case-control studies were smaller and had a much lower percentage of subjects with exposure to glyphosate.
Furthermore, the definition of exposure differed among the studies—ever exposure in some studies and exposure for three or more days/year and 10 or more days/year in others. Four of the case-control studies reported roughly a doubling of NHL risk with exposure to glyphosate, whereas the fifth study found no association.
Two of the studies were from a research team in Sweden, which reported elevated risks for many associations, and which appear to be outliers. I questioned the value of combining the inferior case-control studies with the vastly superior AHS. A recent paper in the journal Risk Analysis has confirmed the validity of this concern, demonstrating that recall bias and selection bias contributed to spuriously elevated estimates for associations of glyphosate with NHL from case-control studies.
My last criticism of the Zhang paper was this. The AHS reported five different risk estimates for the association of the highest glyphosate exposure level and NHL. Four of these were below 1.0, indicating no association (0.87, 0.87, 0.83, and 0.94). Zhang selected the only relative risk that was above 1.0—the 20-year lagged relative risk of 1.12 (95% CI 0.83-1.51).
Although the authors attempted to justify their use of the 20-year lagged risk estimate with reference to the biology, there is much that we don’t know about the natural history of NHL, and there is considerable evidence suggesting a much shorter induction period. A conservative approach would have been to show what the impact of using differently lagged estimates would have been. Because all of the estimates from the case-control studies included in the Zhang paper were based on unlagged analyses, at the very least, the main meta-analysis should have included the unlagged AHS risk estimate for NHL (0.87).
The point in all this is that the authors, who view themselves as irreproachable, disinterested researchers methodically pursuing their hypothesis, repeatedly made choices that just happened to produce a positive result that bore out their belief that glyphosate is dangerous.
By combining the results of the six studies, the authors obtained a “summary” relative risk estimate of 1.41 (95% confidence interval 1.13-1.75). This result was featured in the abstract of the paper as indicating that people exposed to glyphosate had a 41 percent increased risk of NHL. The authors concluded that that their analysis of human epidemiological studies “suggests a compelling link” between glyphosate and NHL.
I pointed out that if Zhang et al. had used the more appropriate unlagged estimate of 0.87 from the AHS, it was likely that the summary relative risk would have been considerably reduced and no longer statistically significant (a conclusion confirmed by the EPA, as we’ll see below). Of course, in that case, their paper would have attracted little attention.
My Forbes column was carefully worded and, in essence, contained the points made above. On February 19, the day after I posted it, however, it was taken down without any warning or discussion.
Facts ignored by Dr. Sheppard
Dr. Sheppard’s contention that my critique of her study was taken down because I “failed to adhere to Forbes’ editorial standards” is laughable. I had written approximately 80 columns on Forbes over the preceding eight years and had never had any problem. What happened, in fact, is that anti-pesticide, anti-GMO, anti-modern agriculture activist Carey Gillam raised a fuss with the editors, and they spinelessly took down the article and severed my connection to Forbes, without any discussion.
Curiously, in referring to Forbes’ “retraction” of my column, Sheppard parroted the same clueless language Gillam used. This was not a retraction, which, as Sheppard should know, is precipitated by evidence that an article includes serious errors or fabrications. No one has pointed out any error in my article in the year since it was published. This scenario was simply an example of a corporation concerned about its “brand” taking the path of least resistance, rather than standing up for a carefully made argument.
Sheppard also failed to note that my article was republished here and here, but her ploy allowed her to evade acknowledging the substantive criticism in my column.
Now, let’s turn to some of the important developments that Sheppard inexplicably left out of her post. First, Sheppard cited the International Agency for Research on Cancer (IARC), which relied on animal carcinogenicity data to declare glyphosate a “probable carcinogen.” However, reporting by award-winning Reuters investigative journalist Kate Kelland and a careful reassessment by a former statistician at the National Cancer Institute found irregularities and even fraud in IARC’s evaluation of the animal studies.
A rigorous and complete synthesis of the rodent studies relied upon by IARC does not support the conclusion that glyphosate is an animal carcinogen. As with her research team’s background discussion of the glyphosate controversy, Sheppard’s Forbes column showed no awareness of, or interest in, careful work that doesn’t support her position.
Second, just as Sheppard and coauthors publicized a questionable 41 percent increase in the risk of NHL, she went on to argue that this increased risk, even though linked to a very rare cancer, could account for a significant number of additional cases of cancer. She further argued that there is no safe level of exposure and that people may be harmed by much lower levels. This revives the scare stories about glyphosate residues in Cheerios and other foods, and goes against all the careful risk assessments performed by the agencies I referred to above.
In fact, we have valuable information relevant to this point. The application of glyphosate to crops has increased about 15-fold since Roundup-ready crops were introduced in 1996. What has happened to the rates of NHL over the past 25 years? As shown in the figure below, while the use of glyphosate increased markedly, the incidence of NHL has remained flat, and mortality from NHL has decreased.
EPA’s reanalysis of the Zhang Paper
In January 2020, the U.S. EPA issued a memorandum updating its 2017 review of studies of glyphosate. 
EPA repeated the Zhang meta-analysis substituting a relative risk of 0.85 (95% CI 0.73-0.99) for unlagged ever exposure in the AHS (instead of 1.12, 95% CI 0.83-1.51 for the highest quartile of glyphosate exposure based on a 20-year lag used in the Zhang analysis). The resulting summary relative risk was 1.14 (95% CI 0.87-1.50). EPA concluded that the reanalysis confirmed its earlier 2017 judgment of no association between glyphosate exposure and NHL.
In addition to Zhang et al.’s selection of the highest hazard ratio in the AHS, the EPA re-review suggested other ways in which Zhang et al. made decisions based on assumptions that had little support in the actual data. Specifically, Zhang et al. emphasized their “a priori hypothesis” that higher exposures to glyphosate-based herbicides would be associated with greater effect sizes, where “higher” was defined as “greater intensity/magnitude, longer duration, or some cumulative combination of the two.” However, EPA pointed out that in the AHS, “the largest study and of the highest quality,”
…. this hypothesis does not appear to be supported …. there is no readily apparent indication of a trend toward higher odds ratios [sic] with higher exposures in Andreotti et al. (2018) so there appears to be little evidence supporting the authors’ stated a priori hypothesis
Here, again, the Zhang paper authors’ judgment seems to have been blinkered by their determination to find evidence supporting their preferred hypothesis. Sheppard’s Forbes post was a belated attempt to neutralize criticism of her meta-analysis, but, as in the paper itself, she preferred to double down on her hypothesis, rather than taking into account the new, high-quality work on the topic.
Sheppard also added that discussion of scientific issues should be confined to the precincts of peer-reviewed journals and letters to the editors of those journals, dismissing commentary on “the internet” as unworthy of a serious response. However, critical problems have been documented in the peer review process, and publication of the Zhang paper shows that highly questionable research can make its way into scientific journals.
The problem is not, as Sheppard tried to convince us, that we have undisciplined and uninformed people addressing scientific issues on social media. Useful and informed criticism is valuable wherever it comes from. Many superb scientists recognize the value of cutting through the medical publishing logjam and writing critical assessments (“tweetorials”) of important articles online. It’s ironic that Sheppard seems completely unaware that she is using social media to discredit the use of social media to discuss scientific issues.
There is an important lesson here. The problem has always been that Sheppard and her co-authors are intent on favoring data that appears to support the existence of a risk from glyphosate exposure, even when the best evidence fails to support the case. Sheppard rejected the charge that she and her co-authors deliberately cherry-picked the data they used in their meta-analysis. I believe her. What is clear, however, is that they were so wedded to their hypothesis and so certain that there must be some signal in the data that they selected estimates according to their hypothesis, while ignoring data that did not fit.
It’s worth mentioning that Sheppard and I were scheduled to discuss the glyphosate-NHL question on Pasedena’s NPR affiliate KPCC on August 13, 2019. A couple of hours before the broadcast, I got a phone call informing me that she had dropped out.
The only surprise about her column is that Sheppard is so unaware of the biases she displays at every turn, and that she could try to justify her paper, which only last month was reviewed carefully by the EPA and revised in a well-conducted and consistent meta-analysis showing no evidence of an association between glyphosate exposure and NHL.
Geoffrey Kabat is a cancer epidemiologist and the author of Hyping Health Risks: Environmental Hazards in Daily Life and the Science of Epidemiology and Getting Risk Right: Understanding the Science of Elusive Health Risks. He is a GLP board member. Follow him on Twitter @GeoKabat
