- A small sample size: only 109 people out of more than 40,000 developed the disease, making it difficult to establish a correlation between sugar and colon cancer
- Study participants reported their soda consumption using food frequency questionnaires, which many epidemiologists have argued are notoriously unreliable sources of data.
- Many previous studies have failed to find an association between sugar and colon cancer, and some research has even found that it could have a protective effect against the disease.
For these and several other reasons, the new study is somewhat of an outlier and thus worthy of heightened scrutiny, cancer epidemiologist Geoffrey Kabat wrote in a recent article for GLP. But this didn’t stop reporters (with the help of the study authors) from speculating about the significance of the paper:
Human beings are so constructed that when we read such a news story, we immediately take the highlighted finding as if it were true and worry that something we had considered relatively harmless—drinking sweetened beverages— could be adversely affecting our health or the health of people close to us. People who have an interest in publicizing this type of result (scientist authors, journal editors, and journalists) know that stories like this provoke a visceral reaction. The logic behind this kind of reporting could be distilled down to the injunction, “Start worrying—details to follow.”
The problem is much greater than a single eye-grabbing paper, Kabat argued. There is an entire sub-discipline within epidemiology, known as risk-factor epidemiology, that tries to correlate particular chemical exposures with complex diseases like cancer. While not inherently flawed, research in this field is prone to exaggeration and risks misleading the public about the risks they face from food, chemicals and other environmental exposures. As Kabat put it:
Over the years, a number of prominent epidemiologists have criticized risk factor epidemiology and “approaches that abstract single elements … from the complexity of the life and times of people and relate these to a single health outcome.” (See also here and here). But such critiques have done little to dampen the appeal of focusing on small, questionable effects – a focus that one epidemiologist wrote “marginalizes us as a field.”
On this episode of Science Facts and Fallacies, Kabat joins geneticist Kevin Folta and GLP contributor Cameron English to examine these troublesome trends in epidemiology and offer some tools you can use to better analyze the quality of the science that generates widespread media coverage.
Geoffrey Kabat is cancer epidemiologist and the author of Hyping Health Risks: Environmental Hazards in Daily Life and the Science of Epidemiology and Getting Risk Right: Understanding the Science of Elusive Health Risks. He can be found on Twitter @GeoKabat
Kevin M. Folta is a professor in the Horticultural Sciences Department at the University of Florida. Follow Professor Folta on Twitter @kevinfolta