During the outbreak of the SARS-CoV-2 pandemic, epidemiology ha assumed an outsized role in the public consciousness. It was pervasive in the news, as scientists struggled to elucidate both how the virus was being transmitted as well as its likely short- and long-term health effects.
Its high profile during the crisis harks back to what epidemiology was in its “heroic” period over the prior century-and-a-half, when disease scourges were a part of everyday life. The rise of epidemiology began in the 1850s led by John Snow, widely considered the ‘father of epidemiology,’ who demonstrated that contaminated water and not “miasma” was the cause of cholera.
Triumph after triumph followed: identifying cigarette smoking as the cause of lung cancer and many other diseases; the eradication of smallpox; pinpointing the handful of risk factors contributing to heart disease; and unraveling the role of human papillomavirus as the cause of cervical cancer. These breakthroughs were made possible by use of the epidemiologic method to address public health crises of global dimensions.
Slipping influence and the politicization of science?
In recent decades, epidemiology has become more specialized, with researchers focusing on the social determinants of health, “life-course epidemiology,” and the contribution of specific genetic variants to the risk of particular diseases, as well as the study of “gene-environment interactions.”
The facet of epidemiology, however, that gets the most attention nowadays is what has been referred to as “risk factor epidemiology.” Typically, studies of this kind isolate a specific factor – a component of diet or exposure to a chemical (often at trace levels)—and examine its association with a specific disease. When the results of a study are reported in the media, we get the message that “factor X is associated with increased risk of disease Y.” Unlike the breakthrough findings that established the critical importance of epidemiology, this type of study speaks directly to the public via the media, as if the results have direct relevance to our health. As a consequence, it has become a staple of our daily news feed.
For example, a 2021 study, by researchers at Washington University and Harvard, found that compared to individuals who drank sugar-sweetened beverages less than once a week those who consumed these beverages two or more times per day experienced a doubling of the risk of early-onset colorectal cancer. This result has potential interest because colorectal cancer, which is mainly a disease of older age, has been occurring more frequently in younger people, i.e., below age 40. But for many reasons, the result of the study is far from telling us that scientists have uncovered a cause—let alone an important cause—of this new phenomenon.
Human beings are so constructed that when we read such a news story, we immediately take the highlighted finding as if it were true and worry that something we had considered relatively harmless—drinking sweetened beverages— could be adversely affecting our health or the health of people close to us. People who have an interest in publicizing this type of result (scientist authors, journal editors, and journalists) know that stories like this provoke a visceral reaction. The logic behind this kind of reporting could be distilled down to the injunction, “Start worrying—details to follow.”
Here is what one of many news reports made of the story.
Seems scary, based on the headline and the highlighted result. In fact, however, epidemiologists are well-aware that results like the one pertaining to early-onset colorectal cancer are merely hints that need to be confirmed or rejected with larger and more in-depth studies, probing this and many other factors in the context of what is known about the causes of colorectal cancer, which is the third most common cancer in men and women. But the paper and the headlines it evoked don’t convey that nuance.
Few people are in a position to fully realize how many questions, qualifications, and complexities lie behind the bald statement of results like this. Below, I list some of the most important questions that need to be asked when we encounter the results of a study that appears to be drawing our attention to a novel factor that may be affecting our health. Use of a checklist like this can provide a needed corrective to the many preliminary results that get into circulation, creating a fog of unvetted and often misleading information.
- What is the finding/main result?
- How strong is the result? How consistent is the result within the study?
- Does the result pertain to the whole study population, or is it only present in a subgroup?
- How many different factors did the researchers examine? (Is the finding the result of “multiple comparisons”/“data dredging”?)
- How good are the data from the study? How well did the researchers measure the study factor?
- Could the result be due to confounding, i.e., could some other factor that is correlated with the study factor be the true cause?
- Is there evidence of a dose-response relationship?
- Do the authors report the “absolute risk,” as well as the relative risk? (“Absolute risk” conveys the population impact of the factor, i.e., how many cases of the disease would be prevented if the risk factor were eliminated). Reporting only the relative risk often makes the factor look more important than if one reported the absolute risk.
- How does the result accord with the results of other studies?
- How does the result comport with what we know about other established risk factors?
- Do the authors acknowledge the limitations of the data?
- Do they put their results in perspective?
- Keep in mind that association does not prove causation.
- Although authors often point to a biological mechanism that could explain their result, we should keep in mind that one can always find a possible biological mechanism to explain a given association.
- Is this result something that we should pay any attention to?
Space limitations prevent us from examining all of the claims in the sweetened beverage paper. But let’s look at a few key points. First, as can be seen in the table below, the number of cases in the different levels of soda intake is quite small, especially at the two higher intake levels (14 and 16 cases), and, as a result, the risk estimates are unstable and imprecise.
Second, a 2-fold relative risk sounds impressive, but we should note that a doubling of the risk is actually a modest increase.
Third, and more important, if we look at how many cases of early-onset colorectal cancer can be attributed to a relatively high intake of sugary beverages, we see that the incidence rate among those drinking these beverages less than once per week was 8.4 cases per 100,000 (45 cases/536,446 person-years) and among those drinking 2 or more sugar-sweetened drinks per day the rate was 11.6 cases per 100,000 (16 cases/138,469 person-years).
In other words, if drinking sugary beverages does in fact increase the risk of early-onset colorectal cancer, it results in an extra 3 cases per 100,000 population. This is the “absolute risk” or “risk difference,” which indicates the population impact of the hypothesized risk factor.
Finally, the role of intake of sugar-sweetened beverages in the development of colorectal cancer generally (not just in younger people) has been investigated extensively, and it has generally not been found to be associated with increased risk.
At the very least, our examination of these few, but important points, raises doubts about the hypothesis that the intake of sugar-sweetened beverages accounts for the observed increase in colorectal cancer among younger people.
In spite of the very weak evidence and the many questions regarding the results, the authors concluded, “In this large prospective US cohort study of younger women, higher SSB consumption in adulthood and adolescence was associated with a substantially higher [emphasis added] risk of EO-CRC,” and that, “Our findings add unique epidemiologic evidence [emphasis added – GK] that SSB intake may partly contribute to the rapid increase of CRC in younger adults.”
When assessing weak and uncertain associations that might potentially be relevant to the risk of a serious disease, epidemiologists are practiced at the art of acknowledging the limitations of their study, while, at the same time, underscoring the potential importance of their findings.
The very length of the checklist tells us just how carefully the results of a single study need to be scrutinized and placed in the context of what is known about a disease in question.
We also need to remind ourselves that well-done studies whose results don’t lend themselves to sensational headlines don’t tend to get media attention. “If it bleeds, it leads.”
The triumphs of epidemiology and public health I mentioned at the beginning of this essay spoke for themselves. They didn’t need to be embellished by exaggerating their potential significance, by appealing to public concerns about harmful components of our diet, or providing environmental activists with ambiguous evidence that exposure to trace amounts of some chemical are having discernable health effects.
Over the years, a number of prominent epidemiologists have criticized risk factor epidemiology and “approaches that abstract single elements … from the complexity of the life and times of people and relate these to a single health outcome.” (See also here and here). But such critiques have done little to dampen the appeal of focusing on small, questionable effects – a focus that one epidemiologist wrote “marginalizes us as a field.”
Once we realize how big the gulf is between the punchline of an epidemiologic study and the many links in the causal chain that need to be filled in, we can begin to take a more skeptical attitude toward results that are a staple of our media ecosystem and that, for the most part, provide little by way of actionable information to improve our health and well-being.
Geoffrey Kabat is cancer epidemiologist and the author of Hyping Health Risks: Environmental Hazards in Daily Life and the Science of Epidemiology and Getting Risk Right: Understanding the Science of Elusive Health Risks. He can be found on Twitter @GeoKabat
This article previously ran on the GLP June 16, 2023.
