In hyped claim organic milk is healthier, activist science meets bungled reporting

& | December 25, 2013 |
Image via Oldways. Image via Oldways.

For the health-obsessed media, a study touting the alleged benefits of drinking organic milk is a made-for-headlines event and a bonanza for the ‘natural food’ industry.

The web erupted last week after Washington State University issued a news release titled “Researchers see added nutritional benefits in organic milk,” promoting a just-published study as a long awaited silver bullet proof of the superior health benefits of organic foods. All the stories lacked were consensus science. Here’s how the media bungled their reporting and what the science really says—and doesn’t say.

The study, co-authored by well-known organic researcher Charles Benbrook, claimed to be “the first large-scale, U.S.-wide comparison of organic and conventional milk”—it wasn’t, but we’ll get to that later. Published in PLOS ONE, the Public Library’s open-access resource, the news struck immediate media gold, spawning a spate of misleading headlines and stories, even in the mainstream press.

“More Helpful Fatty Acids Found in Organic Milk,” headlined Kenneth Chang’s New York Times story. (Fact: organic milk does not have higher concentrations of more helpful fatty acids.) Businessweek‘s Andy Martin’s report was titled “Is Organic Milk Better for You? It Might Be.” (Fact: It’s not.) The NBC News headline for the story by Melissa Dahl, a health writer and editor, was among the least responsible: “Yep, organic milk really is better for you than regular milk.” (Fact: Nope.)

The activist media upped the ante, with stories that read like industry news releases in Mother Jones (Tom Philpott cites the study as ‘proof’—Fact: No way.) and Grist “Organic milk is better for your heart” (Fact: No better than conventional milk.), and the promotion of the story by the Organic Consumers Association, Stonyfield Organic and other companies and organizations set to gain financially from this news.

While the rapturous headlines and the endorsements sound definitive, the science is anything but. The flurry of uncritical stories touched off by the release of this study is a textbook example of how easily science can be manipulated and misrepresented—and why journalistic vigilance is so critical.

Fatty acid conundrum

Charles Benbrook claims that “organic milk is better” than conventional milk because it contains significantly higher concentrations of heart-healthy fatty acids. Testing nearly 400 samples over an 18-month period, the research team found that conventional milk they tested had an average omega-6 to omega-3 fatty acid ratio of 5.8, more than twice the ratio of 2.3 they reported from organic milk. They say that is a more favorable balance.

Independent scientists—those with no financial ties to the organic industry—say that the Benbrook study is rife with methodological problems and questionable scientific assumptions. Rossi Filippo, a nutritionist with the Institute of Food Science and Nutrition at the Catholic University of the Sacred Heart in Milan, pointed out in an email that the omega-6 to omega-3 ratio should be calculated considering the whole diet, not only with one food. But even if the recalculated ratio should hold, the sweeping conclusions—that organic milk is better for us—is almost certainly wrong.

The problems with the study go far beyond poor math. While some scientists argue that higher ratios of omega-6 to omega-3 lead to greater health risks, that’s not the consensus belief. Dr. Walter Willett, chairman of the nutrition department at the Harvard School of Public Health says that studies like this one that claim that omega-6 fatty acids are harmful promote a “false assumption.” According to Willett, omega-6s are actually associated with a lower risk of heart disease. The ratio touted in the study is “irrelevant,” he says, and health conscious consumers should eat more of both kinds of fatty acids—directly contradicting a central assumption in the Benbrook study.

Other scientists are concerned about the crude way Washington State University and Benbrook promoted the research. They make it seem like a breakthrough study, a  theme echoed widely in news stories, and even by the New York Times in its otherwise balanced report. Kenneth Chang calls it “the most clear-cut instance of an organic food’s offering a nutritional advantage over its conventional counterpart.”

In fact, study after study—thousands of them, with the 237 considered most pertinent summarized in a meta-analysis by a team at Stanford University published last year—have found “little evidence of health benefits from organic foods.” The American Academy of Pediatrics and the Mayo Clinic, among dozens of organizations, have said organic foods offer no nutritional advantages over conventional foods, and the National Dairy Council, which represents organic and conventional farmers, states there are no unique benefits from organic milk.

Despite Benbrook’s claim, this is neither the first major nor the most comprehensive study comparing the health differences of organic and conventional milk. In 2010, for example, the Journal of Dairy Science evaluated samples from 48 states and reached the opposite conclusion from Benbrook’s one-off publication. That study also looked at milk labeled as recombinant bovine somatotropin (rbST) free. (rbST is a synthetic version of the protein, which organic activists, including Benbrook, say is unhealthy, although there is no data to support that claim.) The conclusion in JDS:

Overall, when data from our analysis of [fatty acid] composition of conventional milk and milk labeled rbST-free or organic were combined with previous analytical comparisons of the quality and composition of these retail milk samples, results established that there were no meaningful differences that would affect public health and that all milks were similar in nutritional quality and wholesomeness.

Grass fed not organics

“The problem with this study,” animal scientist at the University of California-Davis Alison Van Eenennaam notes in an email, “is that it conflates ‘organic production’ with ‘omega-3 levels in pasture’. Fatty acids found in milk are derived almost equally from two sources: feed and the microbial activity in the rumen of the cow.  It is well known that milk from cows consuming grass has slightly higher levels of omega-3 fatty acids. Grass-fed conventional milk would have the same fatty acid profile. This can actually be seen in the study. Conventional milk from Northern California, where cows are typically grazing pasture, had a fatty acid profile and omega-3 content similar to organic milk.”

What about the claim that organic farmers are more likely to grass feed their cows? That’s true in some cases but not in others. Depending upon the time of the year organic cows may or may not graze on pasture. According to USDA organic guidelines, they can technically be off it for 8 months if raised in a region with a short grazing season.

“In practice, organic farms as well as conventional farms can vary widely in feeding and diet formulations,” writes Van Eenennaam. “This variation was demonstrated in a 2009 study that compared the omega-3 fatty acids milk fat content from three retail sources of organic and conventional milk, as well as one individual organic farm, every other week for 18 months. The individual organic farm had the highest and the lowest omega-3 fatty acids values, illustrating the substantial variation that exists even within a single farm.”

Bottom line: the fatty acid levels in milk (even if the ratios are meaningful) do not necessarily have anything to do with organic methods. Milk composition is determined by breeding (genetics), the feed cows consume, how the cow is raised, lactation stage and season. In this context, the data in the Benbrook study looks like little more than overhyped noise.

Studies in Britain from several years ago have showed that conventional milk from cows fed on a diet of mostly grass tend to produce this exact result, notes Graham Brookes, an economist with the agricultural consultancy PG Economics. In the UK, most milk is produced this way, so whether it’s organic or not is irrelevant.

It’s not even established that fatty acids from milk—organic or conventional—provide any unique health benefits. Studies demonstrating the benefits of certain fatty acids have focused on fish oils, not on dairy products. After the publication of a similarly flawed 2010 study of organic and conventional milk in England (Mother Jones hawked that study too), the UK Food Standards Agency evaluated and rejected claims by organic milk supporters that their products offered a health benefit, recommending oily fish as the only way to get beneficial omega fatty acids.

Conflicts of interest dog organic study

Only the New York Times, among the major media, provided some counterweight to Benbrook’s hyped claims about the alleged benefits of organic milk even though it fell prey to sensationalist journalism in its title. Times reporter Chang was also one of the few journalists to highlight the clear conflict of interest in the funding of the study. (Call out to Philpott—good reporting on this.) Almost all the research dollars came from Organic Valley, a mega farm cooperative that sells organic products and is positioned to reap a huge financial windfall if the publicity over the study influences consumer buying behavior.

Contrast this deafening silence to what happens whenever research is released that links conventional or genetically modified foods to a health or sustainability benefit—for example, when studies found that Golden Rice enhances nutrition or Bt cotton reduces pesticide use. Journalists’ knives come out, usually in the first sentence (“In a study funded by the biotechnology industry, researchers have found….”). Organic and anti-GMO activists dismiss these studies outright (“Scientists Tied to Tobacco Industry Propaganda, and Funding from Monsanto, Turn Attention to Organic Food,”) claiming the entire conventional food industry is corrupt, and therefore all studies they produce are worthless regardless of the independent controls built in to ensure the credibility of the data.

Then there’s the issue of Charles Benbrook himself. I found no mainstream stories that addressed the organic researcher’s highly politicized role in this debate. Benbrook is a well-known activist who regularly savages conventional agriculture, criticizes crop biotechnology and accepts research money from the organic industry. In short he is a classic industry funded scientist whose research activists should reject outright. As GMWatch has noted, “Conflict of interest consistently obfuscates the claims of safety made by industry affiliated scientists.”

Noting the funding sources of a study helps readers contextualize information; but when it comes to reporting on a favorite subject of health and foodie reporters—organics—journalistic responsibility often goes out the window. If this had been a study funded by the conventional ag industry and it had concluded that organics offered no clear health benefits, anti-GMO and and prominent organic advocacy groups would have ridiculed the study and labeled the author a “shill”.

Benbrook’s research is controversial. He is perhaps best known among scientists for releasing multiple studies, most recently last year, claiming that the cultivation of genetically modified crops leads to increased pesticide use, a popular talking point of anti-GMO activists. As we reported at the Genetic Literacy Project in October 2012, many scientists savaged this claim, saying it appeared he had cooked the data to bolster his anti-GMO campaigning.

“I can’t help but get the feeling that Dr. Benbrook started with a conclusion and found data to fit rather than starting with a general review then finding significant conclusions,” noted Anastasia Bodnar, co-founder of the non-profit Biofortified website. She cited far more extensive research that contradicted his extrapolations—Benbrook did not have hard data for much of his analysis—and pointed out that independent of overall annual volume usage numbers, the toxic profile of pesticides used in conjunction with commodity crops, primarily glyphosate, is far less harmful than the pesticides it replaced and even less harmful than some organic alternatives.

In an apparent eagerness to promote a ‘breakthrough’ study that puts a permanent halo on organic foods, many reporters fell victim to “single study syndrome”—the notion that the latest study no matter how one-off or poorly executed trumps all previously published data and deserves promotion. Compounding that problem, in this case, most reporters just didn’t do their homework, or if they did, they did not grasp the fundamental methodological problems with the study.

There could be serious health consequences from poor reporting about organic milk. Organic foods are expensive and by the judgment of independent scientists are not worth the extra cost if one is looking for health benefits. Plus, they can pose unique dangers. For example, unpasteurized organic milk and cheese have become a fad among ‘health conscious’ consumers entranced by organic industry promotions. Just this week, the American Academy of Pediatrics warned that babies and children are at increased risk from raw and unpasteurized milk and cheese that can carry bacteria such as E. coli and Salmonella from sick animals or contact with manure, which is used as fertilizer on organic farms. Some but not all organic milk is sold as raw or unpasteurized. Organic Valley Coop, which funded the Benbrook study and misrepresents it on links from its website (“Organic Valley’s milk healthier than conventional alternatives”) sells raw cheeses that it brags as being made from “organic cultured unpasteurized milk.”

If reporters had done even a minimal amount of research, they would have come across numeorus studies suggesting that organic milk offers no meaningful advantages and might even pose some health challenges. For example, a 2008 study in the Journal of the American Dietetic Association that analyzed retail samples of conventional, rbST-free and organic milk found that organic milk does show a minimal increase in protein. But conventionally-labeled milk had the lowest bacterial counts (although the differences were not biologically meaningful) of the three varieties and conventional milk had less of the endocrine disrupting chemicals estradiol and progesterone than organic milk.

A 2010 study published in the Journal of Dairy Sciences found that organic milk was higher in saturated fatty acids, considered to be less healthy, and lower in heart-healthier monounsaturated fatty acids when compared to both conventional and rbST-free milk. The differences were measurable but minor, and not significant enough to be considered a health issue—as in the case of Benbrook’s findings, although that’s not what he claims and what many reporters hyped.

So, what did we actually learn from the Benbrook milk study touted by the likes of the New York Times and every organic site known to humankind? Milk from cows that graze on pasture, whether on organic or conventional farms, contain more omega-3 fatty acids that do not provide any unique health benefits. That’s headline material?

Like any research undertaken by a true believer, science is vulnerable to taking a back seat to spin. Hundreds of independent studies by dozens of the top independent science organizations around the world have concluded that organic foods offer no unique health benefits when compared to conventional varieties. Botched reporting just promotes cynicism and contributes to efforts by many in the organic establishment to demonize conventional agriculture and biotechnology. That some journalists participated in what amounts to a marketing blitz for the organic industry is unconscionable.

Jon Entine, executive director of the Genetic Literacy Project, is a senior fellow at the Center for Health & Risk Communication and STATS (Statistical Assessment Service) at George Mason University.

Jeremy Summers is a freelance science writer and editor. Having gotten his start covering North Carolina’s extensive biotechnology industry, he currently contributes to and manages the Genetic Literacy Project website.

Additional Resources:

  • RobertWager

    Money from the biotech industry for studies cancels out the studies according to many critics of GE crops, but money from the organic industry funding organic research is just green.

  • stevesavage

    Benbrook also only very recently started working for WSU, but he claims that for credibility. He is still involved with his longer-term job at the Organic Center.

  • goaway

    What a ridiculous wordy unconscionable piece of propoganda. To say the media that you and yours who uses every media possible to make your false claims and sense of security so ppl keep buying your friends products is what is truly sick. everyone knows that all of you people are connected to huge corporatins who have NO INTENTION of spreading and nothing that keeps us healthy. If you all weren’t paid off by huge corporations to “provide these studies” then why are the greedy bastards buying up every organic company that sells out to them?? You want us sick and depleted by your chemicals and man made crap so we think we have to use more of your chemicals to get “well”. If it weren’t true or basic common sense you wouldn’t need to write this crap. Save it and go sit in your man made food and chemicals with that nut job quack.com paid off doctor. We don’t want it. Your man made “facts” are just as detrimental to us as the Crap you force into our foods and skies and soil and water as you turn the earth into a dying dustbowl while blamong the oil industry who really provides what you need to modify everything so that it can survive in an environment ruined by your other chemicals and modifications. Whether you like it or not…something out there bigger and better than you created everything to be dependent on everything else and to interact in positive supportive ways called ECOSYSTEMS and your man made crap is nothing but a sad attempt to be something bigger than you are. I can no longer even give the benefit of the doubt and believe any of you care about anything but power and money. And it’s not Bc of the media. It is because of common sense and your own insistence that what is good is bad and that what is bad is good. You collectively hate that people don’t have to pay you to be alive and thrive so you want us to pay you while you tear down our environments. Why don’t you all descend back into the earth and rule over each other with your “infinite knowledge” and leave the rest of us to our “ignorance”.

  • Rob Wallbridge

    This is going to be a long comment, but after noting on Twitter that this article appeared to be a satire on anti-GMO reporting, Jon Entine has asked me to expand on the misinterpreted data, ad hominem attacks and conflated claims in this piece.

    Misinterpreted data: despite the way it’s profiled in this piece, the Stanford meta-analysis explicitly stated that “evidence from a limited number of studies suggested that organic milk may contain significantly higher levels of omega-3 fatty acids.” This latest study obviously adds to that list, and no matter how you try to paint it, no one is telling us we should be consuming *less* omega-3s.

    Far from reaching the “opposite conclusion” as claimed in this piece, the Journal of Dairy Science study did, in fact, find significantly higher levels of beneficial omega-3s and CLAs in organic milk. The authors also cite several studies suggesting the health benefits of consuming these beneficial fatty acids. Their conclusion, therefore, seems to be based more on their personal opinion than on the findings presented in the paper.

    To add to the irony, Entine & Summers dismiss Benbrook’s work as a “one-off publication” while referring to the JDS work as a “study”. The facts, however, are that Benbrook analyzed almost 400 samples over a 18-month period. The JDS: 292 samples over a 3-wk period (in October and November – which, by the way, is certainly not peak grazing season for much of the contiguous 48). I’m no statistician, but a greater number of samples over a greater time period would seem to indicate greater statistical reliability to me. But finding people to claim a lack of consensus (sound familiar?), selective quotations, and throwing around terms like “one-off” shows just how easy it is for journalists to manipulate and misrepresent science, doesn’t it?

    Another celebrated anti-GMO tactic employed here is to cry “shill”, as Mr. Wager so helpfully points out, seemingly oblivious to the fact that Entine and Summers employ exactly the same tactic to attack Benbrook and Organic Valley, while themselves attacking those who use funding sources or past affiliations to attack researcher’s work. Seems to me like the authors of this piece would like to have it both ways, as long as it’s their way.

    Next, we see the attempt to conflate the issue at hand with other, only peripherally-related issues (another favorite tactic of the anti-GMO camp.) First, Entine and Summers conflate organic dairy with “raw” or unpasteurized dairy. The fact is that unpasteurized dairy products can and do come from both organic and conventional farms. As well, OV’s “raw” cheese is produced according to USDA food safety regulations and has never been connected with a food safety issue. The bottom line is that the “unique dangers” of raw dairy have absolutely nothing to do with organic products.

    Finally, Entine and Summers trot out the old myth about manure use on organic farms posing a health risk. Again, as I’ve discussed on this site (and my blog) before, there is absolutely zero scientific evidence to suggest that this is the case. What’s particularly hilarious in this instance is that dairy cows are the topic of conversation. Here’s a little tip: conventional dairy cows poop, too, and their manure gets spread on those farm fields as fertilizer, too!

    Maybe the “organic establishment” wouldn’t feel so compelled to “demonize conventional agriculture and biotechnology” if the true believers of these methods didn’t consistently botch science and logic to mount attacks on organic food and farming. Why doesn’t this site give it a try?

    • http://www.geneticliteracyproject.org/ Jon Entine

      Rob, lot’s of provocative comments. I’ll try to wade through the issues you raise, though not necessarily in the order that you raised them. First I want to make a key point, which I also made in the story. Charles Benbrook claims that “organic milk is better” than conventional
      milk because it contains significantly higher concentrations of
      heart-healthy fatty acids. His study and the reporting revolve around that claim. Yet we know, I think we call can agree, that such a claim is hyperbolic at best and most likely, based on the known science, flat out wrong. So we know from the start that Benbrook positions himself not as a scientist in the spirit of independent inquiry–let the facts speak for themselves–but as an activist/advocate. He is what he is.

      You claimed in your Twitter post that we made an ad hominem attack on him. You provide no evidence of that from the piece, however. The story we wrote commented not only on the science, which Benbrook misrepresented (he did not provide a review of the literature on milk and fatty acids, which is standard for academic pieces, probably because the literature is so inconclusive and contradicts with his conclusion), but on how the media covered the story. We commented on how the media did not contextualize their stories by noting Benbrook’s history of cherry picking data–which is well known and even notorious in the mainstream science community. We also noted that he had ties to the organic industry–again, not an attack on him at all. Plenty of scientists have connections to one group or another. What we pointed out is that ideologues, as contrasted with independent journalists and activists, call out associations as way to mount an ad hominem attack. We were in fact doing quite the opposite of what you are imputing. We were pointing out how unfair that kind of ad hominem attack is and in fact we underscored in our piece that noting such associations are totally appropriate for journalists to do when covering a story–in fact it’s journalism 101. Associations are to be noted but they do not invalidate someone’s work; research rises or falls on its own merits–and that’s where Benbrook’s work falls so short. I’ll discuss some of those other issues in a subsequent post.

      Meanwhile, let’s keep the dialogue going. I personally have great respect for farmers–organic and conventional. What gets me going is the attempt by organic promoters–you do not seem to fall into this category–to claiming some moral or health superiority from their chosen method. Organics have a place in agriculture…but it’s just a method that provides some benefits and has some drawbacks.

  • Ann Green

    I’m a bit late with this comment, but will post anyway…
    I read the original WSU press release, but not the various articles that ensued. After reading the release, I emailed a nearby conventional dairy farmer I know whose cows would certainly fit the grass fed-criteria of the study and asked if her marketing co-op could tie a grass-fed ribbon on their labels. (No.) Then I called WSU and asked if this study tells us anything about “industrial” (my term) organic products. (No.)

    For me, the take home message of the study is that inputs affect outputs. But that doesn’t help us consumers all that much. I certainly agree that one cannot conclude that all organic milk has the same profile as that of the grass-fed cows in the study. And we don’t really know about how various milk profiles affect our health. Too many leaps of faith there. I happen to believe that more omega-3s might be a good thing, but in general I think we are just at the beginning of understanding nutrition and how ag practices affect the food we eat.

    Something that cracks me up: egg and poultry products that brag that their critters got “all vegetarian diets.” I know for sure those critters were not outdoors living the good life where they eat anything they can wrap their little beaks around. I know that claim stems from days when chickens might have been fed sick cow parts, but I just think it shows how naive foodies are.