I spent over a decade of buying little else than organic food. During that whole time, I never justified my choice by claiming that non-organic vegetables were less nutritious or somehow less healthful. (I just thought that the environment was better off with organic farming. That is another story.)
It was first a couple of years ago that I became aware of people claiming that organic produce would somehow possess a superior wholesomeness. I was confused, and when I learned of an extensive meta-analysis (The Stanford Study) which concluded there was no nutritional difference between conventionally and organically farmed food, I wasn’t fazed. Why would there be a difference?
Still I kept running into arguments insisting that the jury was still out there, or that, indeed, that organic food was most certainly superior. If any evidence is provided, it may often be in the form of an anecdote – that people can taste the difference. This, however, is a very unreliable way to determine nutrition content or truly, even taste perception, as simply being told the food is organic makes people claim it tastes better.
When scientific evidence is presented on this view, so far it has always been in the form of this one meta-study, which is titled: “Higher antioxidant and lower cadmium concentrations and lower incidence of pesticide residues in organically grown crops: a systematic literature review and meta-analyses.” I will call it the High Antioxidant study. I’ve written about this topic earlier as a case study of bias – Am I biased? Are you? – but to cut to the chase, I will include only the summarized review of the science here.
To know more about the robustness of the conclusion reached in the High Antioxidant study, I searched for critical evaluations on it.
Science 2.0, criticized the High Antioxidant study for being too inclusive, for not dismissing studies with flawed methodology.
They used a lot of papers, that is a good thing if there is actually a large body of knowledge and it is rigorous, but in even the most controversial toxicological issue, the EPA will end up disqualifying all but about a dozen papers due to lack of underlying data being included or methodological concern. In a review, they look at no data, of course, and 343 papers becomes the problem rather than the solution when the methodology is flawed.
Meta-analysis, as everyone with statistics knowledge knows, can boost the strength of systematic reviews when done properly but easily suffers from bias unless the researchers are truly interested in controlling eligibility criteria and methodological quality. Without controlled eligibility, it’s easy to find any pattern you want. With Web of Knowledge search terms like ‘organic’ and ‘biodynamic’, it’s really easy to skew the inclusion. Then they synthesized their dramatically different studies using a random effects model.
There was also criticism of the study’s claims that antioxidants reduce risk of chronic illness, both by Science Media Centre, Nutritional content of organic and conventional foods, and pharmacologist Ian Musgrave The Conversation, Organic food is still not more nutritious than conventional food:
While consumption of antioxidant containing fruit and vegetables have been associated with better health outcomes, the antioxidants themselves do not appear to have any role in this effect (see also here, here and here), despite the number of television advertisements that exhort us to buy antioxidant enriched food. Indeed, the major finding is that high concentrations of fat soluble antioxidant vitamins are associated with detrimental effects.
Musgrave pointed out the lack of significance in the difference of vitamin and antioxidant levels – carotenoid in fruits was 50 percent higher, for instance, but then, fruits are a poor source of carotenoid all in all, and our intake mainly comes from vegetables, which have much more carotenoids, and the vegetables showed no difference between organic and conventional. Organic apples had 6% more vitamin C. For recommended daily intake you’d need 5.3 organic vs 5.6 conventional apples. Pesticide and cadmium levels were likewise low for both, with no demonstrable harm from residue of either (the food we have in the western world is the safest humans have ever eaten, and pesticides are one of the least of our worries, see more here and here).
Musgrave continued with the context of natural variability:
[T]he nutritional value of foods is very variable, influenced strongly by local regional factors, variations in growing seasons and rainfall, ripeness of food when harvested and time of harvest. I’m writing this is South Australia, possibly the wine capital of Australia, where we know that having vines on the different sides of a hill will affect sugar and flavor of the grapes. Even different cultivars of the same crop may vary significantly in composition due to the factors above. Nutritional values of crops can vary from between 100% to nearly 200% (which should be kept in mind when the differences reported between conventional crops and organic crops run from 6-69%).
So perhaps the High Antioxidant study didn’t reach such a strong conclusion supporting the benefits of organic produce after all. I had earlier read the Stanford meta-study that came to a different conclusion. Being a non-expert with limited time and resources, I did not actually sit down to read through and understand in detail the studies’ merits. As a layman, my best approach was to see at the collective wisdom of more reviews would bring to the table.
The Conversation mentioned two other reviews that concluded there was no difference between organic and conventional foods. Medical science blogger, Dr. Steven Novella, had summarized the existing meta-analyses on the topic. He had looked at the High Antioxidant study and laid it out with four other reviews on the topic. He begins with a great principle of how to approach a scientific question.
Whenever I am trying to quickly grasp the bottom line of any scientific question, I look for a consensus among several independent systematic reviews. If multiple reviewers are looking at the same body of research and coming to the same conclusion, that conclusion is likely reliable.
In this case, there are three [he later added a fourth, see below] other recent large systematic reviews and meta-analyses of the same research on the nutritional content and safety of organic vs conventional produce. The other three studies all came to the opposite conclusion as the current [High Antioxidant] study.
Here are the studies. A 2009 review of 162 separate studies by a group of scientists led by Alan Dangour from the London School of Hygiene and Tropical Medicine, “Nutritional quality of organic foods: a systematic review,” concluded:
On the basis of a systematic review of studies of satisfactory quality, there is no evidence of a difference in nutrient quality between organically and conventionally produced foodstuffs. The small differences in nutrient content detected are biologically plausible and mostly relate to differences in production methods.
Dangour and his colleagues published a subsequent review the next year:
From a systematic review of the currently available published literature, evidence is lacking for nutrition-related health effects that result from the consumption of organically produced foodstuffs.
After analyzing the data, the researchers found little significant difference in health benefits between organic and conventional foods. No consistent differences were seen in the vitamin content of organic products, and only one nutrient — phosphorus — was significantly higher in organic versus conventionally grown produce (and the researchers note that because few people have phosphorous deficiency, this has little clinical significance).
Novella also noted an earlier 2002 review which found no nutritional differences.
At this point I am satisfied with my conclusion. It must be reasonable to assume, that should there be a stable measurable difference in the nutritional content of organic food, other reviews would be able to arrive at that conclusion independently of each other. Instead, we have four reviews on similar lines – finding no nutritional or health benefits of organic food. Then we have one study stating a different conclusion, but at a closer look demonstrating very little differences for a few nutrients, likely at a level of no consequence for our health or nutrient intake.
Could the four of them be deeply flawed and the High Antioxidant one be the best of the bunch? Could future studies show a much greater difference in nutrient content? That is possible, but based on the evidence at the time, it does not seem likely.