Afraid of Environmental Working Group’s ‘Dirty Dozen’ list? Here’s 12 reasons you shouldn’t be

Many regular food shoppers anxiously await the results of the Environmental Working Group’s annual “Dirty Dozen” list. Even if you don’t know what or who EWG is, you’re probably aware that conventional strawberries and other common produce items are supposedly loaded with toxic pesticides. With all the press this list gets, you might assume the research behind it must be scientifically credible, right? Well…

Developing our ‘Reliability Radar’

Some of our everyday news sources, like social media, flood us with information. And unfortunately, much of it isn’t credible. Many websites try to cloud our objective reasoning by intentionally misrepresenting data to ‘sell’ a perspective, much like the sudden popularity of the celery juice diet that Hayley Philip previously wrote about.

So how can we build our analytical defenses back up? We’ve put together our own unique list — “The Discerning Dozen” — a compilation of tips to help you identify good science from pseudoscience. This way, you can be the judge when catchy news stories like The Dirty Dozen are released.

Credibility:

Though challenging to read, studies from .edu and .gov websites lay the foundation for good research. Try to stick with sources that use respected institutions to verify their practices and reporting.

1. Is it written by someone from a credible establishment? Reports and studies from recognizable institutions (academic, governmental, and/or medical) often have the most detail and are peer-reviewed, meaning other institutions have verified the research. Accredited medical journals with .org and .com sites, of course, can be good resources, too. Newbies to exhaustive reports can read the overviews typically found on the first page to understand the big picture. Still need an interpreter? Check out usefulscience.org; it’s a great resource for deciphering studies and has a simple, intuitive interface. Scholar.google.com is easy to navigate, too.
2. Does it include knowledgeable industry experts and authors? Trustworthy reports and articles come from professionals with credentials in their related industry. These experts usually provide insight garnered from data, rather than opinion and specious claims.
3. Which references does the report cite? Reliable research that’s not written by a credible establishment and doesn’t come from an industry expert should, at the very least, cite credible sources, like the U.S. Department of Agriculture or Massachusetts Institute of Technology, to substantiate any claims.

That’s a nice little circle of trust there, right? But as we know from life, few things are that easy…yes, there’s more to consider.

Accuracy/transparency:

So the site ends in .org or .com and it’s not a medical journal. What’s the next step to check its trustworthiness?

4. Is it cherry-picking data? This is when an organization only shows data that supports their agenda but fails to address conflicting info or cites data out of context, and/or relies on outdated data since nothing more recent aligns with their purpose.
5. Do other sites use the same facts? Hopefully, you can find the same information cited by other credible institutions.
6. How much do they spend on researching their cause – and how much for marketing? All non-profits must publicly disclose financials on their site; you just have to dig for it. For instance, on EWG’s Statement of Activities page, 13% of their expenses went toward marketing and fundraising – not horrible. However, Functional Expenses reveals a much higher figure: in addition to fundraising, each subcategory also has its own marketing expenses. Furthermore, only a paltry 2% of their expenses is going to research and data. That’s not much funding for finding solutions to a problem, is it?

Bias:

Time to take a peek under the hood when the site’s validity is not easily determined.

7. Do a domain double-check: Sites ending in .com and .org aren’t as regulated as the .gov and .edu sites of the world, so you’ll need to dig into the “about us” page for some background. Sometimes it’s hard to tell reputable foundations from organizations peddling questionable products or ideas. So be sure to read the bios of the management team and authors to determine reliability.
8. Is there political pull? Any site can have an agenda, but not all of them explicitly state it. Advocacy websites, like PETA, are quite clear in their intentions. Reading the “About Us” page can tell you which policies, actions, campaigns, and lobbies they promote.
9. Is it clear who wrote the article? This is a simple one we often overlook. If the author isn’t stated and/or doesn’t cite sources used for its research, then you can quickly determine it was written in-house to promote the organization’s stance.
10. Is there only one answer? Does the author address alternative viewpoints on the topic? Good writers don’t omit or contest credible data that conflicts with their intent.

Quality:

If you manufacture a product, you know about quality control – measures and precautions taken to ensure customers that everything is in good working order. This goes for websites, too…

11. How’s the quality of the writing? Typos? Wrong words used? Time to check the author and his/her data.
12. Don’t judge a site by its homepage. Does the site look rather simplistic? Or so polished and bursting with content that you feel like you found a goldmine of good material? No matter the design, the site is only as good as its underlying content, so vet it accordingly.