In my last Source Evaluation Series post, we covered sources and accuracy of information appearing on the American Lung Association website*. Today, we will look at the realism of claims. While somewhat related to source accuracy and validity of data, it does require a slightly different approach. If someone were to offer a diet pill that melts 20 lbs. of fat overnight, the average person would (and should) be skeptical. Even having the most basic understanding of weight loss should be enough to reject the claim of the seller. If something sounds too good to be true, it probably is.
However, the real danger is that some claims seem reasonable at the outset, but only after examining them further, we find that they are unlikely. Many people are interesting in protecting themselves from diseases, which resulted in a very active and lucrative business hawking super-foods, supplements, and cures. The most successful websites mislead by citing individuals with impressive sounding credentials and some of the “latest” research studies. However, if you actually look at the studies, it becomes clear that the sample sizes are inadequate or contain a number of methodological flaws, which ensures that one cannot generalize the findings to the general population. If you have access to the linked research (often they are behind paywalls), read the abstract, then look at the discussion or findings appearing at the end. A good research study is peer reviewed and fully discusses study flaws and their implications. Also, see if you recognize the publishing journal. You do not want to rely on something that appears in a ‘pay for print’ journal. There are courses available to help you gain a better understanding of research evaluation, but I find that for practical purposes, Purdue OWL* offers a great overview of things to think about.
• Be aware of scare tactics that end in a pitch for a product that will protect you against whatever ‘evil’ you encounter.
• Be aware of information that is significantly outside the currently accepted treatment or runs contrary to findings in available literature.
• Be aware of broken links and outdated references.
• Be aware of flashy headlines, especially those including “miracle cure” or “that ______ doesn’t want you to know”.
• Be aware of websites authored by individuals with “scam”, “fraud”, or “quack” in their search results. You may even want to conduct a search of the website with these terms included to see what you find.
Looking at the the ALA website, a little hunting reveals that they take care in providing accurate and realistic data. The general tone of the website is moderate instead of sensational. The website states that they have “a commitment to provide accurate and complete information, to exercise care and act in good faith, to comply with all laws, regulations and organizational policies, and to promote ethical behavior” (American Lung Association, 2005, p. 3). In addition, ALA has an ethics policy located on their website along with a means for reporting suspected violations. They utilize EthicsPoint, Inc. to manage their ethics reporting and mandate that employees and volunteers become familiar with their code of ethics and reporting procedures. The code also includes a section on avoiding conflicts of interest. (American Lung Association, 2005, p. 3). They also avoid the “Danger Signs” violations I provided.
*I am not affiliated with the organizations or website used in this post. All rights belong to their respective owners and authors. Usage does not convey endorsement. Use of the ALA & Purdue OWL websites are for educational purposes only. Purdue OWL linked with permission.
If you have questions about the series or a particular website, please comment below.