Written by Dr. John Carosso
To Believe or not to Believe?
We are deluged daily with research findings concerning everything from A-Z. The findings often come from prestigious journals and universities and from people with lots of letters after their names. However, can we trust the findings?
The Short Answer
No, we can’t.
The Long Answer
Oh, you’re looking for some elaboration. Well, there are problems on so many levels it’s tough to decide where to begin. If, by the grace of God, the research study actually used a sufficient sample size, an adequate research design, stringent statistical methods, and has been replicated (all of which not uncommonly is not the case), the subsequent headline touting the findings still may be misleading.
Examples?
You’d hurt your back picking-up all the articles and books chronicling flawed research. The less strenuous approach of any quick Google-search will demonstrate myriad studies targeting everything from vaccinations and autism, vitamin D and cancer, anti-depressants and depression/autism, climate-change analysis and global cooling from the 1970’s (remember being told we were all going to freeze to death?) and caffeine and Alzheimer’s that have proven inaccurate and/or misleading. Moreover, it seems that the more money and politics involved, the more invalid the research finding.
Recent investigation into this matter, from UCLA and Harvard, found that, for example, between 27% and 37% of the studies of various medications utilized outcome measures that were misleading. For example, the finding that a medication may prevent heart disease by 50% but also increases cancer deaths by the same percentage (guess with statistic was touted); or that a medication reduces risk of heart attack by 50% when, stated another way, the medication lowers risk from two-in-a-million to one-in-one-million (again, guess which statistic was head-lined).
Speaking of Media Hype
I subscribe to a number of listserv journal outlets that email well over a hundred research headlines per week in the mental health and medical arena. I’ve found many of the headlines misleading. Those same misleading headlines are touted all over the news media. When I’ve looked past the headlines to read the actual research article, almost always there are vital distinctions and discrepancies that are not included in the headline and neglected in the subsequent news article. Example: recent finding that mothers with diabetes are more likely to have a child with autism. The media often excluded the lack of statistical significance of that finding. I could provide countless examples of headlines that completely missed the boat.
What to do?
The following might help:
• Don’t rely on headlines; rather, read the actual research article (most are easy to find on-line).
• Look to see if it’s a one-time finding, or replicated.
• On a more technical level, you want to see large sample sizes, control groups, and randomized and double-blind assignment to groups.
• I don’t want to sound conspiratorial, but also consider the source of the research and remember that money and politics can be a corrupting influence.
A healthy Skepticism
There’s a difference between being cynical and skeptical; try for the latter. Don’t get lazy and don’t get caught-up in the hype.
Despite concerns, there are quality research findings that help immensely in our understanding of disorders and best-practices in treatment; you just have to look hard for them.
Please share your comments; I’d love to hear your opinion. God bless you in your quest for the truth.
