Those statistics…
As Mark Twain popularized, “there are three kinds of lies: lies, damn lies, and statistics.” I suppose there is some truth to that saying; statistics can be confounding, confusing, and twisted. However, by the same token, statistics can also add clarity and help to guide our decisions. So, with that in mind, today we’ll focus on a way to help determine whether a given treatment is effective.
‘Significant Difference’ and ‘Treatment Effect’
In research, determining whether there is a statistically significant difference between the two groups is vital (is there a genuine difference between the treated group and untreated group, due to the treatment, or is the difference simply by chance). Once it’s shown that there is a significant difference, knowing what’s called the ‘treatment effect’ helps us understand the magnitude or importance of that difference. In that respect, the ‘treatment effect’ measures the strength of the relationship between two variables, i.e. between the treatment group and the untreated group. Put another way, the treatment effect measures the magnitude of the impact of the treatment compared to those who were not treated. The treatment effect is measured in what’s called an ‘Effect Size’.
How to Interpret Effect Size
Effect Size:
- 0.0 is no different than placebo
- 0.2 small effect size (but better than placebo)
- 0.5 Moderate effect size (and quite a bit better than placebo)
- 0.8 Large effect
- 1.0 or Above – Robust impact of the treatment!!
So, with all that in mind, here is a list of the treatment effect size for various treatments including ADHD medications, removing dyes from the diet, exercise, anti-depressants, psychotherapy, and miscellaneous medications.
- Stimulants in blind clinical trials – 0.95
- Lisdexamfetamine (Vyvanse) – 1.28
- Mydayis – new ADHD med- 0.67 – 1.1 depending on dose
- Methylphenidate (Ritalin, Concerta)- 0.77
- Stimulants in optimized dose trials – 2.2
- Alpha agonists (Clonodine, Guanfacine) – 1.3 (often used for Rejection Sensitive Dysphoria)
Exercise can be an effective treatment for ADHD, but is temporary and lasts for only a few hours depending on how long and how intense. Regular exercise (and perhaps strategically timed exercise such as before one needs to focus for a class) is a good use of this strategy, and its effect size varies across symptoms measured:
Aerobic exercise effect on aspects of ADHD
(kids and teens meta-analysis) 0.56 – 0.84
Attention – 0.84
Hyperactivity – 0.56
Impulsivity – 0.56
Anxiety – 0.66
Executive functioning – 0.58
Social disorders – 0.59
Depression – no effect size number here, but thought to be as effective as Zoloft in one study
- Strattera – 0.44 to 0.68
- Bupropion (Wellbutrin, Zyban) - .34
- Zoloft (for ADHD only) – 0.25
- Removal of artificial colors and dietary restrictions - 0.21
Sources: Dr. William Dodson, ADDitude Magazine Spring, 2013: ADHD Report Oct. 2012; Sept. 2015 & Dec. 2016 Russell Barkley, Ph.D.
What Doesn’t Work for ADHD (did not find effect sizes, but the research outcomes are pretty clear)
- Computer training (CogMed, Lumosity…) – short-term ‘near transfer’ but no ‘far transfer’…which means that you learn what they are teaching, such as memorizing numbers backwards, but it doesn’t ‘spread’ to other parts of your life
- Neurofeedback – works better for anxiety, but less so for ADHD
Depression
Antidepressant medication is clearly beneficial for severe depression, but also in moderate depression. There is less of an impact over placebo in mild depression. Relative antidepressant versus placebo benefit increased from 5% in mild depression to 12% in moderate depression to 16% in severe depression.
Medication Effect size
Buproprion: 0.18
Citalopram: 0.23
Duloxetine: 0.30
Escitalopram: 0.31
Fluoxetine: 0.28
Mirtazapine: 0.33
Nefazodone: 0.28
Paroxetine CR: 0.44
Sertraline: 0.23
Venlafaxine: 0.42
To give you a comparison…
By way of comparison, psychotherapy effect size ranges between 0.22 – 0.80, depending on the experience of the therapist and the relationship between the therapist and client.
As a way of further comparison, effect size of common medications
Gastric Acid mediation (Nexium…): 1.39
Oxycodone: 1.04
Metformin (diabetes): 0.87
Sumatriptan for Migraines: 0.83
Benzodiazepine for anxiety: 0.65
Antihypertensives: 0.65
Corticosteroids for asthma: 0.56
Anti-psychotic medication: 0.51
Statins for cholesterol: 0.15
Aspirin for vascular disease: 0.12
However, treatment Effect Size does not tell the full story
It should be noted that there is more to the story than treatment effect. In that respect, statistics deal with groups, not individuals. Consequently, one individual might get a huge benefit from a treatment approach or medication, while a bunch of other people may only experience a small improvement. If you’re that one individual who responds wonderfully, then an Effect Size of 0.2 does not mean a whole lot. Moreover, for some treatment options, it’s not applicable to compare the effect size to an entire group of ‘depressed’ persons when the medication is shown to work better for depressed females who are also quite anxious and worried about weight gain. In that instance, the Effect Size may be 0.5, but more globally may only be 0.2. This is why doctors make clinically appropriate decisions to treat individuals, not groups. Finally, keep in mind that if only 5% of people experience a notable improvement with a medication or treatment approach, that 5% can add-up to a lot of people benefiting if the condition is wide-spread and the treatment approach is widely used.
Hope that helps
Making informed decisions is vital, and I hope this post helps in that respect. Please feel free to respond or provide any comments on my Facebook page, I would love to hear your thoughts. God bless.
