I don’t know why Archives of Internal Medicine stopped the publication of a paper only moments before its scheduled debut (see my previous post for the details). But a news story on the study (published despite the withdrawal of the paper, and provoking further questions about gullible journalists) provides firm evidence that the study’s authors are clearly guilty of gross scientific exaggeration and misstatement. And a closer look at the paper intended for publication raises additional questions.
Transcendental meditation can cut heart attacks and strokes by 50%, writes Richard Alleyne in a news article in the Telegraph that contains not a single caveat or statement of caution. He quotes Dr. Norman Rosenthal of the NIMH, who calls the study results “a seminal finding.” Even more:
The prevention of heart attack and stroke and actual lengthening of lifespan by an alternative treatment method is exceedingly rare, if not unprecedented. If Transcendental Meditation were a drug conferring so many benefits, it would be a billion-dollar blockbuster.”
Study co-author Theodore Kotchen is also quoted (update: the director of media relations at the Medical College of Wisconsin has informed me that this quote should have been attributed by the Telegraph to the first author of the paper, Robert Schneider):
These findings are the strongest documented effects yet produced by a mind-body intervention on cardiovascular disease. The effect is as large or larger than major categories of drug treatment for cardiovascular disease.
And here’s the study’s first author, Robert Schneider:
This study builds on previous research findings showing that the Transcendental Meditation program reduces high blood pressure, high cholesterol, insulin resistance, psychological stress, and atherosclerosis, and takes it to the next step — lower rates of death, heart attack, and stroke.
Now let’s be clear: even if the data from this study turns out to be completely reliable (a point which we can no longer take for granted), the results are at best hypothesis generating and tell us absolutely nothing about the actual value of TM. Only about 200 people were randomized in the study– most studies with hard clinical endpoints require thousands of patients. And a cursory examination of the actual paper raises all sorts of red flags.
Here are a few other items of concern:
- Although 201 patients are reported in the analysis, the study assessed 451 patients for eligibility and randomized 213 patients.
- Of the 105 patients randomized to TM, 19 didn’t even receive TM.
- 12 patients– 6 in each arm– were randomized but then excluded because they did not meet the trial’s inclusion criteria.
- 41 patients– 20 in the TM arm, 21 in the control arm– were lost to followup.
But my biggest concern is with the analysis of the primary endpoint, which was the composite of all-cause mortality, MI, or stroke. This occurred in 17 patients in the TM group compared with 23 patients in the control group, a difference that the authors claim achieved significance (p=0.03) after adjusting for differences in the age, sex, and use of lipid-lowering drugs between the groups. However, there was no significant difference between the groups in any of these factors. Even worse, there were very significant differences in the amount of education (11.3 years in the TM group versus 9.9 years in the control group, p=0.003) and the CES-D clinical depression scale (13.8 versus 17.7) for which the authors did not make an adjustment, although in both cases the imbalance would appear to favor the TM group. In other words, to use the old cliché, they tortured the data until they made it talk. Strange behavior, perhaps, for supposedly laid back TM types, no?
See more on this story from Erika Check at the Nature.com newsblog.