Yet Another Look At The Transcendental Meditation Paper 84

Editor’s note: Below are two responses to Robert Schneider’s defense of his Transcendental Meditation paper, which Schneider wrote in response to my earlier article about the publication of his paper.  In the first part I respond to some of the general issues raised by Schneider. The second part, from Sanjay Kaul, addresses the statistical issues discussed by Schneider.

I’m grateful for Kaul’s highly technical analysis of the statistical issues raised by Schneider, but I don’t think this case really requires a terribly high level of technical expertise. Common sense actually works pretty well in this case. A trial with barely 200 patients can not be expected to provide broad answers about the health benefits of a novel intervention. As Kaul and others have stated on many other occasions, “extraordinary claims require extraordinary evidence,” and it is quite clear that the evidence in this trial is not extraordinary, at least in any positive sense.

Questions About Trial Reliability And Data– In his response Schneider tries to skate away from the inevitable questions raised about this paper when Archives of Internal Medicine chose to withdraw the paper only 12 minutes before its scheduled publication time. Schneider can pretend that this incident never occurred, but outsider readers can not help but wonder what sparked this extraordinary incident, and will not be satisfied  until the details are fully explained.

There are additional red flags about the trial. Schneider told WebMD that since the Archives incident “the data was re-analyzed. Also, new data was added and the study underwent an independent review.” Said Schneider:

“This is the new and improved version.”

This is an extraordinary claim, because a clinical trial can not be “new and improved” unless there were serious flaws with the earlier version. What exactly does it mean to say that a paper published in 2012 about a trial completed in 2007 is “new and improved”? (According to ClinicalTrials.Gov the study was completed in July 2007, while June 2007 was the “final data collection date” for the primary endpoint.)
Click to continue reading…

About these ads

Investigator Defends Controversial Transcendental Meditation Paper 4

Editor’s Note: Here is a response by Dr. Robert Schneider to my story last week about his controversial paper on Transcendental Meditation that appeared in Circulation: Cardiovascular Quality & Outcomes. I will respond to Dr. Schneider’s post later this week.

Response from Dr. Robert Schneider

We appreciate the interest in our article published in Circulation: Cardiovascular Outcomes and Quality, November 2012.  However, there are several errors and inaccuracies in the blog. Many of these points were addressed in the Circulation Outcomes publication.  Below are highlights.

Robert Schneider, MD, director of the Institute for Natural Medicine and Prevention and dean of Maharishi College of Perfect Health in Fairfield, Iowa. Courtesy MAPI

1.     This was a prospective, randomized, controlled, single-blinded clinical trial.  This report is the first publication of data from the trial. It was revised in response to extensive peer-review over the past months.  Whatever unpublished version of the manuscript the bloggers had access to previously did not have the benefit of the most current peer review and revision.

2. There were a range of check and balances built into the study to ensure veracity.  For example:

  • data were collected blindly at the clinical site in Milwaukee
  • the trial was monitored by independent data safety and monitoring board
  • end point events were adjudicated by an independent reviewer using standardized criteria
  • results were confirmed by independent data analysis
  • limitations are enumerated and discussed in the publication

3. Regarding the hypothesis testing capacity of the study, according to the Consolidated Standards of Reporting Trials (CONSORT) guidelines,

“Randomised clinical trials (RCTs) are generally considered to produce the most solid evidence for the effectiveness of medical interventions”  (Keech 2007).  The CONSORT guidelines are recommended by the International Committee of Medical Journal Editors (ICMJE). The design, implementation, analysis and reporting of this trial followed CONSORT guidelines.  Thus this clinical trial was an experimental study that tested an hypothesis.  By contrast, observational studies, which this was not, generates hypotheses (http://en.wikipedia.org/wiki/Observational_study).

4.  The total number of primary end point events is within one of an earlier, unpublished version of the manuscript (noted in blog update November 18).  In preparation for the revised manuscript, one additional event was identified.  However, this had no meaningful effect on the results or statistical significance. That is, the main results are essentially unchanged.
Click to continue reading…

Mysterious Disappearing Paper Finally Reappears In Another Journal 33

Robert Schneider, MD, director of the Institute for Natural Medicine and Prevention and dean of Maharishi College of Perfect Health in Fairfield, Iowa. Courtesy MAPI

Updated– Last year, in what may have been an unprecedented action, a paper on the effects of Transcendental Mediation (TM) in African Americans was withdrawn by the editors only 12 minutes before the paper’s scheduled publication in Archives of Internal Medicine. No definitive explanation was ever provided, though the editors and authors said that the action was prompted by last minute questions from reviewers at the NIH, which helped fund the study. (The episode was covered in detail on CardioBrief  (here and here) and on RetractionWatch.)

Now a new version of the paper has been published in Circulation: Cardiovascular Quality & Outcomes. The first author is Robert Schneider, from the Institute for Natural Medicine and Prevention at Maharishi University of Management in Maharishi Vedic City, Iowa. The co-authors are from the same institution and from the department of medicine at the Medical College of Wisconsin in Wisconin. It is clearly the same study of 201 African American patients randomized to TM or health education (HE) and followed for 5.4 years, though some of the numbers have changed in important ways between the earlier and later publications. One change involves the primary endpoint: the new paper in Circ:CVQ&O reports  52 primary endpoint events (the composite of death, MI, or stroke). Of these, 20 events occurred in the TM group and 32 in the HE group. By contrast, in the previous Archives version there were 51 primary endpoint events: 20 in the TM group and 31 in the control group.

In both papers the difference in the primary endpoint did not achieve statistical significance until after the investigators adjusted for baseline differences. The unadjusted hazard ratio (HR) for the new study was 0.64, (confidence interval 0.3701.12, p = 0.12); after adjusting for age, gender, and lipid lowering medications the HR drops, presto-chango, to 0.52 (0.29-0.92, p=0.025) and achieves statistical significance..

Dome shaped building on MUM campus, for the group practice of the TM-Sidhi program and Yogic Flying (Photo credit: Wikipedia)

Here are some other key questions that I have asked AHA, Circulation editors, and other experts to address:

– Was the AHA or the Circulation staff aware at any time that an earlier version of this paper had been scheduled for publication in Archives of Internal Medicine and withdrawn only 12 minutes before the scheduled publication time? Should this information have been disclosed by the study authors when submitting the paper?

– As mentioned above, the original publication in Archives appeared to have been cancelled because of questions raised by NIH reviewers. Have the Circulation: Cardiovascular Quality & Outcomes editors received any assurance that these questions have been addressed in the new paper?

– This trial was started all the way back in 1998 but was not registered on ClinicalTrials.Gov until February 2011 (shortly before the intended Archives publication). Why was it not registered earlier than 2011?

– June 2007 is listed in ClinicalTrials.Gov as the final data collection date for primary outcome measure. Why are the primary endpoint numbers different between the Archives and new version of this paper?

Update, November 13-Harlan Krumholz, the editor of  Circulation: Cardiovascular Quality & Outcomes, sent the following statement:

“We had no prior knowledge of what transpired with the Archives of Internal Medicine.  The Schneider paper went through rigorous peer review, statistical review and editorial discussions and the authors of the article were responsive to the review process.  As a result, the paper was accepted for publication and we are going ahead as planned.  If you have any further questions, we suggest you contact the researchers directly.”

Update, November 18–  An earlier version of this article contained several mistakes. First, the difference in the number of primary endpoint events between the two papers is much smaller than I had stated. The original paper reported 51 primary endpoint events. The new paper reports 52 endpoint events. I am still unclear about why there should be any difference between the two, since, as mentioned above, June 2007 was the final data collection date for the primary endpoint. I also incorrectly claimed that some  data reported in the original Archives paper were not included in the new paper. In fact, as a reader pointed out in the comments section below, these data were reported in Figure 1. I apologize for this mistake and regret the error. –LH

I asked Sanjay Kaul for his perspective on the paper. Here are his points:
Click to continue reading…