Editor’s Note: Here is a response by Dr. Robert Schneider to my story last week about his controversial paper on Transcendental Meditation that appeared in Circulation: Cardiovascular Quality & Outcomes. I will respond to Dr. Schneider’s post later this week.
Response from Dr. Robert Schneider
We appreciate the interest in our article published in Circulation: Cardiovascular Outcomes and Quality, November 2012. However, there are several errors and inaccuracies in the blog. Many of these points were addressed in the Circulation Outcomes publication. Below are highlights.
1. This was a prospective, randomized, controlled, single-blinded clinical trial. This report is the first publication of data from the trial. It was revised in response to extensive peer-review over the past months. Whatever unpublished version of the manuscript the bloggers had access to previously did not have the benefit of the most current peer review and revision.
2. There were a range of check and balances built into the study to ensure veracity. For example:
- data were collected blindly at the clinical site in Milwaukee
- the trial was monitored by independent data safety and monitoring board
- end point events were adjudicated by an independent reviewer using standardized criteria
- results were confirmed by independent data analysis
- limitations are enumerated and discussed in the publication
3. Regarding the hypothesis testing capacity of the study, according to the Consolidated Standards of Reporting Trials (CONSORT) guidelines,
“Randomised clinical trials (RCTs) are generally considered to produce the most solid evidence for the effectiveness of medical interventions” (Keech 2007). The CONSORT guidelines are recommended by the International Committee of Medical Journal Editors (ICMJE). The design, implementation, analysis and reporting of this trial followed CONSORT guidelines. Thus this clinical trial was an experimental study that tested an hypothesis. By contrast, observational studies, which this was not, generates hypotheses (http://en.wikipedia.org/wiki/Observational_study).
4. The total number of primary end point events is within one of an earlier, unpublished version of the manuscript (noted in blog update November 18). In preparation for the revised manuscript, one additional event was identified. However, this had no meaningful effect on the results or statistical significance. That is, the main results are essentially unchanged.
5. Regarding statistical adjustment, the discussion (pg. 7) explains: The unadjusted results of the primary end point analysis showed a nonsignificant statistical trend (p=.12). However, after adjustment for covariates of age, sex and antihypertensive medications was significant (p=.025). According to the Consolidated Standards of Reporting Trials (CONSORT) recommendations, “adjusted analyses frequently improve the precision of the estimate of the treatment effect” (Keech et al.). Furthermore, because these factors were used in the stratified randomization procedure, it recommended to adjust for stratification factors to achieve the most efficient treatment comparison (Pocock et al. 2007; Committee for Proprietary Medicine Products, European Agency for the Evaluation of Medicinal Products, 2003). This is particularly relevant when the adjustment factors predict the outcome, as they did in this trial (Pocock 2007). Thus, statistical adjustment is the recommended procedure for this type of study data.
6. The number of patients assessed, excluded, randomized, followed and analyzed are reported in the participant flow diagram, figure 1, pg. 3 (noted in blog update November 18_. This is in accordance with CONSORT guidelines for reporting clinical trials (Keech et al 2007). The results are analyzed by the intention to treat principle that includes all eligible subjects who were randomized to treatment groups regardless drop out or loss to follow up status.
7. The Food and Drug Administration Amendments Act of 2007 that required registration of clinical trials was not enacted until Sept. 27, 2007 (PUBLIC LAW 110–85—SEPT. 27, 2007). It required trials that were ongoing in September 2007 or which began after 2007 to be registered. This trial began in 1998 and ended in June 2007. Thus, registration was not required by the NIH nor was it required by the International Committee of Medical Journal Editors at the time. As stated in their guidelines, ICMJE journals will accept ‘retrospective registration’ of trials that began before July 1, 2005…”. (http://www.icmje.org/faq_clinical.html).
8. Regarding the distribution of fatal and nonfatal events, the methods-outcomes section, pg. 2 describes that, “All clinical end points were adjudicated by a blinded, independent reviewer who applied standardized and validated criteria (ALLHAT, 2002)”. As reported in the acknowledgments, the events adjudicator was at an outside university. It may be that when the sample size approaches the population size, then the observed distribution of events in the sample approaches the population distribution. This general consideration is noted in the discussion-limitations section, pg. 7.
9. In the discussion section, pg. 6, it is proposed that the significant net differences in BP and anger expression may plausibly contribute to lower event rates. However, it is also explained that this magnitude of BP change is with a 15% reduction in CVD clinical events. Therefore, “it is possible that other mechanisms not evaluated in this study contributed to the reduced risk in the TM group”. Moreover, “previous studies have reported reductions in sympathetic nervous system tone, hypothalamic-pituitary-adrenal axis activation, insulin resistance, left ventricular mass, myocardial ischemia, carotid atherosclerosis and heart failure” (Barnes and Orme-Johnson, 2012). It is likely that a set of changes in physiological mechanisms and intermediate outcomes synergistically contributed to the changes in event-free survival.
10. The description of the power analysis included sample size, event rates and duration of follow-up and is provided in methods, pg. 3. “Power calculations were based on the approach of Proschan and Hunsberger for conditional power (1995 ).” “With the review and approval of the data and safety monitoring board, a single interim analysis determined that with 201 subjects……the trial had 80% power to detect a 50% risk reduction in the DSMB approved end point of all-cause mortality, nonfatal MI, and nonfatal stroke.”
11. There was only one primary analysis. Other analysis of survival were clearly labeled as secondary or sensitivity analysis. One does not adjust for multiple comparisons of one pre-specified analysis.
12. The significance for the primary outcome met pre-specified criteria (p <.05). According to Drs. Sanjay Kaul and George Diamond (JACC 2010), “In clinical trials, investigators pre-specify a significance level (most commonly 0.05)…” . Kaul et al then refer to Panagiotakos (2008) when they write, “because p values are dependent on sample size, a p value of 0.001 should not be interpreted as providing more support for rejecting the null hypothesis than one of 0.05.” Furthermore in this trial, the single, primary analysis was confirmed by a series of secondary/sensitivity analysis that met the pre-specified significance requirements (table 3).
13. Kaul and Diamond (2010) point out that clinical significance in a trial may be considered as important as statistical significance. The effect size on the primary outcome (48% risk reduction) in the current trial is similar to or greater than conventional drug therapies used in secondary prevention of cardiovascular disease, e.g. statins and anti-hypertensive medications (Smith 2011). Notwithstanding that the latter have been subjected to larger, phase III trials. Thus, the current findings indicate clinical as well as statistical significance. This is robust.
Robert Schneider, MD, FACC
Institute for Natural Medicine and Prevention