A Guide To The Raging Debate Over The NIH’s TACT Chelation Trial 7

(Updated) The publication in JAMA of the NIH’s Trial To Assess Chelation Therapy (TACT) trial has provoked a fascinating debate in the blogosphere. The vast majority of responsible physicians and healthcare professionals have little interest in chelation therapy per se, but the TACT trial has raised many important questions about the nature of medical evidence. Here’s a brief guide with links to some of the more interesting discussions (let me know if you are aware of other worthwhile discussions):

In the first of two accompanying editorials, the JAMA editors discuss some of the complex issues relating to TACT and explain why they decided to publish the TACT paper.

In a second accompanying editorial, Steve Nissen, while agreeing in principle with the idea that randomized controlled trials should be published, argues that the TACT investigators “fell short of the minimum level of quality necessary to adequately answer the question they sought to investigate.”

TACT investigator Daniel Mark provided CardioBrief with a  detailed response to Nissen’s criticism. (Nissen declined to respond to Mark.)

Harlan Krumholz, in a blog post on Forbes, asked “what to do with inconvenient evidence”? He agrees with just about everyone else that TACT does not provide a reason to administer chelation therapy, but he argues forcefully that the results of the trial should not be simply dismissed.

Responding to Krumholz, Peter Lipson, also on Forbes, takes a position similar to Nissen’s that the trial was poorly performed and the results are highly questionable. The important things here are the responses to Lipson from Krumholz himself, Forbes pharma reporter Matt Herper, and Sanjay Kaul.

The most sustained assault on TACT, and on Krumholz’s position, comes from the highly-regarded skeptic blog Respectful Insolence written by Orac (the pseudonym of David Gorski, a surgical oncologist). In his take-no-prisoners assault on TACT, JAMA, and Krumholz, Orac writes “that JAMA is every bit as guilty as The Lancet was in 1998 when it published Andrew Wakefield’s antivaccine nonsense…. If published at all, TACT should have been published in some crappy, bottom-feeding journal, because that’s all that it deserves.” The comments section includes worthwhile exchanges between Orac and Sanjay Kaul and Matt Herper.

Finally (for now), Sanjay Kaul today summarized his defense of TACT (though he does not, of course, endorse chelation) in a blog post on CardioExchange. “Bottom line,” he writes, “in my opinion, the arguments that the TACT results are dubious or not valid are overstated. While the debate surrounding TACT is clearly warranted and welcome, I hope it generates more light than heat.”

Update:

Responding to the attacks on TACT from Orac and other members of the skeptical community, TACT investigator Dan Mark sent me the following comment on email, which he has agreed to share with my readers. The comment moves the debate in an entirely new and philosophical direction:

Although skepticism has an important role to play in critical debates, it is easy to overplay that hand. The people you mention seem to have a very naïve view of science, very far removed from the messy realities of daily work of people doing science. It is also important to remember that even the most hard core scientists can have some pretty eccentric views when they venture outside their narrow field of expertise. What does that imply about science and the people who wish to guard its borders?

There has been a project in philosophy to identify firm demarcation criteria that will allow for a distinction between science and pseudoscience. While some useful work has resulted, the overall attempt failed. Gets into some deep waters, but the harder the philosophers tried to find that electrified fence that marked off “real science” from the rest of human thought, the more they undermined the borders of science itself. Interestingly, “real scientists” rarely worry about whether they are doing science. They consider the question uninteresting, leaving it for the philosophers, sociologists and (now) the bloggers!

About these ads

‘Bias in choosing the question is a much bigger issue than lying about the data’ Reply

Robert Califf takes questions about conflict of interest from Harlan Krumholz. Here are a few choice quotes. Read the entire Q&A on CardioExchange.

…focusing exclusively on the medical products industry and failing to consider other sources of conflict of interest is a huge mistake and leads to sensationalism that then engenders reactive rules that add to bureaucracy rather than addressing the real issues.

…a despotic, egotistical principal investigator can be just as dangerous as a controlling funding organization. Like our constitution, trials need a balance of power…

Bias in choosing the question is a much bigger issue than lying about the data.

…lying about the data is rare. Lying by omission (not looking at aspects of the data) is much more common.

Missing Data: The Elephant That’s Not in the Room (Guest Post) 2

Editor’s Note: The following guest post by Harlan Krumholz is reprinted with permission from CardioExchange, the cardiology social media website published by the New England Journal of Medicine.

Missing Data: The Elephant That’s Not in the Room

by Harlan M. Krumholz, MD, SM

There is a problem so grave that it threatens the very validity of what we learn from the medical literature. Bad data? Not exactly. Actually, it’s missing data — information, relevant to the risks and benefits of treatments, that is simply not published. In some cases, these data would make a critical difference in the inferences that readers draw from the literature. The absence of the data renders meta-analyses, systematic reviews, and book chapters suspect. Conclusions are made on the basis of incomplete science. In short, publication bias and selective publication are impugning the validity of what we can learn from a PubMed search or even the most careful review of published studies.

This matter demands our immediate attention and speaks to the need to rethink the configuration of clinical medical science. It may be time to adopt strategies to ensure that all relevant studies, results, and supporting documentation are made publicly available. “Out of sight, out of mind” is a dangerous reality in science and medicine. It’s time for a change — and it starts with the recognition that we have a problem.

I urge you to read BMJ this week to explore the evidence of this problem. In full disclosure, the studies include one by me (with others, led by Joe Ross) showing that more than half of trials sponsored by the NIH go unpublished even 30 months after completion. The other articles reveal troubling information, including about how missing data can affect the results of meta-analyses — and how many investigators are ignoring the requirements for mandatory reporting of trial results, raising the question of what “mandatory” actually means.
Click to continue reading…