Mandatory registration of clinical trials was supposed to prevent some troubling abuses of the research system. Two new studies published today suggest that many of these abuses continue.
In the first study, published in JAMA, French investigators found significant discrepancies when they examined registered trials with their published primary outcomes. They concentrated their search in 3 medical areas (cardiology, rheumatology, and gastroenterology) and ten top-rated journals, including Circulation, Circulation Research, and the Journal of the American College of Cardiology.
They found that less than half the trials (45.5%) were adequately registered, which they defined as being registered before the end of the trial and with the primary outcome clearly specified. More than a quarter of the trials were not registered at all, while 13.9% were registered after the study was finished. In the trials that were adequately registered, the French investigators found “discrepancies between the outcomes registered and the outcomes published.” In the papers where they were able to assess the discrepancies, the investigators found that “statistically significant results were favored in 82.6%.”
“A main goal of trial registration is to enhance transparency of research and accountability in the planning, conduct, and reporting of clinical trials, an objective achieved by making available details about the trial,” they write. “Therefore, adequate registration should be a safeguard against publication bias.”
A major step has been achieved with the ICJME initiative for trial registration, and the existence of all trials is now publicly available. However, after this first step, the quality and timing of registration still needs improvement. To obtain the full benefit of clinical trial registration, the active participation of different stakeholders is needed. First, the sponsor and principal investigator should ensure that the trial details are registered before enrolling participants. Second, the comprehensiveness of the registration should be routinely checked by editors and readers, especially regarding the adequate reporting of important items such as the primary outcome. Third, editors and, possibly, peer reviewers should systematically check the consistency between the registered protocol and the submitted manuscript to identify any discrepancies and, if necessary, require explanations from the authors. Finally, the goal of trial registration could be to make available and visible information about the existence and design of any trial and give full access to all trial protocols and the main trial results.
In the second study, appearing in PLoS Medicine, US researchers, including Steve Nissen and senior author Harlan Krumholz, looked at a selection of trials published on ClinicalTrials.Gov and found a high rate of trials completed for at least two years without being published. Industry-sponsored trials were the least likely to be published.
Harlan Krumholz commented on the significance of both papers for CardioBrief:
These papers, in tandem, highlight the shortcomings of the national registry of randomized trials. Much important information is missing from the voluntary registry, wrong information is included in the registry, and many studies remain unpublished even years after they are completed. The findings raise concerns about the quality and completeness of the medical literature.
These studies reveal a major flaw in the medical literature. These articles are a clarion call to action: if we cannot trust the literature – that it is complete and accurate – at least for trials important enough to be registered – then we have a problem of enormous proportions.
Note to readers:
Due to an error in communication, the PLoS Medicine paper discussed in this post was not available on the PLoS Medicine website at the time this story was published. We regret the error. The PLoS paper is now available at this link.
PLoS Medicine has now issued a press release on this article:
New analysis reveals missing information on clinical trials in major international registry
Important information about completed and ongoing clinical trials is missing from the ClinicalTrials.gov, a web-based registry of clinical trials conducted worldwide, according to a research article by Joseph Ross (Mount Sinai School of Medicine and the James J. Peters VA Medical Center, New York, USA) and colleagues. The work, published this week in the open access journal PLoS Medicine, also shows that for many of the completed clinical trials registered at ClinicalTrials.gov results are not available in the medical literature.
Clinicaltrials.gov was established in 2000 following the U.S 1997 FDA Modernization Act was created to provide public access to information on clinical trials; it provides a registry of both federally and privately funded trials. Ross and colleagues sampled a large cross-section of trials (7,515 of over 77,000 trials held on ClinicalTrials.gov) registered after 31 December 1999 and completed by 8 June 2007. All trials reported the essential information required by ClinicalTrials.gov, but optional data—including information such as the primary outcome of the trial and start and end dates—were less complete, report the authors. A random 10% subsample of the trials (677 in total) revealed that less than half were eventually published in the medical literature, and for only a third of these was the publication linked back to the ClinicalTrials.gov registry record.
Selective publication of trials (a preference towards publishing results of studies with a “positive” outcome) can distort the evidence available for clinical decision making. Policy statements— such as the World Medical Association’s Declaration of Helsinki—declare that researchers have a duty to publish the results of all research on human subjects, including both negative and inconclusive studies as well as positive studies.
The authors argue that “without greater attention to reporting of all data elements, the potential for ClinicalTrials.gov to address selective publication of clinical trials will be limited.” Accurate and up-to-date trial information helps researchers, clinicians and patients to access all of the data available and prevents trial results from being hidden, say the authors. They argue that open access to trial data would encourage publication of all results, both positive and negative, from completed trials.
Funding: Joseph Ross is funded by the National Institute of Aging and by the Paul B. Beeson Career Development Award Program from the American Federation of Aging Research, and has received support from the Department of Veteran Affairs Health Services Research and Development Service and the Hartford Foundation during the early stages of the work. The funders had no role in study design, data collection and analysis, or preparation of the manuscript.
Competing interests: Joseph Ross, Gregory Mulvey and Harlan Krumholz were consultants at the request of plaintiffs in litigation against Merck and Co., related to rofecoxib. Steven Nissen has received support for clinical trials through the Cleveland Clinic Coordinating Center for Clinical Research from Pfizer, Astra Zeneca, Daiichi-Sankyo, Takeda, Roche, Novartis, Sanofi-Aventis, and Eli Lilly. Steven Nissen also consults for Novartis, Anylam, Sanofi-Aventis, Astra Zeneca, Forbes Meditech, Ganedon, Genzyme, Hollis-Eden, Eli Lilly, Pfizer, Karo Bio, Novo Nordisk, Roche, Daiichi-Sankyo, Takeda, Resverlogix, and Glaxo-Smith-Kline, but requires them to donate all honoraria or consulting fees directly to charity so that he receives neither income nor a tax deduction. Harlan Krumholz has research contracts with the American College of Cardiology and the Colorado Foundation for Medical Care, has served on the advisory boards of Amgen and UnitedHealthcare, has been a subject expert for VHA,and is academic editor-in-Chief of “Circulation: Cardiovascular Quality and Outcomes”, and “Journal Watch Cardiology” of the Massachusetts Medical Society.
Citation: Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009) Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis. PLoS Med 6(9): e1000144. doi:10.1371/journal.pmed.1000144