Novel biomarkers are the subject of intense controversy, with a bewildering variety of factions and perspectives seeking to elevate or dismiss any of a large number of proposed new measures. Now a new examination of the literature published online in JAMA Internal Medicine suggests that the evidence base used to evaluate novel biomarkers may be seriously compromised by selective reporting bias.
John Ioannidis led a team of researchers who analyzed 56 meta-analyses of new candidate cardiovascular biomarkers. 49 of the studies had statistically significant results, but 9 studies were compromised by very large heterogeneity, 13 studies were compromised by small-study effects, and 29 studies had an excess of studies with statistically significant results. Only 13 studies had more than 1,000 cases, achieved statistical significance, and had none of the other deficiencies listed above. The meta-analyses that emerged unscathed examined the associations of glomerular filtration rate and albumin to creatinine ratio in general and high-risk populations with cardiovascular disease mortality and of non–high-density lipoprotein cholesterol, serum albumin, Chlamydia pneumoniae IgG, glycosylated hemoglobin, nonfasting insulin, apolipoprotein B/AI ratio, erythrocyte sedimentation rate, and lipoprotein- associated phospholipase mass or activity with coronary heart disease.
The authors summarized their finding as suggesting that “the effect of biomarkers is exaggerated because the largest studies— which one would expect to produce the most stable estimates— consistently showed smaller effects. In most meta-analyses, too many single studies had reported ‘positive’ results compared with what would be expected on the basis of the results of the largest studies. This suggests that small studies with ‘negative’ results remain unpublished or that their results are distorted during analysis and reporting to seem more prominent.”
In an invited commentary, Steve Nissen writes that evidence-based medicine has been put on a “golden pedestal” but publication basis “is a dark secret that corrupts nearly every aspect of our profession and undermines societal efforts to promote evidence-based medicine.” He cites carotid intima-medial thickness and apolipoprotein B as just two biomarkers in which “the magnitude of the association is probably much smaller than suggested by the definitive meta-analysis.”
Nissen urges investigators to register their studies with ClinicalTrials.Gov, but points out that the site does not support large data sets. “Therefore,” he argues, “society must consider funding the National Library of Medicine to create a public website where authors can post the detailed results of findings that they were unable to publish despite submitting to multiple journals. Finally, we must emphasize to colleagues and trainees that all studies contribute to scientific understanding. We have a moral obligation to our patients to make all research findings available to the broader scientific community.”
