It’s time for clinical research to join other scientific fields like physics and computer science and encourage preprint publication of manuscripts on the internet, according to three top cardiologists writing in a Lancet comment.
Since 1969 researchers have labored under the constraints of the Ingelfinger rule, in which “posting a paper, data, or key findings on the internet represented presubmission publication and would disqualify it” from publication in a major journal, write the authors of the comment, Michael Lauer (NIH), Harlan Krumholz (Yale), and Eric J Topol (Scripps Translational Science Institute). They cite several advantages for preprints, most notably the speedier delivery of new information to interested readers.
The proposal also appears to respond to some of the many recent well-publicized episodes in which peer review has failed. They note that “preprints allow for more robust peer review, as peer review is not limited to the 2–4 reviewers chosen by one journal editor; instead, any number of interested readers can offer comments and suggestions.”
The authors also write that the traditional form of peer review and publication may represent “a kind of paternalism, which is troubling when many are calling for a democratization of medicine and for widespread adoption of open science.”
One of the authors, Harlan Krumholz, said that “it’s time for us to consider new models of sharing results in a timely and interactive way. Other fields are moving quickly and finding it useful. Medicine has a special set of considerations, including the need to protect the interests of patients, but we also have an obligation to report completely and rapidly. It is a challenge to see if we can do that well.”
The Empire Strikes Back
I asked several editors of traditional journals for their response to the proposal and received thoughtful responses from the New England Journal of Medicine and the editors of Heart, the American Heart Journal, and the Journal of the American College of Cardiology:
New England Journal of Medicine:
We can only speak for NEJM. The spirit of our guidelines is not to obstruct the scientific communication of research but to prevent inappropriate conclusions and recommendations from reaching physicians and their patients before peer review and expert editing can correct them. When a study is of public health importance, we expedite its publication process, getting it through peer review and editing in a matter of days. We complete this thorough process without compromising its rigor. There is a lot of information of varying quality out there. We think the public and patients are best served by the publication of studies that have been carefully reviewed and edited by experts in the field.
Catherine Otto (University of Washington), editor-in-chief of Heart:
We all agree that medical researchers are responsible for prompt reporting and dissemination of new clinical information; the question is how best to accomplish this goal. The goals of pre-publication peer review by medical journals are to ensure that the published paper includes all the essential information, the data presentation and statistical analysis are appropriate and the findings justify the author’s conclusions. A key step in this process is selection of peer reviewers who are expert in the topic and expected to provide an unbiased review; it is not clear to me that this will happen with “open” online posting of manuscripts. It would be a disservice to the public to end up with more than one published version of a clinical research study given the implications for patient care. We already have many forums for early presentation of research results, for example at scientific meetings, which offer an opportunity for comment and discussion. If the journal peer review process is too slow, we need to make it faster, not abandon it.
Dan Mark, editor-in-chief of The American Heart Journal:
Is it useful to ask why a commentary encouraging us to adopt a prepublication culture is being published in a journal that imposes an embargo? The answer, of course, is that the authors wish to have their message as widely read and discussed as possible, so they chose a high impact journal rather than publishing on some internet site with a twitter link to direct reader traffic to it. This is exactly what also happens with the most important papers from the major clinical trials. The Ingelfinger rule use is largely restricted I believe to the most prestigious journals, such as Lancet, JAMA and NEJM. It is part of the theater those journals use to enhance the experience of being selected by the “best of the best”. When you spend years doing an important clinical trial and finally reach the end, the NIH, the investigators, the leadership of the trial, the commercial sponsors, the press, perhaps even the public, all expect a paper in a high impact journal. The reasons for this are complex and beyond the scope of a simple comment. But we in cardiovascular medicine are in a very different situation from physics, chemistry or statistics, fields where internet sites for posting unpublished white papers are well accepted.
The issue of getting more results from important clinical research published/available more quickly is a much deeper issue, and I do not think that prepublication will really have much impact on that, even if we can get past our desire for a prestigious publication whenever possible. Every part of the process of doing clinical trials and other clinical research involves tradeoffs, often related to funding. In addition, figuring out what the data say is not such an easy thing, even if there is a detailed protocol and statistical analysis plan to “guide” the process. The debate at AHA last month on how to interpret SPRINT is a great example of just how complex a process it is. Huge, important, expensive trial, with everything prespecified, and the “experts” are all over the map on how to use the results clinically.
I would also note that for journals that do not worry about embargoes and such, it is typical now for a paper that is accepted by the Editorial Office to be “published” online and available to readers even before final editing and proofs are created. Of course, that is post-peer review, not pre-. But the point is that with web publication the norm now, it is possible to have a paper go from initial submission to accepted and available online in weeks in very high priority situations.
If we rebuilt the research peer review and publication system from the ground up, we would certainly make something very different from what we now have. But that is a topic for another time, I think.
Valentin Fuster (Mt. Sinai), editor-in-chief of the Journal of the American College of Cardiology:
–Cardiovascular medicine and research is different than astronomy, mathematics, and computer science (examples from the Lancet piece), because we are dealing with patients and people’s lives.
- SPRINT trial, as an example, the trial was stopped on 08.20.2015, and it only took 82 days for publication in the NEJM. In that time, they had to inform all the patients, gather all the data from the participating sites, adjudicate the data and write the publication.
–Who will benefit from the release of such raw data earlier: Researchers like the authors of this editorial, who can truly sift through and disseminate it for their own research needs. The public would not benefit because they will not understand the data and the practicing clinician/cardiologist does not have the time to sift through the raw data.
–The suggestion that peer review process is in some way prohibited transparency is inaccurate, because the goal of peer-review journals is to help the practicing clinician with their pressing needs, not confuse them with an overwhelming amount of information that they will never read.