The open data movement has gained its biggest victory yet.
In an editorial published simultaneously in 14 major medical journals today, The International Committee of Medical Journal Editors (ICMJE) delivered a full-throated endorsement, in both actions as well as words, of data sharing by clinical researchers. Here is the core of the ICMJE proposal:
As a condition of consideration for publication of a clinical trial report in our member journals, ICMJE proposes to require authors to share with others the deidentified individual patient data (IPD) underlying the results presented in the article (including tables, figures, and appendices or supplementary material) no later than six months after publication.
Harlan Krumholz, principal investigator of the Yale University Open Data Access (YODA) Project expressed considerable enthusiasm for the development:
This is a remarkable day; the medical editors showed leadership in demonstrating that we medical scientists must put the interests of society first. There are a lot of details to work out – but they have given substantial impetus to the open science movement and to the efforts to have medical research be transparent. If this proposal is accepted, then the medical science world will be forever transformed for the better.
The proposal will also require potential authors to include data sharing plans at the time of trial registration. (Current ICMJE regulations require trial registration before the beginning of patient enrollment.) The ICMJE is inviting comments on the proposal through April 18. Implementation of the rules would go into effect one year after the rule is adopted.
Truth and Consequences
Several experts (contacted by Roger Sergel at MedPage Today) pointed out some possible limitations or unintended consequences of the ICMJE plan.
Zaven Khachaturian, editor-in-chief of Alzheimer’s & Dementia: Journal of the Alzheimer’s Association,
predicted that the rule “very likely will have the opposite effect that the ICMJE expects, it will delay the publication of clinical trials results; in some cases after registration and/or marketing of a drug.”
Milton Packer ((Baylor University, Dallas) agreed in part with Khachaturian, but made the perhaps counter-intuitive point that the rule won’t discourage publication of industry-sponsored trials but might hinder non-industry sponsored trials:
I strongly endorse the newly proposed ICMJE rule on data-sharing of clinical trials. It will NOT discourage publication of industry-sponsored clinical trials, since these undergo such careful quality control. However, I suspect that this new policy may discourage publication of NON-industry sponsored clinical trials, which are generally not audited and do not have quality-controlled databases and whose analyses are not subject to review by regulatory agencies.
Finally, I contacted Ben Goldacre (Oxford University and AllTrials), who said the proposal is “excellent news.” But Goldacre also wanted to emphasize that the new rule will not solve a much bigger problem involving missing data, because it does not and can not address the problem of trial results that go unpublished:
While all progress on all fronts is good news, I am somewhat concerned by the widespread emphasis on IPD (individual patient data) from this and other initiatives, many of which (such as ClinicalStudyDataRequest) have significant budgets. This emphasis may come at the expense of initiatives to improve availability of overall study results, which is where the real demand is.
At AllTrials we have recently conducted an overview of applications for IPD access. The key finding is: “very few applications”. We are hopefully working with Cochrane to survey systematic review authors to find out why they are not requesting such data, but I suspect the answers will be a combination of “too expensive and time-consuming to work with”, “it doesn’t add enough to summary results to justify the extra resource”, and “we didn’t know it was there”.
It will be good to see IPD being shared and used more. However, tens of thousands of researchers are clamouring for access to summary results, while there have only been a couple of hundred requests for IPD. It is perhaps noteworthy that IPD can only be shared after review of applications, and to small numbers of individuals, after permission is granted by the company, because of the very real privacy concerns around sharing pseudonymised individual patient data online. Sharing summary results is simple: they just need to be placed online.
Overall, therefore, while greater support for IPD sharing is definitely great news, it’s very odd that all this resource and effort is now going into IPD sharing, when IPD sharing is expensive and not in demand, with complex governance and access control issues, whereas summary results sharing presents no privacy issues, and has been highly in demand, for many decades, but with multiple ongoing barriers to access, and frustratingly slow progress.