The Impact Of Invisible Data: Analyzing Unreported Clinical Trials
Executive Summary
Despite the scientific, social and ethical implications of withholding clinical trial data, which have been widely debated in the public arena, many studies remain unreported according to data from the AllTrials initiative and Trialtrove.
Despite improvements in the way industry clinical trial results are communicated to professional practice and the patient community, significant gaps remain in making the transition to a state of full and transparent disclosure, in the interest of public health. Implications of the status quo are well known: the withholding of trial data for new therapeutic candidates can result in publication and reporting bias, thus skewing the evidence available to decision makers.
Biopharma companies interviewed by In Vivo agree that transparency of data and the routine publication of results contribute to improvements in R&D efficiency and accelerate approval of new medicines for patients. “It could save billions in duplication of trials that were bound never to succeed. Critically, it could also reduce the number of patients enrolled in futile trials, avoiding a lot of false hope and missed opportunity,” says Jonathan Tobin, investment director at Arix Bioscience.
But is there a consensus on the actual state of the underreporting problem? Viewpoints clearly differ, so In Vivo decided to investigate the issue, relying on a data pool drawn from both our own internal Informa Pharma Intelligence sources – including Trialtrove – as well as a repository of data compiled by the AllTrials campaign. AllTrials is an advocacy coalition comprising the prominent industry skeptic Dr. Ben Goldacre, the British Medical Journal, and the global Cochrane evidence-based medicine alliance, along with more than 700 other participating sponsors – including GlaxoSmithKline PLC – but drawn mainly from the NGO advocacy community. Using sourced information from trials that are late in reporting data, In Vivo has summarized trends in the data gaps of clinical trial reporting.
While the World Medical Association’s Helsinki Declaration, the pre-eminent global ethical standard for clinical trials, affirms that all trials should report results, new evidence from AllTrials suggests around 37% of studies go unreported. Furthermore, the declaration states that every investigator running a clinical trial should register it and report its results. More important, the 2007 FDA Amendments Act (FDAAA) states that applicable clinical trials on US registries due after January 2018 must report their results within 12 months of completion. The mandate is fairly comprehensive, with exceptions only for Phase I trials or trials that do not include drug, biological or device products, such as behavioral interventions.
Despite these guidelines, progress in moving to full reporting remains sketchy. While the US Government and the FDA now have the legal power to impose substantial fines of up to $200 million, they are not publicly tracking compliance and no fines have been imposed by the government or the agency to date.
Informa Pharma Intelligence has analyzed its own data sets on unreported trials to determine whether there are design or operational features that regularly occur in clinical studies, leading to missing data readouts. Data collected by the Evidence Based Medicine Data Lab at the University of Oxford, combined with Trialtrove, the industry’s most comprehensive, accurate and up-to-date source of pharmaceutical clinical trials data, allows us to present a more comprehensive picture of the extent of the unreported trials problem.
As of June 2018, 300 of 806 (37.2%) trials covered by the FDAAA 2007 regulation were unreported. Compared with the comprehensive database of all clinical trials within Trialtrove, unreported clinical trials are more likely to be those in the Phase II stage of development (60% versus 40%), and less likely to be the opposite end of the spectrum, Phase IV (18% versus 35%). But with such a high proportion of unreported trials being in Phase II, it remains unclear whether the compound of interest remains in active development for the indication of interest (see Exhibit 1). Furthermore, it is unclear whether reporting of any side effects and adverse events have been shared with the wider research and development community.
Unreported Clinical Trials By Development Stage
This analysis also reveals there is a small, but potentially important difference in sponsor type (see Exhibit 2). A higher prevalence of unreported clinical trials is backed by a review of data from the ClinicalTrials.gov site (20% versus 10% for all trials), and a lower prevalence is sponsored by academic institutions (33% versus 47%). The roles of responsible parties, be they clinical trial sponsors or principal investigators, in publishing clinical trial results, and the potential explanations or justifications made by different types of trial sponsors for their non-publication of trials results is not clear in the current data held on unreported studies – and is worthy of further investigation.
Unreported Clinical Trials By Sponsor Type
There were no major differences in trials by therapeutic area or underlying disease (see Exhibit 3). However, disrupting these data, the highest proportion of trials are not readily classified into a therapeutic area, and are instead assigned as “N/A” in Trialtrove (30%). These trials include those that fall out of scope of the in-depth Trialtrove coverage but have been captured from the ClinicalTrials.gov site.
Unreported Clinical Trials By Therapy Area
Unreported clinical trials are more likely to be in Phase II according to data from the FDAA TrialsTracker, which is also sponsored by AllTrials and uses the FDAAA 2007 law to determine its listings. But it is not clear from unreported trial information if risk factors can be deduced based on therapeutic area of design.
The conclusion to be drawn from this evidence is relatively straightforward. It remains the duty of all clinical trial sponsors, principal investigators and managers, to ensure all available data, whether seemingly “important” or indeed “uninteresting,” are available to support ongoing clinical decisions. Trial results should include both the intended and unintended effects of healthcare interventions, and inaccessible results may lead to bad treatment decisions, missed opportunities for good medicine, and trials being repeated unnecessarily, alongside their financial, logistical, ethical, and medical burden. Aside from the perceived negative implications of non-reporting, full cross-sector data transparency also has benefits in itself.
This is acknowledged by many in the biopharma industry itself, with a spokesperson from Bayer AG commenting, “We believe that increased transparency while maintaining patient privacy will ultimately encourage innovation and benefit patients.”
However, establishing ground rules that build on this rough consensus remains a stumbling block. There is a vital requirement for further work on how regulations and protocols requiring that data be publicized might be enforced, and how buy-in from the research and development community may be achieved. “It takes the leadership of influential people in a handful of the most successful biopharma companies to set the example, and then others will follow,” Tobin says. For example, GSK proved instrumental a decade ago in persuading the rest of the industry to approach clinical trial reporting and transparency in a joint series of reform initiatives.
The requirements for a joined-up approach between academia, biopharma and publishers, is also under scrutiny. Annalisa Jenkins, CEO of UK biotech PlaqueTec, comments that academia and biopharma must work together to confirm “consistent standards and approaches to reporting of data to ensure an increase in the level of trust the public has on our collective enterprise.” She adds that academic journals must embrace their role in publishing the totality of data, including negative studies. “Academia has often failed to keep up with its societal obligation to actively seek peer reviewed publication of negative results,” says Jenkins.
The responsibility to pursue full data transparency and trial publication lies across the entire sector, and while evidently high on the agenda for many public and private entities, change may require the leadership of a select few of the most successful biopharma companies to set the example, and then others will follow. If the system is to truly benefit from the wealth of data and insights available from clinical trials – good and bad – all players must be held to the same strong standards of disclosure. That is a competitive necessity as scientific progress provides more therapeutic options for companies and their patients.
[Editor's note: The article was edited on September 3, 2018 to better define what "N/A" refers to in the TrialTrove database.]