Pharma Told To Get It Right First Time On Clinical Trial Data

Drug companies are complying with their obligation to submit clinical trial results to the US-based ClinicalTrials.gov database, but major errors in their submissions prevent the results from being uploaded in a timely manner.

The debate around disclosure and clinical trial results never stops. In recent years, regulators around the world have launched their own transparency initiatives, supported with legislative changes to ensure compliance.

In the EU, the upcoming Clinical Trials Regulation (536/2014) will make it mandatory for companies to provide layperson summaries of their clinical trial results so that these can be made public. In addition, the European Medicines Agency plans to examine late next year how and when it can revive its landmark policy on proactive publication of clinical study reports. The policy is on hold as the EMA focuses on the huge job of preparing for Brexit.

Earlier this year, the US Food and Drug Administration launched a pilot on publishing clinical study reports. Johnson & Johnson's prostate cancer drug, Erleada (apalutamide), was the first medicine about which this information was made public. Canada too has drawn up regulations to publicly disclose clinical data contained in new drug submissions and medical device applications.

The initiatives vary to such a degree that a company could find itself having to create multiple different versions of its clinical study report because of the differences in regulatory policies for publishing clinical data.

Although most pharmaceutical companies have created separate departments to deal with disclosure-related requirements, it sometimes takes more to get the job done. This article illustrates how companies are struggling with the US requirement to disclose even basic-level data on clinical trial results.

Drug companies often inadvertently make major errors in their submissions of clinical trial results to the US-based registry, ClinicalTrials.gov, and they lack clear processes for resolving their mistakes quickly.

The National Institutes of Health, which maintains ClinicalTrials.gov, recently started sharing data with individual companies on the types of major errors commonly found in their CT results submissions. For some companies, such as GlaxoSmithKline PLC, this has been a real eye-opener as they had no idea that they were faring so poorly on this front.

Several companies use ClinicalTrials.gov as their "default register" to publicly disclose study results as part of their transparency commitments, and so a direct consequence is that the public disclosure of CT results is delayed.

All CT results submitted to ClinicalTrials.gov undergo strict quality review before they can be posted on to the public site and they can face multiple cycles of rejection unless all major issues and inconsistencies are addressed, according to Heather Dobbins, lead results analyst at ClinicalTrials.gov.

Dobbins explained at a recent Drug Information Association conference that the aim of the quality review process is not to check if the data submitted "is of the highest quality possible," but "simply to ensure that it is free of apparent errors, deficiencies and inconsistencies" and generally makes sense.

Dobbins recalled that last year a sponsor had submitted the results of a study where the outcome measure looked at the change in total sleep time of subjects over several weeks. In the study description, the sponsor specifically mentioned that the data was being "presented in minutes since predicting hourly data in decimals is tedious.” However, the "unit of measure" was listed as "hours" and "it was reported that there was a mean change of 87 hours per night, which simply can't be correct," she said. The NIH returned the submission to the sponsor as the "unit of measure" was not consistent with information in the measure title/study description.

Dobbins was speaking at the DIA's Global Clinical Trials Transparency Conference, which took place in London on September 19-20.

Companies have been submitting results data to ClinicalTrials.gov since September 2008 when it became mandatory under the FDA Amendments Act of 2007. The legislation requires sponsors to submit basic-level information on trial results, namely participant flow, baseline characteristics, outcome measures and statistical analyses, and adverse events.

The results database is now finally mature enough to offer insights on companies' performance with respect to their submissions, Dobbins told In Vivo. The NIH has been sharing these metrics since 2017 to help companies prepare error-free submissions that can be accepted in the first round. This will help reduce the workload both for the NIH and sponsors.

The situation has improved somewhat since then, but major errors are still being detected, Dobbins said. In March 2018, for example, a sponsor submitted study results on the percentage of participants meeting a certain response criterion. Both the study description and the "unit of measure" stated that the results were reporting a "percentage of participants," but the company ended up reporting a response by 121%, 133% and 111% of participants. "These are the types of major issues that our quality control review criteria are designed to catch," Dobbins said. The submission was rejected as the information within the measure was inconsistent.

For some companies, the NIH metrics have come as a major surprise. GSK, for example, held a meeting with NIH representatives on this issue in May 2018. Staff were "really taken aback" to learn that only 28.9% of the company’s 470 CT results submitted between January 2017 and April 2018 were accepted in the first round, said Smita Shukla, director of data sharing and disclosure at GSK.

All the remaining submissions were returned to GSK due to major errors that rendered the data incomprehensible or inconsistent. GSK resubmitted the results after addressing the NIH's concerns; 63.5% of its 189 submissions were accepted in the second round, while 61.4% of its 57 records were accepted after undergoing more than two cycles of review.

GSK's senior management has dubbed the issue as a risk and is following the matter very closely. "We have to provide updates on a quarterly basis to our senior board about how we are doing – so the pressure is on," said Shukla.

Since its meeting with NIH representatives in May, GSK has taken several steps to improve its internal processes and has set for itself a "somewhat lofty" goal of ensuring that 80% of all its CT results submitted to ClinicalTrials.gov get first-time acceptance in 2019, said Shukla. These process improvements, she said, have helped improve the company's first-time acceptance rate to 61% based on data from 75 submissions reviewed between May 15 and September 5 this year.

Although GSK has been submitting CT results to ClinicalTrials.gov since September 2008, Shukla explained that there was no emphasis on improving first-time acceptance rates until now. The NIH, for its part, is issuing updated lists of most common major errors found in CT results submissions. Between Jan. 2017 and April 2018, the most common deficiencies in initial submissions were:

• Missing scale information. ClinicalTrials.gov requires the trial results information to be presented in accordance with a scale so that the data are interpretable to anyone looking at them.

• Unit of measure does not match measure title/description.

• Analysis population is not consistent throughout the record. The number of participants analyzed changes; they can go up and it is not clear where these additional trial participants come from and there is no explanation provided, explained Dobbins.

• Invalid unit of measure.

• Data not presented for each study arm separately.

• Inconsistent data within a measure.

The NIH is sharing tailored metrics with drug companies on their individual performance. For Dobbins, the "most surprising thing" is that several companies fail to address the major errors highlighted in the first review cycle and re-submit the data and it again gets rejected for the very same reasons.

Between January 2017 and April 2018 for previously rejected submissions, more that 80% of the records had to be rejected again as earlier comments had not been addressed. "That's something that I don't understand … and we would like to look more into it," Dobbins said.

On the issue of companies repeating the same errors, Dobbins cited the example of a submission made in June 2018 on a study reporting changes in a depression rating scale. In the study, patients were rated on a scale from 1 to 7, with 1 indicating a "normal state" and 7 indicating "among the most extremely ill patients," but the company then reported mean scale scores of 29.9, 30.2 and 30.9 for various arms/groups, she said.

The NIH returned the initial submission saying that the information within the measure was inconsistent and that more information was needed about the scale to know what was going on. The company re-submitted the study results in August, offering a lot more information about the depression scale that now had a range of 0 to 6 (instead of 1 to 7 in the initial submission), but they still reported mean scores of 29.9, 30.2 and 30.9, said Dobbins. The submission was again rejected as the company "didn’t really address the major issue in the first place and the same major error was present," she said.

Here to Help

The NIH, for its part, is taking steps to help companies deal with the major errors commonly found in their submissions, such as:

• Making improvements to its Protocol Registration and Results (PRS) System, used by companies to submit CT results data, to offer real-time feedback on potential major errors. Dobbins said the NIH is doing its best to try and figure out how the PRS can automatically detect some of the data submission problems. In June, for example, it added a new feature to alert companies if they do not make changes to the section where major errors were previously detected.

• Introducing automated validation rules during data entry.

• Offering targeted on-screen help during data entry. Under this, for example, if a sponsor is reporting a score on a scale, the PRS system would automatically detect this and would state what information should be submitted about the scale.

• Issuing improved help resource materials. In June, the NIH issued updated QC review criteria to help companies prepare and submits information clinical trial results.

• Assessing users' needs to understand where they need help.

• Developing educational online resources.

The NIH is supporting these changes by reaching out to companies to raise awareness about the issue. Specifically, the NIH has made attempts to identify sponsors doing well and sponsors who could use assistance. It is also scheduling conferences with companies to talk about their internal processes and is open to dialogue with sponsors looking for help.

"We are not here to make your life a nightmare. We are here to get the best data quality as possible," Dobbins said.

Some companies have already taken up on NIH's offer and are proactively reaching out to the institute for advice before making initial submissions. At GSK, "this is being done much more than before," said Shukla.

A version of this article was first published by Pink Sheet. It is part of the Outlook 2019 series – an annual collection provided exclusively to subscribers of Informa Pharma Intelligence publications.