Minimizing Luck In Study Feasibility

Two thirds of clinical trial sites are unable to meet original enrollment goals, with half of sites recruiting either a single patient or failing completely. Study sponsors must scrutinize their approaches to feasibility, as recruitment is such a critical part of overall trial success.

Source: Alamy

Clinical trial failure is both inevitable and multi-factorial. Translational research often reaches dead ends as the complexity of human biology and incomplete understanding of disease processes are laid bare. New therapeutic advances are therefore incredibly hard-won, and usually accompanied by a slice of good fortune as scientific leaps of faith are proven correct. These high, intrinsic barriers certainly provide some of the allure – and indeed frustration – that attract the most inquiring and determined minds into the biopharmaceutical industry. Adding to the challenge and risk are the far more practical obstacles, and the ones in which luck should play no part.

The study feasibility process, without sufficient care and attention, itself contributes to clinical trial failure. Latest estimates suggest it takes an average of eight months to fully initiate a trial, from site identification to first patient in, with timelines gradually lengthening. As many as 86% of clinical trials do not reach recruitment targets within their specified time periods, as actual enrollment times typically double the planned estimates. Poor enrollment can lead to the search for additional trial sites and investigators, while disruptive protocol amendments may be required to adjust eligibility criteria, jumping back several steps in the clinical trial timeline (see Exhibit 1). At the very least, slow enrollment and under-performing clinical trial sites add extra costs to the already expensive R&D process. In the worst-case scenario, a trial sponsor may never achieve target enrollment, compromising the entire clinical program and occasionally resulting in a complete write-off of the investment.

Pharma Intelligence

The latest analysis from the Tufts Center for the Study of Drug Development estimates $560m in direct and capitalized spend in Phase I, II, and III research per approved drug, even before preclinical expenses and the sunk cost of failed programs are accounted for. With this, the total cost rises to an incredible $2.6bn. The incentives towards improving any step in the R&D process that contributes towards these costs are therefore clear. A mere 10% improvement in study cycle time would shave $250m off Tufts’ estimate, while an associated 10% gain in clinical success rates adds a further $384m in savings. As study feasibility contributes towards both cycle time and success rate, it is hugely important to adopt best practices to improve R&D efficiency.

In addition to the cost and time savings within individual drug development programs, there are myriad benefits that industry can capitalize on as a downstream effect. An equivalent R&D budget would be able to sustain larger pipelines with more clinical trials, accelerating the pace of innovation, while patients could alternatively benefit from improved access to new treatments via lower drug prices. Equally, trial sponsors can focus on patient centricity and clinical trial diversity – two essential modernizing initiatives that are top of mind – while accurately quantifying the effect on trial timelines. Finally, with treatments brought to market quicker, drug developers will enjoy longer profitable commercial life cycles for their products, tilting the competitive landscape in their favor.

Best Practice Highlights

1. Assess Competitive Scenarios And Benchmark Against Prior Studies: The overwhelming majority of drug development programs follow a well-worn path, whereby there is a strong precedent for past clinical trials and drugs with a shared mechanism or formulation. Regardless of the protocol for your drug, it would be remiss not to study the historical performance of related programs. An expanded assessment within the target disease area helps to understand trial design features that contribute towards speedy recruitment and positive trial outcomes. This process should provide feedback to protocol development, so pitfalls such as restrictive enrollment criteria or under-performing trial sites can be avoided from the outset. Benchmarking of previous trial performance is also essential when it comes to balancing target accrual with the number of sites required. This knowledge can be used to refine which countries and locations to shortlist, while metrics such as enrollment duration and rate ultimately allow for a more accurate projection of project costs.

2. Identify Countries, Clinical Sites And Experienced Investigators: In tandem with protocol development, the feasibility process as it relates to the individual study design begins. The initial goal is to identify the countries in which to test the treatment (and therefore the necessary regulatory bodies), then extending to the clinical centers and investigators that are most suitable to work with. The potential combinations are astronomical, so data sources that enable prioritization are essential. “We don’t look at just the KPIs of a site – start-up, quality, saturation, competition – we look at the attributes that surround the site, such as access to patients, diversity, connections with other sites, and ability to work with certain vendors that support the needs of 21st century trials,” said Oriol Serra, senior director, head of global site intelligence at Pfizer Inc., during a recent Pharma Intelligence panel discussion on study feasibility trends.

Sponsors can also tip the overall balance in their favor through the identification of experienced investigators with proven track records. These experts will likely be in high demand in a competitive disease area but are more likely to produce results on the ground. Up-to-date information concerning their experience, availability, and any recent regulatory actions is therefore a precious commodity. With this in hand, clinical sites can be initiated along the lines of investigator prioritization, resulting in an efficient approach to study start-up.

3. Enhance Recruitment Efforts With Real-World Data: Trial sponsors can also harness the burgeoning wealth of real-world data to support feasibility assessments. Past performance of sites and investigators is no guarantee of future performance, so it is wise to consider innovative factors such as ties to physician referral networks and patient proximity. “Generating human-focused data is just as important as looking backwards at other trial performances … that is something we really prioritize as well,” said Camilla Ramdeen, director, strategic feasibility at Parexel, during the same panel discussion.

The integration of clinical trial intelligence with patient-level data, such as electronic health records and diagnostic codes, allows for a much more targeted approach to feasibility that is bespoke to a given protocol. Furthermore, the concentration of clinical research within established clinical sites may also overlook previously untapped investigators and patient demographics.

4. Promote Diversity: Study sponsors have long struggled with diversity in clinical trials, with analyses of demographic reporting consistently showing the under-representation of women and ethnic minorities. Patients with co-morbidities are also routinely excluded, as are children and the elderly. A feasibility process that relies too heavily on optimizing past learnings runs the risk of reinforcing long-standing biases, which runs contrary to the need to improve clinical trial diversity. In an era when trial diversity is paramount, sponsors must consider equitable access, casting the net wider in order to build study populations that are truly representative of the patients they ultimately wish to target. “We can’t lose sight of the goal, which is to design studies that are ultimately acceptable and focused on the patients that they are trying to serve,” said Ramdeen.

5. Choose The Right Data To Support Feasibility Assessments: Often the difficulty for feasibility assessments is not the availability of data, but rather how to manage the complexity of this information. “Feasibility is inherently inefficient because of the wealth of data,” said Ramdeen. “That is how it needs to be because there isn’t a single source of truth. You always have to have a cynical and an eyes-wide-open approach. Ultimately, the interpretation can be very different depending upon the context.”

With the vast number of potential inputs into an assessment, from historical performance metrics to site assessments and real-world data, selecting the appropriate emphasis on each can be highly subjective. A successful approach in one disease area may be sub-optimal in others, as each trial faces its unique challenges and rate-limiting steps. This is clearly exemplified in the comparison between a well-studied indication, where patient availability is relatively high but competition for sites and investigators may also be fierce, and an ultra-rare disease with a limited patient pool and little precedent for randomized clinical trials.

Successful feasibility should therefore be a flexible, iterative process, where previous learnings can be applied in real-time, both within a single trial where new sites may be needed, or eligibility requirements altered, and also as a broader approach within an organization to inform future best practices. This feedback loop is essential for continuous improvement and allows the most competitive clinical trial sponsors to gain an edge, purely through minimizing the influence of luck on study cycle times. Taken to the extreme through automation and machine learning algorithms, the separate universes of study feasibility and predictive analytics begin to converge, as discussed in the subsequent section.

Innovations In Study Feasibility

The clinical trial ecosystem is awash with data. For example, at the time of writing, Pharma Intelligence lists over 350,000 studies on Trialtrove, just shy of 500,000 investigators within Sitetrove, and 90,000 discrete drugs in the Pharmaprojects R&D database. Each of these is rich with structured data and unstructured free text. Feasibility strategies need to cut through these unique datasets and combine with additional assessments of protocol design, site suitability, patient availability and diversity. It is inevitable that human-derived insights contain a huge degree of subjectivity, with a mix of intuition and guesswork underpinning feasibility assessments and project timelines.

Scenario Generation Allows R&D Decisions To Be Optimized For Cost, Speed Or Risk

An approach that is supported by data science is highly reproducible, enabling the rapid generation of different trial design scenarios that can be scrutinized and validated by experienced feasibility professionals. Such scenario planning is not always practical using traditional approaches, owing to the considerable time spent on data wrangling exercises. When applying automation, this artificial restriction is removed, allowing one study sponsor to reveal that they generated up to 30,000 recruitment simulations for a single trial, each with its own timeline predictions. In this way, conscious decisions can be made about whether to prioritize raw speed, allocate clinical trial budgets with maximum efficiency, or balance a study in such a way as to mitigate risk of missed clinical trial milestones.

Enter the increasing influence of data scientists, supplementing the deep domain-specific expertise of clinical research professionals. “The [data science] culture has come out of wanting to make better decisions to help us move more quickly, to access patients more quickly, and ultimately to get through the clinical development cycle as quickly as we can,” said Travis Caudill, VP, feasibility & clinical informatics at ICON.

But taking advantage of such large and disparate sources of information requires technical know-how, combining data processing, manipulation, and analytics capabilities. With solid foundations in data science, clinical trial sponsors can employ an algorithmic approach to study the factors that lead to rapid patient enrollment. Feasibility strategies are no longer limited by a pre-defined number of inputs that are traditionally thought to lead to successful trials. Rather, machine learning can uncover unique variables, assign the appropriate degree of relevance to each, and suggest sophisticated combinations of sites and investigators that are optimized for reduced cycle times. By its very nature, machine learning algorithms can improve the accuracy of their predictions over time as additional data become available. “We are not there in terms of the capability of AI-ML to fix everything, but if we put the right data sources and the right interpretation into what might occur in the real world, we are certainly going to get somewhere toward being a little bit more efficient in terms of how we evaluate these recommendations,” noted Jamie Lorimer, director, delivery optimization and informatics team at GlaxoSmithKline plc.

Projecting Timelines With Greater Accuracy

The performance of an individual trial can be affected by many different variables, including those outside of the control of a feasibility assessment. Thus, the benefit of employing machine learning to augment site selection recommendations may only become apparent over the course of several clinical trials. However, with an estimated 80% of studies failing to finish on schedule, its value in projecting more realistic enrollment timelines can be realized much more quickly. By combining an assessment of historical trial analogs, site and investigator performance, and patient availability with dozens of other relevant design features, the predictive analytics approach can simulate highly tailored study cycle timelines, with narrow confidence thresholds. “We are now at the intersection of data, analytics, and AI. How do we see these early warning signs, even at the protocol design stage, that we are taking on higher risk with this protocol design decision? These decisions we are making now have implications to your risk profile later,” Caudill noted.

One top 10 pharma has taken the approach of building a proprietary enrollment forecasting tool, embedding a single data-driven methodology across the entire organization. It is designed to solve the problem of inflexible forecasts by providing reliable, transparent, dynamic projections to help plan and assess enrollment. Recognizing that enrollment forecasts are subject to prevailing winds, the model includes 95% confidence intervals so uncertainties in the prediction can be visualized. Iterative improvements in the model refine the accuracy of forecasting over time, while its algorithm also enables suggestions for indication-specific country recommendations and site number projections per country, linking back to the core feasibility process. Blending with live site performance data, trial timelines can be re-forecast in real-time for accurate project planning and course corrections.

Greater accuracy when projecting clinical trial timelines has immediate and practical utility for managing clinical trial budgets. For a contract research organization, this embeds added competitiveness when bidding for prospective projects, while study sponsors can allocate resources across a broader R&D portfolio with much greater efficiency. For both parties and the industry at large, it crucially mitigates against the risk of clinical trials failing owing to poor recruitment, avoiding the costly scenario of a long running trial entering rescue mode. “The art is interconnecting everything and making sure everything works like a well-oiled machine,” said Serra in his closing thoughts. “We need to empower our people, analysts, scientists, engineers and experts with the tools and applications.”

Get more on this topic from Pharma Intelligence