W-38: Defining the Methodology for Interim Analysis and Data Peek for Power in Late Phase Research and Pragmatic Clinical Trials.
Poster Presenter
Thomas Wasser
Senior Principle Scientist, Biostatistics
Consult-Stat: Complete Statistical Services United States
Objectives
To assure patient safety, proper sampling methods and conserve study costs Interim Analysis (IA) or Data Peek for Power (DPP) is often needed in late Phase Research. ICH-E9 guidelines do not specifically address the methods. This research suggests methods for IA/DPP in this context.
Method
A meta-analysis of 21 published clinical trials that reported an IA/DPP were reviewed. The methods used, often loosely described, were compiled. Basic guidelines for methodology are described and placed in temporal order.
Co-Authors:
John Uebersax
Ann Martin
Results
In general, the methodology for conducting IA/DPP in clinical trials are not defined well or at all. Of the 21 Late Phase studies reviewed 2 (9.5%) did not mention the methods or variables used but simply stated that an IA/DPP was performed. Of the remaining 19 studies, 12 (63.2%) described in good detail the variables and methodology used for the IA/DPP analysis. All 12 studies performed an IA/DPP to examine either the existing effect size between groups, perform a power analysis to determine the sample size to be enrolled prospectively or both. Both reasons were stated to relate directly to participant safety. The remaining five studies (36.8%) performed IA/DPP on “outcome variables” or “study objectives”. Eleven of the 19 studies (52.6%) specifically mentioned that the IA/DPP was performed by staff external to the study, either statisticians from a different department within the pharmaceutical company or Contract Research Organization (CRO). One study specifically mentioned hiring a research statistician contractor outside of both the pharmaceutical company or CRO to conduct the analysis.
When mentioned, IA/DPP were performed after 25-35 participants were enrolled so that the statistical assumptions of Central Limit Theorem (the point where variable mean and standard deviations are thought to be stabilized) were in place. Other studies performed their IA/DPP at a fixed point in the study process, from 20% to 50% of participant enrollment or completion. No studies reported more than one IA/DPP was used.
Three studies (15.8%) performed their IA/DPP at the request of a Data Safety Monitoring Board (DSMB) or the organization Institutional Review Board (IRB). One of these three studies was stopped early as a result of the IA/DPP because the desired effect size of medication treatment had been achieved at 72% patient enrollment/completion. These results are preliminary as more studies that used IA/DPP over the last five years are being added for this analysis.
Conclusion
Late Phase studies are prone to design bias due to a lack of a defined methodology for IA/DPP in the real-world setting. In these studies, confounding variables such as copayments, failure to fill, concomitant medications complicate research. Also, outcome measures reach far beyond medication efficacy and include costs, comorbidities, health care utilization and many other variables where little data is available for use in sample size calculations and statistical power. Bias can enter analysis when statistical procedures are not established. Guidelines for conducting IA/DPP in Late Phase research is key to avoiding bias.
The focus on Late Phase methods for conducting and reporting IA/DPP provides for concrete methodology for conducting IA/DPP in Late Phase clinical trials or Pragmatic Clinical trials and aims to avoid bias in pharmaceutical research, particularly where there is less information about how medications will perform outside of the clinical trial setting. Proposed methods are summarized as: (1) Determine the test statistic that will be used on the primary/secondary objective for which the IA/DPP will be conducted; (2) Identify the power formula (or formula for other safety or efficacy measure) that will be used in the analysis; (3) Conduct analysis once an adequate sample has been prospectively collected. No less than 35 subjects per group to assure data stability under the Central Limit Theorem, and no more than 30% of the sample required in priori sample size calculations; (4) Use the data analysis result estimators (standard error, etc.) to conduct the IA/DPP analysis.
Other Late Phase guidelines include: (1) Do not perform an analysis that would test for statistical significance between groups or any measure that would require corrections to p-values for multiple comparisons; (2) When possible have the IA/DPP conducted by individuals external to the study team; (3) Stop the study or perform analysis at any time when requested by a DSMB or IRB.
