We investigated the impact that prioritization versus random assignment of patients to biomarker-defined strata when patients meet criteria for multiple strata in the Master Protocol framework has on treatment effect evaluation and trial performance (e.g. power, type I error, and estimation bias).
Method
A master protocol simulation tool was created using R software to conduct simulations with different strata assignment schemes (random assignment, assignment to lowest prevalence, weighted assignment based on biomarker prevalence, assignment to highest level of evidence) under different settings.
Results
We considered a master protocol design with two biomarkers, Biomarkers A and B, where the prevalence of Biomarker A is 0.6 and prevalence of Biomarker B is 0.4. A subject who is positive for Biomarker A is defined as A+ regardless of the status of other biomarkers, and a subject who is positive for Biomarker B is defined as B+. A subject who is negative for Biomarker A is denoted as A- and a subject who is negative for Biomarker B is denoted as B-. In this setup, we have four sub-populations: A+B-, A-B+, A+B+, A-B-. We assume two active drugs, Treatment 1 (TRT1, 30% response rate (RR) for A+B-) and Treatment 2 (TRT2, 30% RR for A-B+) for investigation, and one common control (standard of care, SOC). 38 subjects per stratum were required to have 80% power for each sub-study. In total, 95 subjects (38 × 2 biomarker strata + 19 × 1 non-match stratum) were accrued for the simulation study.
The true treatment effect is defined as the difference of the true RR between treatment and control for each drug on each subpopulation (A+, B+) respectively. The patients with both biomarkers positive (A+B+) is not viewed as a separate subpopulation as the interactions between the two biomarkers are assumed to be unknown. We considered the following scenarios for treatment 1 and treatment 2 RR on A+B+ population: 1) TRT 1 and TRT 2 RR of 30%, 2) TRT 1 RR of 40% and TRT 2 RR of 20%, and 3) TRT 1 RR of 20% and TRT 2 RR of 40%.
The estimated average treatment RR (for 1000 simulations for each scenario) and true treatment effects were calculated. Some of the strata assignment strategies led to overestimation or underestimation of the treatment effect. For instance, when using the lowest prevalence method in Scenario 2, the estimated treatment effect is 24.97% whereas the true treatment effect is 29%. This is likely due to the fact that A+B+ patients were assigned to Strata 2 (B+) only since the prevalence of Biomarker B (0.4) was less than the prevalence for Biomarker A (0.6).
Conclusion
We created a master protocol simulation tool with a high-level approach that allows users to select choices (by logical or index variables) for different scenarios. The flexible and branching logic structure of the tool will enable investigators to explore different statistical considerations for the master protocol platform.
The impact of the different patient assignment-to-strata strategies is a new problem under the Master Protocol framework. In our simulation study, we found that some of the strata assignment strategies led to overestimation or underestimation of the treatment effect. This was likely due to the fact that the A+B+ patients were sometimes only assigned to one strata and not the other depending on which assignment strategy was used. The bias is likely due to the fact that the estimand and the estimator are not fully matching. For instance, when using the lowest prevalence method, A+B+ patients were assigned to Strata 2 (B+) since the prevalence of Biomarker B (0.4) was less than the prevalence for Biomarker A (0.6).
Investigators who wish to conduct umbrella trials in which they assume that the target biomarkers do not interact may be misled by their findings, if interactions exist and there is a sufficiently high prevalence of double positivity. The problem will also need more investigation. For instance, it would be of benefit to also consider a simulation study that involves more than two biomarkers to be included in the study or that involves adaptive assignment as a strata assignment method to investigate.
There is great promise in the use of the Master Protocol platform to discover, explore and test multiple biomarkers and multiple treatments under one overall protocol. Possible interactions between the biomarkers should be considered when designing and interpreting these studies.
