T-41: Using Causal Inference Modelling to Predict Unbiased Treatment Response for Managed Care Organizations and Drug Manufacturers
Poster Presenter
Denise Meade
Director, Managing Health & Life Sciences
Microsoft United States
Objectives
Our objective was to demonstrate the utility of causal inference models in real-world data and develop a framework for evaluating the performance of causal inference in objectively assessing treatment response. We use Rheumatoid Arthritis (RA) treatment as our example use case.
Method
We built causal inference models for RA clinical and cost outcomes. We analyzed 10 biologic therapies with conventional treatments combined. To evaluate the models, we created 6 visuals for model precision, consistency, calibration, propensity distribution, covariate balancing, and accuracy.
Results
We used the IBM MarketScan Commercial and Medicare Claims database (2010-2017) to build causal inference models for RA clinical (emergency room visits, outpatient visits, hospitalizations) and per-patient per-month (PPPM) cost outcomes. We analyzed 27 outcomes for 10 biologic therapies and for conventional therapies as a single treatment group. To evaluate the models, we created 6 visuals that exhibit model precision, consistency, calibration, propensity distribution, covariate balancing, and accuracy.
Using the evaluation framework we were able to detect positivity issues in the data, and construct robust and reliable outcome predictions by: identifying and removing treatment predictions that were not comparable (by calibration plot); modifying models that were too weak for predicting outcomes; discarding methods that were incompatible with causal inference analysis (e. g. random forests); and identifying outcomes for which there isn’t sufficient predictive power. In the context of rheumatoid arthritis, we identified multiple sub-populations that could benefit from a specific treatment over other treatment. For example, we identified that for the population over the age of 65 Etanercept provides the lowest risk of anemia, while Infliximab provides the lowest overall risk for multiple outcomes while also accruing relatively low cost.
Conclusion
Clinical and cost outcomes from real-world data contain selection bias due to formulary management and provider and patient influences impacting treatment assignment. Causal inference allows learning from real-world data by using observed treatment with corresponding response and adding predicted outcomes given alternative (non-factual) treatments while adjusting for bias and confounding. We sought to develop causal inference models and a framework for evaluating their performance using rheumatoid arthritis (RA) as a use case.
Our approach allows multiple directions of analysis to find the best treatment per population, the best responding population for a treatment, as well as clusters of patients that respond similarly across multiple treatments. By analyzing adjusted predictors of treatment response within a disease, formulary decisions can be made to optimize coverage and manufacturers could gain novel insights into which patient populations may best respond to their treatment.
Finally, we present a framework for evaluating our models to ensure their accuracy and limit bias. Our novel framework is offered as a means to determine the robustness of causal inference models.
