Morgan Stanley
SYSTEM AND METHOD FOR MEASURING MODEL EFFICACY IN HIGHLY REGULATED ENVIRONMENTS

Abstract:

Systems and methods for measuring efficacy of prediction models are described. A processor generates champion scores, variation scores, and challenger scores based on data analyzed using a champion model and a challenger model. The processor uses the variation scores to define a control group and a test group based on the relationship of the variation scores to the champion scores and the challenger scores. The control group and test group are measured to determine an attributable impact on completed actions and based on the attributable impact, one of the champion model and challenger model is selected.