Adobe Inc.
FRAMEWORK THAT ENABLES ANYTIME ANALYSIS OF CONTROLLED EXPERIMENTS FOR OPTIMIZING DIGITAL CONTENT

Abstract:

A computer-implemented method includes instantiating a framework configured to optimize a metric of interest for a website based on interactions by participants with instances of a website in a controlled experiment. The instances of the website include one of two variants of digital content. Test data including an estimate of an effect on the metric of interest is generated based on the interactions. A sequence of confidence intervals is dynamically generated while the controlled experiment is ongoing. The true effect and the estimate effect on the metric of interest are both bounded by the sequence of confidence intervals throughout the controlled experiment. As such, an anytime analysis with anytime-valid test data is enabled while the controlled experiment is ongoing.