Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Making

• DOI: 10.48550/arxiv.2311.02532
• Publication Date: 2023-01-01
• Authors: Ting Li; Chengchun Shi; Jianing Wang; Fan Zhou; Hongtu Zhu
• Keywords: Methodology (stat.ME); FOS: Computer and information sciences; Statistics - Methodology

A/B testing is critical for modern technological companies to evaluate the effectiveness of newly developed products against standard baselines. This paper studies optimal designs that aim maximize amount information obtained from online experiments estimate treatment effects accurately. We propose three allocation strategies in a dynamic setting where treatments are sequentially assigned over time. These designed minimize variance effect estimator when data follow non-Markov decision process or (time-varying) Markov process. further develop estimation procedures based on existing off-policy evaluation (OPE) methods and conduct extensive various environments demonstrate proposed methodologies. In theory, we prove optimality design establish upper bounds mean squared errors resulting estimators.