The Winner’s Curse in Data-Driven Decision-Making: Evidence and Solutions

Data-driven decision-making involves estimating the value of each potential choice and selecting the best one based on these estimates. This approach underpins a wide array of modern marketing applications, including A/B testing, pricing, personalized targeting, and assortment optimization. In many practical settings, it is essential not only to identify the optimal decision but also to accurately assess its incremental value or lift. In this paper, we first provide a theoretical demonstration that selecting the optimal decision based on estimated effects from data leads to systemic overly-optimistic evaluations of that decision’s value — a phenomenon known as the winner’s curse — regardless of the estimation or optimization procedure used. We then empirically show that an economically significant winner’s curse exists across a broad set of marketing applications with realistic parameter settings, including A/B testing, personalized targeting, and assortment optimization. Recognizing the generality of this issue, we propose a correction method based on a non-continuous bootstrap approach that effectively mitigates the winner’s curse in most settings we study. Finally, we benchmark our method against several existing context-specific solutions and offer a general guide for correcting the winner’s curse across a wide range of marketing applications.