Placebo Tests Done Right

Placebo tests, specifically those in which a variable of interest is replaced with a permuted or otherwise randomized version of itself, are commonly employed to assess the validity of a seemingly significant result. When analysing a randomized control trial (RCT), a placebo treatment assignment variable is reassigned according to the experiment’s treatment assignment mechanism and the validity of the ensuing placebo test is immediate. With observational data, however, reassigning the treatment –or, more generally, the tested– variable “as if” it had been experimentally assigned at random can produce very misleading results. We explore the possible distortions such an approach can bring about as well as principled alternatives for valid placebo tests.