Evaluating Replicability: Considerations for Analyses and Implications for Design


As high-profile empirical research has questioned the replicability of scientific findings, it has become clear that there is no standard approach to designing and analyzing studies to evaluate replication. Ambiguity regarding key estimands for ‘replication’ and the purpose of replication research has shaped statistical treatments of the topic and sparked debate in several fields. This talk sheds light on this ambiguity by identifying different possible statistical definitions of ‘replication’ that could be studied. It then highlights relevant analysis methods and derives their statistical properties. Finally, it connects these properties to key implications for the design of primary studies, as well as subsequent replication attempts.

Mar 10, 2022
Jacob M. Schauer
Assistant Professor

My research interests involve statistical methods for the social and health sciences.