Article first appeared online May 20, 2015
Luke Keele, Ph.D., Associate Professor, Political Science Department, Penn State University, Dustin Tingley,Ph.D., Paul Sack Associate Professor of Political Economy, Government Department, Harvard University & Teppei Yamamoto, Ph.D., Assistant Professor of Political Science, Massachusetts Institute of Technology.
What was the genesis of the idea for your research/paper?
There has been a recent explosion in interest in the social and health sciences about how to study causal mechanisms. That is, there is intense interest in explaining WHY some policy interventions work. This discussion how to conduct statistical analyses that show why interventions work has largely been absent in the policy analysis literature. We want to bring together these literatures, especially as the policy analysis community regularly engages with fascinating empirical projects and program evaluations.
What is the main conclusion that becomes evident from your research? (Or, what is your main takeaway?)
Researchers that want to know why their interventions have an effect, not just whether, need to think carefully about how to design their studies in order to satisfy assumptions necessary to make claims about causal relationships. Ignoring these insights about design mean that researchers cannot draw on rich statistical tools for estimating the role of interesting causal mechanisms.
What are some of the more interesting or surprising findings/conclusions did you find in the process of bringing this together?
Randomized experiments are often called the gold standard for policy evaluations. We think one surprising finding is that there isn’t a gold standard method for causal mediation analysis. There are a number of statistical analyses that can be suggestive, but there isn’t a method that can provide definitive answers.
Click here to view article
Authors' Bios
Luke Keele is an Associate Professor in the political science department at Penn State University. In this capacity, he conducts research on statistical methods for program and policy evaluation. He has lectured and taught globally on statistical techniques for program evaluation and comparative effectiveness. He has experience not only in the practice of program evaluation, but also in the development of new statistical techniques for use in evaluating policy interventions. He received his Ph.D from the University of North Carolina at Chapel Hill and was a post-doctoral research fellow at Nuffield College, Oxford University. He has published in Journal of the Royal Statistical Society, Series A, the American Journal of Political Science, the American Political Science Review, and Statistical Science.
Twitter: @ljk50
Dustin Tingley is the Paul Sack Associate Professor of Political Economy in the Government Department at Harvard University. He received a PhD in Politics from Princeton in 2010 and BA from the University of Rochester in 2001. His research interests include international relations, international political economy, experimental approaches to political science, and statistical methodology. Dustin is currently working on new experimental projects on bargaining, attitudes towards global climate change, new methods for the statistical analysis of causal mechanisms and textual data, and a book about the domestic politics of US foreign policy.
Dustin directs IQSS's Undergraduate Research Scholar program, is the founding director of the Program on Experience Based Learning in the Social Sciences, which founded and maintains ABLConnect, and is the former (and founding) editor of the APSA Experimental Section newsletter, The Experimental Political Scientist. Dustin initiated and organized the Harvard Government Department annual poster session, and has organized interdisciplinary conferences on causal mechanisms, climate change politics, and negotiation in international relations. Dustin is a scientific adviser to EconVision.
Twitter: @dustintingley
Teppei Yamamoto is an Assistant Professor of Political Science at Massachusetts Institute of Technology. He obtained a B.A. in Liberal Arts from the University of Tokyo (2006) and a M.A. (2008) and Ph.D. (2011) in Politics from Princeton University, where he received a Charlotte Elizabeth Procter fellowship for the year of 2010 to 2011. His doctoral dissertation won the John T. Williams Dissertation Prize in 2010 from the Society for Political Methodology. He also studied at Lincoln College, the University of Oxford, as a visiting student.
He is broadly interested in the development of quantitative methods for political science data. My research has focused on statistical methods for causal inference, including causal attribution, causal mediation, causal moderation, and causal inference with measurement error. He also studied applied Bayesian statistics, with focus on discrete choice models and empirical applications in electoral studies and comparative political behavior.
His work has appeared in various academic journals, including American Journal of Political Science, American Political Science Review, Journal of the Royal Statistical Society Series A, Political Analysis, PNAS, and Statistical Science.