The Centre for the Study of Democratic Citizenship, in collaboration with the Canada Research Chair in Electoral Democracy & the Research Chair in Electoral Studies, presents:
Graeme Blair (UCLA)
Better pre-analysis plans through design declaration and diagnosis
You can learn more about Graeme Blair by clicking here.
Where and When: Thursday, February 4, at 12pm on Zoom.
RSVP: https://semrasevi.com/montreal-methods-workshops/
Abstract: Pre-analysis plans (PAPs) are used by researchers to
register a set of analyses before they learn about outcomes. PAPs
clarify which choices were made before observing outcomes and which
were made afterwards. Ironically, the set of decisions that should be
specified in a PAP is itself remarkably unclear. PAP templates and
checklists are proliferating, and the number of items they suggest
range from nine to sixty. PAPs are becoming longer and more detailed,
with some in the American Economic Association (AEA) and Evidence in
Governance and Politics (EGAP) study registries reaching hundreds of
pages, as researchers seek to be ever more comprehensive. For all
that, it is hard to assess whether these detailed plans actually
contain key analytically-relevant details. We propose an approach that
clarifies what might belong in a PAP by placing design declaration at
the core of a pre-analysis plan. In particular, we propose that PAPs
include, in addition to other details: (1) A declaration (in code) of
the features of a research design; and (2) A design diagnosis that
analyzes the properties of the design such as power, bias, or coverage
via Monte Carlo simulations.
See more Montreal Methods Workshops events here.
