Home Events Designing Evaluations with Comparison Groups: Part 1 & 2

Designing Evaluations with Comparison Groups: Part 1 & 2

Part I

This introductory webinar on March 19 at 12 p.m. ET equips S-STEM PIs and evaluators with essential design thinking to move from descriptive outcomes to stronger, defensible evidence of program impact. The webinar introduces the foundations of quasi-experimental designs by explaining when and why they are used and the kinds of real-world problems they are meant to solve. It highlights common threats to validity in single- and multiple-group studies and focuses on the central challenge of selection bias—how to think carefully about what would have happened in the absence of a program or intervention. The session then walks through practical examples of comparison groups commonly used in education settings, emphasizing how thoughtful design choices strengthen conclusions and could be applied to the evaluation planning for S-STEM PIs and their evaluators.

 

Part II

The second webinar builds on the introductory session by guiding PIs and evaluators through the analytic techniques needed to strengthen quasi‑experimental designs when random assignment is not possible. This session provides a practical overview of propensity score methods, including matching, weighting, and model diagnostics, to help participants create more credible non‑equivalent comparison groups for S-STEM scholar program evaluations. Attendees will learn some of the technical aspects of these methods for reducing bias, improving balance, and enhancing the rigor of their impact findings.

Speakers

Joseph A. Taylor

Principal Researcher

AIR

Joseph A. Taylor is a principal researcher at AIR where he directs intervention impact studies, syntheses of the effects of science education interventions, and contributes to AIR’s work on the What Works Clearinghouse (WWC). Dr. Taylor is currently a co-Principal Investigator of a US Department of Education funded grant (Education Innovation and Research program) that is studying the effects of a promising professional development program for elementary science teachers. He is also overseeing impact analyses on a US Department of Education funded grant (Supporting Effective Educator Development program) that is also studying the effects of professional development on teacher and student outcomes. He is senior advisor of a NSF funded meta-analysis of the effects of the 5E instructional model. Dr. Taylor also consults on AIR projects with regard to psychometric validation of measurement instruments. In his work, Dr. Taylor draws upon his experience teaching statistics and measurement at the University of Colorado, Colorado Springs, as well as prior experience as a high school physics and math teacher. 

Melissa Rodgers headshot

Melissa A. Rodgers

Melissa A. Rodgers, PhD is a Quantitative Researcher at American Institutes for Research (AIR). Rodgers is the project director of the IES-funded meta-analytic grant evaluating the impact of postsecondary nudging interventions on college enrollment, persistence and achievement. Rodgers is also the quantitative lead on multiple educational research projects including meta-analytic studies, evaluation of teacher preparation and training programs, and student web-based interventions to improve academic achievement in math, English and civic education. In this role, her primary responsibilities include the development and implementation of methodological strategy, management of large relational datasets, conduct analyses using advanced analytic methods, and management of junior staff to assist with these efforts. In August 2026, she obtained her doctoral degree in Quantitative Methods in Educational Psychology from The University of Texas at Austin. Her methodological research focuses on research synthesis in education and psychology, specifically the evaluation and advancement of methodologies to handle the influence of design, selection and publication bias.

Get the S-STEM REC Newsletter

Sign up to receive a monthly S-STEM REC newsletter containing resources and upcoming events!

Secret Link