Authors: Natalie A. Tran, Alan B. Nathan, Mitchell J. Nathan

May 2010, 10 pp.

Abstract: This paper illustrates how propensity score matching (PSM) can provide a robust estimate of a program treatment effect on student learning outcomes. A quasi-experimental design was used to investigate the impact of enrollment in a pre-college engineering curriculum on student achievement on standardized state tests in science and mathematics, controlling for a small number of covariates (prior achievement score, gender, free/reduced-price lunch status). Models using either hand-matched or PSM controls produced comparable substantive results. The estimates for the impact of the pre-college engineering curriculum were statistically significant for math achievement but not for science. PSM was less laborious; it generated a comparison group with more participants and less variability, thus increasing statistical power; and it identified an additional significant predictor (gender).

Keywords: propensity score matching, quasi-experimental design, student achievement, pre-college engineering education, career and technical education

Read the full working paper: Hand Matching vs. Propensity Score Matching: An Empirical Comparison of Results From a Quasi-Experiment (PDF)