Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school [grade point average] GPA as a measure) provides a more holistic picture of students' academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA,
... Show more
Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school [grade point average] GPA as a measure) provides a more holistic picture of students' academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA, questions remain about whether the positive effects of MMA are sustained over time. This study - a follow-up to prior research using the same sample of students - employs a randomized controlled trial to investigate whether algorithmic MMA placement used at seven State University of New York (SUNY) community colleges led to better student outcomes, for up to four and a half years after randomization, than a system based on test scores alone.
Nearly 13,000 incoming students who arrived at the seven colleges in fall 2016, spring 2017, and fall 2017 took placement tests and were randomly assigned to be placed using either the status quo method (business-as-usual group) or the alternative, algorithmic MMA method (program group). Using this sample, we estimate the overall treatment effects on placement into, enrollment in, and completion of college-level math and English as well as effects on other outcomes. We conduct similar analyses on race/ethnicity, Pell recipient status, and gender subgroups. We also descriptively examine the proportion of program group students who were bumped up (i.e., their placement changed from a developmental course placement to a college-level course placement) and bumped down (i.e., their placement changed from a college-level course placement to a developmental course placement) by the MMA algorithm, and we perform a cost-effectiveness analysis.
We find that the MMA method used at the colleges improved access to and success in college-level courses and that lower cut scores in English rather than math are associated with larger and longer lasting impacts on completion of college-level coursework. While MMA improved outcomes among student subgroups, it had little to no impact on gaps in outcomes between subgroups. We also find that bumped-up students had substantially better outcomes in both math and English, while bumped-down students had substantially worse outcomes. Our results suggest that increased access to college-level courses is the driving factor in the positive outcomes experienced by program group students and that placement into standalone developmental courses can have detrimental effects on student outcomes. In the discussion of the study's results, we make recommendations for adopting MMA at colleges. Implemented together with other initiatives to support students, MMA can be a first step on the path to success for incoming students.
Published abstract.
Show less
