Problem Roulette (PR), an online study service at the University of Michigan, offers points-free formative practice to students preparing for examinations in introductory science, technology, engineering, and mathematics courses. Using four years of PR data involving millions of problem attempts by thousands of students, we quantify the benefits of increased practice study volume in introductory physics. After conditioning the mean final grade on standardized (ACT and SAT) math test score, we analyze deviations based on student study volume. We find a strong effect; mean course grade rises quadratically with the logarithm of the total number of PR questions encountered over the term ($N_{\mathrm{Q},\mathrm{tot}}$), with an overall gain of $0.77\pm 0.12$ grade points between $1<N_{\mathrm{Q},\mathrm{tot}}<1000$. The gains are persistent across the range of math test scores represented in our sample. While $N_{\mathrm{Q},\mathrm{tot}}$ surely correlates with other study habits, the benefits of increased study in general still hold. A model for final grade using test score and study volume largely accounts for demographic stratification, including by sex, parental education level, number of parents at home, nationality or underrepresented minority status, and regional income level, with two significant exceptions: students whose parents did not earn a college degree, who earn $-0.27\pm 0.04$ grade points ($6.1\sigma$) below expectations, and underrepresented minority students at $-0.14\pm 0.04$ points ($3.6\sigma$). Residual scatter in final grade remains comparable to the maximal study gains, implying that the model is far from deterministic: individual variation trumps mean trends. Our findings can help motivate students to study more and help teachers to identify which types of students may especially need such encouragement.

• Received 20 December 2022
• Accepted 25 April 2023

DOI:https://doi.org/10.1103/PhysRevPhysEducRes.19.010139

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society