Formative assessment — evidence gathered and used during instruction to adapt teaching and support learning — is among the most heavily cited levers in education, yet its meta-analytic effect estimates span roughly g = 0.2 to g = 0.91 depending on context, synthesis, and evidence base. Three source studies crystallized a coupled trustworthiness problem: a Turkish meta-analysis of 32 studies found a large effect (g = 0.72) with strong feedback-source moderation but untested subject-area, formality, and feedback-procedure moderators; a special-education review found published studies yield larger effects than gray literature (d = 0.89) but estimated this from only eight meta-analyses using the criticized fail-safe N; and a peer-feedback meta-analysis found its moderator effects unstable because the underlying studies are few, heterogeneous, and weakly controlled. The subsequent literature shows: field-level studies confirm published-over-unpublished effect-size inflation in education (d = 0.64) and that education meta-analyses almost never correct for bias; newer syntheses shrink the formative-assessment effect (g = 0.25 across 118 studies) without applying principled bias correction; and meta-research now shows that moderator findings themselves are distorted by selective reporting of null moderator results (RAMSing — reporting after moderators are significant). No located work applies selection-model, PET-PEESE, or robust-Bayesian correction to the formative-assessment achievement literature and aligns its conclusions with corrected estimates, and no synthesis decomposes the residual heterogeneity across instructional and study-context moderators in a bias-aware way. The problem — a bias-corrected pooled estimate plus a verified heterogeneity decomposition of formative-assessment effectiveness — remains open.

Background

Formative assessment — often called assessment for learning — is the collection and use of evidence about student understanding during instruction to adapt teaching and give feedback while learning is still in progress, in contrast to summative assessment, which judges outcomes after instruction ends. 'Student achievement' is used throughout in its ordinary sense of academic learning outcomes, as distinct from the self-regulated-learning, motivational, and affective outcomes that some adjacent syntheses report. Since Black and Wiliam's (1998) influential review, large positive effects of formative assessment on achievement have been widely claimed in research and policy, with meta-analytic estimates of the standardized mean difference (Hedges' gg, the treatment effect in pooled standard-deviation units; Cohen's dd denotes the same quantity without gg's small-sample correction) running from roughly g=0.2g=0.2 in broad achievement syntheses to g=0.91g=0.91 in narrow subdomains such as peer feedback on higher-education academic writing. Kingston and Nash (2011) showed that the frequently cited value of about .70 rests on a thin base: only 13 usable studies (42 effect sizes), a weighted mean effect of .20, and suggestive but fragile subject-area differences (English language arts .32, mathematics .17, science .09).

In a meta-analysis, effect sizes from primary studies are combined under a random-effects model, which allows each study its own true effect; moderator analysis (subgroup analysis or meta-regression) then tests whether study-level covariates — subject, education level, feedback type, study design — explain part of the between-study variance, conventionally indexed by I2I^2 (the share of observed variance due to real differences rather than sampling error). Two structural threats make such syntheses untrustworthy if unaddressed. First, publication bias: studies with large or statistically significant effects are more likely to be published than null or small-effect studies, so syntheses drawing only on the published record — rather than also on gray literature (dissertations, reports, and other unpublished documents) — can overstate effects. Classic diagnostics include the funnel plot (effect size versus precision, where asymmetry suggests missing small studies), Egger's regression test for that asymmetry, trim-and-fill (which imputes the missing studies), and the fail-safe NN (how many null studies would be needed to nullify the result — a method repeatedly criticized as uninformative about effect-size distortion). Principled corrections instead estimate the effect under an explicit model of selective publication: PET-PEESE regression-based adjustment, selection models (re-weighting studies by an assumed probability of publication), and robust Bayesian model-averaged meta-analysis (RoBMA). Second, statistical dependence: most primary studies contribute several effect sizes, which requires robust variance estimation or three-level (multilevel) models rather than naive pooling.

Three source studies crystallize the coupled problem for formative assessment. Karaman (2021) synthesized 32 Turkish experimental and quasi-experimental studies (47 effect sizes), found a large pooled effect (g=0.72g=0.72) with strong moderation by feedback source (student-initiated feedback d=1.16d=1.16 down to computer-initiated d=0.42d=0.42) but no significant education-level or publication-type differences, and no funnel asymmetry — yet the evidence base was small, and moderators such as subject area, the formality of the assessment, and feedback procedures and timing went untested. Gage, Cook, and Reichow (2017) reviewed 109 special-education meta-analyses: only 33% assessed publication bias, meta-analyses that excluded gray literature were more likely to show signs of bias, and in the eight meta-analyses where published and unpublished effect sizes could be compared, published studies yielded clearly larger effects (converted d=0.89d=0.89) — an estimate the authors themselves flagged as resting on a small sample and on the criticized fail-safe NN. Huisman, Saab, van den Broek, and van Driel (2019) meta-analyzed formative peer feedback on academic writing in higher education, finding large pooled effects (for example g=0.91g=0.91 versus no feedback, from only two studies) and moderator tests (feedback nature, number of peers) that were fragile because they rested on eight one-group pretest-posttest studies with I2I^2 above 95%. The open question couples two axes: whether the formative-assessment effect is robust to principled bias correction, and which contextual factors genuinely drive its heterogeneity.

Problem Statement

Do meta-analytic estimates of the effect of formative assessment on student achievement survive correction for publication bias and related selective-reporting biases, and which instructional and study-context factors genuinely account for the substantial residual heterogeneity among studies?

Answering it calls for a systematic, reproducible synthesis of the experimental and quasi-experimental formative-assessment literature — unrestricted by subject area, education level, or country, and not confined to any single design within that class — in which the pooled effect is estimated with models appropriate for the dependent effect sizes that primary studies contribute, and publication bias is addressed by principled correction rather than only by the criticized classic diagnostics, so that the substantive conclusions rest on the corrected estimates and on how these vary across correction methods and selection assumptions. The moderator side of the question is settled in the same synthesis: which contextual factors — including but not limited to subject domain, formality of the assessment, feedback source, procedure and timing, education level, study design and rigor, sample size, and cultural or institutional setting — genuinely explain the remaining between-study variation. A moderator effect counts as genuine — and the resulting decomposition as verified — only if it survives the same principled bias corrections and RAMSing-style selective-reporting checks, not merely a single conventional significance test. What the field lacks, and what the problem asks for, is a bias-corrected pooled estimate of the formative-assessment effect on achievement together with an account of which context moderators are genuine, replacing the current situation of uncorrected, mutually inconsistent syntheses whose pooled estimates span roughly g = 0.2 to g = 0.91.

References

  1. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. DOI: 10.1080/0969595980050102. https://doi.org/10.1080/0969595980050102
  2. Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28-37. DOI: 10.1111/j.1745-3992.2011.00220.x. https://doi.org/10.1111/j.1745-3992.2011.00220.x
  3. Gage, N. A., Cook, B. G., & Reichow, B. (2017). Publication bias in special education meta-analyses. Exceptional Children, 83(4), 428-445. DOI: 10.1177/0014402917691016. https://doi.org/10.1177/0014402917691016
  4. Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students' academic writing: a meta-analysis. Assessment & Evaluation in Higher Education, 44(6), 863-880. DOI: 10.1080/02602938.2018.1545896. https://doi.org/10.1080/02602938.2018.1545896 (first published online December 2018)
  5. Chow, J. C., & Ekholm, E. (2018). Do published studies yield larger effect sizes than unpublished studies in education and special education? A meta-review. Educational Psychology Review, 30(3), 727-744. DOI: 10.1007/s10648-018-9437-7. https://doi.org/10.1007/s10648-018-9437-7
  6. Karaman, P. (2021). The effect of formative assessment practices on student learning: A meta-analysis study. International Journal of Assessment Tools in Education, 8(4), 801-817. DOI: 10.21449/ijate.870300. https://doi.org/10.21449/ijate.870300
  7. Ropovik, I., Adamkovic, M., & Greger, D. (2021). Neglect of publication bias compromises meta-analyses of educational research. PLOS ONE, 16(6), e0252415. DOI: 10.1371/journal.pone.0252415. https://doi.org/10.1371/journal.pone.0252415
  8. Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. DOI: 10.3389/fpsyg.2019.03087. https://doi.org/10.3389/fpsyg.2019.03087
  9. Xuan, Q., Cheung, A., & Sun, D. (2022). The effectiveness of formative assessment for enhancing reading achievement in K-12 classrooms: A meta-analysis. Frontiers in Psychology, 13, 990196. DOI: 10.3389/fpsyg.2022.990196. https://doi.org/10.3389/fpsyg.2022.990196
  10. Lee, A. [Ahhyun Lee]. (2024). Publication bias in special education meta-analyses in Korea. Korean Journal of Learning Disabilities, 21(2), 111-132. DOI: 10.47635/kjld.2024.21.2.111. https://doi.org/10.47635/kjld.2024.21.2.111
  11. Yao, Y., Amos, M., Snider, K., & Brown, T. (2024). The impact of formative assessment on K-12 learning: a meta-analysis. Educational Research and Evaluation, 29(7-8), 452-475. DOI: 10.1080/13803611.2024.2363831. https://doi.org/10.1080/13803611.2024.2363831
  12. Sortwell, A., Trimble, K., Ferraz, R., Geelan, D. R., Hine, G., Ramirez-Campillo, R., Carter-Thuiller, B., Gkintoni, E., & Xuan, Q. (2024). A systematic review of meta-analyses on the impact of formative assessment on K-12 students' learning: Toward sustainable quality education. Sustainability, 16(17), 7826. DOI: 10.3390/su16177826. https://doi.org/10.3390/su16177826
  13. Talbott, E., Maggin, D. M., Chua, M., Ashley, L., Chen, X., Chin, P. A., & Curry, M. K. (2026). Outcome-reporting bias in special education intervention research using experimental and quasi-experimental group designs: A conceptual replication. Remedial and Special Education, 47(1), 25-35. DOI: 10.1177/07419325241240067. https://doi.org/10.1177/07419325241240067 (first published online April 2024)
  14. Pellegrini, M., Pigott, T. D., Chubb, C. S., Pruitt, N., & Scarbrough, H. F. (2025). Exploring the methodological quality of education intervention meta-analyses: A meta-review. Review of Educational Research. Advance online publication. DOI: 10.3102/00346543251389938. https://doi.org/10.3102/00346543251389938
  15. Li, A. W., & Hebert, M. (2026). Effects of peer feedback on adolescent writing: A meta-analysis. Scientific Studies of Reading. Advance online publication. DOI: 10.1080/10888438.2026.2712255. https://doi.org/10.1080/10888438.2026.2712255
  16. Li, J., & Gu, P. (2026). Teacher continuing professional development in formative assessment: A pathway to enhanced student achievement. Teaching and Teacher Education, 174, 105426. DOI: 10.1016/j.tate.2026.105426. https://doi.org/10.1016/j.tate.2026.105426
  17. Yan, Z., Lao, H., Panadero, E., & Fernández-Castilla, B. (2026). A meta-analytical review of the effects of self-assessment and peer-assessment interventions on self-regulated learning strategies and affective outcomes. Assessment in Education: Principles, Policy & Practice, 33(3), 209-247. DOI: 10.1080/0969594X.2026.2681630. https://doi.org/10.1080/0969594X.2026.2681630
  18. Bicakci, M., Heller, F., Stoeger, H., & Ziegler, A. (2026). From RAMSing to rigor: Improving moderator analysis in STEM education meta-research. Educational Psychology Review, 38(1). DOI: 10.1007/s10648-026-10172-1. https://doi.org/10.1007/s10648-026-10172-1