A 2020 meta-analysis of bilingual–monolingual differences on executive-functioning tasks (Ware, Kirkovski & Lum) found positive pooled effects that were task- and age-dependent, but the conclusion about the true effect depended on the publication-bias correction applied: PET-PEESE regression yielded corrected intercepts near or below zero, Trim-and-Fill imputed no missing studies, and fail safe N indicated robustness. The paper's explicitly outstanding question — which approach most accurately corrects the observed effect size in this literature — remains open. It has two linked threads: (i) the methodological question the paper poses as outstanding — establishing, under the conditions that characterize this literature (moderate numbers of studies per analysis, large between-study heterogeneity, mixed published and unpublished evidence, unknown selection mechanism), which correction method (PET-PEESE, Trim-and-Fill, fail-safe estimates, selection models, or robust Bayesian model-averaging) recovers the true effect most accurately; and (ii) the substantive question of the bias-corrected magnitude of the bilingual executive-functioning advantage, which currently flips between near-zero and positive depending on the method trusted. Thread (i) is the open problem; thread (ii) is what gives it its stakes, since the source paper already reports its own task- and age-specific corrected estimates. Later simulation and robust-Bayesian work has reframed but not settled the methodological question, and method-dependent verdicts persist throughout the bilingualism literature, so the open core survives intact.

Background

The 'bilingual advantage' hypothesis holds that managing two languages on a daily basis strengthens domain-general executive functioning — the family of controlled cognitive processes (inhibiting prepotent responses, shifting between task sets, updating working memory) measured by tasks such as Stroop, Simon, flanker, and the Attentional Network Task (ANT, a combined cueing-and-flanker paradigm separating alerting, orienting, and executive-attention networks). Whether the advantage is real has been contested for over a decade, and much of the evidence base consists of small-sample studies aggregated by meta-analysis — the statistical combination of effect sizes (standardized mean differences such as Hedges' gg) across independent studies.

Meta-analyses of small studies are vulnerable to publication bias: studies with statistically significant, 'positive' results are more likely to be published and thus to enter the analysis, inflating the pooled estimate. A family of correction methods exists, each with known weaknesses. Funnel plots scatter effect sizes against precision, and asymmetry suggests missing studies. Egger's regression test detects such asymmetry. Trim-and-Fill iteratively imputes the mirror images of studies thought missing from a funnel plot and re-estimates the pooled effect. Fail-safe N computes how many unpublished null studies would be needed to nullify the pooled effect, compared against a critical value. PET-PEESE is a two-step regression of effect sizes on their standard errors (PET) or variances (PEESE), using the intercept — the extrapolated effect at zero sampling error — as the bias-corrected estimate; it has been argued to over-correct, sometimes yielding negative estimates of effects that are at worst zero. Selection models explicitly model the publication process (which pp-values get selected into print). Robust Bayesian meta-analysis (RoBMA) replaces the choice of a single method with Bayesian model-averaging across fixed- and random-effects models with and without publication-bias components (PET-PEESE-like and selection-model-like), weighting each by its predictive adequacy. All of these methods behave differently depending on the number of studies kk, the amount of between-study heterogeneity (true variation in effects across studies, quantified e.g. by τ2\tau^2 or I2I^2), and the true, unknown selection mechanism — exactly the conditions under which they disagree.

The substantive controversy precedes the source paper. De Bruin, Treccani and Della Sala (2015) showed that conference abstracts on bilingualism reporting positive executive-functioning results were more likely to appear in print than null ones, making publication bias a central concern. Lehtonen et al. (2018), in a meta-analysis of adults, found that after PET-PEESE correction the bilingual advantage in inhibitory control was statistically indistinguishable from zero. Donnelly, Brooks and Homer (2019) used a multiverse meta-analysis (systematically re-analyzing under all defensible analytic choices) of interference-control tasks and found weak evidence for an advantage, with PET-PEESE-based analyses especially skeptical. Against this backdrop, Ware, Kirkovski and Lum (2020) meta-analyzed bilingual–monolingual differences across tasks and age groups, pooled with a random-effects model, and applied Egger's test, Trim-and-Fill, fail-safe N, and PET-PEESE as sensitivity analyses — finding that the different corrections told different stories. On the methods side, Carter, Schönbrodt, Gervais and Hilgard (2019) had already shown by large-scale simulation on realistic psychological data that no single bias-correction method consistently outperforms the others across conditions, recommending sensitivity analysis instead of trusting one method. The question that survives is therefore a methodological one with direct substantive consequences: a defensible corrected estimate of the bilingual advantage requires knowing which correction to trust under this literature's specific conditions, and that knowledge does not yet exist.

Problem Statement

Determine which publication-bias correction approach most accurately corrects the observed effect size in the bilingualism-and-executive-functioning meta-analytic literature — the question posed verbatim by Ware, Kirkovski & Lum (2020): 'which approach most accurately corrects the observed effect size' — where the candidate approaches are those the paper itself applied (Egger's Test of Asymmetry, Duval and Tweedie's Trim and Fill, Classic Fail-Safe N, and PET-PEESE, together with the published-versus-unpublished comparison the authors used as an empirical benchmark) plus the later standard corrections for the same problem (selection models and robust Bayesian model-averaging), and the literature's defining conditions are moderate numbers of studies per analysis, large between-study heterogeneity, a mix of published and unpublished evidence, and an unknown selection mechanism.

Two clarifications fix the meaning of the target. First, the true effect is unobservable in this literature, so 'most accurate' is to be read against an accuracy criterion that the answer makes explicit — for example bias of the corrected estimate, root-mean-square error, interval coverage, or correctness of the null-versus-positive verdict — and the candidates are not all procedures of the same kind: PET-PEESE and Trim-and-Fill return a corrected effect size, whereas Classic Fail-Safe N returns the number of hypothetical un-retrieved studies that would return a significant average effect size to non-significance and Egger's test returns an asymmetry diagnostic, so what is compared is how accurately each procedure informs the true effect size under the criterion adopted. Second, the determination may be condition-indexed: the question does not require showing that one approach dominates uniformly over every selection mechanism compatible with the literature's unknown mechanism, but it does require stating the conditions under which the determination holds.

The source question is methodological. Ware, Kirkovski & Lum (2020) already report their own task- and age-specific pooled and bias-corrected estimates, tentatively suggest that PET-PEESE over-corrects, raise the inflation of average effect sizes as a separate 'key issue', and defer the accuracy question to 'future research evaluating meta-analysis methods'. The bias-corrected magnitude of the bilingual executive-functioning advantage — its size, its task and age dependence, and whether it is distinguishable from zero — is therefore the substantive stake of the determination rather than an additional required deliverable of it.

References

  1. Ware, A. T., Kirkovski, M., & Lum, J. A. G. (2020). Meta-Analysis Reveals a Bilingual Advantage That Is Dependent on Task and Age. Frontiers in Psychology, 11, 582. DOI: 10.3389/fpsyg.2020.01458. https://doi.org/10.3389/fpsyg.2020.01458 (source paper)
  2. de Bruin, A., Treccani, B., & Della Sala, S. (2015). Cognitive Advantage in Bilingualism: An Issue of Bilingual Advantage? Psychological Science, 26(1), 99–107. DOI: 10.1177/0956797614557866. https://doi.org/10.1177/0956797614557866
  3. Lehtonen, M., Soveri, A., Laine, A., Järvenpää, J., de Bruin, A., & Antfolk, J. (2018). Is bilingualism associated with enhanced executive functioning in adults? A meta-analytic review. Psychological Bulletin, 144(4), 394–425. DOI: 10.1037/bul0000142. https://doi.org/10.1037/bul0000142
  4. Donnelly, S., Brooks, P. J., & Homer, B. D. (2019). Is there a bilingual advantage on interference-control tasks? A multiverse meta-analysis of global reaction time and interference cost. Psychonomic Bulletin & Review, 26(4), 1122–1147. DOI: 10.3758/s13423-019-01567-z. https://doi.org/10.3758/s13423-019-01567-z
  5. Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019). Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods. Advances in Methods and Practices in Psychological Science, 2(2), 115–144. DOI: 10.1177/2515245919847196. https://doi.org/10.1177/2515245919847196
  6. Bartoš, F., Maier, M., Quintana, D. S., & Wagenmakers, E.-J. (2022). Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis. Advances in Methods and Practices in Psychological Science, 5(3). DOI: 10.1177/25152459221109259. https://doi.org/10.1177/25152459221109259
  7. Maier, M., Bartoš, F., & Wagenmakers, E.-J. (2023). Robust Bayesian meta-analysis: Addressing publication bias with model-averaging. Psychological Methods, 28(1), 107–122. DOI: 10.1037/met0000405. https://doi.org/10.1037/met0000405
  8. Sladekova, M., Webb, L. E. A., & Field, A. P. (2023). Estimating the change in meta-analytic effect-size estimates after the application of publication bias adjustment methods. Psychological Methods, 28(3), 664–686. DOI: 10.1037/met0000470. https://doi.org/10.1037/met0000470
  9. Bartoš, F., Maier, M., Shanks, D. R., Stanley, T. D., Sladekova, M., & Wagenmakers, E.-J. (2023). Meta-analyses in psychology often overestimate evidence for and size of effects. Royal Society Open Science, 10(7), 230224. DOI: 10.1098/rsos.230224. https://doi.org/10.1098/rsos.230224
  10. Leivada, E. (2023). A Classification Bias and an Exclusion Bias Jointly Overinflated the Estimation of Publication Biases in Bilingualism Research. Behavioral Sciences, 13(10), 812. DOI: 10.3390/bs13100812. https://doi.org/10.3390/bs13100812
  11. Privitera, A. J., Momenian, M., & Weekes, B. S. (2023). Modeling the bilingual advantage: Do results differ between analysis methods? Ampersand, 11, 100134. DOI: 10.1016/j.amper.2023.100134. https://doi.org/10.1016/j.amper.2023.100134
  12. Paap, K. R., Majoubi, J., Balakrishnan, N., & Anders-Jefferson, R. T. (2024). Bilingualism, like other types of brain training, does not produce far transfer: It all fits together. International Journal of Bilingualism, 29(1), 114–127 (online first January 2024). DOI: 10.1177/13670069231214599. https://doi.org/10.1177/13670069231214599
  13. Privitera, A. J., Han, J., Han, K., Yee, J., & Yin, Y. (2026). Bilingual effects on cognitive control: a comprehensive bibliometric analysis of 50 years of research. Frontiers in Psychology, 17, 1905413. DOI: 10.3389/fpsyg.2026.1905413. https://doi.org/10.3389/fpsyg.2026.1905413