Whether bilingualism research systematically suppresses negative results — findings of bilingual disadvantages — remains unresolved. The landmark quantification by de Bruin, Treccani and Della Sala (2015), which traced the fate of 104 conference abstracts and concluded that studies fully supporting the bilingual-advantage hypothesis were more than twice as likely to be published as studies challenging it, was shown by Leivada (2023) to be overinflated by two artifacts: a classification bias that merged null and negative outcomes into one category, and an exclusion bias that omitted published dissertations, book chapters, and conference proceedings. The corrected analyses, however, only show that the originally claimed large bias is unsupported; they cannot establish that no bias against negative results exists, because the underlying corpus contained only four identifiable negative-result abstracts, many older conference abstracts are no longer accessible, and the field lacks an overarching theory of bilingual effects that could justify classifying null and negative outcomes as distinct or predict advantage–disadvantage trade-offs. Later work has analyzed the composition of the published record, has bibliometrically mapped published and unpublished output in the field, and has developed a trade-off framework, but no grey-literature-inclusive, classification-sensitive quantification of the publication bias against negative results has been conducted. The problem is to determine the true publication landscape for positive, mixed, null, and negative bilingual-effects findings.

Background

A widely held view in cognitive psychology holds that speaking two languages improves executive functions — the domain-general cognitive processes (inhibition of irrelevant responses, task switching, and working-memory updating) that control goal-directed behavior. This "bilingual advantage hypothesis" has been contested for over a decade: meta-analyses disagree, with some reporting small advantages that vary by task and age, others reporting no reliable advantage in adults after correcting for small-study effects, and critics attributing positive findings to confirmation bias (Lehtonen et al. 2018, doi:10.1037/bul0000142; Donnelly, Brooks and Homer 2019, doi:10.3758/s13423-019-01567-z; Grundy 2020, doi:10.1007/s41809-020-00062-5; Paap et al. 2020, doi:10.1177/1747021819900098). Publication bias — the phenomenon whereby studies with statistically significant or theory-supportive results are more likely to be published than null results — became a central explanation for the perceived fragility of bilingualism effects (Leivada, Westergaard, Duñabeitia and Rothman 2021, doi:10.1017/S1366728920000358). The landmark quantification is de Bruin, Treccani and Della Sala (2015, doi:10.1177/0956797614557866), who classified 104 conference abstracts on bilingualism and executive control (1999–2012) by outcome — full support for the bilingual advantage, mixed results, or null/negative results — and tracked which studies were later published in journals: 68% of fully supporting abstracts were published versus an estimated 29% of null/negative ones, and a funnel-plot asymmetry test was read as further evidence of bias. (A funnel plot scatters each study's effect size against its precision; if only small studies with large effects get published, the plot becomes asymmetric — but asymmetry can also arise when sample size is correlated with effect size, and the studies pooled here ranged from Simon-task reaction times to age of dementia diagnosis, so the diagnostic is contested.) The claim was immediately disputed by Bialystok, Kroll, Green, MacWhinney and Craik (2015, doi:10.1177/0956797615573759), with a reply from de Bruin et al. (doi:10.1177/0956797615583443), and has since been cited hundreds of times as established evidence that the field suppresses non-supportive results. Leivada (2023, doi:10.3390/bs13100812) re-analyzed the same dataset using Bayesian statistics (a Bayes factor, BF, expresses how probable the data are under the alternative versus the null hypothesis; values near 1 are inconclusive) and found: (i) a classification bias — the original "null/negative" mega-category conflated effects indistinguishable from zero with genuine bilingual disadvantages, yet only the positive-versus-null contrast was robust (BF ≈ 9.3), not positive-versus-negative (BF ≈ 0.51); and (ii) an exclusion bias — the original dataset counted only journal articles, omitting published dissertations, book chapters, and proceedings; once those are included, 33/40 positive versus 14/17 negative-or-null abstracts were published by 2023, with no detectable difference (BF ≈ 0.29). Importantly, Leivada concluded that these corrections show the originally claimed large bias is unsupported, not that a genuine bias against negative results is absent: only four negative-result abstracts could be identified, most 10–20-year-old conference abstracts are no longer accessible (so negative-result studies may have been missed), and — more fundamentally — the field lacks an overarching theory of bilingual effects that could justify treating null and negative outcomes as distinct categories or could predict systematic trade-off relationships between advantages and disadvantages, as proposed in the trade-off framework of Leivada, Dentella, Masullo and Rothman (2023, doi:10.1017/S1366728922000761). Grey literature — unpublished or non-journal outputs such as dissertations, conference proceedings, and book chapters — is precisely where suppressed negative results would surface, and its systematic exclusion is itself a recognized form of publication bias (location bias).

Problem Statement

Does a genuine publication bias against negative results — findings of bilingual disadvantages — exist in bilingualism research, and what is the true publication landscape for positive, mixed, null, and negative bilingual-effects findings — that is, the publication probability of each outcome category, with uncertainty? Here mixed results are those in which some, but not all, findings of a given study support the bilingual-advantage hypothesis (results partly supporting and partly challenging it), following the source dataset's classification, in which the original 'partly supporting' and 'partly challenging' subcategories were merged because no objective criteria distinguished them (Leivada 2023). The landscape in question is the publication probability, with an uncertainty interval, for each of the four outcome categories — positive, mixed, null, and negative — over a systematic corpus that includes grey literature (dissertations, conference proceedings, book chapters, and other non-journal outputs) rather than journal articles alone, and with an outcome classification that treats null results (group differences indistinguishable from zero) and negative results (bilingual disadvantages) as distinct categories. Because studies that were never published, or whose records are no longer retrievable, cannot enter any observable corpus, this landscape cannot by itself prove that a bias against negative results is absent; the existence question is bounded by the attainable record and turns on the negative-result publication probability compared with the positive-result publication probability, so a residual sampling limitation may still hide a bias. No single corpus frame is canonical: its time window, the admissible grey-literature output types (dissertations, conference proceedings, book chapters, and other non-journal outputs), and how 'attainable' is bounded are chosen and justified by the solver.

References

  1. de Bruin, A., Treccani, B., & Della Sala, S. (2015). Cognitive advantage in bilingualism: An example of publication bias? Psychological Science, 26(1), 99–107. DOI: 10.1177/0956797614557866. https://doi.org/10.1177/0956797614557866
  2. Bialystok, E., Kroll, J. F., Green, D. W., MacWhinney, B., & Craik, F. I. M. (2015). Publication bias and the validity of evidence. Psychological Science, 26(6), 944–946. DOI: 10.1177/0956797615573759. https://doi.org/10.1177/0956797615573759
  3. de Bruin, A., Treccani, B., & Della Sala, S. (2015). The connection is in the data. Psychological Science, 26(6), 947–949. DOI: 10.1177/0956797615583443. https://doi.org/10.1177/0956797615583443
  4. Lehtonen, M., Soveri, A., Laine, A., Järvenpää, J., de Bruin, A., & Antfolk, J. (2018). Is bilingualism associated with enhanced executive functioning in adults? A meta-analytic review. Psychological Bulletin, 144(4), 394–425. DOI: 10.1037/bul0000142. https://doi.org/10.1037/bul0000142
  5. Donnelly, S., Brooks, P. J., & Homer, B. D. (2019). Is there a bilingual advantage on interference-control tasks? A multiverse meta-analysis of global reaction time and interference cost. Psychonomic Bulletin & Review, 26(4), 1122–1147. DOI: 10.3758/s13423-019-01567-z. https://doi.org/10.3758/s13423-019-01567-z
  6. Grundy, J. G. (2020). The effects of bilingualism on executive functions: An updated quantitative analysis. Journal of Cultural Cognitive Science, 4(2), 177–199. DOI: 10.1007/s41809-020-00062-5. https://doi.org/10.1007/s41809-020-00062-5
  7. Paap, K. R., Mason, L., Zimiga, B., Ayala-Silva, Y., & Frost, M. (2020). The alchemy of confirmation bias transmutes expectations into bilingual advantages: A tale of two new meta-analyses. Quarterly Journal of Experimental Psychology, 73(8), 1290–1299. DOI: 10.1177/1747021819900098. https://doi.org/10.1177/1747021819900098
  8. Leivada, E., Westergaard, M., Duñabeitia, J. A., & Rothman, J. (2021). On the phantom-like appearance of bilingualism effects on neurocognition: (How) should we proceed? Bilingualism: Language and Cognition, 24(1), 197–210. DOI: 10.1017/S1366728920000358. https://doi.org/10.1017/S1366728920000358
  9. Leivada, E., Dentella, V., Masullo, C., & Rothman, J. (2023). On trade-offs in bilingualism and moving beyond the Stacking the Deck fallacy. Bilingualism: Language and Cognition, 26(3), 550–555. DOI: 10.1017/S1366728922000761. https://doi.org/10.1017/S1366728922000761
  10. Leivada, E. (2023). A classification bias and an exclusion bias jointly overinflated the estimation of publication biases in bilingualism research. Behavioral Sciences, 13(10), 812. DOI: 10.3390/bs13100812. https://doi.org/10.3390/bs13100812
  11. Dentella, V., Masullo, C., & Leivada, E. (2024). Bilingual disadvantages are systematically compensated by bilingual advantages across tasks and populations. Scientific Reports, 14, 2107. DOI: 10.1038/s41598-024-52417-5. https://doi.org/10.1038/s41598-024-52417-5
  12. Declerck, M., & Kirk, N. W. (2023). No evidence for a mixing benefit — A registered report of voluntary dialect switching. PLoS ONE, 18(5), e0282086. DOI: 10.1371/journal.pone.0282086. https://doi.org/10.1371/journal.pone.0282086
  13. Leivada, E., Kelly-Iturriaga, L., Masullo, C., Westergaard, M., & Rothman, J. (2026). The unpredictable role of language distance in bilingual cognition: A systematic review from brain to behavior. Bilingualism: Language and Cognition, 29(4), 1064–1077 (first published online November 2025). DOI: 10.1017/S1366728925100746. https://doi.org/10.1017/S1366728925100746
  14. Privitera, A. J., Han, J., Han, K., Yee, J., & Yin, Y. (2026). Bilingual effects on cognitive control: a comprehensive bibliometric analysis of 50 years of research. Frontiers in Psychology, 17, Article 1905413. DOI: 10.3389/fpsyg.2026.1905413. https://doi.org/10.3389/fpsyg.2026.1905413