Cross-sectional international assessments repeatedly find that students who report more frequent ICT use at school and for schoolwork score lower in mathematics, reading, and science, while some leisure and attitude measures associate positively — a pattern first documented at scale in PISA data and confirmed by Hu, Gong, Lai and Leung (2018) with three-level multilevel models across 44 countries. Whether this negative association reflects genuinely harmful uses of technology, reverse causation (lower-achieving students and compensating schools using ICT more), or confounding and measurement artifacts remains unresolved, as does the question of which specific uses, task types, and instructional designs produce beneficial rather than harmful effects. The problem is to explain and adjudicate this negative association — establish its causal status, decompose it by type, quality, and pedagogical context of use, and test the mediating and moderating roles of teacher and classroom factors. The existing literature is cross-sectional and frequency-based and lacks identification-capable designs, quality- and subject-sensitive ICT-use instruments, and teacher-level data linked to student outcomes.

Background

ICT (information and communication technology) — computers, tablets, the Internet, and digital applications — has been promoted in schools worldwide on the assumption that it improves learning, and governments have invested heavily in educational technology. International large-scale assessments provide the main evidence base on whether this assumption holds. PISA (the OECD's — the Organisation for Economic Co-operation and Development's — Programme for International Student Assessment) tests representative samples of 15-year-olds in mathematics, reading, and science literacy every three years and administers questionnaires on students' ICT availability, frequency of use in different contexts (for example the PISA scales for use at school, academic use outside school, and entertainment use outside school), and attitudes toward ICT. Because students are nested in schools and schools in countries, such data are analyzed with hierarchical linear modeling (multilevel regression), which estimates associations at the student, school, and country level simultaneously. TIMSS (the IEA's — the International Association for the Evaluation of Educational Achievement's — Trends in International Mathematics and Science Study) is a comparable international assessment of younger students' mathematics and science achievement. The pivotal antecedent finding came from the OECD's 2015 report Students, Computers and Learning: countries and students with heavy ICT investment and use showed no advantage, and the raw association between the frequency of school computer use and performance was negative in most participating systems — the 'ICT paradox'. Hu, Gong, Lai and Leung (2018) gave this paradox its most complete multilevel treatment: using PISA 2015 data on 305,414 students in 11,075 schools across 44 countries, they found that ICT availability at school (school average) associated positively with literacy while home availability associated negatively; student academic use of ICT (use at school and schoolwork-related use at home) associated negatively with performance while entertainment use associated positively; and attitude measures showed mixed signs. Crucially, this design — like nearly all work in this literature — is cross-sectional: it measures use and achievement at the same time, on the same students, with frequency-based questionnaire scales. The authors' discussion therefore framed a research agenda: the PISA measures record how often broad categories of activity occur, not their pedagogical quality, subject specificity, or task type; the negative sign cannot be distinguished from reverse causation (lower-achieving students, or schools serving them, using ICT more to compensate); teacher-level ICT practice and classroom pedagogy are absent from the student-outcome datasets, so mediating mechanisms are untested; coverage skews toward developed countries; and interactions between ICT use and other elements of the learning environment were not modeled. A separate econometric literature has attacked adjacent margins with credibly causal designs — randomized provision of home computers (Fairlie and Robinson 2013, in the United States; Beuermann et al. 2015, One Laptop per Child in Peru, a program distributing low-cost laptops to whole schools; and Cueto et al. 2025, the ten-year follow-up of that same randomization), and a regression discontinuity design (comparing students just above and below an eligibility cutoff) in a Romanian government voucher program (Malamud and Pop-Eleches 2011, DOI 10.1093/qje/qjr008) — generally finding null or negative achievement effects of device access. Falck, Mang and Woessmann (2018) exploited within-student, between-subject variation in TIMSS to decompose classroom computer use by type, finding positive associations for using computers to look up information and negative ones for skill practice. These studies establish that causal identification of ICT effects is possible, but they concern device access or specific use types in single (mostly developed) countries, not the international use-frequency constructs whose negative associations motivate the question.

Problem Statement

Explain the negative association between student ICT use and literacy documented in international assessments, along the axes of the research agenda set out by the source study's authors (Hu, Gong, Lai and Leung, 2018): (1) determine whether the negative associations between school ICT use, academic ICT use outside school, and school-average ICT use and mathematics, reading, and science literacy reflect harmful effects of particular uses, reverse causation (lower-achieving students using ICT more to compensate, or compensatory deployment in schools serving weaker intakes), or confounding and selection — a question that the existing cross-sectional, frequency-based designs cannot settle, and that therefore calls for evidence capable of causal identification; (2) identify which specific ICT uses, software features, task types, and instructional designs, at what intensity, duration, and pedagogical quality of use, produce beneficial rather than harmful effects on the three literacy domains — a decomposition that existing frequency-based measures do not capture; (3) test the mediating and moderating roles of teacher-level ICT factors and classroom pedagogical practice, linking teacher and classroom measures to student outcomes, and the interactions of ICT use with content, teaching practices, and assessment; and (4) establish whether the findings generalize across countries at different development levels, beyond the developed-system weighting of PISA 2015.

The four axes are jointly posed and jointly constitute the problem, which is to explain the negative association across all of them.

References

  1. Hu, Xiang; Gong, Yang; Lai, Chun; Leung, Frederick K. S. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education 125:1–13. DOI 10.1016/j.compedu.2018.05.021. https://doi.org/10.1016/j.compedu.2018.05.021
  2. OECD (2015). Students, Computers and Learning: Making the Connection. OECD Publishing. DOI 10.1787/9789264239555-en. https://doi.org/10.1787/9789264239555-en
  3. Falck, Oliver; Mang, Constantin; Woessmann, Ludger (2018). Virtually No Effect? Different Uses of Classroom Computers and their Effect on Student Achievement. Oxford Bulletin of Economics and Statistics 80(1):1–38. DOI 10.1111/obes.12192. https://doi.org/10.1111/obes.12192
  4. Malamud, Ofer; Pop-Eleches, Cristian (2011). Home Computer Use and the Development of Human Capital. The Quarterly Journal of Economics 126(2):987–1027. DOI 10.1093/qje/qjr008. https://doi.org/10.1093/qje/qjr008
  5. Fairlie, Robert W.; Robinson, Jonathan (2013). Experimental Evidence on the Effects of Home Computers on Academic Achievement among Schoolchildren. American Economic Journal: Applied Economics 5(3):211–240. DOI 10.1257/app.5.3.211. https://doi.org/10.1257/app.5.3.211
  6. Beuermann, Diether W.; Cristia, Julian; Cueto, Santiago; Malamud, Ofer; Cruz-Aguayo, Yyannu (2015). One Laptop per Child at Home: Short-Term Impacts from a Randomized Experiment in Peru. American Economic Journal: Applied Economics 7(2):53–80. DOI 10.1257/app.20130267. https://doi.org/10.1257/app.20130267
  7. Cueto, Santiago; Beuermann, Diether W.; Cristia, Julian; Malamud, Ofer; Pardo, Francisco (2025). Laptops in the Long Run: Evidence from the One Laptop per Child Program in Rural Peru. NBER Working Paper 34495. DOI 10.3386/w34495. https://doi.org/10.3386/w34495
  8. Gubbels, Joyce; Swart, Nicole M.; Groen, Margriet A. (2020). Everything in moderation: ICT and reading performance of Dutch 15-year-olds. Large-scale Assessments in Education 8(1):1. DOI 10.1186/s40536-020-0079-0. https://doi.org/10.1186/s40536-020-0079-0
  9. Hu, Jie; Yu, Rushi (2021). The effects of ICT-based social media on adolescents' digital reading performance: A longitudinal study of PISA 2009, PISA 2012, PISA 2015 and PISA 2018. Computers & Education 175:104342. DOI 10.1016/j.compedu.2021.104342. https://doi.org/10.1016/j.compedu.2021.104342
  10. Zhu, Jinxin; Li, Sandy C. (2022). The non-linear relationships between ICT use and academic achievement of secondary students in Hong Kong. Computers & Education 187:104546. DOI 10.1016/j.compedu.2022.104546. https://doi.org/10.1016/j.compedu.2022.104546
  11. Li, Sandy C.; Zhu, Jinxin (2023). Cognitive-motivational engagement in ICT mediates the effect of ICT use on academic achievements: Evidence from 52 countries. Computers & Education 204:104871. DOI 10.1016/j.compedu.2023.104871. https://doi.org/10.1016/j.compedu.2023.104871
  12. Erdogdu, Funda (2022). ICT, learning environment and student characteristics as potential cross-country predictors of academic achievement. Education and Information Technologies 27(5):7135–7159. DOI 10.1007/s10639-021-10848-x. https://doi.org/10.1007/s10639-021-10848-x
  13. Li, Ziying; Zhang, Shan; Huggins-Manley, A. Corinne; Botelho, Anthony F.; Zhang, Huibin (2026). Evaluating the causal effects of digital use quality on students' ICT literacy. Journal of Research on Technology in Education (online first) 1–25. DOI 10.1080/15391523.2025.2603508. https://doi.org/10.1080/15391523.2025.2603508
  14. Caponera, Elisa; Colonnello, Valentina; Palmerio, Laura; Russo, Paolo Maria (2026). The 'golden mean' of ICT use: how moderate use and digital competence predict mathematics achievement. Frontiers in Education 11:1860483. DOI 10.3389/feduc.2026.1860483. https://doi.org/10.3389/feduc.2026.1860483