Large language models (LLMs) produce fluent language, yet the field possesses no unambiguous empirical criterion — independent of human judgment — for deciding whether such a system genuinely understands natural language or merely behaves as if it understands. Havlík (2024) argued that understanding can in principle be attributed to LLMs, grounding meaning not in the world but in the linguistic corpus as a whole ('semantic fragmentism'), while explicitly leaving unresolved the absence of a judgment-independent criterion distinguishing genuine understanding from 'as-if' understanding (simulation vs duplication of a language game). The related ostension issue — whether LLMs can achieve the full variety of human language games, in particular those requiring ostensive definitions (pointing at the world) — appears in Havlík as Søgaard's (2022) concern, which Havlík reports and answers conceptually rather than leaves unresolved: he argues that the referential grounding of meanings is 'an illusion' and that the pointing game has 'a primarily correlative and synchronizing role.' The criterion question remains open in the 2022–2026 literature: the main subsequent proposal (Leyton-Brown and Shoham's pragmatic framework) deliberately replaces the genuine-vs-as-if distinction with a performance-based test rather than resolving it, and grounding/pragmatics work still treats referential grounding and ostensive language games as unsolved.

Background

A large language model (LLM) is a neural network, typically a Transformer (an attention-based neural architecture), trained on massive text corpora to predict the next token (word fragment); its competence is acquired purely from textual statistics, without direct interaction with the world. Whether such a system can be said to understand language is an old question in new clothing. Searle's (1980) 'Chinese Room' argument held that symbol manipulation alone ('syntax') can never constitute understanding ('semantics'), and Harnad's (1990) symbol grounding problem sharpened the complaint: symbols acquired only from other symbols, without perceptual grounding, remain ungrounded. Critics of LLMs repeat this line — fluent output is statistical pattern-matching, not meaning.

Havlík (2024, Synthese; preprint arXiv:2310.17407) pushed back. He argues that (a) understanding must be distinguished from awareness of understanding — consciousness is not a causal precondition for understanding; (b) the sharp syntax/semantics gap is untenable, because minimal semantic content already emerges from syntactic structure; and (c) the meanings of an LLM's expressions can be grounded not in extra-linguistic reality but in the linguistic corpus as a whole, a view he calls semantic fragmentism: the minimal unit of meaning is a 'minimal corpus' that is semantically saturated, i.e., large enough that adding further expressions would not change how the meanings of its parts are grounded. This builds on the meaning-as-use tradition in the philosophy of language, on which the meaning of an expression is determined by its role in rule-governed practices called language games; an ostensive definition is the paradigmatic case of fixing a word's meaning by pointing at instances in the world, rather than by using other words. On this tradition, following Søgaard's (2022, Synthese) analysis of what it takes for models to 'understand' language, LLMs can already play many language games at or above human level, but games hinging on ostension — on live contact with the world being talked about — remain the crucial open case on Søgaard's analysis.

Havlík distinguishes a system that merely simulates a language game from one that duplicates it, and conjectures that understanding could be among the emergent abilities of sufficiently large models — abilities present at large scale but not small scale (Wei et al. 2022), for which circumstantial internal evidence exists, such as world representations discovered inside a trained sequence model (Li et al. 2023). But he concedes a cluster of unresolved issues. First, and most fundamental: 'there is no unambiguous criterion that can determine independently of us whether a system only behaves as if it understands or whether it actually understands.' The intended empirical companion — confirming that LLM mechanisms are intrinsically similar to human language processing — is blocked because, as Bowman (2023) put it, precise explanations of LLM behavior appear too complex for humans to grasp. Second, the paper explicitly disclaims that semantic fragmentism is 'a practical procedure for determining meaning saturation and thus minimum corpus size.' Third, whether corpus-grounded meaning survives corrupted or biased training corpora is unaddressed. Fourth, the relation between consciousness and understanding (whether some forms of understanding require phenomenal consciousness, i.e., the felt, subjective character of experience) is left open. The ostensive language games, by contrast, are not left open by Havlík: he answers Søgaard's concern conceptually, arguing that the referential grounding of meanings is 'an illusion' and that the pointing game has 'a primarily correlative and synchronizing role.'

Adjacent later work reshapes but does not close these questions. Mollo and Millière (2026; preprint 2023) reframe grounding as the 'vector grounding problem' — whether LLM internal states can be about the world independently of human interpreters — and argue affirmatively on teleosemantic grounds (teleosemantics being the view that a system's internal states are about whatever their proper biological or design function tracks), a contested philosophical proposal with no empirical test. Leyton-Brown and Shoham (2024) propose a rigorous pragmatic framework for testing understanding via question-answering performance with statistical confidence bounds — explicitly in 'Turing-test fashion', hence deliberately sidestepping the genuine-versus-as-if distinction at the heart of Havlík's open question.

Problem Statement

The problem, as left open by Havlík (2024) and still open in the 2026 literature, has one objective — putting the attribution of natural-language understanding to LLMs on an objective, human-judgment-independent empirical footing — carried by one core question:

The criterion question. Does there exist an unambiguous empirical criterion, applicable independently of human judgment, that determines whether an LLM genuinely understands natural language or merely behaves as if it understands — equivalently, whether it duplicates rather than only simulates a language game? A resolution may take either form: (a) such a criterion, made operational (what is measured, on which systems, by which protocol and decision rule) and validated, including robustness to systems engineered to mimic understanding superficially; or (b) a principled argument that no such criterion can exist (or that the genuine/as-if distinction is ill-posed), which would itself resolve the question by reconstruction.

Linked context acknowledged by the source, which the same objective touches but which does not constitute independent mandatory completion requirements: (i) turning semantic fragmentism into a practical procedure — the source explicitly disclaims providing one — defining, at least for bounded domains, what a minimal semantically saturated corpus is and how meaning grounding within it is measured; (ii) whether corpus-grounded meaning is robust to corrupted or biased training corpora, extending the source's remark that errors in training data significantly affect model functionality; and (iii) the source's conjecture that understanding-like capacities are genuine emergent properties of sufficiently complex systems, which awaits empirical confirmation. The consciousness question — whether some form of understanding requires phenomenal consciousness — remains a standing caveat on any attribution, since Havlík's case rests on denying it is a precondition. The ostension debate — whether games requiring pointing at, or direct interaction with, the world are achievable by text-trained systems — is background to this question: it is Søgaard's reported concern, which Havlík answers by conceptual argument.

References

  1. Havlík, Vladimír (2024). Meaning and understanding in large language models. Synthese 205(1). DOI: 10.1007/s11229-024-04878-4. https://doi.org/10.1007/s11229-024-04878-4 (preprint: arXiv:2310.17407, https://arxiv.org/abs/2310.17407)
  2. Søgaard, Anders (2022). Understanding models understanding language. Synthese 200(6), 443. DOI: 10.1007/s11229-022-03931-4. https://doi.org/10.1007/s11229-022-03931-4
  3. Harnad, Stevan (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena 42(1–3), 335–346. DOI: 10.1016/0167-2789(90)90087-6. https://doi.org/10.1016/0167-2789(90)90087-6
  4. Searle, John R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences 3(3), 417–424. DOI: 10.1017/S0140525X00005756. https://doi.org/10.1017/S0140525X00005756
  5. Bowman, Samuel R. (2023). Eight things to know about large language models. arXiv:2304.00612. https://arxiv.org/abs/2304.00612
  6. Mollo, Dimitri Coelho; Millière, Raphaël (2026). The vector grounding problem. Philosophy and the Mind Sciences 7(1). DOI: 10.33735/phimisci.2026.12307. https://doi.org/10.33735/phimisci.2026.12307 (preprint: arXiv:2304.01481, https://arxiv.org/abs/2304.01481)
  7. Leyton-Brown, Kevin; Shoham, Yoav (2024). Understanding understanding: A pragmatic framework motivated by large language models. arXiv:2406.10937. https://arxiv.org/abs/2406.10937
  8. Li, Kenneth; Hopkins, Aspen K.; Bau, David; Viégas, Fernanda; Pfister, Hanspeter; Wattenberg, Martin (2023). Emergent world representations: Exploring a sequence model trained on a synthetic task. ICLR 2023. arXiv:2210.13382. https://arxiv.org/abs/2210.13382
  9. Wei, Jason; Tay, Yi; Bommasani, Rishi; Raffel, Colin; Zoph, Barret; Borgeaud, Sebastian; Yogatama, Dani; Bosma, Maarten; Zhou, Denny; Metzler, Donald; Chi, Ed H.; Hashimoto, Tatsunori; Vinyals, Oriol; Liang, Percy; Dean, Jeff; Fedus, William (2022). Emergent abilities of large language models. Transactions on Machine Learning Research. arXiv:2206.07682. https://arxiv.org/abs/2206.07682
  10. García-Valdecasas, Miguel (2026). Are large language models intentional? The limits of referential grounding. Philosophy & Technology 39(2). DOI: 10.1007/s13347-026-01079-4. https://doi.org/10.1007/s13347-026-01079-4
  11. Beckmann, Pierre (2025). New horizons in machine understanding: explanatory and objectual understanding in deep learning video generation models. Synthese 206(6). DOI: 10.1007/s11229-025-05174-5. https://doi.org/10.1007/s11229-025-05174-5
  12. Gvoždiak, Vít (2025). Pragmatics beyond humans: meaning, communication, and LLMs. arXiv:2508.06167. https://arxiv.org/abs/2508.06167