The AI Scientist (Lu et al., 2024) was the first system to automate machine-learning research end to end — idea generation, code, experiments, paper writing, and automated review — but its outputs are unreliable: it occasionally hallucinates entire result tables and experimental facts (e.g., which GPUs were used), miscompares changed metrics against baselines, and its reported numbers are not independently verified. The paper's own discussion identifies automatic verification of reported results — for example, an automated verifier that independently reproduces them — as crucial future work. This problem takes up exactly that question: seeing if an independent automatic verifier, given the artifacts an autonomous research system produces (code, experimental runs, data, manuscript), can re-execute or otherwise directly validate the claimed experiments and automatically detect and reject fabricated, misattributed, or misinterpreted results, at scale and without per-paper human inspection. As of 2026 the problem is open: benchmarks report that frontier coding agents still produce fabricated or invalidated experimental results in most autonomous-research attempts, and a 2026 survey coding 24 runnable AI-scientist systems finds that none demonstrates an externally validated in-loop verification oracle.

Background

Large language models (LLMs) are neural networks trained on text and code that can drive autonomous agents — programs that plan, call tools, write and execute code, and iterate on errors with limited human intervention. 'AI scientist' systems apply such agents to the research pipeline itself: generating hypotheses, implementing and running experiments, analyzing results, and writing manuscripts. The AI Scientist (Lu et al., 2024) was the first end-to-end demonstration: given a short seed idea and a human-written code template (e.g., a small 2D diffusion model or a character-level Transformer), it iterated modifications, ran experiments, produced complete LaTeX papers for under $15 each, and scored them with an automated LLM reviewer validated against human ICLR 2022 reviews. An 'ablation' is a control experiment that removes a component to measure its contribution; a 'baseline' is the reference method a change is compared against; 'hallucination' here means confident text asserting results or facts that were never actually produced or observed. The paper's limitations section documents concrete reliability failures: the system struggled to compare the magnitude of two numbers, sometimes compared a redefined metric against the old baseline's numbers as if they were commensurable, occasionally fabricated an entire ablations table it had not run, and hallucinated experimental details such as the GPU model used. Its mitigations were limited to prompt instructions ('only include results directly observed') and storing copies of executed files; the authors state that one should manually check the implementation before trusting reported results. The discussion names the missing piece: 'future work should address the reliability and hallucination concerns, potentially through a more in-depth automatic verification of the reported results … by directly linking code and experiments, or by seeing if an automated verifier can independently reproduce the results.' Subsequent work has made the need more precise rather than answered it. MLR-Bench (Chen et al., 2025) found that coding agents completing open-ended ML research tasks 'frequently (e.g., in 80% of the cases) produce fabricated or invalidated experimental results,' and contributed MLR-Judge, an LLM-plus-rubric paper evaluator — which scores write-ups against rubrics but does not re-execute or independently check the claimed experiments. PaperBench (Starace et al., 2025) showed that grading replication attempts can be largely automated (its best LLM judge reaches an F1 of 0.83 against gold labels from human expert graders on its JudgeEval auxiliary benchmark, built from human-graded partial replications), but it measures the capability of the replicating agent, not the correctness of a discoverer's claims, and the best agent reached only a 21.0% average replication score across the benchmark's 8,316 individually gradable rubric tasks. A 2026 survey of autonomous research agents (Ding et al., 2026) audited the field along verification dimensions: most systems release code, but few release execution traces or any novelty-verification method, and no LLM-era system in its corpus has an externally validated in-loop oracle that checks results as they are generated. Component-level detectors have begun to appear — CARTOGRAPH (Shah and Kartik, 2026), a steering/refusal layer whose retrospective audit flagged all later-disconfirmed claims from the A-Lab autonomous materials laboratory, and the regime-shift check of Eulig (2026) for agents whose conclusions overgeneralize their own data — but these are pieces of verification machinery, not an at-scale independent verifier for autonomous discovery outputs. The problem below takes up the question the source paper itself posed: seeing if a verifier that is independent of the discovery system can check experimental claims directly (by re-execution or equivalent validation), catch the documented failure modes (fabricated results, misattributed facts, incommensurable baseline comparisons, non-reproducible numbers), and work at scale without per-paper human inspection.

Problem Statement

Determine whether automatic verification can serve as the trust gate that lets autonomous end-to-end discovery be treated as a reliable scientific instrument — the question The AI Scientist's discussion poses as 'seeing if an automated verifier can independently reproduce the results.' The route to an affirmative answer is an independent automatic verifier. Given the complete artifact bundle that an autonomous research system produces for a study — source code, experimental configuration, execution logs, data, and the resulting manuscript — such a verifier must, without human inspection per study: (1) check reported experimental claims directly, by re-executing the experiments or by an equivalent validation that links each reported number, table, and figure to the artifacts and code that produced it; (2) detect and reject the documented failure modes — hallucinated or fabricated results (including entire ablation tables), incorrectly attributed experimental facts, comparisons of a changed metric against a baseline as if commensurable, and numeric results that do not reproduce — while accepting valid reported results rather than flagging them, with its discrimination measured on a labelled corpus that mixes valid and invalid claims and reported as a stated sensitivity and specificity, so that a verifier which flags every study does not satisfy this condition; and (3) operate at scale, defined throughout this task as at least 100 distinct AI-generated artifact bundles with zero per-study human labor and a per-study verification cost (wall-clock and accounted expense) no greater than the per-study cost of producing the study. A working verifier meeting these conditions would answer the source's question affirmatively; a rigorous, well-supported demonstration that independent automatic verification meeting (1)–(3) at that same scale is not achievable would answer it negatively. Sibling open issues raised in the same source — automated-reviewer evaluation, sandboxing and operational governance, generality beyond code-executable ML templates, and whether such systems can produce paradigm-shifting ideas — are separate problems and are not part of this problem's completion boundary.

References

  1. Lu, Chris; Lu, Cong; Lange, Robert Tjarko; Foerster, Jakob; Clune, Jeff; Ha, David (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv:2408.06292, DOI 10.48550/arXiv.2408.06292. https://arxiv.org/abs/2408.06292
  2. Yamada, Yutaro; Lange, Robert Tjarko; Lu, Cong; Hu, Shengran; Lu, Chris; Foerster, Jakob; Clune, Jeff; Ha, David (2025). The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search. arXiv:2504.08066, DOI 10.48550/arXiv.2504.08066. https://arxiv.org/abs/2504.08066
  3. Chen, Hui; Xiong, Miao; Lu, Yujie; Han, Wei; Deng, Ailin; He, Yufei; Wu, Jiaying; Li, Yibo; Liu, Yue; Hooi, Bryan (2025). MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research. arXiv:2505.19955, DOI 10.48550/arXiv.2505.19955. https://arxiv.org/abs/2505.19955
  4. Starace, Giulio; Jaffe, Oliver; Sherburn, Dane; Aung, James; Chan, Jun Shern; Maksin, Leon; Dias, Rachel; Mays, Evan; Kinsella, Benjamin; Thompson, Wyatt; Heidecke, Johannes; Glaese, Amelia; Patwardhan, Tejal (2025). PaperBench: Evaluating AI's Ability to Replicate AI Research. arXiv:2504.01848, DOI 10.48550/arXiv.2504.01848. https://arxiv.org/abs/2504.01848
  5. Ding, Tianyu; Nannapaneni, Aditya; Liu, Bingfan; Zhang, Ling (2026). Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap. arXiv:2608.05179, DOI 10.48550/arXiv.2608.05179. https://arxiv.org/abs/2608.05179
  6. Shah, Neel Tushar; Kartik, Manglam (2026). When Should an AI Scientist Stop? Verifiable Experiment Steering and Refusal for Autonomous Discovery. arXiv:2606.07576, DOI 10.48550/arXiv.2606.07576. https://arxiv.org/abs/2606.07576
  7. Eulig, Steven Young (2026). Position: Correct Answer, Wrong Mechanism — When AI Scientists Defend General Claims Their Own Data Contradicts. arXiv:2606.23175, DOI 10.48550/arXiv.2606.23175. https://arxiv.org/abs/2606.23175
  8. Gottweis, Juraj; Weng, Wei-Hung; Daryin, Alexander; Tu, Tao; Sirkovic, Petar; Myaskovsky, Artiom; Glowaty, Grzegorz; Weissenberger, Felix; Orlandi, Alessio; Popovici, Dan; Palepu, Anil; Rong, Keran; Tanno, Ryutaro; Saab, Khaled; Zhang, Fan; Blum, Jacob; Carroll, Andrew; Kulkarni, Kavita; Tomasev, Nenad; Zverinski, Dina; Rendulic, Ivor; Vedadi, Elahe; Hasler, Florian; Rimanic, Luka; Boia, Marina; Budiselic, Ivan; Feinstein, Ben; Bellaiche, Mathias; Sheffer, Tom; Freyberg, Jan; Ratcliff, Jeremy; Bertolli, Ottavia; Chou, Katherine; Hassidim, Avinatan; Gokturk, Burak; Vahdat, Amin; Guan, Yuan; Dhillon, Vikram; Vaishnav, Eeshit Dhaval; Lee, Byron; Costa, Tiago R D; Penadés, José R; Peltz, Gary; Matias, Yossi; Manyika, James; Hassabis, Demis; Xu, Yunhan; Kohli, Pushmeet; Pawlosky, Annalisa; Karthikesalingam, Alan; Natarajan, Vivek (2025). Accelerating scientific discovery with Co-Scientist. arXiv:2502.18864, DOI 10.48550/arXiv.2502.18864. https://arxiv.org/abs/2502.18864
  9. Si, Chenglei; Yang, Diyi; Hashimoto, Tatsunori (2024). Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. arXiv:2409.04109, DOI 10.48550/arXiv.2409.04109. https://arxiv.org/abs/2409.04109