Code produced by large language models (LLMs) is evaluated almost exclusively for functional correctness, yet practitioners identify the maintainability and readability of generated code as their primary quality concern for long-lived systems. The source study (Sun, Ståhl, Sandahl, and Kessler, Journal of Systems and Software 238:112885, 2026) documents that LLM-generated patches trigger an order of magnitude more static-maintainability warnings than human-written gold patches, that prompting models to improve a quality dimension can instead degrade it, and that practitioners distrust existing maintainability metrics. The open problem: construct a standardized, automatically computable maintainability and readability measurement for LLM-generated code whose values are validated against actual downstream maintenance outcomes (not merely against expert opinion or other static metrics), and demonstrate that it can be enforced as a gate and feedback signal in code-generation and review pipelines without systematic regressions in functional correctness or other quality characteristics. As of August 2026, no published work validates such a metric or demonstrates such enforcement; the newest studies instead show that traditional maintainability metrics fail to explain the downstream maintainability gap of agent-generated code.

Background

Large language models trained on code now generate a substantial share of new software, from function completions to repository-level bug fixes. Evaluation of these models has been dominated by functional correctness: function-level benchmarks such as HumanEval (arXiv:2107.03374) check generated functions against unit tests, and repository-level benchmarks such as SWE-bench (arXiv:2310.06770) ask models to resolve real GitHub issues by producing patches that pass the project's test suite. Whether code that passes tests is also good code is the domain of non-functional quality characteristics (NFQCs), the quality dimensions beyond functional correctness enumerated by the ISO/IEC 25010 international product-quality model, which decomposes software quality into characteristics such as performance efficiency, security, and maintainability. Maintainability denotes the ease with which code can be analyzed, modified, and extended; readability is the closely related ease of understanding. Poor structural quality accumulates technical debt, the implied future rework cost of expedient or fragile code.

Research on NFQCs of generated code has concentrated on security (static vulnerability detection in generated snippets) and performance efficiency (runtime and memory of generated solutions), mostly at the level of isolated, algorithm-style tasks. The source study (Sun et al., Journal of Systems and Software 238:112885, 2026) combined a literature review of 109 papers, two industry workshops with practitioners from seven organizations, and an empirical analysis of patches generated by three LLMs for real-world issues from SWE-bench Lite (a filtered subset of SWE-bench). Its findings motivate this problem: security, performance efficiency, and maintainability dominate the literature while other characteristics are understudied and terminology is inconsistent; practitioners prioritize maintainability and readability for long-lived systems and report that existing maintainability metrics do not reflect practical maintenance effort; and in the empirical study, generated patches triggered far more maintainability warnings under CodeQL (a semantic static-analysis engine that flags rule violations, e.g., 423 maintainability recommendations for GPT-4o patches versus 30 for the corresponding human-written reference, or gold, patches), while instructing models to optimize one quality dimension frequently failed or even degraded that very dimension, a phenomenon the authors call negative optimization.

Two adjacent lines of work exist but do not close the gap. First, static-analysis-in-the-loop generation: Blyth, Licorish, Treude, and Wagner (arXiv:2508.14419) showed that iteratively feeding findings from Bandit and Pylint (two widely used Python static analyzers) back to GPT-4o greatly reduces security, readability, and reliability violations on a function-level security benchmark; similar systems target security alone. These results are single-dimension and function-level, and the source study's repository-level instability findings stand unrefuted. Second, maintainability metrics for human-written code: the Maintainability Index (a composite of lines of code, cyclomatic complexity — the number of independent control-flow paths through the code — and Halstead volume, a size measure derived from counts of operators and operands), SonarQube's maintainability rating (produced by a widely used commercial static-analysis platform), and CodeScene's Code Health (a hotspot and code-smell based measure) are standard, but Borg, Ezzouhri, and Tornhill (arXiv:2408.10754) showed these instruments are validated at best against instantaneous human assessment, not against long-term maintenance cost, and that SonarQube produces enough false positives to undermine studies that use it as ground truth. Most tellingly, Patel et al. (arXiv:2606.21804) recently showed with their CodeThread framework, which evaluates agents on sequences of related coding tasks, that traditional maintainability metrics do not explain the measurably worse downstream maintainability of agent-generated code, and Xia and Miller (arXiv:2607.09902) showed in a longitudinal study of 182 repositories that merged agentic contributions require significantly more corrective maintenance. The burden of generated code is therefore measurable, but no metric can yet predict it, and nothing can yet enforce against it during generation.

Problem Statement

Construct a standardized, quantitative measurement of maintainability and readability for LLM-generated code, covering both standalone snippets and repository-level patches, such that:

(1) The measurement is automatically computable at generation and review time from the code itself (possibly together with repository context), with pinned tooling and reproducible outputs, and is mapped explicitly to the maintainability-related characteristics of the ISO/IEC 25010 product-quality model so that results are comparable across models, languages, and studies, in place of the inconsistent, ad hoc quality columns used by current benchmarks.

(2) The measurement is empirically validated against the actual downstream maintenance outcomes of LLM-generated code, for example corrective-maintenance effort, change- and defect-proneness, or post-merge stability of merged generated code. Validation against instantaneous human preference or against other static metrics alone is insufficient: the metric must be shown to predict real maintenance outcomes better than established baselines (Maintainability Index, SonarQube maintainability rating, CodeScene Code Health, and comparable static analyzers), and practitioners must judge it to reflect practical maintenance effort. Outcome validation is required at the repository-level granule, where such maintenance history exists; snippet-level applicability is a design requirement of the measurement in (1), not a separate outcome-validation requirement.

(3) The validated measurement is demonstrated as an enforcement mechanism: integrated as a gate and structured feedback signal in automated code-generation or code-review pipelines, regenerating or revising code under its guidance must improve the actual downstream maintenance outcomes, measured independently of the metric, without systematic regressions in functional correctness or in other measured quality characteristics, addressing the negative-optimization and cross-dimension trade-off behavior documented in the source study.

These three elements are jointly posed as the research target: a standardized, quantitative measurement of maintainability and readability for LLM-generated code that is automatically computable (1), validated against actual downstream maintenance outcomes (2), and demonstrated as an enforceable gate and feedback signal (3).

References

  1. Xin Sun, Daniel Ståhl, Kristian Sandahl, Christoph Kessler. Quality Assurance of LLM-generated Code: Addressing Non-Functional Quality Characteristics. Journal of Systems and Software 238:112885 (2026). DOI: 10.1016/j.jss.2026.112885; arXiv:2511.10271. https://arxiv.org/abs/2511.10271
  2. Shaswat Patel, Betty Li Hou, Arun Purohit, Kai Xu, Jane Pan, He He, Valerie Chen. Is Agent Code Less Maintainable Than Human Code? arXiv:2606.21804 (2026). https://arxiv.org/abs/2606.21804
  3. Chunqiu Steven Xia, Courtney Miller. Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code. arXiv:2607.09902 (2026). https://arxiv.org/abs/2607.09902
  4. Markus Borg, Marwa Ezzouhri, Adam Tornhill. Ghost Echoes Revealed: Benchmarking Maintainability Metrics and Machine Learning Predictions Against Human Assessments. arXiv:2408.10754 (2024). https://arxiv.org/abs/2408.10754
  5. Scott Blyth, Sherlock A. Licorish, Christoph Treude, Markus Wagner. Static Analysis as a Feedback Loop: Enhancing LLM-Generated Code Beyond Correctness. arXiv:2508.14419 (2025). https://arxiv.org/abs/2508.14419
  6. Joanna Szych, Anne Schwerk. Evaluating LLM-Generated Code: A Benchmark and Developer Study. arXiv:2605.09059 (2026). https://arxiv.org/abs/2605.09059
  7. Rodrigo Pato Nogueira, Marco Vieira, João R. Campos. PROBE: Benchmarking Code Generation in Large Language Models. arXiv:2607.13820 (2026). https://arxiv.org/abs/2607.13820
  8. Seyed Mohammad Mahdi Ghalandarian, Majid Bazargani, Masoumeh Taromirad. Benchmarking the Titans: A Multi-Dimensional Empirical Evaluation of LLM Code Generation Quality in the .NET Ecosystem. arXiv:2608.22529 (2026). https://arxiv.org/abs/2608.22529
  9. Houjun Liu, Lisa Einstein, John Yang, Joachim Baumann, Duncan Eddy, Christopher D. Manning, Mykel Kochenderfer, Diyi Yang. SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization. arXiv:2605.08382 (2026). https://arxiv.org/abs/2605.08382
  10. Jiazheng Quan, Xiaodong Li, Bin Wang, Guo An, Like Liu, Degen Huang, Lin Liu, Chengbin Hou. Learning to Generate Secure Code via Token-Level Rewards. arXiv:2602.23407 (2026). https://arxiv.org/abs/2602.23407
  11. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv:2310.06770 (2023). https://arxiv.org/abs/2310.06770
  12. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, Wojciech Zaremba. Evaluating Large Language Models Trained on Code (the HumanEval benchmark). arXiv:2107.03374 (2021). https://arxiv.org/abs/2107.03374