Developers of deep-learning (DL) systems routinely leave suboptimal temporary choices in code — hardware workarounds, untuned hyper-parameters, brittle training pipelines — and admit them in source comments as self-admitted technical debt (SATD). A 2024 taxonomy study (Pepe et al.) elicited 41 types of DL-specific SATD but analyzed only a qualitative snapshot of 443 comments in 54 Python projects. Whether admitted DL-specific debt corresponds to a measurable maintenance cost — how long it survives, when and how it is removed, and whether it affects model quality or operational failures — has not been quantified. A 2025 first-look study (Recupito et al.) began measuring persistence on the same small corpus family but explicitly excluded resolved debt and any impact analysis. The open problem is a large-scale empirical quantification of the full lifecycle of DL-specific SATD (introduction, survival, removal dynamics) and of its relationship to model quality and operational failures, on corpora larger and more diverse than the original 54 Python/TensorFlow/PyTorch projects.

Background

Technical debt (TD) is the metaphor for short-term engineering shortcuts whose long-term cost compounds like unpaid interest. When developers explicitly acknowledge such a shortcut in a source-code comment (e.g., '#TODO: hard-coded batch size, fix later'), the instance is called self-admitted technical debt (SATD); comments are mined with keyword patterns or learned classifiers, and SATD is conventionally typed as design, requirement, documentation, or defect debt. For conventional software, Bavota and Russo (Mining Software Repositories, MSR 2016) mined thousands of commits and showed that SATD is usually repaid quickly but that its presence correlates with heightened maintenance activity. Machine-learning software raises the stakes: beyond the early, widely cited warning that deployed machine-learning systems quietly accumulate hidden technical debt, empirical work found ML projects carry roughly twice the median SATD percentage of non-ML projects, with data-preprocessing and model-generation components most susceptible, and survival analysis — the statistical study of how long each debt instance persists before removal — has been run on 318 ML projects (Bhatia et al., 2023) and, under the '23 shades' study (O'Brien et al., ESEC/FSE 2022), ML-specific categories such as configuration debt and inadequate tests were added. Separately, Liu and colleagues characterized the presence, introduction, and removal of conventional SATD categories inside seven popular DL frameworks (TensorFlow, PyTorch, etc.), finding design debts repaid fastest and documentation debts repaid least. None of these lines addresses debt that is specific to the DL content of applications: in 2024, Pepe, Zampetti, Mastropaolo, Bavota, and Di Penta manually coded 443 SATD comments from 54 open-source Python projects depending on TensorFlow or PyTorch and derived a taxonomy of DL-specific SATD with seven categories (Hardware, API, Data, Model, Training, Inference, Pipeline) and 41 leaf types; they showed DL-specific SATD is mostly suboptimal-solution debt rather than blocking bugs, and that state-of-the-practice Python static analyzers do not surface it. That study was deliberately a qualitative snapshot: a single point-in-time reading of comments, on a limited, single-language, two-framework, open-source-only corpus. What it did not do — and what its threats-to-validity and related-work discussion expose as the gap — is follow the debt through time: survival times, removal dynamics, and any measurable consequence for the trained model (accuracy, latency, reproducibility) or for operational behavior (crashes, incidents, failed releases). Mining software repositories provides the standard machinery for this: SATD instances are linked to the commits that introduced and (when it happens) removed them, survival distributions are estimated per taxonomy category, and outcomes are correlated with debt presence and persistence.

Problem Statement

Quantify, empirically and at scale, the relationship between admitted deep-learning-specific self-admitted technical debt (SATD, classified under the Pepe et al. 2024 taxonomy of seven categories and 41 types) and the actual evolution of the systems carrying it, on corpora substantially larger and more diverse than the original 54 Python/TensorFlow/PyTorch open-source projects. The single overarching objective is to establish whether self-admitted DL-specific debt is a measurable maintenance cost or largely benign annotation, and it decomposes into three supporting questions: (1) Survival — how long do DL-specific SATD instances persist from introduction (introduction time, project phase, and developer task context) to removal or project end, per taxonomy category, extending beyond the 185-instance first look (Recupito et al., 2025) that excluded resolved debt? (2) Removal dynamics — which DL-specific SATD categories are actually repaid, how quickly, through what kinds of changes (e.g., refactoring versus feature rewrites versus abandonment), and which accumulate indefinitely? (3) Impact — is the presence or persistence of admitted DL-specific SATD associated with measurable downstream consequences for model quality (accuracy, latency, reproducibility of training) and, where operational evidence exists (crashes, incidents, failed releases), for operational behaviour under explicit observable definitions for the operational measures, i.e. the 'quality issues, harming the maintenance process' that Recupito et al. leave open? Pepe et al. report that DL-specific SATD concerns 'sub-optimal solutions, maintainability problems or, at most, errors occurring in corner-case circumstances' whereas DL bugs 'may cause crashes or functional errors in common system usages'.

The 41 types were elicited exclusively from Python TensorFlow/PyTorch open-source comments, and Pepe et al. note that 'it is possible that projects developed with other pieces of technology may exhibit different forms of SATD'. The classification requirement and the corpus-diversity objective are therefore reconciled by the following priority: instances are classified under the seven categories and 41 types wherever they map; instances that do not map are reported explicitly as unmapped rather than silently discarded; and any extension or retyping of the taxonomy is documented separately, so that per-category results remain comparable with the 443-comment and 185-instance baselines to which questions (1)-(3) are indexed.

The corpus is constructed so that its project selection criteria and its language and framework coverage enlarge and diversify beyond the 54 Python/TensorFlow/PyTorch open-source projects and 443 sampled comments of the taxonomy study and the 40-project/185-instance lifecycle sample; where a specific language or framework axis cannot be extended, any generalization of the results along that axis is correspondingly limited. DL-specific SATD is identified and classified under the seven categories and 41 types, with reliability safeguards.

References

  1. Federica Pepe, Fiorella Zampetti, Antonio Mastropaolo, Gabriele Bavota, Massimiliano Di Penta. A Taxonomy of Self-Admitted Technical Debt in Deep Learning Systems. ICSME 2024. DOI: 10.1109/ICSME58944.2024.00043. https://doi.org/10.1109/ICSME58944.2024.00043 (also arXiv:2409.11826, https://arxiv.org/abs/2409.11826)
  2. Gilberto Recupito, Vincenzo De Martino, Dario Di Nucci, Fabio Palomba. A First Look at the Lifecycle of DL-Specific Self-Admitted Technical Debt. SANER-C 2025. DOI: 10.1109/SANER-C66551.2025.00030. https://doi.org/10.1109/SANER-C66551.2025.00030 (also arXiv:2510.03802, https://arxiv.org/abs/2510.03802)
  3. Aaditya Bhatia, Foutse Khomh, Bram Adams, Ahmed E Hassan. An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software. 2023. arXiv:2311.12019. https://arxiv.org/abs/2311.12019
  4. Jiakun Liu, Qiao Huang, Xin Xia, Emad Shihab, David Lo, Shanping Li. Is using deep learning frameworks free? Characterizing technical debt in deep learning frameworks. ICSE-SEIS 2020. DOI: 10.1145/3377815.3381377. https://doi.org/10.1145/3377815.3381377
  5. Jiakun Liu, Qiao Huang, Xin Xia, Emad Shihab, David Lo, Shanping Li. An exploratory study on the introduction and removal of different types of technical debt in deep learning frameworks. Empirical Software Engineering 26, 2021. DOI: 10.1007/s10664-020-09917-5. https://doi.org/10.1007/s10664-020-09917-5
  6. Gabriele Bavota, Barbara Russo. A large-scale empirical study on self-admitted technical debt. MSR 2016. DOI: 10.1145/2901739.2901742. https://doi.org/10.1145/2901739.2901742
  7. David O'Brien, Sumon Biswas, Sayem Imtiaz, Rabe Abdalkareem, Emad Shihab, Hridesh Rajan. 23 shades of self-admitted technical debt: an empirical study on machine learning software. ESEC/FSE 2022. DOI: 10.1145/3540250.3549088. https://doi.org/10.1145/3540250.3549088
  8. Niruthiha Selvanayagam, Taher A. Ghaleb, Manel Abdellatif. Self-Admitted Technical Debt in LLM Software: An Empirical Comparison with ML and Non-ML Software. 2026. arXiv:2601.06266. https://arxiv.org/abs/2601.06266