Vulnerability reintroduction — a code change that fixes a security vulnerability while inadvertently re-introducing a previously patched one (a security regression) — has so far been studied only through single-project exploratory case studies: Mozilla (Braz et al., ESEM 2022) and ImageMagick (Shimmi et al., 2025/2026), the latter correlating longitudinal software-engineering process metrics with 76 reintroduced vulnerabilities. Whether these socio-technical correlations generalize across projects, languages, and domains, and whether vulnerability-fixing commits likely to reintroduce a vulnerability can be predicted at the time they are proposed, remains unresolved. The problem is to (i) build a reliable multi-project ground-truth dataset of vulnerability-reintroducing fixes with quantified identification-pipeline error rates, (ii) determine which process indicators (bus factor and finer-grained contributor-knowledge measures, issue density, issue spoilage, pull-request review metrics) are project-general versus project-specific, and (iii) develop and validate a just-in-time classifier for reintroducing fixes combining process, code-level, and commit-metadata features on multi-project data. No study resolving any of these components is known in the literature to date.

Background

A security (regression) vulnerability arises when a code change — often an ordinary bug fix — is made to software that was previously free of that flaw. In vulnerability reintroduction the pattern is cyclic: a vulnerability is reported to a database such as CVE (Common Vulnerabilities and Exposures) or the NVD (U.S. National Vulnerability Database), fixed, and then later reintroduced by a subsequent change, frequently the fix of another vulnerability. Two exploratory studies document the phenomenon. Braz et al. (ESEM 2022, the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement) analyzed 78 regression vulnerabilities and 72 bug reports at Mozilla plus developer interviews, finding that security is rarely discussed during ordinary bug fixes and that developers assume tools will catch regressions. Shimmi, Synovic, Rahimi, and Thiruvathukal (arXiv:2510.26676; published at the SVM@ICSE 2026 workshop, the IEEE/ACM International Workshop on Software Vulnerability Management co-located with the International Conference on Software Engineering) conducted the first case study correlating longitudinal software-engineering process metrics with reintroduction in ImageMagick, identifying 76 reintroduced vulnerabilities and observing that reintroduction periods coincide with rising issue spoilage. Key terms: the SZZ algorithm (introduced by Šliwerski, Zimmermann, and Zeller at the Mining Software Repositories (MSR) workshop, 2005) identifies the commit that originally introduced a bug by tracing back from fixing commits, and its extension to security fixes is how candidate reintroducing commits are found, here combined with labeling by an LLM (large language model) and manual review; bus factor (Cosentino et al., SANER 2015, the International Conference on Software Analysis, Evolution and Reengineering) is the number of contributors whose knowledge keeps a project maintainable, approximated by counting unique contributors to the repository; issue density measures open issues relative to development activity; issue spoilage measures the accumulation of unresolved issues (issues open past their intended resolution window). The ImageMagick study computed these only at project level, on a single project, with no predictive model; its authors explicitly list generalization beyond ImageMagick, predictive modeling of risky fixes, and pull-request review dynamics as future work, and acknowledge as threats to validity that their LLM-based labeling was adopted because a full manual analysis was too costly and that their project-level bus factor does not capture who contributes to individual directories or files. Adjacent research on just-in-time vulnerability prediction — predicting at commit time whether a change introduces a vulnerability in general (e.g., Nguyen et al., ICSME 2025, the International Conference on Software Maintenance and Evolution) — does not target the reintroduction-of-a-fixed-vulnerability event nor process-metric screening of fix commits, so the reintroduction-specific question is untouched.

Problem Statement

Establish, beyond a single-project case study, whether software-engineering process indicators correlate with and predict vulnerability reintroduction in open-source software, with three jointly required components serving one objective: (1) Ground truth — construct a multi-project dataset of vulnerability-reintroducing fixes (fixes that reintroduce a previously patched vulnerability) spanning diverse projects, languages, and domains, with the identification pipeline's error rates quantified (SZZ-based bug-introducing-commit tracing plus automated labeling, validated against manually reviewed samples). (2) Generalization — test which process correlations found at ImageMagick (bus factor, issue density, issue spoilage, and reintroduction-rate trends), together with finer-grained contributor-knowledge measures at file or directory level and pull-request review metrics (review latency, reviewer-to-contributor ratios, review comment volume, pull-request spoilage), hold across projects and which are project-specific. (3) Prediction — develop and validate a classifier that, at the time a vulnerability-fixing commit is proposed, predicts whether it is likely to reintroduce a vulnerability, combining longitudinal process metrics with code-level features and commit metadata, trained and evaluated on held-out multi-project data under a just-in-time protocol (using only information available at proposal time) against reasonable baselines. The answer reports the measured incremental predictive contribution of the process indicators over code-level and commit-metadata baselines — positive, marginal, or a well-supported null result — together with the classifier's practical triage value (e.g., precision/recall trade-offs meaningful for regression-test triage), and these may be reported as absent.

References

  1. Shimmi, Samiha; Synovic, Nicholas M.; Rahimi, Mona; Thiruvathukal, George K. (2026). Process-based Indicators of Vulnerability Re-Introducing Code Changes: An Exploratory Case Study. Proceedings of the 2026 IEEE/ACM 4th International Workshop on Software Vulnerability Management (SVM@ICSE 2026). DOI: 10.1145/3786165.3788438. https://doi.org/10.1145/3786165.3788438 (preprint arXiv:2510.26676, Oct 2025: https://arxiv.org/abs/2510.26676)
  2. Braz, Larissa; Fregnan, Enrico; Arora, Vivek; Bacchelli, Alberto (2022). An Exploratory Study on Regression Vulnerabilities. ESEM 2022. DOI: 10.1145/3544902.3546250. https://doi.org/10.1145/3544902.3546250
  3. Śliwerski, Jacek; Zimmermann, Thomas; Zeller, Andreas (2005). When Do Changes Induce Fixes? On Friday Typos and the Need for SZZ. MSR 2005. DOI: 10.1145/1083142.1083147. https://doi.org/10.1145/1083142.1083147
  4. Cosentino, Valerio; Izquierdo, Javier Luis Cánovas; Cabot, Jordi (2015). Assessing the bus factor of Git repositories. SANER 2015. DOI: 10.1109/SANER.2015.7081864. https://doi.org/10.1109/SANER.2015.7081864
  5. Nguyen, Duong; Le-Cong, Thanh; Le, Triet Huynh Minh; Babar, M. Ali; Huynh, Quyet-Thang (2025). Toward Realistic Evaluations of Just-In-Time Vulnerability Prediction. ICSME 2025. DOI: 10.1109/ICSME64153.2025.00023. https://doi.org/10.1109/ICSME64153.2025.00023
  6. Shimmi, Samiha; Rahimi, Mona (2022). Mining software repositories for patternizing attack-and-defense co-evolution. MSR APFS 2022. DOI: 10.1145/3549035.3561181. https://doi.org/10.1145/3549035.3561181