When a vulnerability in open-source software is fixed through a sequence of patches rather than one, intermediate versions remain partially vulnerable, and it is hard to tell at which point remediation is complete. A 2026 empirical study of 1,646 multi-patch fixes from the U.S. National Vulnerability Database (NVD) showed that this situation is common and systematic, and that existing techniques cannot handle it: seven representative vulnerability-detection models score below random-guessing accuracy when asked to recognize vulnerabilities that remain after an incomplete fix, and two vulnerable-code-clone detectors lose roughly half of their true-positive rate when recurring vulnerabilities appear at multiple code locations. The open problem is to build techniques that verify complete remediation throughout a multi-patch fixing process: predicting whether an initial patch will require follow-up patches, and detecting the remaining vulnerable code locations of a multi-location fix.

Background

A Common Vulnerabilities and Exposures (CVE) record is a public identifier for a specific disclosed security flaw; the National Vulnerability Database (NVD), maintained by NIST, links CVE records to the corresponding security patches (code changes that remove the flaw). Ideally one patch fully resolves one vulnerability, but in practice remediation often proceeds through several patches (a multi-patch fix): the first patch may cover only some of the affected code locations, may leave the root cause partially unfixed (an incomplete fix), or may itself introduce new defects. A large empirical literature mines security patches at scale — most notably Frank Li and Paxson's CCS 2017 study of security patches, which noticed multi-patch fixes but analyzed only about 100 samples — but until 2026 no systematic study of the multi-patch phenomenon existed.

Qi, Li, and Wang (arXiv:2607.13206, accepted at ESORICS 2026) manually examined 1,646 multi-patch fix records from the NVD (1999–2025) and built a taxonomy of root causes with three primary categories: multi-location fixes (the vulnerability affects several code locations — within the same branch or across different branches or projects — and each patch covers only some of them), fix-and-surrounding work (the fixing change is interleaved with documentation, workaround, and clean-up commits), and defective fixes (the initial patch is incomplete or itself buggy). They then asked whether existing techniques can verify, at each point in a patch sequence, whether the vulnerability is fully remediated — and showed they cannot. First, for incomplete fixes: vulnerability-detection (VD) models — neural classifiers such as CodeBERT, UniXcoder, VulBERTa, LineVul, PDBERT, Devign, and ReVeal that are trained to label a function vulnerable or not — were trained on standard single-patch data and then evaluated on the task 'classify the code before the final fixing patch as vulnerable and the code after it as non-vulnerable'. All seven models dropped by 7 to 28 true-positive-rate points relative to single-patch detection, and all scored accuracy and F1 below 50% (worse than random guessing), typically assigning the same label to the pre-fix, intermediate, and post-fix versions of a function, which differ by only small code edits. Second, for multi-location fixes: vulnerable code clone (VCC) detectors — tools that take a known vulnerable function as a fingerprint and find similar code sites that retain the vulnerability (ReDeBug, IEEE S&P 2012; VUDDY, IEEE S&P 2017; FIRE, USENIX Security 2024) — were evaluated with the first patch of each sequence as the fingerprint and the remaining patches as ground truth; both tested tools degraded sharply (FIRE's true-positive rate fell from 90 on general VCC benchmarks to 52.46, ReDeBug's from 37.56 to 16.75), while their false-positive rates remained low — the tools tend to miss related vulnerabilities rather than over-predict them.

This matters beyond patch management because much of the vulnerability-detection research infrastructure is built by patch reversal: datasets such as Big-Vul and DiverseVul recover 'vulnerable' code by undoing a security patch. When a fix consists of several patches, undoing a later patch yields an intermediate state that is neither the original vulnerable code nor fully fixed code, so naive labels are wrong. Related threads exist but do not address the multi-patch setting: silent-fix identification (VFDelta, GRAPE, 2024) finds fixes applied without a CVE to broaden dataset coverage; similar-but-patched-code filtering (FVF, ICSE 2025) reduces false alarms in recurring-vulnerability detection; VulCoCo (2025) improves vulnerable-code-clone detection with embedding retrieval plus large language model (LLM) validation; HYDRA (accepted at AST 2026) flags latent vulnerabilities that may persist in patched functions; and patch-correctness evaluation has been studied in restricted domains (KLAUS for Linux, PatchVerif for robotic vehicles, both USENIX Security 2023). A large-scale 2025 analysis of publicly reported security patches (Woo, Choi, and Lee, Computers & Security) additionally documents that silent fixes leave multi-patch data incomplete in public databases.

Problem Statement

Develop and empirically validate techniques that verify complete remediation throughout a multi-patch vulnerability fixing process in open-source software. Concretely, given a security patch (or the initial segment of a patch sequence) for a reported vulnerability, the two linked capabilities required are: (1) incomplete-fix prediction — decide whether the initial patch fully remediates the vulnerability or leaves it only partially fixed, that is, the incomplete-fix case on which the cited evaluation was measured (a fix that is merely accompanied by documentation, workaround, or clean-up commits, without leaving the vulnerability unfixed, is outside this capability's positive class), a task on which all seven vulnerability-detection models evaluated in the source study score accuracy and F1 below 50% (worse than random guessing); the study's failure analysis traces this to the models often assigning the same label to the pre-fix (vulnerable), intermediate (partially fixed, still vulnerable), and post-fix (fully remediated) versions of the affected code, which differ by only small edits; and (2) multi-location recurring-vulnerability detection — given the first patch of a multi-location fix, identify the other code locations that still contain the vulnerability, where, in line with the source study's ground truth, these locations may lie within the same branch (the study's category A2, fixes at different locations within the same branch) or across different branches or projects (its category A1, fixes across different branches or projects); on this task the evaluated vulnerable-code-clone detectors lose roughly half or more of their true-positive rate (FIRE 90 to 52.46, ReDeBug 37.56 to 16.75). Both capabilities jointly serve the single objective stated by the source study — validating complete remediation during multi-patch fixing — since incomplete fixes and multi-location recurrence are the two dominant mechanisms by which multi-patch fixes arise. Both capabilities remain unsolved on the source study's benchmarks, where the reported baseline values above are the levels that existing techniques currently reach rather than targets; the vulnerable-code-clone detectors keep low false-positive rates, so the difficulty lies in missing related vulnerable locations rather than in over-predicting them.

References

  1. Weiliang Qi, Youpeng Li, Xinda Wang. Why Not Fix It Once and for All? An Empirical Study of Multiple Patches for Vulnerability Fixes in Open-Source Software. arXiv:2607.13206, accepted at ESORICS 2026 (2026). DOI: 10.48550/arXiv.2607.13206, https://arxiv.org/abs/2607.13206
  2. Frank Li, Vern Paxson. A Large-Scale Empirical Study of Security Patches. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS 2017). DOI: 10.1145/3133956.3134072, https://doi.org/10.1145/3133956.3134072
  3. Siyue Feng, Yueming Wu, Wenjie Xue, Sikui Pan, Deqing Zou, Yang Liu, Hai Jin. FIRE: Combining Multi-Stage Filtering with Taint Analysis for Scalable Recurring Vulnerability Detection. 33rd USENIX Security Symposium (2024). https://www.usenix.org/conference/usenixsecurity24/presentation/feng-siyue
  4. Seulbae Kim, Seunghoon Woo, Heejo Lee, Hakjoo Oh. VUDDY: A Scalable Approach for Vulnerable Code Clone Discovery. 2017 IEEE Symposium on Security and Privacy. DOI: 10.1109/SP.2017.62, https://doi.org/10.1109/SP.2017.62
  5. Jiyong Jang, Abeer Agrawal, David Brumley. ReDeBug: Finding Unpatched Code Clones in Entire OS Distributions. 2012 IEEE Symposium on Security and Privacy. DOI: 10.1109/SP.2012.13, https://doi.org/10.1109/SP.2012.13
  6. Tan Bui, Yan Naing Tun, Thanh Phuc Nguyen, Yindu Su, Ferdian Thung, Yikun Li, Han Wei Ang, Yide Yin, Frank Liauw, Lwin Khin Shar, Eng Lieh Ouh, Ting Zhang, David Lo. VulCoCo: A Simple Yet Effective Method for Detecting Vulnerable Code Clones. arXiv:2507.16661 (2025). https://arxiv.org/abs/2507.16661
  7. Zixuan Tan, Jiayuan Zhou, Xing Hu, Shengyi Pan, Kui Liu, Xin Xia. Similar but Patched Code Considered Harmful — The Impact of Similar but Patched Code on Recurring Vulnerability Detection and How to Remove Them. ICSE 2025, arXiv:2412.20740 (2024). https://arxiv.org/abs/2412.20740
  8. Mohammad Farhad, Sabbir Rahman, Shuvalaxmi Dass. HYDRA: A Hybrid Heuristic-Guided Deep Representation Architecture for Predicting Latent Zero-Day Vulnerabilities in Patched Functions. arXiv:2511.06220, accepted at AST 2026 (2025). https://arxiv.org/abs/2511.06220
  9. Xu Yang, Shaowei Wang, Jiayuan Zhou, Xing Hu. VFDelta: A Framework for Detecting Silent Vulnerability Fixes by Enhancing Code Change Learning. arXiv:2409.16606 (2024). https://arxiv.org/abs/2409.16606
  10. Mei Han, Lulu Wang, Jianming Chang, Bixin Li, Chunguang Zhang. Learning Graph-based Patch Representations for Identifying and Assessing Silent Vulnerability Fixes. arXiv:2409.08512 (2024). https://arxiv.org/abs/2409.08512
  11. Seunghoon Woo, Eunjin Choi, Heejo Lee. A large-scale analysis of the effectiveness of publicly reported security patches. Computers & Security 148:104181 (2025). DOI: 10.1016/j.cose.2024.104181, https://doi.org/10.1016/j.cose.2024.104181
  12. Yuhang Wu, Zhenpeng Lin, Yueqi Chen, Dang K. Le, Dongliang Mu, Xinyu Xing. Mitigating Security Risks in Linux with KLAUS: A Method for Evaluating Patch Correctness. 32nd USENIX Security Symposium (2023). https://www.usenix.org/conference/usenixsecurity23/presentation/wu-yuhang
  13. Hyungsub Kim, Muslum Ozgur Ozmen, Z. Berkay Celik, Antonio Bianchi, Dongyan Xu. PatchVerif: Discovering Faulty Patches in Robotic Vehicles. 32nd USENIX Security Symposium (2023). https://www.usenix.org/conference/usenixsecurity23/presentation/kim-hyungsub