A 2018 empirical study of 252 Mozilla security bugs (Sun, Peng, Zhang, Liu and Cai) distilled how security bugs are fixed — ten local fixing patterns, five multi-commit fixing strategies, eight reopening causes, and three blocking-relation types — and concluded with four open directions for improving security bug fixing: automated program repair leveraging the identified fixing patterns; security-specific regression testing and fixing-solution recommendation; program analysis or clone detection to improve the quality of security patches; and prediction of blocking bugs that distinguishes the three blocking-relation types. The subsequent 2018–2026 literature treats the four directions unevenly: automated vulnerability repair and patch-correctness testing have become active research lines (including tools from the study's own authors); clone-based detection of incomplete security fixes advanced sharply in 2025, but its authors state that detection for most root-cause categories remains open; generic blocking-bug prediction matured before and after the study, while the typed variant appears unaddressed. No direction has been resolved outright, so the question stands open as posed. The directions jointly serve the single objective of converting the study's empirical findings into technique and tool support that reduces the cost, risk, and incompleteness of security bug fixing.

Background

A security bug (vulnerability) is a software defect whose exploitation can violate the confidentiality, integrity, or availability of a system. In large open-source projects such as Mozilla Firefox, bugs are managed in an issue tracker, where each report carries status transitions (fixed, reopened), dependency links (a bug 'blocks' another when the blocked bug cannot be fixed first), and references to the code commits that implement the fix. Several pieces of background technology matter for what follows. Automated program repair (APR) generates a patch for a reported defect automatically, either by searching candidate edits validated against a test suite or by learning patch shapes from historical fixes. Regression testing re-runs existing or newly generated tests after a change to confirm that previously working behavior is preserved; deciding whether a candidate patch's behavior is acceptable requires a test oracle, and for security properties such oracles are notoriously hard to state. Code clone detection finds syntactically or semantically similar code fragments; it matters for security because when a vulnerable fragment is duplicated across modules or projects, fixing one copy leaves 'missed' vulnerable clones at the other sites — an incomplete fix, which is the canonical way a security patch is of poor quality. Blocking bugs are bugs whose resolution is a prerequisite for resolving other bugs, recorded (often incompletely) through tracker dependency metadata.

Against this background, Sun, Peng, Zhang, Liu and Cai (Science China Information Sciences 62(1):19102, DOI 10.1007/s11432-017-9459-5) manually analyzed the full change history — bug reports, comments, and commits — of 252 security bugs fixed in Mozilla between July 2005 and August 2015. They established four findings. First, fixes follow ten local fixing patterns, organized by the phase of data computation inside a code module: data-preprocessing patterns placed at a module's entrance (checking data, filtering illegal data, strengthening or weakening data constraints), data-processing patterns (adding a code branch, changing an object's type to its parent class or to a subclass, adding a try-catch module, changing a weak reference to a strong reference), and a data-ensuring pattern (checking data before transmission to another module); most patches were local and small. Second, 138 of the 252 fixes (55%) required multiple commits, following five strategies: fix-then-verify, temporary patch first, deliberate division of a difficult fix, re-fixing after an incorrect fix, and refactoring of the previous fix. Third, 68 bugs were reopened, for eight identified causes — most notably incomplete fixes with a 'coordinate' relation (the follow-up fix differs from the original in pattern, functionality, and code location) or a 'linear' relation (same pattern and functionality, different location), but also patches made wider or stronger, over-strong patches later relaxed, incorrect fixes corrected, and reopenings that changed no code. Fourth, 88 of the bugs were blocked by other bugs, and the blocking relations fall into three types: in-functionality (the blocking bug lies in the same functionality), cross-functionality (the blocking bug lies in a different functionality), and platform (a bug in the underlying platform prevents fixing dependent components).

The study's concluding discussion turns these findings into a request to the research community: convert the empirical characterization of security-bug fixing into concrete technique and tool support. Four directions are posed. Automated program repair that leverages the identified fixing patterns would speed up and standardize fixing. Security-specific regression testing and fixing-solution recommendation would reduce the cost and risk of the multi-commit, reopen-prone fixing process. Program analysis or clone detection applied to security patches would counter the incomplete-fix failure mode that the reopening analysis exposed. Prediction of blocking bugs that accounts for the three blocking-relation types would let maintainers anticipate and manage dependencies during fixing. The study states that these areas had not yet been addressed. Prior context shows the request was never entirely virgin territory: Yin et al. (ESEC/FSE 2011) had already documented how fixes become bugs through incomplete fixes and fix-induced regressions, An, Khomh and Adams (SCAM 2014) had studied supplementary (follow-up) patches versus reopened bugs, Xia et al. (Information and Software Technology 2015) had built the ELBlocker blocking-bug predictor, and Felderer and Fourneret (STTT 2015) had systematically classified security regression testing approaches. The question is what remains of the Mozilla study's four-direction agenda after the subsequent literature.

Problem Statement

As posed by the source, the question is how to convert the empirical findings of the Mozilla security-bug-fixing study into concrete technique and tool support that improves security bug fixing, along four directions stated by Sun, Peng, Zhang, Liu and Cai (2018): (1) develop automated program repair techniques or tools that leverage the ten identified fixing patterns, in order to speed up and standardize the fixing process; (2) develop security-specific regression testing and fixing-solution recommendation techniques, in order to reduce the cost and risk of fixing security bugs, given that 55% of the studied fixes required multiple commits and 68 bugs were reopened; (3) use program analysis or clone detection tools to improve the quality of security bug patches, whose canonical failure mode is the incomplete fix that leaves similar vulnerable code sites unfixed (the 'coordinate' and 'linear' reopening relations found in the study); and (4) develop techniques for predicting blocking bugs that consider the three types of blocking relations (in-functionality, cross-functionality, platform), in order to better anticipate and manage dependencies during security bug fixing. The four requested results jointly serve the single stated objective of reducing the cost, risk, and incompleteness of security bug fixing, and each direction is anchored in a specific empirical finding of the study.

References

  1. Sun X, Peng X, Zhang K, Liu Y, Cai Y. How security bugs are fixed and what can be improved: an empirical study with Mozilla. Science China Information Sciences 62(1):19102, published online 17 December 2018. DOI: 10.1007/s11432-017-9459-5. https://doi.org/10.1007/s11432-017-9459-5
  2. Yin Z, Yuan D, Zhou Y, Pasupathy S, Bairavasundaram L. How do fixes become bugs? In: Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE 2011), 2011, pp. 26–36. DOI: 10.1145/2025113.2025121. https://doi.org/10.1145/2025113.2025121
  3. Xia X, Lo D, Shihab E, Wang X, Yang X. ELBlocker: Predicting blocking bugs with ensemble imbalance learning. Information and Software Technology 61:93–106, 2015. DOI: 10.1016/j.infsof.2014.12.006. https://doi.org/10.1016/j.infsof.2014.12.006
  4. Felderer M, Fourneret E. A systematic classification of security regression testing approaches. International Journal on Software Tools for Technology Transfer 17(3):305–319, 2015. DOI: 10.1007/s10009-015-0365-2. https://doi.org/10.1007/s10009-015-0365-2
  5. An L, Khomh F, Adams B. Supplementary Bug Fixes vs. Re-opened Bugs. In: 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation (SCAM), 2014, pp. 205–214. DOI: 10.1109/SCAM.2014.29. https://doi.org/10.1109/SCAM.2014.29
  6. Pinconschi E, Abreu R, Adao P. A Comparative Study of Automatic Program Repair Techniques for Security Vulnerabilities. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), 2021, pp. 196–207. DOI: 10.1109/ISSRE52982.2021.00031. https://doi.org/10.1109/ISSRE52982.2021.00031
  7. Wei Y, Sun X, Bo L, Cao S, Xia X, Li B. A comprehensive study on security bug characteristics. Journal of Software: Evolution and Process 33(10):e2376, 2021. DOI: 10.1002/smr.2376. https://doi.org/10.1002/smr.2376
  8. Cheng X, Liu N, Guo L, Xu Z, Zhang T. Blocking Bug Prediction Based on XGBoost with Enhanced Features. In: 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC), 2020, pp. 902–911. DOI: 10.1109/COMPSAC48688.2020.0-152. https://doi.org/10.1109/COMPSAC48688.2020.0-152
  9. Zhou Z, Bo L, Wu X, Sun X, Zhang T, Li B, Zhang J, Cao S. SPVF: security property assisted vulnerability fixing via attention-based models. Empirical Software Engineering 27(7):171, 2022. DOI: 10.1007/s10664-022-10216-4. https://doi.org/10.1007/s10664-022-10216-4
  10. Ghanbari A, Marcus A. Patch correctness assessment in automated program repair based on the impact of patches on production and test code. In: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022), 2022, pp. 654–665. DOI: 10.1145/3533767.3534368. https://doi.org/10.1145/3533767.3534368
  11. Li B, Wei Y, Sun X, Bo L, Chen D, Tao C. Towards the identification of bug entities and relations in bug reports. Automated Software Engineering 29(1):24, 2022. DOI: 10.1007/s10515-022-00325-1. https://doi.org/10.1007/s10515-022-00325-1
  12. Chen Z, Ju X, Shen Y, Chen X. Improving Blocking Bug Pair Prediction via Hybrid Deep Learning. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C), 2021, pp. 727–732. DOI: 10.1109/QRS-C55045.2021.00110. https://doi.org/10.1109/QRS-C55045.2021.00110
  13. Wei Y, Bo L, Wu X, Li Y, Ye Z, Sun X, Li B. VulRep: vulnerability repair based on inducing commits and fixing commits. EURASIP Journal on Wireless Communications and Networking 2023(1):34, 2023. DOI: 10.1186/s13638-023-02242-7. https://doi.org/10.1186/s13638-023-02242-7
  14. Ren H, Li Y, Chen L, Zhou Y, Nie C. Why and How Bug Blocking Relations are Breakable: An Empirical Study on Breakable Blocking Bugs. SSRN preprint, 2023. DOI: 10.2139/ssrn.4411397. https://doi.org/10.2139/ssrn.4411397
  15. Molina F, Copia JM, Gorla A. Improving Patch Correctness Analysis via Random Testing and Large Language Models. In: 2024 IEEE Conference on Software Testing, Verification and Validation (ICST), 2024, pp. 317–328. DOI: 10.1109/ICST60714.2024.00036. https://doi.org/10.1109/ICST60714.2024.00036
  16. Zhang Y, Wang J, Berzin D, Mirchev M, Liu D, Arya A, Chang O, Roychoudhury A. Fixing Security Vulnerabilities with AI in OSS-Fuzz. arXiv:2411.03346, 2024. https://arxiv.org/abs/2411.03346
  17. Zhang Y, Wang J, Berzin D, Mirchev M, Roychoudhury A. Fixing Security Vulnerabilities with Agentic AI in OSS-Fuzz. In: Proceedings of the IEEE/ACM 48th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2026, pp. 374–384. DOI: 10.1145/3786583.3786880. https://doi.org/10.1145/3786583.3786880 (published version of the preceding arXiv report)
  18. Alomari HW, Vendome C, Gyawali H. A Slicing-Based Approach for Detecting and Patching Vulnerable Code Clones. In: 2025 IEEE/ACM 33rd International Conference on Program Comprehension (ICPC), 2025, pp. 60–72. DOI: 10.1109/ICPC66645.2025.00015. https://doi.org/10.1109/ICPC66645.2025.00015 (preprint: arXiv:2505.02349)
  19. Bui T, Tun YN, Nguyen TP, Su Y, Thung F, Li Y, Ang HW, Yin Y, Liauw F, Shar LK, Ouh EL, Zhang T, Lo D. VulCoCo: A Simple Yet Effective Method for Detecting Vulnerable Code Clones. arXiv:2507.16661, 2025. https://arxiv.org/abs/2507.16661
  20. Liu Q, Zhang W, Jiang M, Wu L, Zhou Y. Characteristics, Root Causes, and Detection of Incomplete Security Bug Fixes in the Linux Kernel. arXiv:2511.17799, 2025. https://arxiv.org/abs/2511.17799