Automated dependency-update bots such as Dependabot and Greenkeeper are now default infrastructure in open-source ecosystems, yet the literature offers two unreconciled bodies of evidence: bots accelerate the patching of vulnerable dependencies, while also generating substantial maintainer overhead through false alarms, breaking updates, and alert fatigue. The open problem is a joint, comparable quantification of both effects: does bot-mediated updating, on net, reduce or increase software supply-chain risk for dependent projects, when reduced vulnerability exposure is weighed against newly introduced risk and maintenance burden on a common risk measure, at the natural generality across bots and ecosystems?

Background

Modern applications are built on top of large numbers of third-party libraries, called dependencies, distributed through package registries such as npm (the dominant JavaScript package registry). When a security vulnerability — a flaw that an attacker can exploit, typically catalogued as a CVE (Common Vulnerabilities and Exposures) entry — is found in a dependency, every project relying on the affected version is exposed until it updates to a patched release. Empirical studies have repeatedly found that developers are slow to track and apply such updates, leaving known-vulnerable dependencies in place for long periods (see, e.g., the framing in Rebatchi et al., EMSE 2024, DOI 10.1007/s10664-024-10523-y). Dependency-update bots are automated services that watch a project's dependencies and open pull requests — proposals to change the project's code that maintainers must merge (accept) or close — whenever a new release or a security fix becomes available; Dependabot is GitHub's native bot and Greenkeeper was a widely used npm-focused bot. Mirhosseini and Parnin (ASE 2017, DOI 10.1109/ASE.2017.8115621) first asked whether such automated pull requests actually encourage developers to upgrade out-of-date dependencies. Subsequent large-scale studies split into two poles that have not been reconciled. On the benefit side, Alfadel et al. (MSR 2021, DOI 10.1109/MSR52588.2021.00037) studied 2,904 JavaScript projects subscribing to Dependabot and found that 65.42% of bot-created security pull requests are accepted, often merged within a day, with only 3.2% of manually examined pull requests suffering build breakages (the project failing to compile or pass its checks after the update); Mohayeji et al. (MSR 2023, DOI 10.1109/MSR59073.2023.00042, extended in EMSE 2025, DOI 10.1007/s10664-025-10638-w) showed that developers largely delegate vulnerability fixing to Dependabot, that projects with tests and continuous integration (automated building and testing of the project on every change) are more likely to merge security updates, but that when a bot update is not merged the manual fix can take months; and Jeong and Yoo (MSOM 2026, DOI 10.1287/msom.2024.0827), analyzing 476,738 vulnerable-dependency instances across 1,963,957 JavaScript packages with survival analysis (a statistical method for time-to-event data, in which the hazard is the instantaneous rate at which the event occurs), found Dependabot adopters resolve vulnerable dependencies 60% faster (2.499 times the resolution hazard of non-adopters), yet with a still-slow median of 82 days, attributing the bounded benefit to human-driven constraints. On the cost side, Rombaut et al. (ACM TOSEM, DOI 10.1145/3522587) analyzed 93,196 issue reports opened by Greenkeeper and found the bot responsible for half of all issues in its client projects, many of them false alarms (issues raised by incorrect bot actions), with Greenkeeper's automatic reverting of broken updates proving ineffective; He et al. (TSE 2023, DOI 10.1109/TSE.2023.3278129) obtained explicitly mixed findings — Dependabot adoption reduces technical lag (the distance between used and newest dependency versions) and its pull requests are well received, but its compatibility scores (a Dependabot-provided signal of how safely an update can be applied) are too scarce to reduce update suspicion, developers configure it to reduce notification volume, and 11.3% of projects eventually deprecate it; and Kula (BotSE@ICSE 2025, DOI 10.1109/BOTSE67031.2025.00010) argues that bot-induced alert fatigue — maintainers disengaging under a flood of automated alerts — remains an open design problem. Erlenhov et al. (PeerJ Computer Science 2022, DOI 10.7717/peerj-cs.849) document that bots, mainly dependency-management ones, are pervasive and that projects frequently experiment with and switch between them. What no existing study provides is an evaluation that places both poles on a single scale: a common measure of supply-chain risk under which reduced vulnerability exposure (shorter exposure windows, faster patching) can be compared against added risk and maintenance overhead (false alarms, breaking updates, unreviewed merges, alert fatigue), with attribution that addresses the fact that projects adopting a bot are not a random sample.

Problem Statement

Determine, with empirical evidence, whether automated dependency-update bots on net reduce or increase software supply-chain risk for the projects that use them. Answering this requires weighing both documented effects on a common, defensible measure of supply-chain risk and maintenance burden: the bot-attributable reduction in vulnerability exposure for projects using update bots (e.g., patch latency, vulnerability exposure windows, resolution hazard as in Jeong and Yoo, DOI 10.1287/msom.2024.0827) against, on the same scale, the bot-attributable added risk and overhead (false alarms, breaking updates, unreviewed merges, alert fatigue, and maintainer effort, as documented by Rombaut et al., DOI 10.1145/3522587, and He et al., DOI 10.1109/TSE.2023.3278129). Since projects adopting a bot are self-selected rather than randomly assigned, attributing the net effect to the bot turns on addressing that selection. The question is posed at its natural generality — across bots (e.g., Dependabot, Greenkeeper, Renovate) and ecosystems (e.g., npm, Maven, RubyGems — the JavaScript, Java, and Ruby package ecosystems) — and its answer is the sign and magnitude of the net effect, the bots and ecosystems over which it holds, and how it generalizes beyond them; where the scope is restricted to a single bot or ecosystem, the representativeness of that instance for the others bounds that generalization.

References

  1. Mahmoud Alfadel, Diego Elias Costa, Emad Shihab, Mouafak Mkhallalati. On the Use of Dependabot Security Pull Requests. MSR 2021. DOI 10.1109/MSR52588.2021.00037, https://doi.org/10.1109/MSR52588.2021.00037
  2. Benjamin Rombaut, Filipe R. Cogo, Bram Adams, Ahmed E. Hassan. There's no Such Thing as a Free Lunch: Lessons Learned from Exploring the Overhead Introduced by the Greenkeeper Dependency Bot in npm. ACM Transactions on Software Engineering and Methodology, 2023. DOI 10.1145/3522587, https://doi.org/10.1145/3522587
  3. Hocine Rebatchi, Tégawendé F. Bissyandé, Naouel Moha. Dependabot and security pull requests: large empirical study. Empirical Software Engineering, 2024. DOI 10.1007/s10664-024-10523-y, https://doi.org/10.1007/s10664-024-10523-y
  4. Hamid Mohayeji, Andrei Agaronian, Eleni Constantinou, Nicola Zannone, Alexander Serebrenik. Securing dependencies: A comprehensive study of Dependabot's impact on vulnerability mitigation. Empirical Software Engineering, 2025. DOI 10.1007/s10664-025-10638-w, https://doi.org/10.1007/s10664-025-10638-w
  5. Hamid Mohayeji, Andrei Agaronian, Eleni Constantinou, Nicola Zannone, Alexander Serebrenik. Investigating the Resolution of Vulnerable Dependencies with Dependabot Security Updates. MSR 2023. DOI 10.1109/MSR59073.2023.00042, https://doi.org/10.1109/MSR59073.2023.00042
  6. Runzhi He, Hao He, Yuxia Zhang, Minghui Zhou. Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabot. IEEE Transactions on Software Engineering, 2023. DOI 10.1109/TSE.2023.3278129, https://doi.org/10.1109/TSE.2023.3278129
  7. Seongkyoon Jeong, Eunae Yoo. Managing Software Component Quality with Automation: Evidence from Dependabot. Manufacturing & Service Operations Management, 2026. DOI 10.1287/msom.2024.0827, https://doi.org/10.1287/msom.2024.0827
  8. Damien Jaime, Pascal Poizat, Joyce El Haddad, Thomas Degueule. Balancing the Quality and Cost of Updating Dependencies. ASE 2024. DOI 10.1145/3691620.3695595, https://doi.org/10.1145/3691620.3695595
  9. Linda Erlenhov, Francisco Gomes de Oliveira Neto, Philipp Leitner. Dependency management bots in open-source systems—prevalence and adoption. PeerJ Computer Science, 2022. DOI 10.7717/peerj-cs.849, https://doi.org/10.7717/peerj-cs.849
  10. Raula Gaikovina Kula. Reducing Alert Fatigue via AI-Assisted Negotiation: A Case for Dependabot. BotSE@ICSE 2025. DOI 10.1109/BOTSE67031.2025.00010, https://doi.org/10.1109/BOTSE67031.2025.00010 (preprint arXiv:2502.06175, https://arxiv.org/abs/2502.06175)
  11. Samim Mirhosseini, Chris Parnin. Can automated pull requests encourage software developers to upgrade out-of-date dependencies? ASE 2017. DOI 10.1109/ASE.2017.8115621, https://doi.org/10.1109/ASE.2017.8115621