A survey of 75 industry and open-source practitioners by Kudrjavets and Rastogi found that time-to-merge is the essential code review metric to improve, that quick reviewer reaction time is of utmost importance, and that practitioners regard the controlled application of a commit-then-review model (integrating changes before they are reviewed) as the most promising lever for increasing code velocity. The authors explicitly proposed three follow-up research questions that remain empirically untested: (a) how to apply commit-then-review selectively in industry, including which engineers should qualify to commit without prior review; (b) whether reward-based incentives (monetary or recognition) make engineers react faster to code review requests; and (c) whether formally scheduling dedicated code review time improves review speed and planning precision.

Background

Modern code review (MCR) is the lightweight, tool-supported practice — dominant on platforms such as GitHub, Gerrit, and Phabricator — in which peers inspect code changes before or around integration, providing both defect finding and knowledge transfer (Bacchelli and Bird 2013). Its default ordering is review-then-commit: a change must pass review before it is merged. In the opposite ordering, commit-then-review (also called post-commit review), a developer integrates the change first and it is reviewed afterwards; a large-scale historical example is the Apache HTTP server project, where most changes were committed and then reviewed, with reviews tending to be short and focused (Rigby, German, and Storey 2008). Code velocity denotes how quickly code progresses from authoring to running in production, and the survey literature converges on time-to-merge — the elapsed time from submission of a change to its integration — as its central review-related measure; cross-company studies at AMD, Google, and Microsoft find a norm of reacting to review requests within roughly 24 hours (Rigby and Bird 2013). Prior empirical work has mapped which factors slow reviews (author and reviewer experience, change size, reviewer workload) and has examined adjacent process levers: process simulation has compared pre-commit and post-commit review orderings under idealized assumptions (Baum et al. 2016), and a study of explicit reviewer-strategy policies found no performance improvement (Gonçalves et al. 2020). Against this backdrop, Kudrjavets and Rastogi surveyed developers' beliefs and practices about code velocity (75 completed surveys: 39 industry, 36 open source). Their findings: industry and open-source respondents hold similar beliefs; respondents strongly oppose abandoning review but find commit-then-review acceptable under conditions (experienced engineers with deep product knowledge using judgment about when to request review); time-to-merge is the metric most worth improving; and engineers hold differing views on whether higher personal code velocity pays off in career growth. The authors read these findings as motivation for a concrete empirical agenda, stated in their conclusions: 'In our future research, we plan to investigate the following topics: (a) the selective application of the commit-then-review model in the industry, (b) the benefit of reward-based incentives to motivate engineers to react faster to code reviews, and (c) the benefit of scheduling dedicated code review time to achieve a more precise planning outcome.' For the first of these topics, the authors observe that open-source projects designate individuals as committers with write access to the repository, and they suggest that industry could adopt a comparable process to designate who may commit without prior review. Their data being self-reported survey responses — the authors flag that a survey, due to its nature, is a self-report instrument as a primary threat to internal validity — broader observational or longitudinal designs would be needed to confirm the reported beliefs and prevalences. The three proposed topics remain open.

Problem Statement

As stated in the conclusions of Kudrjavets and Rastogi's survey ('Does code review speed matter for practitioners?', Empirical Software Engineering, online first November 2023, journal issue 29(1), article 7, January 2024): empirically investigate (a) the selective application of the commit-then-review model in industry, (b) the benefit of reward-based incentives to motivate engineers to react faster to code review requests, and (c) the benefit of scheduling dedicated code review time to achieve a more precise planning outcome. Concretely, for one or more of these levers, carry out an empirical study of the lever — for (a), a policy under which qualified engineers may integrate changes before review, including the qualification criteria (which engineers or levels are eligible) and the conditions the survey identified as prerequisites; for (b), a monetary or recognition-based incentive scheme for reviewer responsiveness; for (c), formally reserved review time in developers' schedules — and measure its effect on code review speed (e.g., time-to-merge and first reviewer reaction time) and on software quality (e.g., regression and post-release defect rates). Negative results (a lever demonstrably does not help, or hurts quality unacceptably) fully answer the question.

References

  1. Kudrjavets, Gunnar; Rastogi, Ayushi (2023, online first 22 November 2023; journal issue Empirical Software Engineering 29(1), article 7, January 2024). Does code review speed matter for practitioners? Empirical Software Engineering. DOI 10.1007/s10664-023-10401-z. https://doi.org/10.1007/s10664-023-10401-z (arXiv:2311.02489, https://arxiv.org/abs/2311.02489)
  2. Rigby, Peter C.; German, Daniel M.; Storey, Margaret-Anne (2008). Open source software peer review practices. ICSE 2008. DOI 10.1145/1368088.1368162. https://doi.org/10.1145/1368088.1368162
  3. Rigby, Peter C.; Bird, Christian (2013). Convergent contemporary software peer review practices. ESEC/FSE 2013. DOI 10.1145/2491411.2491444. https://doi.org/10.1145/2491411.2491444
  4. Bacchelli, Alberto; Bird, Christian (2013). Expectations, outcomes, and challenges of modern code review. ICSE 2013. DOI 10.1109/ICSE.2013.6606617. https://doi.org/10.1109/ICSE.2013.6606617
  5. Baum, Tobias; Kortum, Fabian; Schneider, Kurt; Brack, Arthur; Schauder, Jens (2016). Comparing pre commit reviews and post commit reviews using process simulation. ICSSP 2016. DOI 10.1145/2904354.2904362. https://doi.org/10.1145/2904354.2904362
  6. Gonçalves, Pavlína Wurzel; Fregnan, Enrico; Baum, Tobias; Schneider, Kurt; Bacchelli, Alberto (2020). Do explicit review strategies improve code review performance? MSR 2020. DOI 10.1145/3379597.3387509. https://doi.org/10.1145/3379597.3387509
  7. Yang, Lanxin; Liu, Bohan; Jia, Junyu; Xu, Jinwei; Xue, Junming; Zhang, He; Bacchelli, Alberto (2024, online first 12 November 2024; journal issue Empirical Software Engineering 30(1), article 24, January 2025). Prioritizing code review requests to improve review efficiency: a simulation study. Empirical Software Engineering. DOI 10.1007/s10664-024-10575-0. https://doi.org/10.1007/s10664-024-10575-0
  8. Lee, Carol S.; Hicks, Catherine M. (2024). Understanding and effectively mitigating code review anxiety. Empirical Software Engineering. DOI 10.1007/s10664-024-10550-9. https://doi.org/10.1007/s10664-024-10550-9
  9. Silva, Ruben; Silva, Publio; Bezerra, Carla; Uchôa, Anderson; Garcia, Alessandro (2025). Unveiling the relationship between Continuous Integration and code review: a study with 10 closed-source projects. SBES 2025. DOI 10.5753/sbes.2025.9965. https://doi.org/10.5753/sbes.2025.9965
  10. Yang, Zezhou; Gao, Cuiyun; Guo, Zhaoqiang; Li, Zhenhao; Liu, Kui; Xia, Xin; Zhou, Yuming (2026). A roadmap for modern code review: challenges and opportunities. ACM Transactions on Software Engineering and Methodology. DOI 10.1145/3800963. https://doi.org/10.1145/3800963
  11. Kučera, Matej; Castelluccio, Marco; Feitosa, Daniel; Rastogi, Ayushi (2026). Group versus individual review requests: tradeoffs in speed and quality at Mozilla Firefox. ICSE-SEIP 2026 (Proceedings of the IEEE/ACM 48th International Conference on Software Engineering: Software Engineering in Practice). DOI 10.1145/3786583.3786888. https://doi.org/10.1145/3786583.3786888
  12. Chouchen, Moataz; Begoug, Mahi; Ouni, Ali (2026). Beyond single code changes: an empirical study of topic-based code review practices in Gerrit for OpenStack. MSR 2026 (Proceedings of the 23rd International Conference on Mining Software Repositories). DOI 10.1145/3793302.3793381. https://doi.org/10.1145/3793302.3793381
  13. Indriasari, Theresia Devi; Luxton-Reilly, Andrew; Denny, Paul (2021). Improving student peer code review using gamification. ACE 2021. DOI 10.1145/3441636.3442308. https://doi.org/10.1145/3441636.3442308