The change propagation probability (cpp) metric assigns to each ordered pair of components in a software architecture the probability that a change arising in one necessitates a change in the other; aggregated over the architecture, it yields a design-time indicator of how well a design localizes change. The metric family was introduced by Shereshevsky et al. (2001) and fully analyzed by Abdelmoez et al. (2005); Shaik et al. (WICSA 2005) argued that it is effective for assessing the design quality of software architectures, but their evidence was a single small case study — a Job Application system examined in two architectural variants, one using design patterns and one not — compared against coupling-based structural metrics. The authors' own companion work (the SACPT tool, 2004, and Shaik's 2006 thesis) largely settled the automation of the computation, but no published study has validated the metric on case studies beyond the original small example. The open problem is the empirical validation — or empirical refutation — of change propagation probability as an indicator of software architecture design quality on substantially larger case studies, including whether it carries predictive value beyond established coupling-based metrics.

Background

A software architecture decomposes a system into components and the connectors between them; in component-based development, the quality of this decomposition largely determines how expensive future changes will be. Change propagation is the phenomenon by which a change made to one component forces changes in other components that depend on it, directly or through intermediate components, and it is a principal driver of software maintenance cost. A line of work from West Virginia University and the New Jersey Institute of Technology sought to quantify change propagation at design time, before any code exists. Shereshevsky et al. (2001) introduced information-theoretic metrics for software architectures, among them change propagation between components; Abdelmoez et al. (2005) gave the full analysis of the change propagation probability attribute; and the same group built SACPT, a web-based CASE (computer-aided software engineering) tool that automates the capture, modeling, and inspection of an architecture in order to derive and display the metric (Abdelmoez et al., 2004).

Formally, for an architecture with components C1,…,CnC_1,\dots,C_n, the change propagation probability cppij∈[0,1]cpp_{ij}\in[0,1] is the probability that a change arising in component CjC_j necessitates a change in component CiC_i; when several independent dependency paths can carry a change from one component to another, the propagation probability is combined across paths as the complement of the probability that no path propagates it. Collecting these values yields a change propagation matrix over the architecture, from which aggregate indicators are read — the probability that a change propagates beyond its originating component, the expected size of change, and the most propagation-prone components. These are interpreted as measures of a design's changeability: a good architecture is one that localizes change. Because the probabilities are conditional on architectural dependencies rather than on implementation details, the metric is computable from design models alone.

Shaik et al. (WICSA 2005) instantiated this on a single small example: a Job Application system studied in two architectural variants, one employing design patterns (reusable, well-tested object-oriented design solutions) and one not. The pattern-based variant showed substantially lower change propagation probabilities for the affected components, and the paper compared the change propagation metric against coupling-based metrics — the traditional structural indicators of changeability, which count dependency links between components. On this basis the authors argued the metric assesses architecture design quality, while explicitly leaving open whether the evidence generalizes: the study involved one small application, and the analysis workflow needed automated support to scale to complex systems. Shaik's 2006 thesis consolidated the line, implementing algorithms for error propagation, change propagation, and size of change on UML-RT (UML for Real-Time, a component-based modeling notation) models inside a web-based tool — so the computation itself is automatable.

Meanwhile, mainstream change-propagation research took a different route: mining version histories to predict which modules change together (Hassan and Holt, 2004), simulating propagation in dependency networks (Li et al., 2009; Pan et al., 2019), and correlating design dependencies with observed co-change (Leigh et al., 2021). These approaches address propagation empirically as systems evolve, but they neither compute nor validate the architecture-level probability metric as a design-time design-quality indicator. Reviews covering architecture change and architecture metrics (Williams and Carver, 2010; Coulin et al., 2019) describe the metric but record no follow-up validation. The unresolved question is thus empirical: does the change propagation probability metric, demonstrated once on a small textbook example, actually track design quality on larger systems?

Problem Statement

Does the change propagation probability metric validly assess the design quality of software architectures on case studies substantially larger than the single small Job Application example on which it was demonstrated?

Validating the metric on systems substantially beyond that example therefore involves deriving the change propagation probability matrix for the architecture of at least one real system whose scale, complexity, or evolution history is substantially beyond that of the original example, with a documented and reproducible source for the pairwise probabilities — historical change data, elicitation from engineers, or explicit model assumptions; testing the metric's validity against independent evidence of change behavior or design quality in that system; and determining whether the metric carries design-quality information beyond established coupling-based structural metrics. A rigorously supported negative result — the metric failing to discriminate design quality or to add value over coupling metrics at scale — answers the question as fully as a positive one. The question does not prescribe a particular system, size threshold, domain, or statistical method, and any such validation stands or falls by the standards of empirical software engineering. Tool support for computing the metric at scale already exists in the source group's SACPT and thesis tooling and is part of the practical means, not the open scientific question.

References

  1. I. Shaik, W. Abdelmoez, R. Gunnalan, M. Shereshevsky, A. Zeid, H. H. Ammar, A. Mili, C. Fuhrman, 'Change Propagation for Assessing Design Quality of Software Architectures', 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05), 2005, pp. 205-208. DOI: 10.1109/WICSA.2005.21, https://doi.org/10.1109/WICSA.2005.21
  2. W. Abdelmoez, M. Shereshevsky, R. Gunnalan, H. H. Ammar, B. Yu, S. Bogazzi, M. Korkmaz, A. Mili, 'Quantifying software architectures: an analysis of change propagation probabilities', 3rd ACS/IEEE International Conference on Computer Systems and Applications (AICCSA), 2005, pp. 660-667. DOI: 10.1109/AICCSA.2005.1387113, https://doi.org/10.1109/AICCSA.2005.1387113
  3. W. Abdelmoez, M. Shereshevsky, R. Gunnalan, H. H. Ammar, B. Yu, S. Bogazzi, M. Korkmaz, A. Mili, 'Software Architectures Change Propagation Tool (SACPT)', 20th IEEE International Conference on Software Maintenance (ICSM), 2004, p. 517. DOI: 10.1109/ICSM.2004.1357860, https://doi.org/10.1109/ICSM.2004.1357860
  4. I. Shaik, W. Abdelmoez, R. Gunnalan, A. Mili, C. Fuhrman, M. Shereshevsky, A. Zeid, H. H. Ammar, 'Using Change Propagation Probabilities to Assess Quality Attributes of Software Architectures', 4th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), 2006, pp. 704-711. DOI: 10.1109/AICCSA.2006.205167, https://doi.org/10.1109/AICCSA.2006.205167
  5. Israr Pasha Shaik, 'Web-based tool for software architecture metrics', MS Thesis, West Virginia University, 2006. DOI: 10.33915/etd.1721, https://researchrepository.wvu.edu/etd/1721
  6. M. Shereshevsky, H. Ammari, N. Gradetsky, A. Mili, H. H. Ammar, 'Information theoretic metrics for software architectures', 25th Annual International Computer Software and Applications Conference (COMPSAC), 2001, pp. 151-157. DOI: 10.1109/CMPSAC.2001.960611, https://doi.org/10.1109/CMPSAC.2001.960611
  7. Théo Coulin, Maxence Detante, William Mouchère, Fabio Petrillo, 'Software Architecture Metrics: a literature review', arXiv:1901.09050, 2019, https://arxiv.org/abs/1901.09050
  8. Byron J. Williams, Jeffrey C. Carver, 'Characterizing software architecture changes: A systematic review', Information and Software Technology 52(1), 2010, pp. 31-51. DOI: 10.1016/j.infsof.2009.07.002, https://doi.org/10.1016/j.infsof.2009.07.002
  9. Andrew Leigh, Michel Wermelinger, Andrea Zisman, 'Evaluating the effectiveness of risk containers to isolate change propagation', Journal of Systems and Software 182, 2021. DOI: 10.1016/j.jss.2021.110947, https://doi.org/10.1016/j.jss.2021.110947
  10. Ahmed E. Hassan, Richard C. Holt, 'Predicting change propagation in software systems', 20th IEEE International Conference on Software Maintenance (ICSM), 2004, pp. 284-293. DOI: 10.1109/ICSM.2004.1357812, https://doi.org/10.1109/ICSM.2004.1357812
  11. Noraini Ibrahim, Wan M. N. Wan-Kadir, Safaai Deris, 'Comparative Evaluation of Change Propagation Approaches towards Resilient Software Evolution', 3rd International Conference on Software Engineering Advances (ICSEA), 2008, pp. 198-204. DOI: 10.1109/ICSEA.2008.71, https://doi.org/10.1109/ICSEA.2008.71
  12. Lin Li, Guanqun Qian, Li Zhang, 'Evaluation of Software Change Propagation Using Simulation', WRI World Congress on Software Engineering (WCSE), 2009, pp. 28-33. DOI: 10.1109/WCSE.2009.22, https://doi.org/10.1109/WCSE.2009.22
  13. Weifeng Pan, Haibo Jiang, Hua Ming, Chunlai Chai, Bi Chen, Hao Li, 'Characterizing Software Stability via Change Propagation Simulation', Complexity 2019, Article 9414162. DOI: 10.1155/2019/9414162, https://doi.org/10.1155/2019/9414162