Rodriguez, Kwiat and Kamhoua (IEEE CISDA 2015) proposed a qualitative framework that scores the design diversity of a set of N functionally equivalent redundant variants by a single degree k=(1−s/σ)ck=(1-s/\sigma)c, where ss is the number of states shared by the variants' common finite-state machine, σ\sigma the state count of the largest possible machine, and cc a constant meant to capture diversity of development teams, tools, platforms and languages. From kk they derived qualitative relationships bounding the maximum extent ωmax\omega_{max} of common-mode failures, the minimum probability of undetected failure, and diversity-dependent synchronization and service-disruption delays. The framework, however, leaves the development constant cc with no estimation or calibration method, derives its undetected-failure bound only under severe simplifying assumptions (deterministic common-FSM, perfect decider, majority voting, full failure propagation), and none of the derived relationships has been validated against real multi-version systems. The open problem is to turn this qualitative framework into a calibrated, empirically validated quantitative theory: give an operational method for estimating kk and cc from observable properties of real redundant systems, re-derive the key relationships under relaxed decision-scheme and decider-failure assumptions, and test the predicted tradeoffs against experimental or operational data.

Background

Design diversity is a classical fault-tolerance technique in which several functionally equivalent versions of a component (differing in development team, language, platform, or design) are run in parallel and their outputs combined by a decider — an adjudication mechanism such as a majority voter — so that a flaw in one version is outvoted or detected by the others. It underlies N-version programming (Avizienis's approach of independently developing multiple software versions from a common specification) and N-modular redundancy with diverse hardware, and it has been advocated for security as well, since diverse replicas blunt both accidental common-mode faults and monoculture attacks that exploit a single implementation weakness.

A recurring difficulty is that diversity is hard to quantify and costly. Mitra, Saxena and McCluskey (IEEE Transactions on Computers, 2002) earlier introduced a quantitative design-diversity metric and used it in reliability analysis of redundant systems; other lines of work measured the cost of software design diversity empirically or studied how rarely independent versions actually fail independently. The classic N-version programming studies of Knight and Leveson (1986), Shimeall and Leveson (1988), and Townend, Xu and Munro (2001) report per-version failure counts and inter-version agreement patterns from controlled multi-version experiments. The metric framework under study, by Manuel Rodriguez, Kevin A. Kwiat and Charles A. Kamhoua (IEEE CISDA 2015), works as follows. The redundant variants share a common service specification modeled as a common finite-state machine (common-FSM) with ss states; σ\sigma denotes the number of states of the largest possible FSM implementing the same specification (the no-diversity extreme), and c∈[0,1]c\in[0,1] is a development-related constant reflecting how independently the variants were produced (teams, tools, platforms, languages). The degree of diversity is the scalar k=(1−s/σ)c∈[0,1]k=(1-s/\sigma)c\in[0,1]. The paper then derives, qualitatively: (1) a bound 1≤ωmax≤1+⌊(1−k)(N−1)⌋1\le\omega_{max}\le 1+\lfloor(1-k)(N-1)\rfloor on the maximum number of variants a single common-mode failure (a fault that simultaneously affects several replicas through a shared design flaw) can hit; (2) a necessary condition f≤⌊(N−1−ωmax)/ωmax⌋f\le\lfloor(N-1-\omega_{max})/\omega_{max}\rfloor on the number of sequentially tolerated failures under majority voting; (3) a bound on the probability of detecting a failure, which under the paper's conservative assumptions simplifies to Pdetect≤s/σP_{detect}\le s/\sigma, so that the probability of an undetected (latent) failure satisfies Pundetected≥1−s/σP_{undetected}\ge 1-s/\sigma, which grows with diversity — the security-relevant insight that more diverse variants leave fewer commonly monitored decision points (events or data from the variants that the decider adjudicates), and hence raise the chance that a coincident failure escapes notice; and (4) models Dsync=τ+μkΔτD_{sync}=\tau+\mu k\Delta_\tau and Ddisrupt=δ+μχε+μ(1−χ)(ζ+kΔζ)D_{disrupt}=\delta+\mu\chi\varepsilon+\mu(1-\chi)(\zeta+k\Delta_\zeta) showing synchronization and service-disruption delays increasing with diversity, with μ\mu a factor depending on the number of variants, χ\chi the fraction of spares kept hot (powered on and ready for immediate switchover), and the remaining symbols timing constants.

The authors themselves state the framework's unresolved issues: the metrics assume a deterministic common-FSM with one input and one output per state, a decider that never fails, and a majority-voting decision scheme, with no extension to imprecise voting or arbitrary consensus protocols; the constant cc is defined only qualitatively, with no method for its empirical estimation or calibration (so two designs with different subjective cc choices cannot be compared); the performance models rest on specific assumed parameter values; the influence of the failure-propagation probability PpropP_{prop} and of more complex decider behaviors on the detection bound is unexplored; and the whole analysis is static and unvalidated against real systems.

Problem Statement

Turn the qualitative design-diversity framework of Rodriguez, Kwiat and Kamhoua (2015) into a calibrated and empirically validated quantitative theory of design diversity for redundant fault-tolerant and secure systems. Concretely, and addressing the unresolved issues the authors themselves state, for systems comprising NN functionally equivalent variants whose outputs are combined by a decider (an adjudication mechanism such as a majority voter): (1) provide an operational method to estimate the degree of diversity k=(1−s/σ)ck=(1-s/\sigma)c — in particular the development-related constant cc, covering diversity of development teams, tools, platforms and languages — from observable properties of real variants or from experimental failure/agreement data, so that diversity values are reproducible rather than designer-chosen; (2) re-derive the framework's key relationships — the bound 1≤ωmax≤1+⌊(1−k)(N−1)⌋1\le\omega_{max}\le 1+\lfloor(1-k)(N-1)\rfloor on the maximum extent of a common-mode failure, the tolerated-failure condition under voting, the detection bound Pdetect≤s/σP_{detect}\le s/\sigma (equivalently Pundetected≥1−s/σP_{undetected}\ge 1-s/\sigma), and the diversity-dependent synchronization and service-disruption delay models — under the relaxed assumptions the authors identify as limitations (non-deterministic or richer common-FSM structure, deciders that can fail or behave imprecisely, decision algorithms beyond precise majority voting (imprecise voting, consensus protocols), and failure-propagation probability Pprop<1P_{prop}<1), re-deriving each relationship under the relaxation(s) that enter its derivation; and (3) validate the resulting calibrated relationships (or their corrected forms, expressed in terms of the framework's diversity quantities together with any relaxation-specific parameters the corrected form requires) against data from real multi-version systems or controlled multi-version experiments, characterizing the diversity–fault-tolerance–performance tradeoff the framework predicts.

References

  1. Manuel Rodriguez, Kevin A. Kwiat, Charles A. Kamhoua, "On the use of design diversity in fault tolerant and secure systems: A qualitative analysis", 2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), pp. 1–8, 2015. DOI: 10.1109/cisda.2015.7208639, https://doi.org/10.1109/cisda.2015.7208639 (source paper)
  2. Manuel Rodriguez, Kevin A. Kwiat, Charles A. Kamhoua, "Modeling fault tolerant architectures with design diversity for secure systems", MILCOM 2015 — 2015 IEEE Military Communications Conference, pp. 1254–1263, 2015. DOI: 10.1109/milcom.2015.7357618, https://doi.org/10.1109/milcom.2015.7357618 (companion paper by the same authors)
  3. Qi Cheng, Kevin Kwiat, Charles A. Kamhoua, "Design Diversity for Mitigating Monoculture Induced Threats", 2016 IEEE 17th International Symposium on High Assurance Systems Engineering (HASE), 2016. DOI: 10.1109/hase.2016.37, https://doi.org/10.1109/hase.2016.37
  4. Silvano Chiaradonna, Felicita Di Giandomenico, Giulio Masetti, "Stochastic Modeling of Intrusion Tolerant Systems Based on Redundancy and Diversity", IEEE Transactions on Computers 74(12):4123–4137, 2025. DOI: 10.1109/tc.2025.3606189, https://doi.org/10.1109/tc.2025.3606189
  5. Peng Zhou, Decheng Zuo, Kun Mean Hou, Zhan Zhang, Jian Dong, Jianjin Li, Haiying Zhou, "A Comprehensive Technological Survey on the Dependable Self-Management CPS: From Self-Adaptive Architecture to Self-Management Strategies", Sensors 19(5):1033, 2019. DOI: 10.3390/s19051033, https://doi.org/10.3390/s19051033
  6. Qisheng Zhang, Abdullah Zubair Mohammed, Zelin Wan, Jin-Hee Cho, Terrence J. Moore, "Diversity-by-Design for Dependable and Secure Cyber-Physical Systems: A Survey", IEEE Transactions on Network and Service Management 19(1):706–728, 2022 (preprint arXiv:2007.08688). DOI: 10.1109/tnsm.2021.3091391, https://doi.org/10.1109/tnsm.2021.3091391
  7. S. Mitra, N. R. Saxena, E. J. McCluskey, "A design diversity metric and analysis of redundant systems", IEEE Transactions on Computers 51(5):498–510, 2002. DOI: 10.1109/tc.2002.1004589, https://doi.org/10.1109/tc.2002.1004589
  8. Orcun Temizkan, Sungjune Park, Cem Saydam, "Software Diversity for Improved Network Security: Optimal Distribution of Software-Based Shared Vulnerabilities", Information Systems Research 28(4):828–849, 2017. DOI: 10.1287/isre.2017.0722, https://doi.org/10.1287/isre.2017.0722
  9. Huangxin Wang, Fei Li, Songqing Chen, "Towards Cost-Effective Moving Target Defense Against DDoS and Covert Channel Attacks", Proceedings of the 2016 ACM Workshop on Moving Target Defense (MTD), pp. 15–25, 2016. DOI: 10.1145/2995272.2995281, https://doi.org/10.1145/2995272.2995281
  10. Subhash Lakshminarayana, David K. Y. Yau, "Cost-Benefit Analysis of Moving-Target Defense in Power Grids", 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 139–150, 2018. DOI: 10.1109/dsn.2018.00026, https://doi.org/10.1109/dsn.2018.00026
  11. Md Mazharul Islam, Qi Duan, Ehab Al-Shaer, "Specification-driven Moving Target Defense Synthesis", Proceedings of the 6th ACM Workshop on Moving Target Defense (MTD), pp. 13–24, 2019. DOI: 10.1145/3338468.3356830, https://doi.org/10.1145/3338468.3356830
  12. Zubaida Rehman, Iqbal Gondal, Mengmeng Ge, Hai Dong, Mark Gregory, Zahir Tari, "Proactive defense mechanism: Enhancing IoT security through diversity-based moving target defense and cyber deception", Computers & Security 139:103685, 2024. DOI: 10.1016/j.cose.2023.103685, https://doi.org/10.1016/j.cose.2023.103685
  13. J. C. Knight, N. G. Leveson, "An experimental evaluation of the assumption of independence in multiversion programming", IEEE Transactions on Software Engineering SE-12(1):96–109, 1986. DOI: 10.1109/tse.1986.6312924, https://doi.org/10.1109/tse.1986.6312924
  14. T. J. Shimeall, N. G. Leveson, "An empirical comparison of software fault tolerance and fault elimination", [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis, pp. 180–187, 1988. DOI: 10.1109/wst.1988.5373, https://doi.org/10.1109/wst.1988.5373
  15. P. Townend, J. Xu, M. Munro, "Building dependable software for critical applications: multi-version software versus one good version", Proceedings Sixth International Workshop on Object-Oriented Real-Time Dependable Systems, pp. 103–110, 2001. DOI: 10.1109/words.2001.945120, https://doi.org/10.1109/words.2001.945120