Design diversity — building several independently developed versions of a function and adjudicating their outputs — is the classical route to dependable systems out of undependable components, but its reliability gain cannot currently be predicted or certified, because failures of independently developed versions are correlated. Kharchenko, Sklyar and Volkoviy (DepCoS-RELCOMEX 2007) consolidated the unresolved problems that determine whether multi-version technologies are practically effective: ensuring and certifying independence of versions against common-mode failures; defining, measuring and validating a practical metric of the degree of diversity applicable to heterogeneous software and platform stacks; designing cost-effective multi-version processes that meet dependability targets under realistic budgets; and architecting and verifying diagnostic, reservation and reconfiguration mechanisms across diverse versions under mixed physical, design and interaction faults. The load-bearing core — a validated, practical, pre-deployment measure of version independence with predictive power for coincident failures — remains open: from the Knight–Leveson experiment and the Eckhardt–Lee/Littlewood–Miller covariance models, through the 2001 survey verdict that evaluating the reliability of a specific diverse system remains difficult, to 2024–2026 work on cloud microservices and LLM-generated code in which ensembles still realize well under half of the independence-ideal reliability gain.

Background

A computing system is called dependable when it delivers a service that can justifiably be trusted; the standard taxonomy (Avizienis, Laprie, Randell and Landwehr, 2004) decomposes this into attributes such as reliability (continuity of correct service), availability, safety, integrity and confidentiality, and treats fault tolerance — redundancy, fault detection, diagnosis and recovery — as the engineering means for attaining them. For design faults (software bugs), one route to fault tolerance is design diversity, also called multi-version technology: several versions of a program, platform or service are developed independently — by separate teams, methods, programming languages or tools — and executed in parallel behind an adjudication mechanism (typically majority voting), so that the failure of one version is masked by the others. N-version programming (NVP), introduced by Avizienis (1985), is the canonical software instance; the idea extends to multi-channel hardware, FPGA (field-programmable gate array) designs and service-oriented architectures in which a middleware redirects requests among functionally equivalent but diverse services.

The reliability gain of such 1-out-of-N structures rests on the assumption that version failures are approximately independent. This assumption failed both theoretically and empirically. Eckhardt and Lee (1985) modeled a version as drawn at random from a development process and introduced the difficulty function θ(x)\theta(x), the probability that a version produced by that process fails on input xx; because some inputs are harder than most, failures of independently developed versions are positively correlated (common-mode or coincident failures). Knight and Leveson's 1986 experiment on independently developed versions of the same specification found exactly such correlated failures. Littlewood and Miller (1989) extended the theory to forced methodological diversity; in their formulation the probability of failure on demand (pfd — the probability that the system fails to deliver correct service on a randomly drawn demand) of a two-version pair is

pfdAB=pfdA⋅pfdB+cov⁡X(θA(X),θB(X)),\mathrm{pfd}_{AB}=\mathrm{pfd}_{A}\cdot\mathrm{pfd}_{B}+\operatorname{cov}_{X}\bigl(\theta_{A}(X),\theta_{B}(X)\bigr),

so the independence product is exact only when the difficulty functions are uncorrelated — and the covariance term is precisely what cannot be observed before deployment. Large experiments confirmed the picture: the PODS experiment (Bishop, 1988), the second-generation multi-version experiment (Kelly, Eckhardt, Vouk, McAllister and Caglayan, 1988), and van der Meulen and Revilla's (2008) study of roughly 89,000 independently written programs across 60 specifications found that 1-out-of-2 diversity yields on average around a hundredfold pfd reduction for reliable program pools, but with very large variability across specifications: language-forced diversity adds only on the order of ten percent, and a single dominant input that most programs fail can erase the diversity benefit entirely. The survey by Littlewood, Popov and Strigini (2001) states the field's standing verdict: diversity demonstrably helps on average, but it remains difficult to evaluate exactly how reliable a particular diverse fault-tolerant system is, because the actual level of dependence between versions is hard to assess.

The source paper (Kharchenko, Sklyar and Volkoviy, 2007) analyzes approaches to developing dependable systems out of undependable components via multi-version technologies. It proposes a multi-version lifecycle model (MVLC) — a stage-indexed plan specifying, for each development stage, how many versions to produce and which diversity-introducing and version-processing operations to apply; a 3M conception combining multi-version diversity (MV), multi-parametric adaptation (MA — run-time reconfiguration control contours) and multi-level controlled degradation (MD — graded redistribution of resources and goals under failure); and a scalar multiplicative dependability estimator over fault kinds (physical, design, interaction) and fault-tolerance operations (forecasting, prevention, detection, diagnosis, tolerance, removal, recovery), valid under independence assumptions between those contributors. It also considers principles of the development and assessment of dependable systems by means of multi-version technologies, including assessment methods such as Markov models and F(I)MEA (failure (impact) mode and effect analysis). The paper closes by consolidating the unresolved problems that determine the practical effectiveness of multi-version technologies.

Problem Statement

As consolidated by Kharchenko, Sklyar and Volkoviy (2007), the problem is to turn multi-version (design-diversity) engineering of dependable systems out of undependable components from a heuristic into a predictable, certifiable discipline — one coherent objective covering heterogeneous software and platform stacks and mixed fault classes (physical faults, design faults, interaction faults such as intrusions and mis-integration). Concretely, an answer must resolve:

(a) how to ensure and certify maximal independence of different versions and of their faults, so that common-mode failures are suppressed by construction and by evidence rather than assumed away;

(b) how to define, measure and validate a practical metric of the degree of version independence — the degree of diversity — applicable to heterogeneous software and platform stacks, and usable before deployment to predict coincident-failure behavior, not only to measure it post hoc;

(c) how to design cost-effective multi-version development processes, toolchains and lifecycle plans (how many versions of what kind, at which stages) that meet required dependability targets under realistic budget constraints — an optimization that is undefined until (b) yields a usable measure; and

(d) how to architect and verify diagnostic, reservation and reconfiguration mechanisms that operate correctly across diverse versions under combined physical, design and interaction faults, so that detected version failures are actually tolerated and recovered in heterogeneous deployments.

Items (a) and (b) are the load-bearing core: a validated, practical notion of degree of diversity with predictive power for coincident failures is the prerequisite on which certification in (a), cost optimization in (c), and tolerance mechanism design in (d) all depend. The core is the pair (a)+(b), not (b) alone: a measure that is never turned into an independence-certification procedure leaves (a) unresolved, and a certification claim with no validated measure does not resolve (b). Completion requires all four items (a)–(d); for each, resolution means: (a) a procedure that suppresses common-mode failures by construction and certifies the independence of a given set of versions by evidence rather than by assumption; (b) a metric that is defined, applies to heterogeneous software and platform stacks, and is usable before deployment to predict coincident-failure behavior, not only to measure it post hoc; (c) a lifecycle and cost model that fixes how many versions of what kind at which stages and shows the plan meets a stated dependability target under stated budget constraints; and (d) an architecture with a verification argument that detected version failures are actually tolerated and recovered under combined physical, design and interaction faults.

References

  1. Vyacheslav S. Kharchenko, Vladimir V. Sklyar, Andriy V. Volkoviy, 'Multi-version Information Technologies and Development of Dependable Systems out of Undependable Components', 2nd International Conference on Dependability of Computer Systems (DepCoS-RELCOMEX '07), IEEE, 2007, pp. 43–50. DOI: 10.1109/DEPCOS-RELCOMEX.2007.34. https://doi.org/10.1109/DEPCOS-RELCOMEX.2007.34
  2. A. Avizienis, 'The N-Version Approach to Fault-Tolerant Software', IEEE Transactions on Software Engineering, SE-11(12), 1985. DOI: 10.1109/TSE.1985.231893. https://doi.org/10.1109/TSE.1985.231893
  3. D. E. Eckhardt, L. D. Lee, 'A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors', IEEE Transactions on Software Engineering, SE-11(12), 1985. DOI: 10.1109/TSE.1985.231895. https://doi.org/10.1109/TSE.1985.231895
  4. John C. Knight, Nancy G. Leveson, 'An Experimental Evaluation of the Assumption of Independence in Multiversion Programming', IEEE Transactions on Software Engineering, SE-12(1), 1986. DOI: 10.1109/TSE.1986.6312924. https://doi.org/10.1109/TSE.1986.6312924
  5. B. Littlewood, D. R. Miller, 'Conceptual Modeling of Coincident Failures in Multiversion Software', IEEE Transactions on Software Engineering, 15(12), 1989. DOI: 10.1109/32.58771. https://doi.org/10.1109/32.58771
  6. P. G. Bishop, 'The PODS Diversity Experiment', in Dependable Computing and Fault-Tolerant Systems, Springer, 1988. DOI: 10.1007/978-3-7091-8932-0_4. https://doi.org/10.1007/978-3-7091-8932-0_4
  7. J. P. J. Kelly, D. E. Eckhardt, M. A. Vouk, D. F. McAllister, A. Caglayan, 'A Large Scale Second Generation Experiment in Multi-Version Software: Description and Early Results', The Eighteenth International Symposium on Fault-Tolerant Computing (FTCS-18), 1988. DOI: 10.1109/FTCS.1988.5290. https://doi.org/10.1109/FTCS.1988.5290
  8. Bev Littlewood, Peter Popov, Lorenzo Strigini, 'Modeling Software Design Diversity: A Review', ACM Computing Surveys, 33(2), 2001. DOI: 10.1145/384192.384195. https://doi.org/10.1145/384192.384195
  9. A. Avizienis, J.-C. Laprie, B. Randell, C. Landwehr, 'Basic Concepts and Taxonomy of Dependable and Secure Computing', IEEE Transactions on Dependable and Secure Computing, 1(1), 2004. DOI: 10.1109/TDSC.2004.2. https://doi.org/10.1109/TDSC.2004.2
  10. M. J. P. van der Meulen, M. A. Revilla, 'The Effectiveness of Software Diversity in a Large Population of Programs', IEEE Transactions on Software Engineering, 34(6), 2008. DOI: 10.1109/TSE.2008.70. https://doi.org/10.1109/TSE.2008.70
  11. Vyacheslav Kharchenko, Volodymyr Sklyar, 'Scalable Diversity-Oriented Decisions and Technologies for Dependable SoPC-Based Safety-Critical Computer Systems and Infrastructures', International Journal of Critical Computer-Based Systems, 2010. DOI: 10.1504/IJCCBS.2010.031890. https://doi.org/10.1504/IJCCBS.2010.031890
  12. Vyacheslav Kharchenko, Olexandr Siora, Volodymyr Sklyar, 'Multi-Version FPGA-Based Nuclear Power Plant I&C Systems: Evolution of Safety Ensuring', in Nuclear Power – Control, Reliability and Human Factors, IntechOpen, 2011. DOI: 10.5772/16554. https://doi.org/10.5772/16554
  13. Per Larsen, Andrei Homescu, Stefan Brunthaler, Michael Franz, 'SoK: Automated Software Diversity', 2014 IEEE Symposium on Security and Privacy, 2014. DOI: 10.1109/SP.2014.25. https://doi.org/10.1109/SP.2014.25
  14. Ievgen Babeshko, Vyacheslav Duzhiy, Oleg Illiashenko, Alexander Siora, Vladimir Sklyar, Artem Panarin, Eugene Brezhniev, 'Diversity for NPP I&C Systems Safety and Cyber Security', in Cyber Security and Safety of Nuclear Power Plant Instrumentation and Control Systems, IGI Global, 2020. DOI: 10.4018/978-1-7998-3277-5.CH010. https://doi.org/10.4018/978-1-7998-3277-5.CH010
  15. Nazanin Akhtarian, Hamzeh Khazaei, Marin Litoiu, 'Employing Software Diversity in Cloud Microservices to Engineer Reliable and Performant Systems', arXiv:2407.07287, 2024. https://arxiv.org/abs/2407.07287
  16. Rodrigo Pato Nogueira, Karthik Pattabiraman, Marco Vieira, João R. Campos, 'A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code', arXiv:2607.02808, 2026. https://arxiv.org/abs/2607.02808