Fisher's 1936 chi-square analysis found Gregor Mendel's 1866 segregation data closer to theoretical expectation than chance allows, concluding the data had been "falsified so as to agree closely with Mendel's expectations." Pires and Branco (2010) proposed a reconciliation: if each experiment whose goodness-of-fit p-value fell below a threshold α was repeated once and the better-fitting run reported (Model A), the published p-values acquire a distribution between the uniform distribution and its square, and Fisher's aggregate too-good effect disappears. The authors themselves left open which selection mechanisms can produce this signature, whether they are distinguishable from Mendel's 84 published p-values, and — their Appendix D sketch — how far such selection can go in making an incorrect theory appear correct. These questions remain open: subsequent work develops the marginal distribution theory without identifying the generating mechanism, and independent re-analyses pursue different reconciliation routes. The precise open core is (i) identifiability of the selection mechanism from the published data and (ii) a systematic characterization of when historically plausible selection rules can mask a false theory.

Background

In 1866 Gregor Mendel published his pea-hybridization experiments, from which the laws of segregation and independent assortment were later derived (English translation reproduced in Franklin et al. 2008). In 1936 Ronald Fisher re-analyzed Mendel's published counts using the chi-square goodness-of-fit test — a standard test comparing observed frequencies to theoretically expected ones via the statistic χ2=∑(Oi−Ei)2/Ei\chi^2=\sum (O_i-E_i)^2/E_i, whose p-value (the probability, under a correct theory, of a fit at least as extreme as the one observed) should be uniformly distributed on [0,1][0,1] when the tested hypothesis is true. Fisher found the aggregate fit far too close to expectation — the values of χ2\chi^2 were systematically too small — and concluded that "the data of most, if not all, of the experiments have been falsified so as to agree closely with Mendel's expectations" (Fisher 1936). Edwards (1986) organized Mendel's data as 84 binomial experiments (comparisons of observed ratios of two classes against a binomial success probability) and confirmed the too-good pattern in the corresponding 84 p-values. The ensuing Mendel–Fisher controversy — fraud, unconscious bias, or innocent explanation — is surveyed in the documentary volume of Franklin, Edwards, Fairbanks, Hartl and Seidenfeld (2008). Pires and Branco (2010) introduced an explicit probabilistic model: in Model A, whenever an experiment's p-value falls below a threshold α\alpha (estimated at α^=0.201\hat\alpha=0.201, with 90% confidence interval (0.094, 0.362)(0.094,\,0.362)), the experiment is repeated once and the run with the larger p-value (the better fit) is reported. Under a true hypothesis this gives reported p-values a cumulative distribution function (c.d.f.) lying between xx and x2x^2, the fitted model passes a Kolmogorov–Smirnov test (a standard test comparing an empirical distribution to a specified c.d.f. via the maximum vertical distance), and — verified by Monte Carlo simulation, i.e. repeated random generation under the model — Fisher's aggregate statistic becomes unremarkable. The paper also studied Model B, reporting a mixture c.d.f. (1−β)x+βx2(1-\beta)x+\beta x^2 corresponding to repeating a random fraction β\beta of experiments: it fits the 84 marginal p-values acceptably (β^=0.45\hat\beta=0.45; Kolmogorov–Smirnov P=0.51P=0.51) yet fails to remove the aggregate too-good effect — so the marginal distribution alone does not determine which mechanism operated. Appendix D then asked, with only a normal-approximation sketch, whether the same selection applied to an incorrect theory can make it appear correct: if the original p-values have some non-uniform c.d.f. F0F_0, the reported ones have Fα∗(x)=[F0(x)]2F_\alpha^*(x)=[F_0(x)]^2 for x≤αx\le\alpha and Fα∗(x)=[1+F0(α)]F0(x)−F0(α)F_\alpha^*(x)=[1+F_0(\alpha)]F_0(x)-F_0(\alpha) otherwise, and the authors state that a detailed answer to how much falseness selection can mask (as a function of sample size, the deviation between true and hypothesized probabilities, and the selection intensity) "is beyond the scope of this paper," noting that allowing more than two repetitions would make "any theory look correct" easy. A later line of work by Brilhante, Gomes, Mendonça, Pestana and Santos (2024, 2025) named the whole [x,x2][x,x^2] family of laws "Mendel(mm)" distributions and developed their order-statistics theory and robustness of combined p-value tests; and Ellis, Hofer, Swain and van Dijk (2019) pursued a different, data-analytic reconciliation challenging Fisher's experimental reconstruction. None of these, however, identifies the mechanism behind Mendel's data or quantifies the masking power of selection.

Problem Statement

The problem, as left open by Pires and Branco (2010), is to give a complete account of the selection-mechanism explanation of the too-good fit of Mendel's data, comprising two jointly supporting questions that share one objective. (1) Identifiability: characterize the selection mechanisms — e.g., repeat up to k≥1k\ge 1 times when a goodness-of-fit p-value falls below a threshold, with binary, randomized, or sample-size-dependent triggers — that, when applied to faithful experiments, (a) give reported p-values a marginal c.d.f. between xx and x2x^2 under a true hypothesis, and (b) remove Fisher's aggregate too-good effect in the 84-experiment decomposition of Mendel's data; and determine which of these mechanisms are statistically distinguishable from the 84 published p-values (which vary in sample size) and which are not. The known boundary datum is that Model B (random repetition, c.d.f. (1−β)x+βx2(1-\beta)x+\beta x^2) satisfies (a) but not (b); the question is where the line of observational equivalence actually lies, so that one can state whether Model A's reconciliation is identifiable from the published data or is one of an indistinguishable family. (2) Distortion limits (the Appendix D question): for the repeat-if-significant, report-the-better-run rule and its kk-repetition generalizations, quantify systematically — analytically and/or by reproducible computation — how close the distribution of reported p-values can be made to the uniform distribution when the tested theory is incorrect, as a function of the sample size nn (in particular at Mendel-scale experiments of order 100 observations), the deviation between the true and hypothesized success probabilities, the selection intensity α\alpha (in particular within the historically plausible range (0.094, 0.362)(0.094,\,0.362) estimated from Mendel's own data), and the number of permitted repetitions kk; and thereby determine whether selection rules of historical plausibility could have converted a substantially incorrect theory into an apparently correct one, with an explicit characterization of the masking region in this parameter space. Sub-question (2) is resolved by an explicit characterization of the masking boundary as a function of all four parameters over their stated ranges — in particular, for every selection intensity α\alpha in the historically plausible interval (0.094, 0.362)(0.094,\,0.362), every repetition count k≥1k\ge 1 admitted by the repeat-and-report-best family, and the indicated deviations and Mendel-scale sample sizes, a determination of whether masking occurs — rather than only at isolated grid points of a numerical study. Each sub-question stands independently as a partial result, while the common objective is a defensible verdict on the selection-mechanism account: which mechanisms can produce Mendel's signature, whether the data can tell them apart, and how far their distortion power extends.

References

  1. Pires, Ana M.; Branco, João A. (2010). A Statistical Model to Explain the Mendel–Fisher Controversy. Statistical Science 25(4), 545–565. DOI: 10.1214/10-STS342. https://doi.org/10.1214/10-STS342 (author version: arXiv:1104.2975, https://arxiv.org/abs/1104.2975)
  2. Fisher, R. A. (1936). Has Mendel's work been rediscovered? Annals of Science 1(2), 115–137. DOI: 10.1080/00033793600200111. https://doi.org/10.1080/00033793600200111
  3. Edwards, A. W. F. (1986). Are Mendel's results really too close? Biological Reviews 61(4), 295–312. DOI: 10.1111/j.1469-185x.1986.tb00656.x. https://doi.org/10.1111/j.1469-185x.1986.tb00656.x
  4. Franklin, Allan; Edwards, A. W. F.; Fairbanks, Daniel J.; Hartl, Daniel L.; Seidenfeld, Teddy (2008). Ending the Mendel–Fisher Controversy. University of Pittsburgh Press. DOI: 10.2307/j.ctv10tq47g; ISBN 9780822943198. https://doi.org/10.2307/j.ctv10tq47g
  5. Ellis, T. H. Noel; Hofer, Julie M. I.; Swain, Martin T.; van Dijk, Peter J. (2019). Mendel's pea crosses: varieties, traits and statistics. Hereditas 156, article 33. DOI: 10.1186/s41065-019-0111-y. https://doi.org/10.1186/s41065-019-0111-y
  6. Brilhante, M. Fátima; Gomes, M. Ivette; Mendonça, Sandra; Pestana, Dinis; Santos, Rui (2024). Two P or Not Two P: Mendel Random Variables in Combining Fake and Genuine p-Values. AppliedMath 4(3), 1128–1142. DOI: 10.3390/appliedmath4030060. https://doi.org/10.3390/appliedmath4030060
  7. Brilhante, M. Fátima; Gomes, M. Ivette; Mendonça, Sandra; Pestana, Dinis; Santos, Rui (2025). Meta-analysis of Genuine and Fake p-Values. Journal of Statistical Theory and Practice 19(2), article 29. DOI: 10.1007/s42519-025-00445-3. https://doi.org/10.1007/s42519-025-00445-3
  8. Green, P. J.; Gill, R.; Mackenzie, N.; Mortera, J.; Thompson, W. (2022). Healthcare serial killer or coincidence? Statistical issues in investigation of suspected medical misconduct. arXiv:2210.00962. https://arxiv.org/abs/2210.00962