The lottery ticket hypothesis (LTH) claims that a dense neural network contains a sparse, identically initialized subnetwork that can be trained in isolation to match the full network. Ma et al. (NeurIPS 2021) showed that under a rigorous definition — requiring non-trivial sparsity, superiority over an equally-parameterized small dense network and over randomly reinitialized masks (themselves tuned over learning rates), matching of the pretrained dense network's accuracy at the same learning rate, and accuracy within a small tolerance of the well-trained dense network — no "Jackpot" winning ticket is found by iterative or one-shot magnitude pruning across ResNet/VGG/MobileNet models on CIFAR-10/100, Tiny-ImageNet and ImageNet-1K. They conjectured that the degree of over- or under-parameterization governs ticket existence and left the loss-landscape mechanism for why larger datasets reverse the relative effectiveness of one-shot versus iterative pruning unexplained. No later work (2021–2026) exhibits a Jackpot ticket, characterizes the size regime that admits one, or explains the dataset-scale reversal: the problem remains open.

Background

The lottery ticket hypothesis (LTH) (Frankle and Carbin, ICLR 2019) states that a dense, randomly initialized neural network contains a sparse subnetwork — a "winning ticket" — that, when trained in isolation from its original initial weights, reaches test accuracy comparable to ("matches") the full dense network. The standard search procedure is iterative magnitude pruning (IMP): repeatedly train the network, delete a fraction of the smallest-magnitude weights, rewind the surviving weights to their original initialization, and retrain. A cheaper alternative is one-shot magnitude pruning (OMP): train once, then delete the smallest-magnitude weights to the target sparsity in a single step. Because IMP from initialization fails on larger networks, weight rewinding (Renda et al., ICLR 2020) — returning weights to an early training iteration rather than iteration zero — is often used instead. Early empirical claims turned out to be highly sensitive to training hyperparameters, especially the learning rate and the number of training epochs, and sanity-check studies such as Su et al. (NeurIPS 2020) showed that randomly reinitialized masks can sometimes match, questioning whether the original initialization carries special information at all.

Ma et al. (NeurIPS 2021) reconciled these controversies with a rigorous definition of the LTH. A subnetwork (identified by a binary mask over the weights) is a Jackpot winning ticket if: (i) it is trained with a non-trivial sparsity level — unstructured sparsity (individual weights pruned anywhere in the network, rather than whole channels or layers) above roughly 60%, so that the mask could actually accelerate training or inference — and a sufficient number of epochs; (ii) a small dense network with the same parameter count, trained from scratch, is clearly less accurate (so the ticket is not trivially explained by parameter count); (iii) for some learning rate, training the masked subnetwork from its original initialization beats training the same mask from a random reinitialization — itself trained with any learning rate — by a clear gap (over 0.5% accuracy); (iv) for some learning rate, the subnetwork achieves accuracy similar to or higher than the pretrained dense network at the same learning rate; and (v) for some decent learning rate, the subnetwork's accuracy is within a small tolerance of the well-trained original dense network (within 0.5% on CIFAR-10, 1% on CIFAR-100 and Tiny-ImageNet, 1.5% on ImageNet-1K). Subnetworks satisfying conditions (i)–(iv) but not (v) are called secondary prize tickets — and the "winning tickets" of prior work are at most of this kind. Under this definition, and using IMP and OMP across ResNet, VGG, and MobileNet architectures on CIFAR-10/100, Tiny-ImageNet, and ImageNet-1K, the authors found no clear Jackpot ticket; only boundary cases (e.g., MobileNet-v1 on CIFAR-10 at about 83% sparsity) come close. They conjectured that the degree of over- or under-parameterization relative to the dataset (how much the network's trainable capacity exceeds, or falls short of, what the dataset requires to be fit) governs existence — extremely under-parameterized or extremely redundant networks are both unlikely to admit tickets — and observed that larger datasets (Tiny-ImageNet, ImageNet-1K) reverse the small-dataset patterns: OMP beats IMP and small learning rates are no longer preferable. The authors state that the loss-landscape and optimization-theoretic reasons for this dataset-dependent behavior "are still remaining mysterious", and their findings "open up many new questions for future work". (This paper is distinct from the similarly titled Su et al. 2020 "Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot", which concerns a different, weaker set of sanity checks.) Existence proofs of the LTH in idealized regimes, e.g., Malach et al. (ICML 2020) showing that sufficiently over-parameterized randomly weighted networks contain pruned subnetworks approximating any target function, address approximation capacity but not the empirical conditions above, which quantify practical usefulness.

Problem Statement

Under the rigorous definition of Ma et al. (NeurIPS 2021), determine whether Jackpot winning tickets exist — and under what conditions. Concretely, for neural network classifiers: (a) determine, for at least one combination of architecture, dataset (which may include scales and structures beyond those evaluated in the source), sparsity level, and mask-search procedure (magnitude-based IMP/OMP or an alternative), whether a subnetwork satisfying all five Jackpot conditions (non-trivial sparsity with sufficient training epochs; clear superiority to an equally-parameterized small dense network trained from scratch; a clear gap over training the same mask from a random reinitialization whose learning rate is itself tuned; accuracy similar to or above the pretrained dense network at the same learning rate; and accuracy within a justified tolerance of the well-trained dense network at a decent learning rate) exists — an existence claim whose refutation consists in establishing and quantifying the obstruction to its existence — with the "similar accuracy" tolerances for datasets not covered by the source fixed in advance of the search and justified independently of the searched-for result (the source fixes 0.5% on CIFAR-10, 1% on CIFAR-100 and Tiny-ImageNet, and 1.5% on ImageNet-1K); (b) characterize quantitatively the network-size regime — the degree of over- or under-parameterization relative to the dataset — in which Jackpot tickets are likely to exist, ideally as a predictive indicator for whether a given model/dataset pair admits one; and (c) provide a mechanistic, loss-landscape/optimization-theoretic explanation of why these factors modulate ticket existence, in particular why increasing dataset scale (e.g., from CIFAR to Tiny-ImageNet/ImageNet-1K) reverses the relative effectiveness of one-shot versus iterative magnitude pruning and shifts the preferred learning-rate regime. These components correspond to the open questions raised by the source paper — whether and under what conditions a genuine, practically useful winning ticket exists (a), its network-size conjecture (b), and the dataset-scale mechanism it states "are still remaining mysterious" (c). Components (a)–(c) are posed jointly.

References

  1. Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, and Yanzhi Wang. Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot? NeurIPS 2021, pp. 12749–12760. DOI: 10.48550/arXiv.2107.00166, https://arxiv.org/abs/2107.00166 (proceedings: https://proceedings.neurips.cc/paper/2021/hash/6a130f1dc6f0c829f874e92e5458dced-Abstract.html)
  2. Jonathan Frankle and Michael Carbin. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. ICLR 2019. DOI: 10.48550/arXiv.1803.03635, https://arxiv.org/abs/1803.03635
  3. Alex Renda, Jonathan Frankle, and Michael Carbin. Comparing Rewinding and Fine-tuning in Neural Network Pruning. ICLR 2020. DOI: 10.48550/arXiv.2003.02389, https://arxiv.org/abs/2003.02389
  4. Jingtong Su, Yihang Chen, Tianle Cai, Tianhao Wu, Ruiqi Gao, Liwei Wang, and Jason D. Lee. Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot. NeurIPS 2020. DOI: 10.48550/arXiv.2009.11094, https://arxiv.org/abs/2009.11094
  5. Eran Malach, Gilad Yehudai, Shai Shalev-Shwartz, and Ohad Shamir. Proving the Lottery Ticket Hypothesis: Pruning is All You Need. ICML 2020. DOI: 10.48550/arXiv.2002.00585, https://arxiv.org/abs/2002.00585
  6. Bohan Liu, Zijie Zhang, Peixiong He, Zhensen Wang, Yang Xiao, Ruimeng Ye, Yang Zhou, Wei-Shinn Ku, and Bo Hui. A Survey of Lottery Ticket Hypothesis. 2024. DOI: 10.48550/arXiv.2403.04861, https://arxiv.org/abs/2403.04861
  7. Advait Gadhikar and Rebekka Burkholz. Masks, Signs, And Learning Rate Rewinding. ICLR 2024. DOI: 10.48550/arXiv.2402.19262, https://arxiv.org/abs/2402.19262
  8. Junghun Oh, Sungyong Baik, and Kyoung Mu Lee. Find A Winning Sign: Sign Is All We Need to Win the Lottery. ICLR 2025. DOI: 10.48550/arXiv.2504.05357, https://arxiv.org/abs/2504.05357
  9. Tanay Arora and Christof Teuscher. Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket Search. 2025. DOI: 10.48550/arXiv.2512.07142, https://arxiv.org/abs/2512.07142
  10. Alan Jeffares and Mihaela van der Schaar. Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable. ICML 2025. DOI: 10.48550/arXiv.2506.23286, https://arxiv.org/abs/2506.23286
  11. Yanwei Yue, Guibin Zhang, Haoran Yang, and Dawei Cheng. Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks. AAAI 2025. DOI: 10.48550/arXiv.2412.07605, https://arxiv.org/abs/2412.07605