The Elastic Lottery Ticket Hypothesis (E-LTH) showed that a winning ticket — a sparse, trainable-at-initialization subnetwork found by iterative magnitude pruning (IMP) — can be elastically transformed between deeper and shallower networks of the same design family by replicating or dropping layers, avoiding a full IMP run on the target network. The source paper itself left the central generalization open: transforming tickets across network width (channel counts), across mixed or topological architectural changes, and across disparate design families was found to be much harder than depth transformation and remained unsolved, with no theory predicting when a transferred ticket stays trainable. The open problem is to extend elastic ticket transformation beyond the depth-only, same-family regime and to establish a predictive account (an architectural-similarity criterion and/or loss-basin analysis) of transferability.

Background

The lottery ticket hypothesis (LTH) states that a dense, randomly initialized network contains a sparse subnetwork — a 'winning ticket', comprising both a parameter mask and its initialization — that, trained in isolation, matches or exceeds the full network's accuracy (Frankle and Carbin, arXiv:1803.03635). The standard way to find tickets is iterative magnitude pruning (IMP): repeatedly train a network, remove a fraction of the smallest-magnitude weights, rewind the survivors to their initialization, and retrain. IMP is expensive because it must be re-run from scratch for every architecture, which motivates reusing tickets across networks. Prior work had shown that ticket initializations generalize across datasets and optimizers (Morcos et al., arXiv:1906.02773), and that independently trained tickets for the same network tend to optimize into a common, linearly connected minimum — 'linear mode connectivity', where straight-line interpolations between two solutions retain accuracy — which is used as evidence that tickets found by different runs lie in the same loss basin (Frankle, Dziugaite, Roy, and Carbin, arXiv:1912.05671). A loss basin is a connected region of the loss landscape around one minimum within which an optimizer tends to stay.

Chen, Cheng, Wang, Gan, Liu, and Wang (NeurIPS 2021; arXiv:2103.16547) introduced the Elastic Lottery Ticket Hypothesis (E-LTH): by 'mindfully replicating (or dropping) and re-ordering layers', the winning ticket of one network can be stretched or squeezed into a winning ticket for a deeper or shallower network of the same design family (for example, within the ResNet family, where a network is a sequence of residual blocks — layers that learn a residual correction added to their input — organized into stages of repeated blocks). The resulting transformed subnetworks, called elastic ticket transformations (ETTs), were shown on CIFAR-10 and ImageNet to be nearly as competitive as the target network's own IMP-found ticket, and the paper observed that tickets transferred over smaller depth gaps retain better linear mode connectivity, suggesting transfer works when source and target tickets land in the same basin. The source's experiments ran on CIFAR-10 and ImageNet using ResNet and VGG — the classic plain convolutional network family — and compared elastic transformations against the target networks' own IMP tickets and against pruning-at-initialization baselines, which seek sparse trainable masks without full training, and dynamic-sparse-training baselines, which learn the sparsity pattern during training. However, E-LTH scales a ticket only along the depth axis. The paper's discussion (Appendix E) states this as its foremost limitation: preliminary attempts to stretch or squeeze network width (the number of channels, i.e., convolutional filters or neurons per layer) were 'much more challenging than depth transformations', and the authors conjecture that width-oriented transforms will need tools different from the depth-oriented ones they used. They further ask whether more sophisticated topological transforms are possible, noting that this requires defining what makes two architectures 'similar' enough to belong to one transferable design family (beyond repeating blocks and stage counts), and they leave a rigorous theory of why replicated or dropped masks remain trainable — their dynamical-systems rationale (viewing a deep residual network as a discretized dynamical system) and unrolled-estimation rationale (viewing an architecture as an unrolled iterative estimation algorithm) being preliminary and largely ResNet-specific — to future work. (One nuance from the source: transfer across fully-connected-layer counts and across MLP hidden widths on MNIST did succeed, so the open difficulty concerns chiefly convolutional channel width and general width transformation in modern architectures.) The paper also flagged, as a separate open issue it explicitly did not address, that winning tickets with hardware-friendly structured (e.g., channel-level rather than individual-weight) sparsity were unknown; that sub-issue has since been partially resolved (Chen et al., arXiv:2202.04736) and is treated here as context, not as the core question.

Problem Statement

Extend elastic winning-ticket transformation beyond the depth-only, same-family regime established by E-LTH, and characterize when such transfer works. Concretely, the open question posed by the source paper's own discussion comprises the following: (1) Construct and validate transformations that carry a winning ticket (mask plus initialization) found in one network to a target network that differs from the source in width (convolutional channel counts, where the authors' preliminary stretching/squeezing attempts already failed), in depth and width simultaneously, in broader topological structure, or that belongs to a different design family — such that the transferred ticket, trained in isolation, performs nearly as well as the target network's own IMP-found ticket at matched sparsity (the same fraction of remaining weights). (2) Provide a predictive account of transferability: a measurable criterion of architectural similarity — what makes two networks belong to one transferable design family — and/or an analysis of the loss-basin relationship that predicts, before training, which source-target pairs and transform ranges yield trainable transferred tickets, replacing the source's preliminary ResNet-centric rationales.

References

  1. Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang. The Elastic Lottery Ticket Hypothesis. Advances in Neural Information Processing Systems 34 (NeurIPS 2021), pp. 26609-26621. arXiv:2103.16547, DOI: 10.48550/arXiv.2103.16547, https://arxiv.org/abs/2103.16547 (proceedings page: https://proceedings.neurips.cc/paper/2021/hash/dfccdb8b1cc7e4dab6d33db0fef12b88-Abstract.html)
  2. Jonathan Frankle, Michael Carbin. The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. arXiv:1803.03635 (2018), DOI: 10.48550/arXiv.1803.03635, https://arxiv.org/abs/1803.03635
  3. Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, Michael Carbin. Linear Mode Connectivity and the Lottery Ticket Hypothesis. arXiv:1912.05671 (2019), DOI: 10.48550/arXiv.1912.05671, https://arxiv.org/abs/1912.05671
  4. Ari S. Morcos, Haonan Yu, Michela Paganini, Yuandong Tian. One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. arXiv:1906.02773 (2019), DOI: 10.48550/arXiv.1906.02773, https://arxiv.org/abs/1906.02773
  5. Tianlong Chen, Xuxi Chen, Xiaolong Ma, Yanzhi Wang, Zhangyang Wang. Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets. arXiv:2202.04736 (2022), DOI: 10.48550/arXiv.2202.04736, https://arxiv.org/abs/2202.04736
  6. William T. Redman, Tianlong Chen, Zhangyang Wang, Akshunna S. Dogra. Universality of Winning Tickets: A Renormalization Group Perspective. arXiv:2110.03210 (2021), DOI: 10.48550/arXiv.2110.03210, https://arxiv.org/abs/2110.03210
  7. Edward Prideaux-Ghee. Transferability of Winning Lottery Tickets in Neural Network Differential Equation Solvers. arXiv:2306.09863 (2023), DOI: 10.48550/arXiv.2306.09863, https://arxiv.org/abs/2306.09863
  8. Bohan Liu, Zijie Zhang, Peixiong He, Zhensen Wang, Yang Xiao, Ruimeng Ye, Yang Zhou, Wei-Shinn Ku, Bo Hui. A Survey of Lottery Ticket Hypothesis. arXiv:2403.04861 (2024), DOI: 10.48550/arXiv.2403.04861, https://arxiv.org/abs/2403.04861
  9. Hong-Yi Wang, Di Luo, Tomaso Poggio, Isaac L. Chuang, Liu Ziyin. A universal compression theory for lottery ticket hypothesis and neural scaling laws. arXiv:2510.00504 (2025), DOI: 10.48550/arXiv.2510.00504, https://arxiv.org/abs/2510.00504
  10. Rebekka Burkholz. Convolutional and Residual Networks Provably Contain Lottery Tickets. arXiv:2205.02343 (2022), DOI: 10.48550/arXiv.2205.02343, https://arxiv.org/abs/2205.02343
  11. Israel Adewuyi, Solomon Okibe, Vladmir Ivanov. The Multiple Ticket Hypothesis: Random Sparse Subnetworks Suffice for RLVR. arXiv:2602.01599 (2026), DOI: 10.48550/arXiv.2602.01599, https://arxiv.org/abs/2602.01599