Wang and Farnia (Uncertainty in Artificial Intelligence, UAI 2023) proved uniform-convergence generalization bounds showing that the gap between the training and population transferability scores of adversarial examples crafted on a substitute network is controlled by spectral-norm complexity terms of the substitute and target networks, suggesting that a better-generalizing substitute yields more transferable attacks. Their analysis rests on strong smoothness and Lipschitz assumptions on the loss and network activations and belongs entirely to the classical uniform-convergence regime, which is vacuous for the overparameterized, interpolating networks used in practice. The open problem is to extend this generalization-based analysis of adversarial transferability along the two directions stated in the source's conclusion: (1) to overparameterized function spaces — benign overfitting and neural-tangent-kernel/lazy-training analysis — to determine how interpolation and benign overfitting of the substitute model affect the transferability of the adversarial examples it generates, for example through bounds analogous to the source's Theorem 1; and (2) to determine how other popular regularization and architectural mechanisms such as batch normalization and dropout influence substitute-model generalization and the resulting transferability. No prior work is known to resolve these questions: adjacent results concern ensemble attacks, flat-minima bounds, or the adversarial robustness (rather than cross-model transferability) of benignly overfitted models, and later empirical work reports findings that conflict with the simple generalization-helps-transferability prediction, leaving the theoretical core open.

Background

Adversarial examples are inputs formed by adding a small, carefully chosen perturbation that flips a model's prediction while leaving the input semantically unchanged. A black-box or transfer attack sidesteps the need for access to the target model: the attacker crafts adversarial examples on a locally trained substitute network and submits them unchanged to an unseen target network. The transferability rate — the fraction of samples on which such a perturbation fools the target — is the empirical measure of success, and understanding what drives transferability is a central open problem in adversarial machine learning.

Wang and Farnia (Uncertainty in Artificial Intelligence, UAI 2023) gave one of the first generalization-theoretic answers. They model the attack as a max–min game in which the adversary perturbs an input to maximize the loss of a substitute classifier hwh_{\mathbf w} while a regularization weight λ\lambda controls the perturbation magnitude; the resulting attack perturbation δw∗(x,y)\delta^{*}_{\mathbf w}(\mathbf x,y) is the fixed point of the gradient-descent map on the substitute loss. Their Theorem 1 bounds ϵgen(δw∗)\epsilon_{\mathrm{gen}}(\delta^{*}_{\mathbf w}) — the gap between the empirical and population transferability scores of the crafted perturbation, where the score of an attack is the worst case (minimum) over the target classifiers of the target's loss on the perturbed inputs. The proof first bounds, uniformly over substitute parameters and target classifiers, the gap between each target's training and population loss via a Rademacher-complexity argument; Rademacher complexity measures the capacity of a function class by its ability to fit random noise, and the standard uniform-convergence technique converts such capacity bounds into gap bounds. The resulting bound scales with products of the spectral norms (largest singular values) of the weight matrices of both the substitute and the target networks, degrades with the depth and Lipschitz constant (a bound on how much the network's output can change per unit change in its input) of the substitute, and improves with the number of training samples nn. It holds under three assumptions (Assumptions 1–3): that the loss is bounded, Lipschitz, and smooth; that the substitute class consists of LL-layer networks with Lipschitz, smooth activations and bounded layer dimensions; and that the target class consists of KK-layer networks with Lipschitz activations and bounded layer dimensions — together with the bounded-data condition ∥X∥2≤B\|X\|_2\le B on the n×dn\times d data matrix XX and the condition λ(1−τ)≥(∏iγi∥Wi∥2)∑i∏jγj∥Wj∥2\lambda(1-\tau)\ge(\prod_i\gamma_i\|W_i\|_2)\sum_i\prod_j\gamma_j\|W_j\|_2, where WiW_i are the substitute's weight matrices, γi\gamma_i bound the derivatives of its activation functions, and τ>0\tau>0 is an absolute constant, so the condition ties the attack regularization parameter λ\lambda to the substitute's Lipschitz constant. The fixed-point characterization of δw∗\delta^{*}_{\mathbf w} itself rests on the source's Proposition 1: for L2L_2-bounded attack power, if ℓ∘hw\ell\circ h_{\mathbf w} is λ\lambda-smooth the fixed point exists and is unique. The interpretation: a substitute that generalizes better should produce more transferable perturbations, and operator-norm (spectral/Lipschitz) regularization of the substitute should raise transferability — a prediction their experiments on standard image benchmarks support for L2L_2- and L∞L_\infty-bounded attacks (perturbations limited in total Euclidean length, respectively in maximum per-coordinate change) generated by projected gradient descent (PGD, an iterative first-order attack that projects each step back into the allowed perturbation ball) and the fast gradient method.

This framework, however, sits entirely in the classical uniform-convergence regime. Modern deep networks are overparameterized and typically interpolate — fit their training data exactly. Benign overfitting names the phenomenon whereby an interpolating estimator still achieves near-optimal test risk; in this regime uniform-convergence bounds are provably vacuous, and the analysis tools are different: covariance-eigensystem and random-matrix arguments, or the neural tangent kernel (NTK), which describes the training of very wide networks as kernel regression in a 'lazy' regime in which the weights barely move from initialization. The source paper explicitly states as future work applying the recently developed generalization analysis of overparameterized function spaces to understand the role of benign overfitting in adversarial transferability, and studying how other popular regularization methods — batch normalization (a technique that rescales each layer's activations during training, widely used to stabilize optimization) and dropout (randomly deactivating units during training as a regularizer) — affect the transferability of designed adversarial examples.

The question is scientifically live rather than merely formal. Later work (Zhang et al., IEEE Symposium on Security and Privacy, S&P 2024) developed an alternative surrogate-side theory — a trade-off between surrogate smoothness and gradient similarity between substitute and target — and reported evidence conflicting with the simple generalization-helps-transferability prediction: data augmentations that yield the best-generalizing surrogates were found to perform worst in transfer attacks. Whether interpolation and benign overfitting of the substitute raise or lower transferability, and what replaces the spectral-norm bound in regimes where it is vacuous, is therefore genuinely unsettled.

Problem Statement

Extend the generalization-based analysis of adversarial example transferability of Wang and Farnia (UAI 2023) beyond its uniform-convergence, spectral-norm framework, in their max–min attack setting where perturbations δw∗\delta^{*}_{\mathbf w} are crafted on a substitute network hwh_{\mathbf w} and evaluated on unseen target networks. The problem has two components, corresponding to the two future directions stated in the source's conclusion; they are linked directions of one objective, and the open question is that objective — a generalization-theoretic explanation of when and why adversarial examples crafted on one model transfer to others, valid for the overparameterized networks used in practice:

(1) Overparameterized, interpolating regimes. Apply the modern generalization analysis of overparameterized function spaces — benign overfitting and neural-tangent-kernel/lazy-training analysis — to determine how benign overfitting and interpolation of the substitute model affect the transferability of the adversarial examples it generates. In particular, determine how the gap between the training and population transferability scores behaves in this regime, and which quantities, if any, take the place of the spectral-norm complexity terms of Theorem 1 that are vacuous here; either a bound analogous to Theorem 1 or a rigorous argument that such a bound cannot hold resolves this component.

(2) Other regularization and architectural mechanisms. Determine how other standard mechanisms — batch normalization, dropout, and related explicit or implicit regularizers — influence substitute-model generalization and the resulting transferability.

The model class under study is a well-specified overparameterized one — for example, overparameterized linear models or kernel/NTK regression — consistent with the source's call to extend the generalization analysis to overparameterized function spaces, and the transferability notion at issue is the substitute-to-target transfer of the source framework. The two components above are linked directions of the single overarching objective of explaining substitute-to-target transferability, rather than mechanisms or regimes considered in isolation from it. Empirical transfer-attack improvements that say nothing about the relation between substitute generalization and transferability, and analyses of a single model's own adversarial robustness, lie outside the scope of this objective.

References

  1. Yilin Wang and Farzan Farnia, 'On the Role of Generalization in Transferability of Adversarial Examples', Proceedings of the 40th Uncertainty in Artificial Intelligence Conference (UAI 2023), Proceedings of Machine Learning Research (PMLR) 216:2259-2270, 2023, https://proceedings.mlr.press/v216/wang23g.html; arXiv:2206.09238, DOI 10.48550/arXiv.2206.09238, https://arxiv.org/abs/2206.09238
  2. Yechao Zhang, Shengshan Hu, Leo Yu Zhang, Junyu Shi, Minghui Li, Xiaogeng Liu, Wei Wan, Hai Jin, 'Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability', IEEE Symposium on Security and Privacy (S&P) 2024, DOI 10.1109/SP54263.2024.00010; arXiv:2307.07873, DOI 10.48550/arXiv.2307.07873, https://arxiv.org/abs/2307.07873
  3. Wei Yao, Zeliang Zhang, Huayi Tang, Yong Liu, 'Understanding Model Ensemble in Transferable Adversarial Attack', Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), Proceedings of Machine Learning Research (PMLR) 267:71798-71833, 2025, https://proceedings.mlr.press/v267/yao25f.html; arXiv:2410.06851, DOI 10.48550/arXiv.2410.06851, https://arxiv.org/abs/2410.06851
  4. Meixi Zheng, Kehan Wu, Yanbo Fan, Rui Huang, Baoyuan Wu, 'Seeking Flat Minima over Diverse Surrogates for Improved Adversarial Transferability: A Theoretical Framework and Algorithmic Instantiation', IEEE Transactions on Pattern Analysis and Machine Intelligence (2026), DOI 10.1109/TPAMI.2026.3679726; arXiv:2504.16474, DOI 10.48550/arXiv.2504.16474, https://arxiv.org/abs/2504.16474
  5. Nikolaos Tsilivis and Julia Kempe, 'What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?', Advances in Neural Information Processing Systems 35 (NeurIPS 2022), https://openreview.net/forum?id=KBUgVv8z7OA; arXiv:2210.05577, DOI 10.48550/arXiv.2210.05577, https://arxiv.org/abs/2210.05577
  6. Yifan Hao and Tong Zhang, 'The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness', IEEE Transactions on Information Theory (2026), DOI 10.1109/TIT.2026.3702992; arXiv:2401.12236, DOI 10.48550/arXiv.2401.12236, https://arxiv.org/abs/2401.12236
  7. Philipp Benz, Chaoning Zhang, In So Kweon, 'Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective', 2021 IEEE/CVF International Conference on Computer Vision (ICCV 2021), DOI 10.1109/ICCV48922.2021.00772; arXiv:2010.03316, DOI 10.48550/arXiv.2010.03316, https://arxiv.org/abs/2010.03316
  8. Tao Wu, Tie Luo, Donald C. Wunsch, 'LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate', Proceedings of the AAAI Conference on Artificial Intelligence 38 (AAAI 2024), DOI 10.1609/aaai.v38i6.28430; arXiv:2312.13118, DOI 10.48550/arXiv.2312.13118, https://arxiv.org/abs/2312.13118
  9. Zeliang Zhang, Wei Yao, Xiaosen Wang, 'Bag of Tricks to Boost Adversarial Transferability', arXiv:2401.08734, 2024, DOI 10.48550/arXiv.2401.08734, https://arxiv.org/abs/2401.08734