Monitoring-based adaptive differential-privacy defense against query-flooding model extraction at DNN scale and under colluding accounts

Machine-learning-as-a-service (MLaaS) prediction APIs can be queried repeatedly, and an adversary who duplicates queries can average away independent differential-privacy (DP) noise to extract model parameters. Yan et al. (IEEE TDSC 2022) demonstrated this with their query-flooding parameter duplication (QPD) attack and answered it with the monitoring-based DP (MDP) defense, in which a real-time Monitor estimates accumulated information leakage about the training data and an adaptive allocation rule (APBA) shrinks the per-query privacy budget as leakage grows, under a provable cap on the total allocated privacy budget. Validated only on logistic-regression and small tabular neural models, the authors themselves flag that the approach's costs and performance degrade on high-dimensional deep neural networks and that per-account leakage accounting does not address adversaries coordinating queries across multiple accounts. The core problem — making monitor-driven adaptive DP budgeting sound at DNN scale and against collusion, with bounded aggregate privacy loss — remains open.

Background

Machine-learning-as-a-service (MLaaS) platforms expose trained models through public prediction APIs, usually paid per query. Because anyone can submit inputs and observe the returned predictions, an adversary can attempt a model extraction attack: reconstructing the model's parameters, or training a functionally equivalent substitute, purely from query-response pairs. A particularly cheap extraction strategy exploits randomness in the defense itself: when the service adds independent, identically distributed noise to each response — the standard deployment of differential privacy (DP) as output perturbation — the law of large numbers lets the adversary submit the same query $r$ times and average the $r$ noisy answers, recovering the true response with variance shrinking as $1/r$. DP formalizes the protection: a randomized mechanism satisfies $\epsilon$-differential privacy if, for any two training datasets differing in one record, the distributions of its outputs differ by at most a multiplicative factor $e^\epsilon$; smaller $\epsilon$ means more noise and stronger privacy. Earlier DP-based extraction defenses such as BDPL (a boundary DP layer that perturbs the confidence scores returned for a query) allocate a fixed per-query $\epsilon$, which is exactly what repeated-query averaging defeats.

Yan et al. (IEEE Transactions on Dependable and Secure Computing 19(4):2680–2694, 2022) made both the attack and a defense concrete. Their QPD attack submits $n+1$ linearly independent queries for an $n$-dimensional linear model, duplicates each query $r$ times, estimates each true response by the sample mean, and solves the resulting linear system for the model coefficients; for non-linear targets it trains a surrogate model on the collected pairs. Their MDP defense closes the loop between measurement and allocation. A Monitor module estimates, for each query-response pair $u$, the information gained about the training dataset $D$ as $I_u = \operatorname{PCC}(D,u) \times I_D - \operatorname{PCC}(Q_A,u) \times I_{Q_A}$, where PCC is the Pearson correlation coefficient, $I_D$ is the entropy of the training data, and $Q_A$ is the matrix of previously answered pairs; the accumulated leakage $L_i$ then drives APBA (adaptive privacy-budget allocation), which sets the per-query budget

$\epsilon_i = \sqrt{p\,(L_t - L_i)},$

a concave allocation that shrinks as leakage approaches a tolerable threshold $L_t$, with scale parameter $p$ (recomputed at each query from the remaining budget $\epsilon_{re} = \epsilon - \sum_{k<i}\epsilon_k$ as $p = 9\epsilon_{re}^2/(4L_t^3)$) chosen so that the total privacy budget is exactly exhausted at the threshold. This budget-exhaustion identity provably caps the sum of the allocated per-query budgets; it is distinct from a composed differential-privacy guarantee over an adversary-controlled query stream. On logistic-regression and small neural-network models over tabular datasets, MDP held QPD's extraction fidelity around 50–60% (versus above 80% under fixed-budget DP) while keeping benign-user accuracy between 84% and 96%.

The authors' own Discussion section, however, delimits what remains unsolved. First, QPD's optimization and duplication costs grow expensive for very high-dimensional models such as computer-vision deep neural networks (DNNs) with hundreds of thousands of dimensions, and MDP's defensive performance degrades there too; all reported evidence concerns low-dimensional tabular models. Second, all measurement and budgeting is per account, while adversaries can coordinate queries across multiple accounts; the authors state that collusion needs to be re-examined from the service-mechanism side and offer only the conjecture that Monitor could measure colluding knowledge accumulation, with no mechanism or policy. A broader agenda — extending the defense to evasion and poisoning attacks and moving from parameter protection to function-level protection — is also sketched but has no completion boundary. Fundamental reconstruction results for output perturbation (polynomial-time attacks on real-valued sanitizers) additionally imply that output DP can slow, not eliminate, extraction, so the realistic goal for this defense family is bounded cumulative leakage at acceptable utility, not impossibility of extraction.

Problem Statement

Extend the monitoring-based adaptive differential-privacy defense paradigm — a real-time leakage measurement driving per-query privacy-budget allocation under a provable cap on the sum of allocated per-query budgets — so that it remains sound under the two deployment conditions Yan et al. explicitly left open.

(1) High-dimensional targets: the defense, and the query-flooding threat it is evaluated against, must work when the protected model is a large deep neural network rather than a low-dimensional tabular model. Monitor's leakage estimate must remain computable in real time and faithful at DNN scale, the adaptive allocation must still provably bound the cumulative privacy loss composed across queries rather than merely the sum of allocated per-query budgets, and benign utility must remain acceptable.

(2) Coordinated multi-account adversaries: leakage measurement and budget accounting must remain sound when extraction queries are split across multiple colluding accounts, so that the budget bound controls the knowledge accumulated jointly by the collusion rather than the per-account leakage. This may require joint accounting mechanisms and service-level policies, which the source says still need to be designed.

The common objective uniting both requirements is the accounting validity of the Monitor-to-budget loop: measured leakage and allocated budget must continue to track the adversary's true accumulated knowledge precisely in the settings where per-account, low-dimensional accounting was never validated. The two conditions are independent axes of this accounting problem: (1) validity and real-time computability of the composed-across-queries bound at DNN scale, and (2) joint accounting across colluding accounts. Establishing the composed bound at DNN scale leaves the collusion condition open, and an account-agnostic composed bound settles (2) as well while a per-account composition leaves it to separate treatment. No numeric fidelity tolerance is fixed for "sound" or "faithful" accounting; soundness is a property of tracking the adversary's accumulated knowledge, not of meeting a prescribed numeric threshold. The source's broader agenda of comprehensive function-level protection against evasion and poisoning is recorded as context, not as part of the completion boundary.

References

1. Haonan Yan, Xiaoguang Li, Hui Li, Jiamin Li, Wenhai Sun, Fenghua Li. Monitoring-Based Differential Privacy Mechanism Against Query Flooding-Based Model Extraction Attack. IEEE Transactions on Dependable and Secure Computing 19(4):2680–2694, 2022. DOI: 10.1109/TDSC.2021.3069258. https://doi.org/10.1109/TDSC.2021.3069258
2. Haonan Yan, Xiaoguang Li, Hui Li, Jiamin Li, Wenhai Sun, Fenghua Li. Monitoring-based Differential Privacy Mechanism Against Query-Flooding Parameter Duplication Attack. arXiv:2011.00418, 2020. https://arxiv.org/abs/2011.00418 (author preprint of the TDSC paper above, under its earlier title)
3. Xiaoguang Li, Hui Li, Haonan Yan, Zelei Cheng, Wenhai Sun, Hui Zhu. Mitigating Query-Flooding Parameter Duplication Attack on Regression Models with High-Dimensional Gaussian Mechanism. arXiv:2002.02061, 2020. https://arxiv.org/abs/2002.02061
4. Yanjiao Chen, Rui Guan, Xueluan Gong, Jianshuo Dong, Meng Xue. D-DAE: Defense-Penetrating Model Extraction Attacks. 2023 IEEE Symposium on Security and Privacy, pp. 382–399, 2023. DOI: 10.1109/SP46215.2023.10179406. https://doi.org/10.1109/SP46215.2023.10179406
5. Xueluan Gong, Rubin Wei, Ziyao Wang, Yuchen Sun, Jiawen Peng, Yanjiao Chen, Qian Wang. Beowulf: Mitigating Model Extraction Attacks Via Reshaping Decision Regions. Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security, pp. 3838–3852, 2024. DOI: 10.1145/3658644.3670267. https://doi.org/10.1145/3658644.3670267
6. Minxue Tang, Anna Dai, Louis DiValentin, Aolin Ding, Amin Hass, Neil Zhenqiang Gong, Yiran Chen, Hai (Helen) Li. ModelGuard: Information-Theoretic Defense Against Model Extraction Attacks. 33rd USENIX Security Symposium, 2024. https://www.usenix.org/conference/usenixsecurity24/presentation/tang
7. Yubo Tang, Wenjian Luo, Zipeng Ye, Qi Zhou. PVMTA: Prediction Vector Monitoring and Adaptive Temperature Adjustment Against Model Extraction. 2025 IEEE 10th International Conference on Data Science in Cyberspace, pp. 431–440, 2025. DOI: 10.1109/DSC67331.2025.00062. https://doi.org/10.1109/DSC67331.2025.00062
8. Quazi Rian Hasnaine, Yaodan Hu, Mohamed I. Ibrahem, Mostafa M. Fouda. A Comprehensive Survey of Model Extraction Attacks: Current Trends, Defenses, and Future Directions. 2025 1st International Conference on Secure IoT, Assured and Trusted Computing (SATC), pp. 1–6, 2025. DOI: 10.1109/SATC65530.2025.11137084. https://doi.org/10.1109/SATC65530.2025.11137084
9. Daryna Oliynyk, Rudolf Mayer, Andreas Rauber. I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences. ACM Computing Surveys 55:1–41, 2023. DOI: 10.1145/3595292. https://doi.org/10.1145/3595292
10. Martin M. Merener. Polynomial-time Attack on Output Perturbation Sanitizers for Real-valued Databases. Journal of Privacy and Confidentiality 2(2), 2011. DOI: 10.29012/jpc.v2i2.591. https://doi.org/10.29012/jpc.v2i2.591