Deep-learning automatic speech recognition (ASR) systems can be fooled by adversarial examples (AEs) — audio with small added perturbations that is transcribed incorrectly. Many practical AE attacks are query-based: the adversary iteratively submits continuous, similar queries to the target model, and Guo, Sun, Chen and Ju (Cybersecurity, 2023) built a deployable 'process-based' defense on exactly this signal: a cache of the most recent kk queries, landmark-based audio fingerprints, and an alarm when the weighted cosine fingerprint similarity of a new query to the cached memory exceeds a threshold δ\delta. The defense identifies query-based AE generation with over 90% accuracy and withstands an adaptive random-noise attack, but its detection success degrades under adaptive fake-query injection, and its authors explicitly leave its robustness open against adversaries that break the defense's core assumptions: single-step/one-shot AE generation that emits no correlated query sequence, fingerprint fraud that spoofs or mimics benign fingerprints, resource-rich adversaries, and adversaries who attack the defense system itself. This record asks whether process-based audio AE detection can be made robust against — or must be fundamentally reconsidered under — these adversary classes, and what detection/false-positive/evasion trade-off governs it.

Background

An adversarial example (AE) against an automatic speech recognition (ASR) system is a waveform obtained by adding a small, often imperceptible perturbation to benign audio so that the model transcribes it as text chosen by the attacker; foundational targeted digital attacks include Carlini and Wagner's optimization-based attack on DeepSpeech, and physical 'over-the-air' attacks such as CommanderSong, which hides commands in music carriers. Many black-box attacks are query-based: the adversary repeatedly queries the target model, each time adjusting the perturbation using the returned transcription, which forces a long sequence of highly similar, temporally correlated queries. This query pattern is the exploitable weakness that Guo, Sun, Chen and Ju turned into a defense (Cybersecurity, 2023; arXiv:2304.10088). Their scheme is 'process-based' — it detects the attack-generation process rather than the final malicious result: the server keeps a memory of the last kk received queries, computes an audio fingerprint of each input (a compact, perceptually robust descriptor: short-time Fourier transform, selection of spectral peaks, and time–frequency peak pairs {f,t}\{f,t\} of the kind used in music-identification systems such as Shazam), and computes the weighted cosine similarity ss between the fingerprint of each new query and the fingerprints in memory; if s>δs>\delta for threshold δ\delta, the input is flagged as part of an AE-generation attempt. Because fingerprints are stable under small noise while being sensitive to audio identity, correlated queries from one attacker score high similarity, whereas queries from benign users do not. On four attacks (two built on the over-the-air attacks CommanderSong and Devil's Whisper, and two further query attacks on speech corpora) the defense identifies adversary intent with over 90% accuracy, without retraining the ASR model. The authors evaluated two adaptive adversaries who know the defense exists: adding random noise to queries (which perturbs fingerprint extraction — moderate noise with SNR in [25, 75] dB even improved robustness, and the authors report that noise strong enough to break detection also destroys attack success) and injecting a proportion pfakep_{fake} of decoy 'fake queries' into the memory; under fake-query injection detection degraded substantially — the detection success rate for the two music-carrier attacks dropped linearly when pfake∈[25,50]%p_{fake}\in[25,50]\% and fell to approximately 10%10\% or 0%0\% at pfake≥60%p_{fake}\ge 60\% — which the authors argue raises the cost and pressure of the attack, since attackers are uncertain whether they can ultimately obtain a successful AE. A closely related defense family exists in computer vision, where 'stateful defenses' similarly cache recent image queries and reject overly similar ones; that line has an active attack–defense literature, including adaptive attacks that defeat such defenses (Oracle-guided Adaptive Rejection Sampling) and theoretical characterizations of the trade-off between attack detection and false-positive rates. What is missing, and what the source work itself flags, is any comparable robustness account for the audio fingerprint-memory defense: universal adversarial perturbations for ASR (single fixed perturbations added to any input, generated once in advance and requiring no queries at attack time) and other single-step or transferable attacks produce no correlated query stream at all, so the signal the defense relies on can disappear. The source authors' own discussion of strong adaptive attacks notes that fingerprint fraud can also create vulnerabilities in the scheme: an attacker could modify the loss function for generating adversarial examples to include the fingerprint similarity score between the current speech and the speech in the local memory, which would reduce the fingerprint similarity between adjacent submitted speeches and so bypass the defense.

Problem Statement

The fingerprint-memory defense of Guo, Sun, Chen and Ju (2023) detects the process of query-based audio adversarial-example generation by exploiting exactly one signal: temporal correlation among an adversary's queries, measured as weighted cosine similarity between landmark-based audio fingerprints of the current query and a memory of the last kk queries, thresholded at δ\delta. The authors themselves state that this leaves the defense's robustness unresolved against adversary classes that break that assumption. The open problem is to determine whether process-based detection of audio AE generation can be made robust against — or must be fundamentally reconsidered under — the adversary classes the source work identifies: (1) single-step or one-shot AE generation, including universal and transferable perturbations that are computed in advance or need at most negligible query access, and hence produce no temporally correlated query sequence for the defense to observe; (2) fingerprint fraud — distinct from the random-noise adaptive attack already tested by the source, in which noise strong enough to break detection also destroyed attack success — i.e., queries deliberately crafted so that the attack remains effective while the measured similarity of correlated attack queries stays below the detection threshold; (3) adversaries with very large resources: the source's own fake-query experiments show detection success already collapsing at the highest tested injection ratios, and the authors concede that the scheme will be fooled if the attacker has many resources, yet what remains detectable — and at what cost to the attacker — against adversaries able to sustain high decoy-injection ratios or large-scale repeated probing is unresolved; (4) adversaries who attack the defense system itself rather than the target ASR system (for example flooding or poisoning the query memory, or reverse-engineering the cache and threshold policy). The four classes delimit the scope of a single question rather than four separate questions, and an adequate resolution for the audio/ASR setting must go beyond a restatement of the authors' concessions or of the fact that an attack emitting no query sequence leaves the process signal absent. Because classes (2)–(4) submit queries, the affirmative branch requires a process-based detector satisfying the deployability properties below that preserves the source's reported detection success (over 90% against the source's attack suite) at its reported false-positive level, while the negative branch requires a demonstration that such a class defeats any process-based detector meeting those properties — driving detection success below that level or forcing the false-positive rate above it. Class (1), single-step or one-shot generation, emits no query sequence for the process signal to observe, so the question it raises is whether the scheme can be extended to cover it while preserving the deployability properties; the detection/false-positive/evasion trade-off is what governs these branches. Any candidate hardening of the detector considered under these classes should preserve the deployability properties claimed for the source scheme (no ASR retraining, compatibility with model ensembles, lightweight operation). The question is asked for the audio/ASR setting of the source work; evidence from the vision-domain stateful-defense literature suggests the general defense class is vulnerable, but that transfer is itself unestablished for audio fingerprints.

References

  1. Feng Guo, Zheng Sun, Yuxuan Chen, Lei Ju, 'Towards the Universal Defense for Query-Based Audio Adversarial Attacks', arXiv:2304.10088, 2023, DOI: 10.48550/arXiv.2304.10088, https://arxiv.org/abs/2304.10088
  2. Feng Guo, Zheng Sun, Yuxuan Chen, Lei Ju, 'Towards the universal defense for query-based audio adversarial attacks on speech recognition system', Cybersecurity 6, 2023, DOI: 10.1186/s42400-023-00177-6, https://doi.org/10.1186/s42400-023-00177-6
  3. Nicholas Carlini, David Wagner, 'Audio Adversarial Examples: Targeted Attacks on Speech-to-Text', 2018 IEEE Security and Privacy Workshops (SPW), 2018, pp. 1-7, DOI: 10.1109/SPW.2018.00009, https://doi.org/10.1109/SPW.2018.00009 (arXiv:1801.01944)
  4. Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, Carl A. Gunter, 'CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition', USENIX Security 2018, 2018, https://www.usenix.org/conference/usenixsecurity18/presentation/yuan (arXiv:1801.08535)
  5. Paarth Neekhara, Shehzeen Hussain, Prakhar Pandey, Shlomo Dubnov, Julian McAuley, Farinaz Koushanfar, 'Universal Adversarial Perturbations for Speech Recognition Systems', Interspeech 2019, DOI: 10.21437/interspeech.2019-1353, https://www.isca-archive.org/interspeech_2019/neekhara19b_interspeech.html
  6. Ryan Feng, Ashish Hooda, Neal Mangaokar, Kassem Fawaz, Somesh Jha, Atul Prakash, 'Stateful Defenses for Machine Learning Models Are Not Yet Secure Against Black-box Attacks', Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS '23), 2023, pp. 786-800, DOI: 10.1145/3576915.3623116, https://doi.org/10.1145/3576915.3623116 (arXiv:2303.06280)
  7. Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash, 'Theoretically Principled Trade-off for Stateful Defenses against Query-Based Black-Box Attacks', 2nd AdvML Frontiers Workshop at ICML 2023, arXiv:2307.16331, 2023, DOI: 10.48550/arxiv.2307.16331, https://arxiv.org/abs/2307.16331
  8. Xin Wang, Kai Chen, Xingjun Ma, Zhineng Chen, Jingjing Chen, Yu-Gang Jiang, 'AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning', ACM Multimedia 2024, DOI: 10.1145/3664647.3681032, arXiv:2408.01978, https://arxiv.org/abs/2408.01978
  9. Zheng Sun, Jinxiao Zhao, Feng Guo, Yuxuan Chen, Lei Ju, 'CommanderUAP: a practical and transferable universal adversarial attacks on speech recognition models', Cybersecurity 7, 2024, DOI: 10.1186/s42400-024-00218-8, https://doi.org/10.1186/s42400-024-00218-8
  10. Renyang Liu, Wei Zhou, Xin Jin, Song Gao, Yuanyu Wang, Ruxin Wang, 'DTA: distribution transform-based attack for query-limited scenario', Cybersecurity 7, 2024, DOI: 10.1186/s42400-023-00197-2, https://doi.org/10.1186/s42400-023-00197-2
  11. Yanru Feng, Qingjie Liu, Jing Li, 'Audio Adversarial Example Detection Scheme via Re-Attack', Electronics 15(7):1411, 2026, DOI: 10.3390/electronics15071411, https://doi.org/10.3390/electronics15071411