Auditory-nerve fibers can fire at preferred phases of low-frequency sounds with sub-millisecond precision (phase locking), but whether this precise temporal code is genuinely necessary for human hearing remains unresolved. Saddler and McDermott (2024) provided the strongest evidence to date: neural networks optimized for sound localization, voice recognition, and word recognition on simulated auditory-nerve input reproduced human behavior best when phase locking was left intact, with task-dependent effects of degrading it. However, the authors themselves identify caveats that leave the necessity claim open: the models are not guaranteed to be near-optimal, they lack decision-stage noise, and the task instantiations were simplified (a 794-word vocabulary, static sources in simplified rooms, separate models per task); moreover no evidence was found that phase locking above 320 Hz is used for speech. The open problem is to settle whether human-like hearing behavior is achievable without fine timing — or whether the necessity of precise temporal coding survives once models approach task optimality, incorporate realistic noise, and face ecologically valid task demands — and to determine whether phase locking above 320 Hz contributes to speech and pitch perception.

Background

The auditory nerve encodes sound partly through spike timing: at low frequencies, auditory-nerve fibers fire at preferred phases of the stimulus waveform with sub-millisecond precision, a phenomenon called phase locking. The waveform can be decomposed into a slowly varying envelope (amplitude modulation) and the rapid temporal fine structure it rides on; phase locking can convey both, and at low carrier frequencies the fine structure is the timing of oscillations of the basilar membrane within each cochlear frequency channel. Whether this precision matters for perception is a long-standing question about the auditory neural code. Classic results frame the tension. Shannon et al. (1995) showed that speech recognition in quiet survives with only a few spectral channels carrying mainly envelope cues, suggesting coarse timing suffices for speech. Smith, Delgutte and Oxenham (2002), using 'chimaeric' sounds that swap the envelope of one sound onto the fine structure of another, found that speech intelligibility follows the envelope whereas sound localization and pitch follow the fine structure. Lorenzi et al. (2006) showed that hearing-impaired listeners lose the ability to exploit temporal fine structure in noise, linking fine timing to real-world listening difficulties. A complementary computational tradition uses ideal-observer analyses on auditory-nerve models (Heinz, Colburn and Carney, 2001) to bound achievable discrimination from nerve representations. Because the human upper limit of phase locking is not directly measurable, its value remains debated (Verschooten et al., 2019, compile opposing viewpoints). More recently, the field has used task-optimized deep neural networks as behavioral models: networks trained on real-world auditory tasks reproduce human behavior and cortical responses (Kell et al., 2018), sound-localization behavior (Francl and McDermott, 2022), and aspects of pitch perception, where models estimating fundamental frequency (F0, the repetition rate of a periodic sound that determines its pitch) require phase locking well above 1000 Hz to match human sensitivity (Saddler, Gonzalez and McDermott, 2021). Saddler and McDermott (2024) extended this approach to three real-world hearing tasks — sound localization, voice (talker) recognition, and word recognition — by optimizing networks on simulated auditory-nerve input (the Bruce et al. auditory-nerve model) in which the phase locking limit was manipulated by lowering the inner hair cell low-pass filter cutoff from the default 3000 Hz to 1000, 320, or 50 Hz. Models with high-fidelity phase locking best reproduced human behavior; lowering the cutoff worsened localization (mean absolute error by 6.4°) and voice recognition (accuracy by 10.5%), and human-model behavioral similarity degraded at low cutoffs for all three tasks, though for speech only the 50 Hz cutoff clearly hurt, yielding no evidence that phase locking above 320 Hz is used for speech. Interaural time differences (ITDs), the small differences in a sound's arrival time between the two ears that underlie low-frequency sound localization, are the cue most directly dependent on sub-millisecond timing. The authors' conclusions, however, rest on models whose optimality cannot be guaranteed: a model without fine timing might fail because precise timing is strictly necessary, or because the model is insufficiently optimized — and the models were deterministic apart from spike sampling, used a 794-word vocabulary, static sources in simplified rooms, and separate models per task.

Problem Statement

Determine whether sub-millisecond phase-locked spike timing in the auditory nerve is genuinely necessary to account for human hearing behavior — or whether human-like behavior is achievable without fine timing once the caveats identified by Saddler and McDermott (2024) are removed. The single overarching objective is a defensible verdict on the necessity (and its task-dependent degree) of precise temporal coding in human hearing, with the following jointly required components: (1) Optimality — establish whether models operating on simulated auditory-nerve input with degraded or eliminated phase locking can reproduce human behavior on sound localization, voice recognition, and word recognition once architectures and optimization are pushed toward task optimality (requiring all of: multiple expressive architectures, demonstrated convergence, and, where possible, comparison to ideal-observer bounds), since present conclusions could reflect suboptimal models rather than genuine necessity. (2) Noise — determine how adding realistic decision-stage noise, calibrated so that overall model performance matches human levels, alters the effect of the phase-locking limit on both task performance and human-model behavioral similarity. (3) Ecological validity — test whether more realistic task demands (requiring all of: larger vocabularies, dynamic sources in reverberant rooms, and single models performing localization and speech recognition concurrently) increase or decrease the temporal precision required for human-like behavior. (4) High-frequency phase locking — determine whether phase locking above 320 Hz contributes to speech and pitch perception, using behavioral assays more sensitive than those used to date, including stimuli derived from model predictions that dissociate phase-locking limits. A model without fine timing that matches the human behavioral pattern across the benchmark tasks under the optimality, noise, and ecological-validity conditions of components (1)–(3) would refute necessity; a persistent performance and similarity gap under those three conditions would establish task-dependent necessity. Component (4) is adjudicated separately: it is resolved by its own more sensitive behavioral assays, where either a demonstrated contribution of phase locking above 320 Hz to speech and pitch or evidence against one counts as a determination.

References

  1. Saddler, Mark R.; McDermott, Josh H. (2024). Models optimized for real-world tasks reveal the task-dependent necessity of precise temporal coding in hearing. Nature Communications 15:10590. DOI: 10.1038/s41467-024-54700-5. https://doi.org/10.1038/s41467-024-54700-5 (published version)
  2. Saddler, Mark R.; McDermott, Josh H. (2024). Models optimized for real-world tasks reveal the task-dependent necessity of precise temporal coding in hearing. bioRxiv preprint. DOI: 10.1101/2024.04.21.590435. https://doi.org/10.1101/2024.04.21.590435 (preprint version; title differs from the published version only by omission of 'task-dependent')
  3. Shannon, Robert V.; Zeng, Fan-Gang; Kamath, Vivek; Wygonski, John; Ekelid, Michael (1995). Speech Recognition with Primarily Temporal Cues. Science 270(5234):303-304. DOI: 10.1126/science.270.5234.303. https://doi.org/10.1126/science.270.5234.303
  4. Smith, Zachary M.; Delgutte, Bertrand; Oxenham, Andrew J. (2002). Chimaeric sounds reveal dichotomies in auditory perception. Nature 416:87-90. DOI: 10.1038/416087a. https://doi.org/10.1038/416087a
  5. Lorenzi, Christian; Gilbert, Gaëtan; Carn, Héloïse; Garnier, Stéphane; Moore, Brian C. J. (2006). Speech perception problems of the hearing impaired reflect inability to use temporal fine structure. Proceedings of the National Academy of Sciences 103(50):18866-18869. DOI: 10.1073/pnas.0607364103. https://doi.org/10.1073/pnas.0607364103
  6. Heinz, Michael G.; Colburn, H. Steven; Carney, Laurel H. (2001). Evaluating Auditory Performance Limits: I. One-Parameter Discrimination Using a Computational Model for the Auditory Nerve. Neural Computation 13(10):2273-2316. DOI: 10.1162/089976601750541804. https://doi.org/10.1162/089976601750541804
  7. Verschooten, Eric; Shamma, Shihab; Oxenham, Andrew J.; Moore, Brian C. J.; Joris, Philip X.; Heinz, Michael G.; Plack, Christopher J. (2019). The upper frequency limit for the use of phase locking to code temporal fine structure in humans: A compilation of viewpoints. Hearing Research 377:109-121. DOI: 10.1016/j.heares.2019.03.011. https://doi.org/10.1016/j.heares.2019.03.011
  8. Kell, Alexander J. E.; Yamins, Daniel L. K.; Shook, Erica N.; Norman-Haignere, Sam V.; McDermott, Josh H. (2018). A Task-Optimized Neural Network Replicates Human Auditory Behavior, Predicts Brain Responses, and Reveals a Cortical Processing Hierarchy. Neuron 98(3):630-644. DOI: 10.1016/j.neuron.2018.03.044. https://doi.org/10.1016/j.neuron.2018.03.044
  9. Saddler, Mark R.; Gonzalez, Ray; McDermott, Josh H. (2021). Deep neural network models reveal interplay of peripheral coding and stimulus statistics in pitch perception. Nature Communications 12:7278. DOI: 10.1038/s41467-021-27366-6. https://doi.org/10.1038/s41467-021-27366-6
  10. Francl, Andrew; McDermott, Josh H. (2022). Deep neural network models of sound localization reveal how perception is adapted to real-world environments. Nature Human Behaviour 6:111-133. DOI: 10.1038/s41562-021-01244-z. https://doi.org/10.1038/s41562-021-01244-z
  11. Saddler, Mark R.; Dau, Torsten; McDermott, Josh H. (2025). Towards individualized models of hearing-impaired speech perception. The 6th Clarity Workshop on Improving Speech-in-Noise for Hearing Devices (Clarity-2025). DOI: 10.21437/clarity.2025-3. https://doi.org/10.21437/clarity.2025-3
  12. Griffith, Ian M.; Hess, R. Preston; McDermott, Josh H. (2026). Optimized feature gains explain and predict successes and failures of human selective listening. Nature Human Behaviour. DOI: 10.1038/s41562-026-02414-7. https://doi.org/10.1038/s41562-026-02414-7
  13. Alavilli, Sagarika; McDermott, Josh H. (2026). From sound to source: Human and model recognition of environmental sounds. bioRxiv. DOI: 10.64898/2026.03.12.711349. https://doi.org/10.64898/2026.03.12.711349
  14. Panzeri, Stefano; Engel, Nicola Marie; Celotto, Marco (2026). Contribution of spike timing to the neural code: from fast to slow timescales. Biological Cybernetics. DOI: 10.1007/s00422-026-01042-8. https://doi.org/10.1007/s00422-026-01042-8
  15. Drakopoulos, Fotios; Pellatt, Lloyd; Sabesan, Shievanie; Xia, Yiqing; Fragner, Andreas; Lesica, Nicholas (2025). Modelling neural coding in the auditory midbrain with high resolution and accuracy. Nature Machine Intelligence. DOI: 10.1038/s42256-025-01104-9. https://doi.org/10.1038/s42256-025-01104-9
  16. Graves, Jackson E.; Guest, Daniel R.; Starr, Tess M.; Li, Angie; Mehta, Anahita H. (2025). Dissociating the neural codes for multiple pitch perception in humans. bioRxiv. DOI: 10.1101/2025.11.21.689749. https://doi.org/10.1101/2025.11.21.689749
  17. Bagur, Sophie; Bourg, Jacques; Kempf, Alexandre; Tarpin, Thibault; Bergaoui, Khalil; Guo, Yin; Ceballo, Sebastian; Schwenkgrub, Joanna; Verdier, Antonin; Puel, Jean Luc; Bourien, Jérôme; Bathellier, Brice (2025). A spatial code for temporal information is necessary for efficient sensory learning. Science Advances 11. DOI: 10.1126/sciadv.adr6214. https://doi.org/10.1126/sciadv.adr6214