Jipp and Ackerman (2016) showed that novices' information-processing ability (IPA) and working-memory capacity (WMC) moderate the effects of higher levels of automation (LOA) on performance and situation awareness (SA) in simulated air traffic control: as LOA of information-analysis and decision-selection automation rose, performance improved but SA declined, with higher-ability operators profiting and lower-ability operators suffering. The unresolved scientific question is the boundary conditions of this moderation: does it persist or strengthen when automation fails and failure diagnosis imposes added cognitive demand; do the ability-by-LOA interactions grow when information-analysis and decision-selection automation exceed the moderate levels tested, or appear when the LOA of information-acquisition and action-implementation automation varies; does the pattern generalize from novice samples to expert operators such as certified air traffic controllers; and can design mechanisms — adaptive automation that adjusts LOA to the operator's cognitive profile or transparency enhancements that make the automation's reasoning explicit — keep operators across the ability spectrum both performing well and situationally aware. Later work has encircled parts of this question with a related but distinct construct (multi-tasking ability), finding moderation of performance but not of SA under imperfect automation, and neighboring lines have established ability-SA links without manipulating automation; no study has directly crossed IPA or WMC with LOA and automation failure on both performance and SA, tested beyond-moderate LOA or other automation stages with ability moderators, or replicated the paradigm with experts.

Background

Human operators increasingly supervise automated systems rather than performing tasks manually. A standard framework (Parasuraman, Sheridan, and Wickens, 2000) describes automation along four stages of information processing — information acquisition (obtaining raw data), information analysis (transforming data into interpreted state), decision selection (choosing among options), and action implementation (executing the choice) — and assigns each stage a level of automation (LOA) ranging from fully manual to fully automatic. A robust empirical regularity is that raising the LOA of information-analysis and decision-selection automation improves routine performance but degrades situation awareness (SA), the operator's perception of task-relevant elements, comprehension of their meaning, and projection of their near-future status. This is the out-of-the-loop problem: supervisors of highly automated systems disengage from task state and are slower to notice and diagnose when something goes wrong (Endsley, 2024). SA is typically measured either by freeze-probe recall (the Situation Awareness Global Assessment Technique, SAGAT, in which the simulation pauses and the operator answers questions about the current traffic picture) or by online probe latency (the Situation Present Awareness Method, SPAM).

A second regularity is that these automation effects are not uniform across operators. Individual differences in cognition — information-processing ability (IPA), the general speed and capacity with which a person processes task information, and working-memory capacity (WMC), the ability to maintain and manipulate information over short intervals, usually indexed by tasks such as operation span — moderate both the benefits and the costs of automation. Jipp (2014) found individual differences moderated LOA effects on wheelchair-control performance and acceptance; Jipp (2016, Human Factors 58(1)) found IPA and WMC moderated how information versus decision automation affected expertise development; and Jipp (2016, Human Factors 58(8)) found that operators with better WMC and sustained attention reacted faster to consecutive automation failures in a monitoring task, suggesting that the extra cognitive demand of diagnosing failures engages exactly these abilities.

Jipp and Ackerman (2016) measured IPA and WMC in 60 novice participants who repeatedly controlled simulated air traffic with different LOAs of information-analysis and decision-selection automation. Performance rose but SA fell as LOA increased; participants with higher IPA and WMC profited from the higher LOA, while those with lower IPA and WMC suffered — automation amplified ability-based inequality.

Recent neighboring results make the question sharper rather than settled. Greenwell-Barnden and colleagues, studying a related construct, multi-tasking ability (MTa, a latent factor from several simultaneous-task tests), found that poorer multi-taskers benefited more from reliable low-degree automation in an air traffic control (ATC) simulation — a direction opposite to Jipp and Ackerman's at their LOA range — and that with imperfect automation (30% missed events), MTa moderated hand-off performance benefits and costs and stress and trust calibration but did not moderate SA or workload effects. Other work has tied ability to SA without manipulating automation (cue-utilization predicts SA in simulated ATC; WMC and expertise predict offline SA in professional pilots), probed ability-by-reliability interactions in smart-factory tasks without SA measurement, and examined stage-specific LOA effects without individual differences. Whether the specific IPA/WMC-by-LOA moderation of performance and SA holds under automation failure, at higher LOA, across automation stages, and for expert operators therefore remains empirically unresolved, as does the design question of matching automation to the operator's cognitive profile.

Problem Statement

Determine, experimentally, the boundary conditions under which individual differences in information-processing ability (IPA) and working-memory capacity (WMC) moderate the effects of automation level (LOA) on operator performance and situation awareness (SA), and identify and empirically evaluate design mechanism(s) intended to ensure that automation supports all operators. The unified objective — identifying when and for whom automation supports versus harms operators, so that automation can be made to support all operators — extends the findings of Jipp and Ackerman (2016), along four related open boundary-condition questions: (1) Failure regime: test whether the moderating effects of IPA and WMC on performance and SA persist or strengthen when automation fails at a controlled rate, so that operators must detect and diagnose failures; the source predicts failure diagnosis adds cognitive demand that engages these abilities. (2) LOA range and stage: test whether the observed ability-by-LOA interaction effects on performance and SA become larger when information-analysis and decision-selection automation increase beyond the moderate levels tested by Jipp and Ackerman (2016), and whether analogous interactions arise when the LOA of information-acquisition and action-implementation automation is varied. (3) Operator expertise: test whether the moderation pattern replicates with higher-proficiency operators, such as certified expert air traffic controllers, rather than the novice samples used originally, which bears on training and personnel-selection implications. (4) Support mechanism: develop and empirically evaluate at least one mechanism intended to ensure automation supports all operators — for example adaptive automation that adjusts LOA to the operator's measured cognitive profile, or transparency enhancements that make the automation's reasoning explicit — assessed against both performance and SA across the ability distribution. The four sub-questions (1)–(4) are posed jointly, and their resolution lies in a concrete experimental design with prespecified, falsifiable predictions about the direction and pattern of the IPA/WMC-by-LOA moderation of performance and SA that the design would confirm or refute, together with a specified support mechanism, a concrete plan for its evaluation, and its predicted boundary conditions, rather than in newly collected empirical results.

References

  1. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics — Part A: Systems and Humans, 30(3), 286–297. DOI: 10.1109/3468.844354. https://doi.org/10.1109/3468.844354
  2. Jipp, M., & Ackerman, P. L. (2016). The Impact of Higher Levels of Automation on Performance and Situation Awareness. Journal of Cognitive Engineering and Decision Making, 10(2), 138–166. DOI: 10.1177/1555343416637517. https://doi.org/10.1177/1555343416637517
  3. Jipp, M. (2014). Levels of automation: effects of individual differences on wheelchair control performance and user acceptance. Theoretical Issues in Ergonomics Science, 15(5), 479–504. DOI: 10.1080/1463922X.2013.815829. https://doi.org/10.1080/1463922X.2013.815829
  4. Jipp, M. (2016). Expertise Development With Different Types of Automation. Human Factors, 58(1), 92–106. DOI: 10.1177/0018720815604441. https://doi.org/10.1177/0018720815604441
  5. Jipp, M. (2016). Reaction Times to Consecutive Automation Failures. Human Factors, 58(8), 1248–1261. DOI: 10.1177/0018720816662374. https://doi.org/10.1177/0018720816662374
  6. Falkland, E. C., & Wiggins, M. W. (2019). Cross-task cue utilisation and situational awareness in simulated air traffic control. Applied Ergonomics, 74, 24–30. DOI: 10.1016/j.apergo.2018.07.015. https://doi.org/10.1016/j.apergo.2018.07.015
  7. Cak, S., Say, B., & Misirlisoy, M. (2020). Effects of working memory, attention, and expertise on pilots' situation awareness. Cognition, Technology & Work, 22(1), 85–94. DOI: 10.1007/s10111-019-00551-w. https://doi.org/10.1007/s10111-019-00551-w
  8. Endsley, M. R. (2024). Understanding Automation Failure. Journal of Cognitive Engineering and Decision Making, 18(4), 386–393. DOI: 10.1177/15553434231222059. https://doi.org/10.1177/15553434231222059
  9. Greenwell-Barnden, J. N., Loft, S., Bowden, V. K., Bender, A. D., Whitney, S. J., Lipp, O. V., & Visser, T. A. W. (2025). Individual differences in multi-tasking ability moderate the benefits of using low-degree automation. Theoretical Issues in Ergonomics Science, 26(4), 478–507. DOI: 10.1080/1463922X.2024.2446848. https://doi.org/10.1080/1463922X.2024.2446848
  10. Greenwell-Barnden, J. N., Visser, T. A. W., Loft, S., Whitney, S. J., & Bowden, V. K. (2026). How Multi-Tasking Ability Impacts Performance, Workload, Situation Awareness, Stress and Trust with Simulated Imperfect Automation. Human Factors, 68(8), 1056–1077. DOI: 10.1177/00187208261431968. https://doi.org/10.1177/00187208261431968
  11. Yang, C., Pang, L., Cao, X., Xue, P., Zhao, B., & Wu, W. (2026). Impact of Automation Levels in Different Information Processing Stages on Submersible Mechanical Support Operational Work Performance Under Unbalanced Workloads. International Journal of Human–Computer Interaction, 42(1), 357–373. DOI: 10.1080/10447318.2025.2508304. https://doi.org/10.1080/10447318.2025.2508304
  12. Zhang, L., Wu, X., Yan, B., & Liu, X. (2026). The Impact of Automation Conditions and Cognitive Abilities on Performance in Smart Factories under Different Task Types. CHI Conference on Human Factors in Computing Systems Extended Abstracts, 1–8. DOI: 10.1145/3772363.3798703. https://doi.org/10.1145/3772363.3798703