Towards a Multimodal Model of Cognitive Workload Through Synchronous Optical Brain Imaging and Eye Tracking Measures
Open Access
- 23 October 2019
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Human Neuroscience
- Vol. 13, 375
- https://doi.org/10.3389/fnhum.2019.00375
Abstract
Recent advances in neuroimaging technologies have rendered multimodal analysis of operators’ cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts’ neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices’ response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field.Keywords
This publication has 48 references indexed in Scilit:
- The Structure of Cognition: Attentional Episodes in Mind and BrainNeuron, 2013
- Functional cerebral reorganization: a signature of expertise? Reexamining Guida, Gobet, Tardieu, and Nicolas' (2012) two-stage frameworkFrontiers in Human Neuroscience, 2013
- Automated classification and scoring of smooth pursuit eye movements in the presence of fixations and saccadesBehavior Research Methods, 2012
- Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial NavigationJournal of Visualized Experiments, 2011
- Optical Brain Imaging to Enhance UAV Operator Training, Evaluation, and Interface DevelopmentPublished by Springer Science and Business Media LLC ,2010
- Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flightErgonomics, 2008
- Near-infrared spectroscopy: does it function in functional activation studies of the adult brain?International Journal of Psychophysiology, 2000
- Increased Cortical Representation of the Fingers of the Left Hand in String PlayersScience, 1995
- Human factors and safety in the design of intelligent vehicle-highway systems (IVHS)Journal of Safety Research, 1993
- A capacity theory of comprehension: Individual differences in working memory.Psychological Review, 1992