Hybrid Eye-Tracking on a Smartphone with CNN Feature Extraction and an Infrared 3D Model
Open Access
- 19 January 2020
- Vol. 20 (2), 543
- https://doi.org/10.3390/s20020543
Abstract
This paper describes a low-cost, robust, and accurate remote eye-tracking system that uses an industrial prototype smartphone with integrated infrared illumination and camera. Numerous studies have demonstrated the beneficial use of eye-tracking in domains such as neurological and neuropsychiatric testing, advertising evaluation, pilot training, and automotive safety. Remote eye-tracking on a smartphone could enable the significant growth in the deployment of applications in these domains. Our system uses a 3D gaze-estimation model that enables accurate point-of-gaze (PoG) estimation with free head and device motion. To accurately determine the input eye features (pupil center and corneal reflections), the system uses Convolutional Neural Networks (CNNs) together with a novel center-of-mass output layer. The use of CNNs improves the system’s robustness to the significant variability in the appearance of eye-images found in handheld eye trackers. The system was tested with 8 subjects with the device free to move in their hands and produced a gaze bias of 0.72°. Our hybrid approach that uses artificial illumination, a 3D gaze-estimation model, and a CNN feature extractor achieved an accuracy that is significantly (400%) better than current eye-tracking systems on smartphones that use natural illumination and machine-learning techniques to estimate the PoG.Keywords
Funding Information
- Natural Sciences and Engineering Research Council of Canada (480479)
This publication has 46 references indexed in Scilit:
- ATTENTION BIAS OF ANXIOUS YOUTH DURING EXTENDED EXPOSURE OF EMOTIONAL FACE PAIRS: AN EYE-TRACKING STUDYDepression and Anxiety, 2012
- Is banner blindness genuine? Eye tracking internet text advertisingApplied Cognitive Psychology, 2010
- Looking at movies and cartoons: eye‐tracking evidence from Williams syndrome and autismJournal of Intellectual Disability Research, 2009
- Eye Movements Drive Steering: Reduced Eye Movement Distribution Impairs Steering and Driving PerformanceJournal of Motor Behavior, 2008
- The eye movements of dyslexic children during reading and visual search: Impact of the visual attention spanVision Research, 2007
- Pilots' Monitoring Strategies and Performance on Automated Flight Decks: An Empirical Study Combining Behavioral and Eye-Tracking DataHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2007
- General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal ReflectionsIEEE Transactions on Biomedical Engineering, 2006
- Estimating the eye gaze from one eyeComputer Vision and Image Understanding, 2005
- Eye movements in reading and information processing: 20 years of research.Psychological Bulletin, 1998
- Eye-gaze word processingIEEE Transactions on Systems, Man, and Cybernetics, 1990