Neuromorphic sensing for yawn detection in driver drowsiness

Paul Kielty; Mehdi Sefidgar Dilmaghani; Cian Ryan; Joe Lemley; Peter Corcoran

doi:10.1117/12.2680327

7 June 2023 Neuromorphic sensing for yawn detection in driver drowsiness

Paul Kielty, Mehdi Sefidgar Dilmaghani, Cian Ryan, Joe Lemley, Peter Corcoran

Proceedings Volume 12701, Fifteenth International Conference on Machine Vision (ICMV 2022); 127010Z (2023) https://doi.org/10.1117/12.2680327
Event: Fifteenth International Conference on Machine Vision (ICMV 2022), 2022, Rome, Italy

Abstract

Driver monitoring systems (DMS) are a key component of vehicular safety and essential for the transition from semiautonomous to fully autonomous driving. A key task for DMS is to ascertain the cognitive state of a driver and to determine their level of tiredness. Neuromorphic vision systems, based on event camera technology, provide advanced sensing of facial characteristics, in particular the behavior of a driver’s eyes. This research explores the potential to extend neuromorphic sensing techniques to analyze the entire facial region, detecting yawning behaviors that give a complimentary indicator of tiredness. A neuromorphic dataset is constructed from 952 video clips (481 yawns, 471 not-yawns) captured with an RGB colour camera, with 37 subjects. A total of 95,200 neuromorphic image frames are generated from this video data using a video-to-event converter. From these data 21 subjects were selected to provide a training dataset, 8 subjects were used for validation data, and the remaining 8 subjects were reserved for an ‘unseen’ test dataset. An additional 12,300 frames were generated from event simulations of a public dataset to test against other methods. A convolutional neural network (CNN) with self-attention and a recurrent head was trained and tested with these data. Respective precision and recall scores of 95.9% and 94.7% were achieved on our test set, and 89.9% and 91% on the simulated public test set, demonstrating the feasibility to add yawn detection as a sensing component of a neuromorphic DMS.

Citation Download Citation

Paul Kielty, Mehdi Sefidgar Dilmaghani, Cian Ryan, Joe Lemley, and Peter Corcoran "Neuromorphic sensing for yawn detection in driver drowsiness", Proc. SPIE 12701, Fifteenth International Conference on Machine Vision (ICMV 2022), 127010Z (7 June 2023); https://doi.org/10.1117/12.2680327

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Mouth

Cameras

Video

Education and training

Eye

Feature extraction

Sensors

Show All Keywords

Keywords/Phrases

Search In:

Publication Years