Deception in the eyes of deceiver: a computer vision and machine learning based automated deception detection

Khan, Wasiq, Crockett, Keeley ORCID: https://orcid.org/0000-0003-1941-6201, O'Shea, Jim ORCID: https://orcid.org/0000-0001-5645-2370, Hussain, Abir and Khan, Bilal (2021) Deception in the eyes of deceiver: a computer vision and machine learning based automated deception detection. Expert Systems with Applications, 169. p. 114341. ISSN 0957-4174

Preview

Published Version Available under License Creative Commons Attribution. Download (5MB) | Preview

Abstract

There is growing interest in the use of automated psychological profiling systems, specifically applying machine learning to the field of deception detection. Several psychological studies andmachine-based models have been reporting the use of eye interaction, gaze and facial movements as important clues to deception detection. However, the identification of very specific and distinctive features is still required. For the first time, we investigate the fine-grained level eyes and facial micro-movements to identify the distinctive features that provide significant clues for the automated deception detection. A real-time deception detection approach was developed utilizing advanced computer vision and machine learning approaches to model the non-verbal deceptive behavior. Artificial neural networks, random forests and support vector machines were selected as base models for the data on the total of 262,000 discrete measurements with 1,26,291 and 128,735 of deceptive and truthful instances, respectively. Analysis of the above models revealed that eye movements carry relatively important clues to distinguish truthful and deceptive behaviours. The research outcomes align with the findings from forensic psychologists who also reported the eye movements as distinctive for the truthful and deceptive behavior. The research outcomes and proposed approach are beneficial for human experts and has many applications within interdisciplinary domains.