Near real-time comprehension classification with artificial neural networks: decoding e-Learner non-verbal behaviour

Holmes, Mike, Latham, Annabel ORCID: https://orcid.org/0000-0002-8410-7950, Crockett, Keeley ORCID: https://orcid.org/0000-0003-1941-6201 and O'Shea, James ORCID: https://orcid.org/0000-0001-5645-2370 (2018) Near real-time comprehension classification with artificial neural networks: decoding e-Learner non-verbal behaviour. IEEE Transactions on Learning Technologies, 11 (1). pp. 5-12. ISSN 1939-1382

Preview

Accepted Version Available under License In Copyright. Download (891kB) | Preview

Abstract

Comprehension is an important cognitive state for learning. Human tutors recognise comprehension and non-comprehension states by interpreting learner non-verbal behaviour (NVB). Experienced tutors adapt pedagogy, materials and instruction to provide additional learning scaffold in the context of perceived learner comprehension. Near real-time assessment for e-learner comprehension of on-screen information could provide a powerful tool for both adaptation within intelligent e-learning platforms and appraisal of tutorial content for learning analytics. However, literature suggests that no existing method for automatic classification of learner comprehension by analysis of NVB can provide a practical solution in an e-learning, on-screen, context. This paper presents design, development and evaluation of COMPASS, a novel near real-time comprehension classification system for use in detecting learner comprehension of on-screen information during e-learning activities. COMPASS uses a novel descriptive analysis of learner behaviour, image processing techniques and artificial neural networks to model and classify authentic comprehension indicative non-verbal behaviour. This paper presents a study in which 44 undergraduate students answered on-screen multiple choice questions relating to computer programming. Using a front-facing USB web camera the behaviour of the learner is recorded during reading and appraisal of on-screen information. The resultant dataset of non-verbal behaviour and question-answer scores has been used to train artificial neural network (ANN) to classify comprehension and non-comprehension states in near real-time. The trained comprehension classifier achieved normalised classification accuracy of 75.8%.