e-space
Manchester Metropolitan University's Research Repository

    3D-CNN for facial micro- and macro-expression spotting on long video sequences using temporal oriented reference frame

    Yap, Chuin Hon, Yap, Moi Hoon ORCID logoORCID: https://orcid.org/0000-0001-7681-4287, Davison, Adrian ORCID logoORCID: https://orcid.org/0000-0002-6496-0209, Kendrick, Connah, Li, Jingting, Wang, Su-Jing and Cunningham, Ryan (2022) 3D-CNN for facial micro- and macro-expression spotting on long video sequences using temporal oriented reference frame. In: MM '22: The 30th ACM International Conference on Multimedia, 10 October 2022 - 14 October 2022, Lisboa, Portugal.

    [img]
    Preview
    Accepted Version
    Available under License In Copyright.

    Download (1MB) | Preview

    Abstract

    Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. Current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than tracking frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression duration. As baseline for MEGC2021 using leave-one-subject-out evaluation method, we show that our solution performed better in a high frame-rate (200 fps) SAMM long videos dataset (SAMM-LV) than a low frame-rate (30 fps) (CAS(ME)2) dataset. We introduce a new unseen dataset for MEGC2022 challenge (MEGC2022-testSet) and achieves F1-Score of 0.1531 as baseline result.

    Impact and Reach

    Statistics

    Activity Overview
    6 month trend
    235Downloads
    6 month trend
    63Hits

    Additional statistics for this dataset are available via IRStats2.

    Altmetric

    Repository staff only

    Edit record Edit record