e-space
Manchester Metropolitan University's Research Repository

    Synthesising facial macro-and micro-expressions using reference guided style transfer

    Yap, CH, Cunningham, R ORCID logoORCID: https://orcid.org/0000-0001-6883-6515, Davison, AK and Yap, MH (2021) Synthesising facial macro-and micro-expressions using reference guided style transfer. Journal of Imaging, 7 (8). p. 142. ISSN 2313-433X

    [img]
    Preview
    Published Version
    Available under License Creative Commons Attribution.

    Download (1MB) | Preview

    Abstract

    Long video datasets of facial macro-and micro-expressions remains in strong demand with the current dominance of data-hungry deep learning methods. There are limited methods of generating long videos which contain micro-expressions. Moreover, there is a lack of performance metrics to quantify the generated data. To address the research gaps, we introduce a new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality. For synthetic long video generation, we use the state-of-the-art generative adversarial network style transfer method—StarGANv2. Using StarGANv2 pre-trained on the CelebA dataset, we transfer the style of a reference image from SAMM long videos (a facial micro-and macro-expression long video dataset) onto a source image of the FFHQ dataset to generate a synthetic dataset (SAMM-SYNTH). We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace. For quantitative measurement, our findings show high correlation on two Action Units (AUs), i.e., AU12 and AU6, of the original and synthetic data with a Pearson’s correlation of 0.74 and 0.72, respectively. This is further supported by evaluation method proposed by OpenFace on those AUs, which also have high scores of 0.85 and 0.59. Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements. With this article, we publish our dataset to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.

    Impact and Reach

    Statistics

    Activity Overview
    6 month trend
    229Downloads
    6 month trend
    58Hits

    Additional statistics for this dataset are available via IRStats2.

    Altmetric

    Repository staff only

    Edit record Edit record