e-space
Manchester Metropolitan University's Research Repository

    StyleBabel: artistic style tagging and captioning

    Ruta, Dan, Gilbert, Andrew, Aggarwal, Pranav, Marri, Naveen, Kale, Ajinkya, Briggs, Jo ORCID logoORCID: https://orcid.org/0000-0002-4041-1918, Speed, Chris, Jin, Halin, Faieta, Baldo, Filipkowski, Alex, Lin, Zhe and Collomosse, John (2022) StyleBabel: artistic style tagging and captioning. In: ECCV 2022: 17th European Conference on Computer Vision, 23 October 2022 - 27 October 2022, Tel Aviv, Israel.

    [img]
    Preview
    Accepted Version
    Available under License In Copyright.

    Download (6MB) | Preview

    Abstract

    We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools. StyleBabel was collected via an iterative method, inspired by ‘Grounded Theory’: a qualitative approach that enables annotation while co-evolving a shared language for fine-grained artistic style attribute description. We demonstrate several downstream tasks for StyleBabel, adapting the recent ALADIN architecture for fine-grained style similarity, to train cross-modal embeddings for: 1) free-form tag generation; 2) natural language description of artistic style; 3) fine-grained text search of style. To do so, we extend ALADIN with recent advances in Visual Transformer (ViT) and cross-modal representation learning, achieving a state of the art accuracy in fine-grained style retrieval.

    Impact and Reach

    Statistics

    Activity Overview
    6 month trend
    198Downloads
    6 month trend
    56Hits

    Additional statistics for this dataset are available via IRStats2.

    Altmetric

    Repository staff only

    Edit record Edit record