e-space
Manchester Metropolitan University's Research Repository

    Srr-lgr: local–global information-reasoned social relation recognition for human-oriented observation

    Qing, Linbo ORCID logoORCID: https://orcid.org/0000-0003-3555-0005, Li, Lindong, Wang, Yuchen ORCID logoORCID: https://orcid.org/0000-0002-8697-365X, Cheng, Yongqiang ORCID logoORCID: https://orcid.org/0000-0001-7282-7638 and Peng, Yonghong ORCID logoORCID: https://orcid.org/0000-0002-5508-1819 (2021) Srr-lgr: local–global information-reasoned social relation recognition for human-oriented observation. Remote Sensing, 13 (11). 2038.

    [img]
    Preview
    Published Version
    Available under License Creative Commons Attribution.

    Download (9MB) | Preview

    Abstract

    People’s interactions with each other form the social relations in society. Understanding human social relations in the public space is of great importance for supporting the public admin-istrations. Recognizing social relations through visual data captured by remote sensing cameras is one of the most efficient ways to observe human interactions in a public space. Generally speaking, persons in the same scene tend to know each other, and the relations between person pairs are strongly correlated. The scene information in which people interact is also one of the important cues for social relation recognition (SRR). The existing works have not explored the correlations between the scene information and people’s interactions. The scene information has only been extracted on a simple level and high level semantic features to support social relation understanding are lacking. To address this issue, we propose a social relation structure-aware local–global model for SRR to exploit the high-level semantic global information of the scene where the social relation structure is explored. In our proposed model, the graph neural networks (GNNs) are employed to reason through the interactions (local information) between social relations and the global contextual information contained in the constructed scene-relation graph. Experiments demonstrate that our proposed local–global information-reasoned social relation recognition model (SRR-LGR) can reason through the local–global information. Further, the results of the final model show that our method outperforms the state-of-the-art methods. In addition, we have further discussed whether the global information contributes equally to different social relations in the same scene, by exploiting an attention mechanism in our proposed model. Further applications of SRR for human-observation are also exploited.

    Impact and Reach

    Statistics

    Activity Overview
    6 month trend
    92Downloads
    6 month trend
    28Hits

    Additional statistics for this dataset are available via IRStats2.

    Altmetric

    Repository staff only

    Edit record Edit record