Prior-less 3D human shape reconstruction with an earth mover’s distance informed CNN

Zhang, J, Shum, HPH, McCay, K and Ho, ESL (2019) Prior-less 3D human shape reconstruction with an earth mover’s distance informed CNN. In: MIG '19: Motion, Interaction and Games, 28 October 2019 - 30 October 2019, Newcastle upon Tyne, United Kingdom.

Preview

Accepted Version Available under License In Copyright. Download (1MB) | Preview

Abstract

We propose a novel end-to-end deep learning framework, capable of 3D human shape reconstruction from a 2D image without the need of a 3D prior parametric model. We employ a “prior-less” representation of the human shape using unordered point clouds. Due to the lack of prior information, comparing the generated and ground truth point clouds to evaluate the reconstruction error is challenging. We solve this problem by proposing an Earth Mover’s Distance (EMD) function to find the optimal mapping between point clouds. Our experimental results show that we are able to obtain a visually accurate estimation of the 3D human shape from a single 2D image, with some inaccuracy for heavily occluded parts.