Face reconstruction on mobile devices using a height map shape model and fast regularization - Maninchedda, F; Häne, C; Oswald, MR; Pollefeys, Marc (2016 Fourth International Conference on 3D Vision (3DV), 2016)
We present a system which is able to reconstruct human faces on mobile devices with only on-device processing using the sensors which are typically built into a current commodity smart phone. Such technology can for example be used for facial authentication purposes or as a fast preview for further post-processing. Our method uses recently proposed techniques which compute depth maps by passive multi-view stereo directly on the device. We propose an efﬁcient method which recovers the geometry of the face from the typically noisy point cloud. First, we show that we can safely restrict the reconstruction to a 2.5D height map representation. Therefore we then propose a novel low dimensional height map shape model for faces which can be ﬁtted to the input data efﬁciently even on a mobile phone. In order to be able to represent instance speciﬁc shape details, such as moles, we augment the reconstruction from the shape model with a distance map which can be regularized efﬁciently. We thoroughly evaluate our approach on synthetic and real data, thereby we use both high resolution depth data acquired using high quality multi-view stereo and depth data directly computed on mobile phones.