Automatic Generation of Interactive 3D Characters and Scenes for Virtual Reality from a Single-Viewpoint 360-Degree Video

Grégoire Dupont de Dinechin (1) , Alexis Paljic (1)

1 CAOR - Centre de Robotique

This work addresses the problem of using real-world data captured from a single viewpoint by a low-cost 360-degree camera to create an immersive and interactive virtual reality scene. We combine different existing state-of-the-art data enhancement methods based on pre-trained deep learning models to quickly and automatically obtain 3D scenes with animated character models from a 360-degree video. We provide details on our implementation and insight on how to adapt existing methods to 360-degree inputs. We also present the results of a user study assessing the extent to which virtual agents generated by this process are perceived as present and engaging.

Domaines: Interface homme-machine [cs.HC]; Apprentissage [cs.LG]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Grégoire Dupont de Dinechin, Alexis Paljic. Automatic Generation of Interactive 3D Characters and Scenes for Virtual Reality from a Single-Viewpoint 360-Degree Video. 26th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2019), Mar 2019, Osaka, Japan. ⟨hal-02114141v1⟩