Volumetric Video Compression Through Neural-based Representation - Calcul Intensif, Simulation, Optimisation
Conference Papers Year : 2024

Volumetric Video Compression Through Neural-based Representation

Abstract

Volumetric video offers immersive exploration and interaction in 3D space, revolutionizing visual storytelling. Recently, Neural Radiance Fields (NeRF) have emerged as a powerful neural-based technique for generating high-fidelity images from 3D scenes. Building upon NeRF advancements, recent works have explored NeRF-based compression for static 3D scenes, in particular point cloud geometry. In this paper, we propose an end-to-end pipeline for volumetric video compression using neural-based representation. We represent 3D dynamic content as a sequence of NeRFs, converting the explicit representation to neural representation. Building on the insight of significant similarity between successive NeRFs, we propose to benefit from this temporal coherence: we encode the differences between consecutive NeRFs, achieving substantial bitrate reduction without noticeable quality loss. Experimental results demonstrate the superiority of our method for dynamic point cloud compression over geometry-based PCC codecs and comparable performance with state-of-the-art PCC codecs for high-bitrate volumetric videos. Moreover, our proposed compression based on NeRF generalizes to arbitrary dynamic 3D content.
Fichier principal
Vignette du fichier
Neural_based_Volumetric_Video_Compression.pdf (1.3 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04550588 , version 1 (17-04-2024)

Identifiers

Cite

Yuang Shi, Ruoyu Zhao, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi. Volumetric Video Compression Through Neural-based Representation. 16th International Workshop on Immersive Mixed and Virtual Environment Systems @ ACM Multimedia Systems Conference (MMSys 2024), ACM, Apr 2024, Bari, Italy. pp.85-91, ⟨10.1145/3652212.3652220⟩. ⟨hal-04550588⟩
431 View
76 Download

Altmetric

Share

More