All Predictions Matter: an Online Video Prediction Approach - Département Image, Données, Signal Access content directly
Conference Papers Year : 2023

All Predictions Matter: an Online Video Prediction Approach


To effectively manage and utilize the massive amount of visual data generated by the surging number of videos, decision-making systems must predict and reason about future outcomes. This paper proposes a novel online approach for video prediction that enables continual learning in the presence of new data, as periodic training of neural networks may not be practical. We utilize all predictions, including intermediate computations obtained during the inference process, to improve the performance of video prediction. To achieve this, we incorporate a weighting scheme in the loss that accounts for all the predictions during the learning process. Additionally, we leverage semantic segmentation to assess the performance of extrapolated frames by focusing on the position of the objects in the scene. Our approach stands out from state-of-the-art methods as it uses intermediate predictions, which are available due to the iterative nature of forecasting future frames. Our method improves the offline counterpart for the same network by 1.45 dB for predicting five steps in the future.
Fichier principal
Vignette du fichier
EUVIP_2023_Melan_final_camera_ready.pdf (431.03 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04204130 , version 1 (11-09-2023)


  • HAL Id : hal-04204130 , version 1


Melan Vijayaratnam, Marco Cagnazzo, Giuseppe Valenzise, Enzo Tartaglione. All Predictions Matter: an Online Video Prediction Approach. 11th European Workshop on Visual Information Processing, Sep 2023, Gjøvik, Norway. ⟨hal-04204130⟩
110 View
43 Download


Gmail Mastodon Facebook X LinkedIn More