Frame synthesis for video compression

Nicola Giuliani; Biao Wang; Elena Alshina; Laura Leal-Taixé

doi:10.1117/12.2596884

1 August 2021 Frame synthesis for video compression

Nicola Giuliani, Biao Wang, Elena Alshina, Laura Leal-Taixé

Proceedings Volume 11842, Applications of Digital Image Processing XLIV; 118420R (2021) https://doi.org/10.1117/12.2596884
Event: SPIE Optical Engineering + Applications, 2021, San Diego, California, United States

Abstract

Traditional video compression techniques heavily rely on the concept of motion compensation to predict frames in a video given previous or future frames. With the recent advances in artificial intelligence powered techniques, new approaches to video coding become apparent. We describe a video compression scheme that employs neural network based image compression and frame generation. The proposed method encodes key frames (I frames) as still images and entirely skips compression and signalling of intermediate frames (S frames), which are conversely synthesized on the decoder side exclusively using I frames. Varying complexities of motion can occur within a sequence, which can have a strong impact on the quality of the generated frames. In order to address this challenge, we propose to let the encoder dynamically adjust the group of pictures (GOP) structure. This adjustment is performed based on the quality of predicted S frames. The achieved performance of the proposed method suggests that entirely skipping coding and instead synthesizing frames is promising and should be considered for future developments of learning based video codecs.

Conference Presentation

Citation Download Citation

Nicola Giuliani, Biao Wang, Elena Alshina, and Laura Leal-Taixé "Frame synthesis for video compression", Proc. SPIE 11842, Applications of Digital Image Processing XLIV, 118420R (1 August 2021); https://doi.org/10.1117/12.2596884

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available