18 June 2021 MetaFlow: a meta-learning-based network for optical flow estimation
Zhiyi Gao, Yonghong Hou, Yan Liu, Xiangyu Li
Author Affiliations +
Abstract

Convolutional neural networks (CNNs) have achieved success in optical flow estimation using labeled datasets, but they fail to build an internal representation to fast adapt to the specific task. On the other hand, limited by the lack of ground truth, existing CNNs-based methods suffer from high noise sensitivity and inferior generalization performance. We integrate the meta-learning technique with optical flow estimation, which can learn internal features to search optimal initial state parameters of the network. Meanwhile, we devise an enhanced network termed MetaFlow to further improve performance. MetaFlow extracts per-pixel features, builds correlation volumes for all pairs of pixels, and iteratively updates optical flow through optical flow predictor using meta-learning. In addition, we propose a meta-transfer pretraining approach to obtain initial network weights, which can efficiently avoid network overfitting. Empirical experiments on MPI Sintel and KITTI benchmarks have shown that the proposed MetaFlow achieves the state-of-the-art results and performs outstanding in challenging scenarios such as textureless regions and abrupt motions.

© 2021 SPIE and IS&T 1017-9909/2021/$28.00© 2021 SPIE and IS&T
Zhiyi Gao, Yonghong Hou, Yan Liu, and Xiangyu Li "MetaFlow: a meta-learning-based network for optical flow estimation," Journal of Electronic Imaging 30(3), 033029 (18 June 2021). https://doi.org/10.1117/1.JEI.30.3.033029
Received: 17 February 2021; Accepted: 3 June 2021; Published: 18 June 2021
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Optical flow

Optical networks

Computer programming

Network architectures

Visualization

Feature extraction

Convolutional neural networks

Back to Top