School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China
binfan@mail.nwpu.edu.cn, daiyuchao@nwpu.edu.cn
The vast majority of modern consumer-grade cameras are equipped with an electronic rolling shutter (RS), which is becoming increasingly important in computer vision, robotics, and autonomous driving applications. Although ubiquitous, its progressive exposure mechanism causes geometric distortions in the acquired images whenever either the camera or the target is moving. In the last two decades, researchers have made significant progress in developing tractable RS models, optimization methods, and learning approaches, aiming to remove the geometry distortion and improve the visual quality. In particular, since the tutorial entitled ''Multi-view geometry for rolling shutter cameras'' in ACCV 2016, many robust and efficient RS motion models and non-linear optimization methods have emerged. At the same time, the rise of deep learning techniques has also pushed the research on RS-related problems in the past five years. In this half-day tutorial, we will give a broad overview of the latest advances in this area from three aspects, i.e., motion modeling, optimization-based solutions, and deep learning-based solutions. Specifically, we will first systematically present geometric motion models (like discrete, continuous, and special motions), recently proposed in the computer vision community, starting from their theoretical descriptions up to their typical applications. Then, we will give an introduction to deep learning-based RS image processing methods, such as RS image correction and RS temporal super-resolution, with new results and benchmarks that have recently appeared.
(I) Introduction and Overview of the Tutorial (15 minutes)
(II) Geometric Modeling and Non-Linear Optimization of Rolling Shutter Camera (40 minutes)
(III) Geometric Problems with Rolling Shutter Model (40 minutes)
(IV) New Paradigm with Deep Learning (60 minutes)
(V) Challenges and Future Trends (15 minutes)
(VI) Open Questions and Discussions (15 minutes)
More details can be found in our recent rolling shutter review paper published in Machine Intelligence Research.
@article{fan2022overview,
title={Rolling shutter camera: modeling, optimization and learning},
author={Fan, Bin and Dai, Yuchao and He, Mingyi},
journal={Machine Intelligence Research},
volume={},
number={},
pages={},
year={2022},
publisher={Springer}
}