A Survey on Rain Removal from Video and Single Image
宗苗宣
Hong Wang, Yichen Wu, Minghan Li, Qian Zhao, and Deyu Meng
Citation
@article{WangA,
title={A Survey on Rain Removal from Video and Single Image},
author={Wang, Hong and Wu, Yichen and Li, Minghan and Zhao, Qian and Meng, Deyu},
journal={arXiv preprint arXiv:1909.08326},
year={2019}
}
Physical Properties of Raindrops
- Gemometric Property
- Brightness Property
- Chromatic Property
- Spatial and Temporal Propety
Video Deraining Methods
-
Time Domain
- Detection and removal of rain from videos (CVPR2004), Garg et al [Project][PDF]
- When does camera see rain? (ICCV2005), Garg et al [Project][PDF]
- Rain removal using kalman filter in video (ICSMA2008), Park et al [PDF]
- Using the shape characteristics of rain to identify and remove rain from video (S+SSPR2008), Brewer et al [PDF]
- The application of histogram on rain detection in video (JCIS2008), Zhao et al [PDF]
- Rain or snow detection in image sequences through use of a histogram of orientation of streaks (IJCV2011), Bossu et al [PDF]
- A probabilistic approach for detection and removal of rain from videos (IETE JR2011), Tripathi et al [PDF]
- Video post processing: low latency spatiotemporal approach for detection and removal of rain (IET IP2012), Tripathi et al [PDF]
- Removal of rain from videos: a review (SIVP2014), Tripathi et al [PDF]
- Stereo video deraining and desnowing based on spatiotemporal frame warping (ICIP2014), Kim et al [PDF]
-
Frequency Domain
-
Low Rank and Sparsity
- A generalized low-rank appearance model for spatio-temporally correlated rain streaks (ICCV2013), Chen et al [PDF]
- A rain pixel recovery algorithm for videos with highly dynamic scenes (TIP2013), Chen et al [PDF]
- Video deraining and desnowing using temporal correlation and low-rank matrix completion (TIP2015), Kim et al [PDF] [Code]
- Adherent raindrop modeling, detection and removal in video (TPAMI2016), You et al. [Project] [PDF]
- Video desnowing and deraining based on matrix decomposition (CVPR2017), Ren et al [PDF] [Code]
- A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors (CVPR2017), Jiang et al [PDF]
- Should We encode rain streaks in video as deterministic or stochastic? (ICCV2017), Wei et al [PDF] [Code]
- A directional global sparse model for single image rain removal (AMM2018), Deng et al [PDF] [Code]
- Video rain streak removal by multiscale convolutional sparse coding (CVPR2018), Li et al [Project] [PDF] [Code]
- Fastderain: A novel video rain streak removal method using directional gradient priors (TIP2019), Jiang et al [PDF] [Code]
-
Deep Learning
- Robust video content alignment and compensation for rain removal in a cnn framework (CVPR2018), Chen et al [PDF] [Code]
- Erase or fill? deep joint recurrent rain removal and reconstruction in videos (CVPR2018), Liu et al. [Project][PDF] [Code]
- D3R-Net: dynamic routing residue recurrent network for video rain removal (TIP2018), Liu et al. [PDF]
Single Image Deraining Methods
-
Filter based methods
- Guided image filtering (ECCV2010), He et al. [Project] [PDF] [Code]
- Removing rain and snow in a single image using guided filter (CSAE2012), Xu et al. [PDF]
- An improved guidance image based method to remove rain and snow in a single image (CIS2012), Xu et al. [PDF]
- Single-image deraining using an adaptive nonlocal means filter (ICIP2013), Kim et al. [PDF]
- Single-image-based rain and snow removal using multi-guided filter (NIPS2013), Zheng et al. [PDF]
- Single image rain and snow removal via guided L0 smoothing filter (Multimedia Tools and Application2016), Ding et al. [PDF]
-
Prior based methods
- Automatic single-image-based rain streaks removal via image decomposition (TIP2012), Kang et al [PDF] [Code]
- Self-learning-based rain streak removal for image/video (ISCS2012), Kang et al. [PDF]
- Single-frame-based rain removal via image decomposition (ICA2013), Fu et al. [PDF]
- Exploiting image structural similarity for single image rain removal (ICIP2014), Sun et al. [PDF]
- Visual depth guided color image rain streaks removal using sparse coding (TCSVT2014), Chen et al [PDF]
- Removing rain from a single image via discriminative sparse coding (ICCV2015), Luo et al [PDF] [Code] pwd: d229
- Rain streak removal using layer priors (CVPR2016), Li et al [PDF] [Code]
- Single image rain streak decomposition using layer priors (TIP2017), Li et al [PDF]
- Error-optimized dparse representation for single image rain removal (IEEE TIE2017), Chen et al [PDF]
- A hierarchical approach for rain or snow removing in a single color image (TIP2017), Wang et al. [PDF]
- Joint bi-layer optimization for single-image rain streak removal (ICCV2017), Zhu et al. [PDF]
- Convolutional sparse and low-rank codingbased rain streak removal (WCACV2017), Zhang et al [PDF]
- Joint convolutional analysis and synthesis sparse representation for single image layer separation (CVPR2017), Gu et al [PDF] [Code]
- Single image deraining via decorrelating the rain streaks and background scene in gradient domain (PR2018)， Du et al [PDF]
-
Deep Learning
- Restoring an image taken through a window covered with dirt or rain (ICCV2013), Eigen et al. [Project] [PDF] [Code]
- Attentive generative adversarial network for raindrop removal from a single image (CVPR2018), Qian et al [Project] [PDF]
- Clearing the skies: A deep network architecture for single-image rain streaks removal (TIP2017), Fu et al. [Project] [PDF] [Code]
- Removing rain from single images via a deep detail network (CVPR2017), Fu et al. [Project] [PDF] [Code]
- Image de-raining using a conditional generative adversarial network (Arxiv2017), Zhang et al [PDF] [Code]
- Deep joint rain detection and removal from a single image (CVPR2017), Yang et al.[Project] [PDF] [Code]
- Residual guide feature fusion network for single image deraining (ACMMM2018), Fan et al. [Project] [PDF]
- Fast single image rain removal via a deep decomposition-composition network (Arxiv2018), Li et al [Project]) [PDF] [Code]
- Density-aware single image de-raining using a multi-stream dense network (CVPR2018), Zhang et al [PDF] [Code]
- Recurrent squeeze-and-excitation context aggregation net for single image deraining (ECCV2018), Li et al. [PDF] [Code]
- Rain streak removal for single image via kernel guided cnn (Arxiv2018), Wang et al [PDF]
- Physics-based generative adversarial models for image restoration and beyond (Arxiv2018), Pan et al [PDF]
- Learning dual convolutional neural networks for low-level vision (CVPR2018), Pan et al [Project] [PDF] [Code]
- Non-locally enhanced encoder-decoder network for single image de-raining (ACMMM2018), Li et al [PDF] [Code]
- Unsupervised single image deraining with self-supervised constraints (ICIP2019), Jin et al [PDF]
- Progressive image deraining networks: A better and simpler baseline (CVPR2019), Ren et al [PDF] [Code]
- Spatial attentive single-image deraining with a high quality real rain dataset (CVPR2019), Wang et al [Project] [PDF] [Code]
- Lightweight pyramid networks for image deraining (TNNLS2019), Fu et al [PDF] [Code]
- Joint rain detection and removal from a single image with contextualized deep networks (TPAMI2019), Yang et al [PDF] [Code]
-
Joint Model-driven and Data-driven
Datasets and the Usage
-
Video
- Synthetic Datasets: highway and park.
- Real Datasets: compfinal and night. Please download from [Baidu Netdisk] provided by Li Minghan.
-
Single Image
- Synthetic Datasets: RainTrainL/Rain100L, RainTrainH/Rain100H, Rain12600/Rain1400, and Rain12. Please download from [Baidu Netdisk] provided by Ren Dongwei.
- Real Datasets: Please download SPA-Data from [Baidu Netdisk, key: 4fwo] provided by Wang Tianyu and Internet-Data from the link provided by Weiwei .
*We note that:
i. RainTrainL/Rain100L and RainTrainH/Rain100H are synthesized by Yang Wenhan. Rain12600/Rain1400 is from Fu Xueyang and Rain12 is from Li Yu.
ii. In video experiment, the rain-removed results of the deep learning method are provided by the author Yang Wenhan. Really thanks!
iii. In single image experiment, we seperately retrain all the recent state-of-the-art methods via the three training datasets: RainTrainL(200 input/clean image pairs), RainTrainH(1800 pairs), and Rain12600(12600 pairs), and then evaluate their rain removal performance based on the correponding test datasets: Rain100L(100 pairs), Rain100H(100 pairs), and Rain1400(1400 pairs). Besides, the trained model obtained by RainTrainL is adpoted to predict rain-removed results of Rain12(12 pairs). Moreover, we utilize the Internet-Data(147 input images) and SPA-Data(1000 pairs) to compare the generalization ability.
iiii. In single image experiment, when training the semi-supervised method--SIRR, we always utilize Internet-Data as unsupervised samples.
Image Quality Metrics
- PSNR (Peak Signal-to-Noise Ratio) [PDF] [Matlab Code] [Python Code]
- SSIM (Structural Similarity) [PDF] [Matlab Code] [Python Code]
- VIF (Visual Quality) [PDF] [Matlab Code]
- FSIM (Feature Similarity) [PDF] [Matlab Code])
*Please note that all quantitative results are computed based on Y channel.
Contact
If you have any question, please feel free to concat Hong Wang (Email: hongwang01@stu.xjtu.edu.cn).
类似资料：