Visualizing Image Priors
Our algorithm determines how images should be deformed so as to better comply with a given image model (exemplified here on a Brain Coral image). The deformed images give insight into the elementary geometric features to which the prior resonates. As can be seen, different image models (BM3D, EPLL, FoE, KSVD, Multi-Layer Perceptron, Nonlocal means, Total Variation, Shrinkage Fields with pairwise cliques) have quite different geometric preferences.
Image priors play a key role in low-level vision tasks. Over the years, many priors have been proposed, based on a wide variety of principles. While different priors capture different geometric properties, there is currently no unified approach to interpreting and comparing priors of different nature. This limits our ability to analyze failures or successes of image models in specific settings, and to identify potential improvements. In this paper, we introduce a simple technique for visualizing image priors. Our method determines how images should be deformed so as to best conform to a given image model. The deformed images constructed this way, highlight the elementary geometric structures to which the prior resonates. We use our approach to study various popular image models, and reveal interesting behaviors, which were not noticed in the past. We confirm our findings through denoising experiments. These validate that the structures we reveal as ‘optimal’ for a specific prior are indeed better denoised by this prior.
We used our algorithm to visualize several different natural image models by generating their GEMs. The GEM image stress out which spatial image features are more plausible under a given prior model. As can be seen, different models have quite different geometric preferences.
[BM3D] K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, "Image denoising with block-matching and 3D filtering," Electronic Imaging, 2006.
[EPLL] D. Zoran, Y. Weiss, "From learning models of natural image patches to whole image restoration," ICCV 2011.
[FoE] S. Roth, M. J. Black, "Fields of experts: A framework for learning image priors," CVPR 2005.
[KSVD] M. Elad, M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Transactions on Image Processing, 2006.
[Multi-Layer Perceptron] H. C. Burger, C. J. Schuler, S. Harmeling, "Image denoising: Can plain neural networks compete with BM3D?," CVPR 2012.
[Nonlocal Means] A. Buades, B. Coll, J. M. Morel, "A Non-local algorithm for image denoising," CVPR 2005.
[Shrinkage Fields] U. Schmidt, S. Roth, "Shrinkage fields for effective image restoration," CVPR 2014.
[Total Variarion] L. I. Rudin, S. Osher, E. Fatemi, "Nonlinear total variation based noise removal algorithms," Physica D: Nonlinear Phenomena, 1992.
[Cross-Scale Patch Recurrence] T. Michaeli, M. Irani, "Blind deblurring using internal patch recurrence," ECCV 2014. (We use the same priror for denoising, see this explanation).