Yoav Y. Schechner: Research

Virtual Periscope

Distortions that are random in space, time and viewpoint are created when viewing objects through a wavy water-air interface. Such distortions are also created in turbulence. In both cases, the distortion is caused by dynamic refraction. We study how this situation can be handled. This is important to some predators. It is also beneficial to submariners, to avoid the salient use of a physical periscope. We have just devised a passive system (STELLA MARIS) to counter distortions caused by water waves, in a single-viewpoint and instant. This single-frame approach is deterministic: it can be stand-alone, or be an enabler for stochastic methods.

A stocahstic method we have developed localizes objects in three dimensions (3D) despite this random distortion. We take a multiview approach to this task. Refracted distortion statistics induce a probabilistic relation between any pixel location and a line of sight in space. Measurements of an object's random projection from multiple views and times lead to a likelihood function of the object's 3D location. The likelihood leads to estimates of the 3D location and its uncertainty. Furthermore, multiview images acquired simultaneously in a wide stereo baseline have uncorrelated distortions. This helps reduce the acquisition time needed for localization.

Moerover, in many cases, the moving objects are of interest, as they reveal animal, human, or machine activity. We show that moving objects can be detected very simply, with low false-positive rates, even when the distortions are very strong and dominate the object motion. While the object and distortion motions are random and unknown, they are mutually independent. This is expressed by a simple motion feature which enables discrimination of moving object points versus the background.

Publications

M. Alterman, Y. Y. Schechner and Y. Swirski, "Triangulation in random refractive distortions," IEEE Trans. Pattern Analysis & Machine Intelligence 39:3, pp. 603-616 (2017).
M. Alterman, Y. Swirski and Y. Y. Schechner, "STELLA MARIS: Stellar Marine Refractive Imaging Sensor, Proc. IEEE ICCP - International Conference on Computational Photography (2014)
Y. Y. Schechner, " A view through the waves, Marine Technology Society Journal 47, 148-150 (2013)
M. Alterman, Y. Y. Schechner and Y. Swirski, "Triangulation in random refractive distortions," Proc. IEEE ICCP - International Conference on Computational Photography (2013).
M. Alterman, Y. Y. Schechner, P. Perona and J. Shamir, "Detecting motion through dynamic refraction," IEEE Transactions on Pattern Analysis and Machine Intelligence 35, No. 1, pp. 245-251 (2013).