Yoav Y. Schechner: Research

Multiplexed illumination

Imaging of objects under variable lighting directions is a frequent procedure in computer vision and computer graphics. We developed an approach that significantly improves the quality of images taken under such varying conditions. Traditionally, only a single source is used per acquired image. In contrast, our approach is based on multiplexing: multiple sources illuminate the object simultaneously from different directions. Thus, the object irradiance is higher. The acquired images are then computationally demultiplexed.

This approach does not increase the number of image acquisitions. It is useful for imaging dim object areas and highly specular objects. The demultiplexed images have enhanced signal-to-noise ratio, relative to traditionally acquired images. This advantage may be translated into faster acquisition or increased density of illumination direction samples. It also enables the use of systems having high directional resolution, which is easy to achieve using a digital projector, as we demonstrated experimentally. Moreover, multiplexed illumination enables imaging of diffuse reflectance while avoiding saturation of specular highlights. It is also more robust to stray light.

For systems having signal-independent noise, multiplexing is optimized by using Hadamard codes. However, we analyzed limitations of the approach and identified scenarios in which it should be avoided. In our most recent publications we developed multiplexing codes that are optimal under signal dependent noise such as photon noise (shot noise), as well as saturation. This analysis also yielded theoretical performance bounds of general codes. These theoretical advancements apply beyond illumination, to any technological field employing multiplexing, such as spectrometry.

Publications

1. Yoav Y. Schechner, Shree K. Nayar and Peter N. Belhumeur, “A theory of multiplexed illumination,” Proc. IEEE ICCV, Vol. 2, pp. 808-815 (2003)
2. Yoav Y. Schechner, Shree K. Nayar, Peter N. Belhumeur and H. S. Peri, “Imaging in multiplexed illumination,” SPIE: Nonimaging Optics and Efficient Illumination Systems, pp. 198-205 (2004)
3. Yoav Y. Schechner, Shree K. Nayar, and Peter N. Belhumeur, “Multiplexing for optimal lighting,” IEEE Trans. Pattern Analysis & Machine Intelligence, Vol. 29, No. 8 , pp. 1339-1354 (2007).
4. Nenanel Ratner and Yoav Y. Schechner, “Illumination multiplexing within fundamental limits,” Proc. IEEE CVPR (2007)
5. Nenanel Ratner, Yoav Y. Schechner and F. Goldberg, “Optimal multiplexed sensing: bounds, conditions and a graph theory link,” Optics Express, Vol. 15 , No. 25, pp. 17072-17092 (2007).

Presentations

1. “Multiplexed Illumination” (6.2 Mb, PowerPoint)
2. The presentation above references the following movies:
   1. Illumination Patterns (1.4 Mb, AVI)
   2. The Setup (8.7 Mb, AVI)
   3. Raw Images (36.1 Mb, AVI)
   4. Single-Source Images (36.1 Mb, AVI)
   5. High Definition Specularities (12.5 Mb, AVI)