Light Fields

From Light Fields

(Redirected from Main Page)
Jump to: navigation, search

CVPR 2009 Course: Light Fields: Present and Future

The ray–based 4D lightfield representation, based on simple 3D geometric principles, has led to a range of new algorithms and applications in Computer Vision and Graphics. They include digital refocusing, depth estimation, synthetic aperture, and glare reduction within a camera or using an array of cameras.

The lightfield representation is, however, inadequate to describe interactions with diffractive or phase–sensitive optical elements. Fourier optics principles are used to represent wavefronts with additional phase information. This course reviews the current and future directions in exploiting higher dimensional representation of light transport. We hope the course will inspire researchers in computer vision comfortable with ray–based analysis to develop new tools and algorithms based on joint exploration of geometric and wave optics concepts.


We encourage graphics, vision, optics community to edit and contribute to these pages. Please add short description of and references to your own and others' work.

Contents

Course Schedule

  1. Introduction to Light Fields: Ramesh Raskar
  2. Wigner Distribution Function to explain Light Fields: Zhengyun Zhang
  3. Augmenting Light Fields to explain Wigner Distribution Function: Se Baek Oh
  4. Q&A
  5. Break
  6. Light Fields with Coherent Light: Anthony Accardi
  7. New Opportunities: Raskar and Oh
  8. Q&A: All


Introduction to Light Fields: Ramesh Raskar

Using Wigner Distributions to Explain Light Fields: Zhengyun Zhang

Augmenting Light Fields to explain Wigner Distribution Function: Se Baek Oh

Quasi Light Fields for Coherent Radiation: Anthony Accardi

New Opportunities: Ramesh Raskar and Se Baek Oh

References

For descriptions below, please include the following (i) problem statement and solution (ii) input and output (iii) limitations and scope. Pl be very thorough about the limitations and possibly later papers that address them. Please include a URL.

Light Field Capture

  • P. P. Sokolov Autostereoscopy by Professor Lippmann's Method.. Журнал Общества любителей естествознания, 1911. One of the earliest papers describing a light field capture. Under the guise of autostereoscopy, mathematically derives ideal optics and describes the implementation of a pinhole light field capture and display system. Translated/edited by Daniel Reetz (NDSU) and Ekaterina Avramova (IATE).
Describes a way to capture multiple viewpoints of a scene (a.k.a. light field) through a single main lens by focusing the main lens on a microlens array and capturing its back focal plane. For a fixed sensor, there's an inherent tradeoff between angular ("parallax") and spatial resolution.
This report showed a modern designs for a camera to record a 4D light field on a 2D sensor in a single photographic exposure. To sample the rays, they place a microlens array very close to the sensor. For simplicity one can imagine, each microlens is a pin-hole lens. Each microlens measures not just the total amount of light deposited at that location, but how much light arrives along each ray. By re-sorting the measured rays of light to where they would have terminated in slightly different, synthetic cameras, they can compute sharp photographs focused at different depths. The key results in such lightfield capturing setup is that a linear increase in the number of pixels under each microlens results in a linear increase in the depth over which one can refocus photographs. Extended depth of field is possible without reducing the aperture (i.e. same amount of light). The lightfield can also be used for small parallax change in macrophotography regime. The benefit is that, to the photographer, the plenoptic camera operates exactly like an ordinary hand-held camera. The limitations include (i) The total 2D resolution of the refocussed photo is decreased by the same factor (ii) Microlens must create a focused image of the main lens on the sensor. But the single layer microlens introduces geometric and chromatic aberrations plus distorted pixels in the peripheral area of each microlens. Plus the focus setting (distance between lens and sensor) is fixed for a given microlens position.
This paper explores the design of a real-time system consisting of a large array of (inexpensive) cameras and possible applications. The main applications it presents are high dynamic range, high resolution and high speed video capture if the cameras are packed closely together, or view interpolation, motion/scene analysi and synthetic aperture photography if the cameras are spread apart. Synthetic aperture photography allows the user to see through objects smaller than the virtual aperture of the imaging system, equal to the spread of the cameras in the array. Some drawbacks of the system include poor noise characteristics due to the inexpensive cameras. Furthermore, this system is not very portable and has large space requirements.
This paper describes a system to capture the light field by time-multiplexing the aperture of the static camera. In each exposure, the angular distribution of the light field is multiplexed by the 2D pattern on the aperture, which can be generated by a liquid crystal array. The light field can be demultiplexed from the images captured with different multiplexing patterns. In this way, the SNR loss due to exposure time reduction is compensated. The main advantages of this method over other light field cameras are (i) the image resolution remains the same to the sensor resolution (ii) the angular resolution is adjustable, and thus fully backward compatible, and (iii) the manufacture cost is relatively lower. The limitation is that the image is captured sequentially. Therefore, while the moving object would appear blurry in the single-shot cameras, the demultiplexed light field would have unnatural artifacts which need to be further processed. Finally, the paper describes an algorithm to remove the inherent vignetting distortion of the light field, and an multi-view depth estimation algorithm for anti-aliasing rendering. Both algorithms are applicable to the data captured by other light field cameras.

Light Field Theory and Analysis

  • J.-X. Chai, S.-C. Chan, H.-Y. Shum, and X. Tong. Plenoptic sampling. In SIGGRAPH, pages 307–318, 2000.
  • A. Gershun, The light field (translated by P. Moon and G. Timoshenko), J. Math. and Physics, 18:51-151, 1939.
  • A. Isaksen, L. McMillan, and S. Gortler. Dynamically reparameterized light fields. In SIGGRAPH, pages 297–306, 2000.
  • D. Lanman, R. Raskar, A. Agrawal, and G. Taubin. Shield Fields: Modeling and Capturing 3D Occluders. In SIGGRAPH ASIA ’08, (Singapore), 2008.

Light Field Synthesis

Light Field Applications

Displays

  • Chun, W. and Cossairt, O. S. (2009). Data processing for three-dimensional displays, United States Patent 7,525,541.
  • Matusik, W., Pfister, H. (2004). "3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes", Proc. ACM SIGGRAPH, ACM Press.

Glare

  • Raskar, R., Agrawal, A., Wilson, C., Veeraraghavan, A. (2008). "Glare Aware Photography: 4D Ray Sampling for Reducing Glare Effects of Camera Lenses", Proc. ACM SIGGRAPH.

Multimodal Capture

  • R. Horstmeyer, G.W. Euliss, R.A. Athale, and M. Levoy. Flexible Multimodal Camera Using a Light Field Architecture. Proc. IEEE ICCP, April 2009

This paper presents a modified conventional camera that is able to collect multimodal images in a single exposure. Utilizing a light field architecture in conjunction with multiple filters placed in the pupil plane of a main lens, synthetic images containing specific spectral, polarimetric, and other optically filtered data are digitally reconstructed. The ease with which these filters can be exchanged and reconfigured provides a high degree of flexibility in the type of information that can be collected with each image. This paper explores the various tradeoffs involved in implementing a pinhole array in parallel with a pupil-plane filter array to measure multi-dimensional optical data from a scene. It also examines the design space of a pupil-plane filter array layout. Images are shown from different multimodal filter layouts, and techniques to maximize resolution and minimize error in the synthetic images are proposed.

General (Wave) Optics

  • V. Arriz´on and J. Ojeda-casta˜neda. Irradiance at Fresnel Planes of a Phase Grating. J. Opt. Soc. Am. A, 9(10):1801–1806, 1992.

The WDF of a phase grating was analyzed.

  • M. Born and E. Wolf, Principles of Optics, 7th. ed., Cambridge University Press, Cambridge, UK, 2005.

One of the most important reference in optics.

  • J. T. Foley and E. Wolf, Radiometry as a short-wavelength limit of statistical wave theory with globally incoherent sources. Opt. Commun., 55(4):236-241, Sept. 1985.
  • A. T. Friberg, On the existence of a radiance function for finite planar sources of arbitrary states of coherence, J. Opt. Soc. Am., 69(1):192-198, Jan. 1979.
  • J. W. Goodman, Statistical Optics, John Wiley and Sons, Inc. 2000

Introductory reference on statistical optics.

  • J. W. Goodman, Introduction to Fourier Optics, Robert & Co., Englewood, Color. 3rd Ed. 2005

Introductory reference on Fourier optics.

  • K. Halbach, Matrix representation of Gaussian optics, American Journal of Physics, 32(2):90-108, February 1964.
  • K. Kim and E. Wolf, Propagation law for Walther's first generalized radiance function and its short-wavelength limit with quasi-homogeneous sources, J. Opt. Soc. Am. A, 4(7):1233-1236, July 1987.
  • E. W. Marchand and E. Wolf, Walther's definitions of generalized radiance, J. Opt. Soc. Am., 64(9):1273-1274, Sept. 1974.
  • A. Walther, Radiometry and coherence, J. Opt. Soc. Am., 58(9):1256-1259, Sept 1968.

The generalized radiance was proposed, whose definition is similar to the WDF.

  • E. Wolf, Coherence and radiometry, J. Opt. Soc. Am., 68(1):6-17, Jan. 1978.

The connection between optics and radiometry was reviewed. Especially, the reason why the generalized radiance (or WDF) cannot be interpreted as radiance was described.

Wigner Distribution and Related Constructs

  • M. A. Alonso. Wigner functions for nonparaxial, arbitrarily polarized electromagnetic wave fields in free space. J. Opt. Soc. Am. A, 21(11):2233–2243, 2004.

Wigner DF can also be used for polarization but it is defined for each polarization: s, p as well as cross-talk between s-p creating a 2x2 tensor for representation.

  • H. O. Bartelt, K.-H. Brenner, and A. W. Lohmann, The Wigner distribution function and its optical production, Opt. Commun., 32(1):32-38, Jan, 1980.
  • M. J. Bastiaans. Wigner Distribution Function and Its Application to 1st-order Optics. J. Opt. Soc. Am., 69(12):1710–1716, 1979.
  • M. J. Bastiaans. Application of the Wigner Distribution Function to Partially Coherent-Light. J. Opt. Soc. Am. A, 3(8):1227–1238, 1986.
  • M. J. Bastiaans. Application of the Wigner distribution function in optics. In W. M. F. Hlawatsch, editor, The Wigner Distribution - Theory and Applications in Signal Processing, pages 375–426. Elsevier Science, Amsterdam, 1997.
  • M. J. Bastiaans and P. G. J. van de Mortel. Wigner distribution function of a circular aperture. J. Op. Soc. Am. A, 13(8):1698–1703, 1996.
  • K.-H. Brenner, A. W. Lohmann, and J. Ojeda-Casta~neda, The ambiguity function as a polar display of the OTF, Opt. Commun., 44(5):323-326, Feb. 1983.
  • K.-H. Brenner and J. Ojeda-casta˜neda. Ambiguity Function and Wigner Distribution Function Applied to Partially Coherent Imagery. Optica Acta, 31(2):213–223, 1984.
  • S. Cho, J. C. Petruccelli, and M. A. Alonso. Diffraction effects in wigner functions for paraxial and nonparaxial fields. In Frontiers in Optics (FiO), volume OSA Technical Digest (CD), page FMK8. Optical Society of America, 2008.
  • R. Casta˜neda. Phase space representation of spatially partially coherent imaging. App. Opt., 47(22):E53–E62, August 2008.
  • G. Cristobal, C. Gonzalo, and J. Bescos, Image filtering and analysis through the Wigner distribution function, in Advances in Electronics and Electron Physics, P. W. Hawkes, Ed. Boston, MA: Academic, 1991, vol. 80, pp. 309–397
  • W. D. Furlan, M. Mart'inez-Corral, B. Javidi, and G. Saavedra, Analysis of 3-D integral imaging displays using the Wigner distribution, J. Display Technol., 2(2):180-185, June, 2006.
  • Y. Li, G. Eichmann, and M. Conner, Optical Wigner distribution and ambiguity function for complex signals and images, Opt. Commun., 67(3):177-179, July 1988.
  • Adolf W. Lohmann, "Image rotation, Wigner rotation, and the fractional Fourier transform," J. Opt. Soc. Am. A 10, 2181-2186 (1993)
  • J. Ojeda-Castaneda, L. R. Berriel-Valdos, and Emma Montes, Ambiguity function as a design tool for high focal depth, Appl. Opt. 27, 790-795 (1988)
  • J. Ojeda-Castaneda, M. E. Testorf, and J-P. Guigay, Phase-space representations in optics: introduction to the feature issue, Appl. Opt. 47(22): PSO1, 2008.
  • A. Papoulis, Ambiguity function in Fourier optics, J. Opt. Soc. Am., 64(6):779-788, June 1974.
  • M. E. Testorf, J. Ojeda-Castaneda, A. W. Lohmann, Selected papers on phase-space optics, SPIE milestone series on optical science and engineering, editors: Vol. MS 181, 2006.
  • E. Wigner, On the quantum correction for thermodynamic equilibrium, Physical Review, 40(5):749-759, 1932.
  • P. M. Woodward, Probability and information theory with applications to radar, Pergamon, London, U.K., 1953.

Wavefront Manipulation and Capture

  • S. Bagheri, P. E. X. Silveira, R. Narayanswamy, and D. P. de Farias. Analytical optial solution of the extension of the depth of field using cubic–phase wavefront coding. part ii. J. Opt. Soc. Am. A, 25(5):104–1074, 2008.
  • N. Caron and Y. Sheng. Polynomial phase masks for extending the depth of field of a microscope. Appl. Opt., 47(22):E39–E43, 2008.
  • A. Castro and J. Ojeda-Casta˜neda. Asymmetric phase masks for extended depth of field. Appl. Opt., 43(17):3474– 3479, 2004.
  • W. T. Cathey and E. R. Dowski. New paradigm for imaging systems. Appl. Opt., 41(29):6080–6092, 2002.
  • E. R. Dowski, Jr. and W. T. Cathey, Extended depth of field through wave-front coding. Appl. Opt., 34(11):1859-1866, Apr. 1995.
  • S. S. Sherif, W. T. Cathey, and E. R. Dowski, Phase plate to extended the depth of field of incoherent hybrid imaging systems, Appl. Opt. 43(13): 2709-2771, 2004.

Miscellaneous

  • A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, Interactive digital photomontage, In Proc. ACM SIGGRAPH, 2004.
  • N. A. Ahuja and N. K. Bose, Spatiotemporal-bandwidth product of m-dimensional signals, In IEEE Signal Processing Letters, pages 123-125, 2005.
  • R. N. Bracewell, The Fourier transform and its applications, 2nd. ed., McGraw-Hill, Reading, NY, 1986.
  • L. Cohen, Time-Frequency Analysis, Prentice Hall, Upper Saddle River, NJ, 1995.
  • P. Hanrahan, Radiosity and Realistic Image Synthesis, chapter Rendering Concepts, pages 13-40. Academic Press Professional, 1993.
Retrieved from "http://scripts.mit.edu/~raskar/lightfields/index.php?title=Light_Fields"
Personal tools