Light Fields

Light Fields: The intensity of light traveling in any direction from any position

The Plenoptic Function (全光函数)

Grayscale snapshot: $P(\theta,\phi)$ is intensity of light

• Seen from a single view point
• At a single time
• Averaged over the wavelengths of the visible spectrum

Color snapshot: $P(\theta,\phi,\lambda)$ is intensity of light

• Seen from a single view point
• At a single time
• As a function of wavelength

A movie: $P(\theta,\phi,\lambda,t)$​ is intensity of light

• Seen from a single view point
• Over time
• As a function of wavelength

Holographic movie: $P(\theta,\phi,\lambda,t,V_X,V_Y,V_Z)$​ is intensity of light

• Seen from ANY viewpoint
• Over time
• As a function of wavelength

The Plenoptic Function: $P(\theta,\phi,\lambda,t,V_X,V_Y,V_Z)$

• Can reconstruct every possible view, at every moment, from every position, at every wavelength
• Contains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! Not bad for a function...

Ray

ray: $P(\theta,\phi,V_X,V_Y,V_Z)$ is 5D

• 3D position
• 2D direction

Infinite line: Assume light is constant (vacuum), 4D:

• 2D direction
• 2D position
• non-dispersive medium

Light Field Camera

Integral Imaging ("Fly's Eye" Lenslets): Spatially-multiplexed light field capture using lenslets

• Impose fixed trade-off between spatial and angular resolution

The Lytro Light Fields Camera: Microlens design. Most significant function is Computational Refocusing (Virtually changing focal length & aperture size, etc. after taking the photo)

Understanding:

How to get a "regular" photo from thelight field photo?

• A simple case — always choose the pixel at the bottom of each block
• Then the central ones & the top ones
• Essentially "moving the camera around"

Computational / digital refocusing

• Same idea: visually changing focal length, picking the refocused ray directions accordingly

In all, all these functionalities are available because: The light field contains everything

Any problems to light field cameras?

• Insufficient spatial resolution(same film used for both spatial and directional information)
• High cost (intricate design of microlenses)
• Computer Graphics is about trade-offs

Color

Light

The Visible Spectrum of Light: Electromagnetic radiation

• Oscillations of different frequencies (wavelengths)

  

Spectral Power Distribution (SPD, 谱功率密度)

Salient property in measuring light

• The amount of light present at each wavelength.

• Units:

  • radiometric units / nanometer (e.g. watts / nm)
  • Can also be unit-less

• Often use "relative units" scaled to maximum wavelength for comparison across wavelengths when absolute units are not important

Linearity of Spectral Power Distribution:

Color

Color is a phenomenon of human perception; it is not a universal property of light. Different wavelengths of light are not "colors"

Retinal Photoreceptor Cells (感光细胞): Rods and Cones

• Rods are primary receptors in very low light ("scotopic" conditions), e.g. dim moonlight

  • ~120 million rods in eye
  • Perceive only shades of gray, no color

• Cones are primary receptors in typical light levels ("photopic")

  • ~6-7 million cones in eye

  • Three types of cones, each with different spectral sensitivity:

    S, M, and L (corresponding to peak response at short, medium, and long wavelengths)

    

    $S=\int r_S(\lambda)s(\lambda)d\lambda$

    $M=\int r_M(\lambda)s(\lambda)d\lambda$

    $L=\int r_L(\lambda)s(\lambda)d\lambda$

  • Provide sensation of color

Metamerism (同色异谱): Metamers are two different spectra ($\infty$-dim) that project to the same (S,M,L) (3-dim) response.

• These will appear to have the same color to a human

The existence of metamers is critical to color reproduction

• Don't have to reproduce the full spectrum of a real world scene
• Example: A metamer can reproduce the perceived color of a real-world scene on a display with pixels of only three colors

Additive Color:

• Given a set of primary lights, each with its own spectral distribution (e.g. R,G,B display pixels): $s_R(\lambda),s_G(\lambda),s_B(\lambda)$
• Adjust the brightness of these lights and add them together: $R_{s_R}(\lambda)+G_{s_G}(\lambda)+B_{s_B}(\lambda)$
• The color is now described by the scalar values: $R,G,B$

Color Spaces

A Universal Color Space: CIE XYZ

• Imaginary set of standard color primaries X,Y,Z
• Primary colors with these matching functions do not exist
• Y is luminance (brightness regardless of color)

Designed such that

• Matching functions are strictly positive
• Span all observable colors

Separating Luminance, Chromaticity:

• Luminance: Y

• Chromaticity: x,y,z, defined as

  since $x+y+z=1$, we only need to record two of the three

  usually choose x and y, leading to (x,y) coords at a specific brightness Y.

The curved boundary

• named spectral locus
• corresponds to monochromatic light (each point representing a pure color of a single wavelength)

Any color inside is less pure.

Gamut

• Gamut is the set of chromaticities generated by a set of color primaries
• Different color spaces represent different ranges of colors
• So they have different gamuts, i.e. they cover different regions on the chromaticity diagram

Standardized RGB (sRGB):

• makes a particular monitor RGB standard
• other color devices simulate that monitor by calibration
• widely adopted today
• gamut (色域) is limited

HSV Color Space (Hue-Saturation-Value)

Axes correspond to artistic characteristics of color. Widely used in a "color picker"

Hue (色调):

• the "kind" of color, regardless of attributes
• colorimetric correlate: dominant wavelength
• artist's correlate: the chosen pigment color

Saturation (饱和度):

• the "colorfulness"
• colorimetric correlate: purity
• artist's correlate: fraction of paint from the colored tube

Lightness (or value) (亮度):

• the overall amount of light
• colorimetric correlate: luminance
• artist's correlate: tints are lighter, shades are darker

CIELAB Space (L*a*b*): A commonly used color space that strives for perceptual uniformity

• L* is lightness (brightness)

• a*and b* are color-opponent pairs

  • a* is red-green
  • b* is blue-yellow

Opponent Color (互补色) Theory: There's a good neurological basis for the color space dimensions in CIELAB

• the brain seems to encode color early on using three axes: white — black, red —green, yellow — blue
• the white — black axis is lightness; the others determine hue and saturation

CMYK: A Subtractive Color Space

Subtractive color model: The more you mix, the darker it will be

Cyan, Magenta, Yellow, and Key widely used in printing