Computational Photography

Computational Displays

by dr. Francho Melendez

computational displays

recap

  • $L = L(x,y,z, \theta, \phi, \lambda, t)$
  • plenoptic imaging
  • coded photography
  • light fields
  • compressing imaging

computational displays

  • the plenoptic function
  • perceptual considerations of Human Visual System (HVS)
  • architectures
  • content generation: computation & taking advantage of the HVS

dimensions of the plenoptic function

  • contrast and luminance
  • color gamut
  • spatial resolution
  • temporal resolution
  • angular resolution
  • other designs

contrast and luminance range

perceptual considerations

  • cones -> color (most sensitive in photopic: day)
  • rods -> intensity (most sensitive in scotopic: night)
  • HVS percieve 4 orders of magniture
  • CRT 2 orders of magnitude
  • limited for local contrast boundaries (scattering inside the eye)

perceptual considerations

limited for local contrast boundaries (veiling glare)

dual modulation

local dimming

micro dimming

dual modulation

  • low-res layer very bright
  • high-res layer low dynamic range
  • theoretical contrast is multiplied
  • perceived local contrast is smaller than global

HDR displays

HDR Projector

Light Reallocation

content generation

  • tone-mapping
  • reverse tone-mapping
  • apparent brightness enhancement

bleaching effect

apparent brightness

afterimages

color gamut

color gamut

expand the color gamut

  • more saturated primaries
  • larger number of primaries

faithful color reproduction

  • take into account the viewing conditions
  • take into account the surface

shader lamps

projection mapping

radiometric correction

spatial resolution

spatial resolution (HVS)

  • cones (color) faster response
  • concentrated in the fovea (no rods there)
  • peri-foveal region, mostly rods, much lower-res
  • saccadic and micro-saccadic movements
  • necessary for vision...
  • stimulus in retina fades when constant
  • temporal integration in retina

retina resolution

find your blind spot

increase resolution

  • optical superposition
  • temporal superposition
  • takes advantage of temporal integration in retina

retina resolution

retina resolution

wobbing display

cascaded displays

cascaded displays

optical pixel sharing

optical pixel sharing

temporal resolution

temporal resolution (HVS)

  • Critical flicker frequency
  • point when flickering is interpreted as continuous

  • depends on temporal contrast, luminance adaptation, spatial extend of the stimuli

hold-type blur

temporal upsampling

  • black data insertion: limits hold-type blur
  • on-off backlight: possilble with LED backlights
  • interpolation along motion: needs optical flow estimation
  • warp between frames

perceptually-motivated temporal upsampling

  • Start with rendered content
  • Attach a coarse grid
  • Snap to depth discontinuities
  • Morph grid
  • Extrapolated frame
  • perceptually-motivated temporal upsampling

    extrapolated warped frame

    perceptually-motivated temporal upsampling

    perceptually-motivated temporal upsampling

    perceptually-motivated temporal upsampling

    Low Frequencies interleaved with High Frequencies

    perceptually-motivated temporal upsampling

    integration

    perceptually-motivated temporal upsampling

    integrated by the HVS

    perceptually-motivated temporal upsampling

    integrated by the HVS

    angular resolution

    taxonomy 3D glass-free displays

    paralax barriers

    losses light and resolution, versatile

    paralax barriers

    losses light and resolution, versatile

    integral imaging

    not as versatile

    directional backlighting

    volumetric displays

    no oclussions, no global lighting

    volumetric displays

    no oclussions, no global lighting

    glasses free 3D display

    automutltistereoscopic displays

    layered 3D display

    generalize paralax barriers

    transparencies

    what to put in the layers? computation

    tomographic LF display

    simple display

    objects should be outside the volume of the display

    key is to design the atenuator

    tomographic LF display

    tomographic LF display

    extrapolated frame

    tomographic LF display

    volumetric atenuator

    tomographic LF display

    not possible without computation

    tomographic LF display

    in 4D...

    5 layers

    tomographic LF display

    static, very continuous

    3-5 layers as good as it gets

    high contrast

    tomographic LF display

    polarization fields

    polarization fields

    stacked LCD

    typical LCD

    polarization fields

    same as before, very dim display

    remove redundant polarizers

    remove color panels

    polarization fields

    polarization fields

    not an atenuator device anymore

    polarization fields

    polarization fields

    solver needs to work in realtime

    polarization fields

    dual layered display

    2 layers, very dim

    dual layered display

    takes advantage of temporal integration

    dual layered display

    simulation

    tensor display

    Light Field Glasses

    others

    extended DOF

    multiple projectors

    extended DOF

    coded projection

    extended DOF

    conclusion

    next week

    • Send me the slides before hand (TUESDAY)
    • PDF
    • libre office, ppt, keynote
    • others? please, let me know in advance

    credits and references and aditional readings

    These slides have been prepared with materials, slides, and discussions from the following.