Computational Photography

Computational Imaging

by dr. Francho Melendez

lab 1

  • difficulties?
  • questions?
  • RGB to YCbCr?
  • Edges?
  • Mosaic Patterns?

RGB to YCbCr

today's schedule

  • Recap
  • Burst Photography
  • Techniques overview
  • HDR, Tone Mapping (TMO)
  • Lab 2

recap

pinhole camera

dimensionality reduction

aperture

depth of field

exposure

field of view - zoom

how to capture color

photons to RAW

demosaicing


white balance

gamma correction


compresion JPEG

RAW to JPEG

video

Instead of independent steps, all as one optimization problem

video

burst photography

burst photography

idea: take 2 or more images (bursts of images) and combine them

  • super-resolution
  • focal stack
  • aperture stack - confocal stereo
  • blurry/noisy
  • flash/no flash
  • HDR

super resolution

idea: capture multiple low-res (LR) images and fuse them into a single super-resolved (SR) image

super resolution

LR must be sub-pixel shifted

super resolution

example for 1 linescan

solve it as a minimization problem: normally well-conditioned for factors 2-3x

super resolution

focal stack

use highest gradient

focal stack

Focal Stack Compositing for Depth of Field Control

aperture stack

what changes? exposure and depth of field –> extract depth!

aperture stack

Confocal Stereo [Hasinoff and Kutulakos 2007]

aperture stack

idea: intensity of in-focus point remains constant for varying aperture

aperture stack

Confocal Stereo [Hasinoff and Kutulakos 2007]

high + low resolution pairs

[Ben-Ezra and Nayar, 2003]

secondary, fast, noisy, low-res camera for motion PSF estimation

high + low resolution pairs

captured data

high + low resolution pairs

blur estimation

high + low resolution pairs

deconvolution vs tripod

high + low resolution pairs

captured, estimated, ground truth

burry & noisy pairs

same idea

super short, high ISO noisy exposure for motion PSF estimation

longer exposure with camera shake -> deblur

burry & noisy pairs

synthetic experiment

very good kernell estimation

burry & noisy pairs

another examples

look at the highight -> blur kernell

burry & noisy pairs

de-rigging

flash / no-flash photography

[Pettschnigg et al., 2004]

flash / no-flash photography

flash / no-flash photography

flash / no-flash photography

flash / no-flash photography

[Eisemann, Durand 2004]

flash / no-flash photography

flash / no-flash photography

non-photorealistic camera

[Raskar et al. 2004]

four pictures, four lighting directions, different shadows

non-photorealistic camera

Edge detection -> non-photorealistic rendering

Similar setup for photometric stereo

high dynamic range imaging

high dynamic range imaging

dynamic range: ratio between brightest and darkest value

what's the problem?

high dynamic range imaging

high dynamic range imaging

What do we vary?

  • shutter speed: linear and reliable, can be noise (long exposures)
  • aperture: change DoF, less orders of magnitude
  • ISO: noisy, even less orders of magnitude
  • filters: need to move the camera, good range, color shift

high dynamic range imaging

overview

high dynamic range imaging

estimate curve response

high dynamic range imaging

estimate curve response

no need if we have RAW

use a color chart

high dynamic range imaging

estimate curve response

no need if we have RAW

use a color chart

high dynamic range imaging

don't have a color chart?

[Debevec and Malik 1997]

high dynamic range imaging

capture exposure, apply lookup table

high dynamic range imaging

how to merge the LDRs?

individual exposure is radiance (X) * exposure time (t): $I_{lin_i} = t_i X$

high dynamic range imaging

radiance up to a scale... use a reference

high dynamic range imaging

Image Based Lighting

Light Probes

tone mapping

tone mapping

HDR data -> LDR displays...

tone mapping

sun overexposed

foreground too dark

tone mapping

gamma correction

colors are washed out!

tone mapping

gamma in intensity only

intensity details lost

tone mapping

compute gradients, scale them, integrate (Poisson eq.)

[Fattal et al., 2002]

tone mapping

[Durand and Dorsey, 2002]

tone mapping

results are quite similar

Durand's looks a bit better

tone mapping

a lot of work in Tone Mapping

most looking for "tone reproduction" : perceptually based

inverse tone mapping

can we expand LDR to HDR?

[Masia et al] “Evaluation of reverse tone mapping through varying exposure conditions”

HDR Display

recap

  • super-resolution (shifted stack)
  • increase DoF (focal stack)
  • extract depth (aperture/focus stack)
  • Motion blurred (blurred + Low res)
  • Low light photography (blurred + noisy)
  • Relight - denoise (flash - no flash)
  • NP Camera (4 flash camera)
  • HDR (exposure stack)
  • Tone mapping

today's lab (Lab 2)

Due: 27 October

  • Focal Stack
  • HDR
  • Tone Mapping
  • Whenever possible, send me emails with [COMPHO] Pleeeease

focal stack: use highest gradient


HDR:

announcements

http://franchomelendez.com/Uwr/teaching/COMPHO/_LECTURES/L3/computational_imaging.html

http://franchomelendez.com/Uwr/teaching/COMPHO/Labs/Lab3.zip


franchomelendez@cs.uni.wroc.pl

credits and references and aditional readings

These slides have been prepared with materials, slides, and discussions from the following.

coming soon...