Computational Photography
Computational Imaging
by dr. Francho Melendezlab 1
- difficulties?
- questions?
- RGB to YCbCr?
- Edges?
- Mosaic Patterns?
today's schedule
- Recap
- Burst Photography
- Techniques overview
- HDR, Tone Mapping (TMO)
- Lab 2
recap
pinhole camera
dimensionality reduction
aperture
depth of field
exposure
field of view - zoom
how to capture color
photons to RAW
demosaicing
white balance
gamma correction
compresion JPEG
RAW to JPEG
burst photography
burst photography
idea: take 2 or more images (bursts of images) and combine them
- super-resolution
- focal stack
- aperture stack - confocal stereo
- blurry/noisy
- flash/no flash
- HDR
super resolution
idea: capture multiple low-res (LR) images and fuse them into a single super-resolved (SR) image
super resolution
LR must be sub-pixel shifted
super resolution
example for 1 linescan
solve it as a minimization problem: normally well-conditioned for factors 2-3x
super resolution
focal stack
use highest gradient
focal stack
Focal Stack Compositing for Depth of Field Control
aperture stack
what changes? exposure and depth of field –> extract depth!
aperture stack
Confocal Stereo [Hasinoff and Kutulakos 2007]
aperture stack
idea: intensity of in-focus point remains constant for varying aperture
aperture stack
Confocal Stereo [Hasinoff and Kutulakos 2007]
high + low resolution pairs
secondary, fast, noisy, low-res camera for motion PSF estimation
high + low resolution pairs
captured data
high + low resolution pairs
blur estimation
high + low resolution pairs
deconvolution vs tripod
high + low resolution pairs
captured, estimated, ground truth
burry & noisy pairs
same idea
super short, high ISO noisy exposure for motion PSF estimation
longer exposure with camera shake -> deblur
burry & noisy pairs
synthetic experiment
very good kernell estimation
burry & noisy pairs
another examples
look at the highight -> blur kernell
burry & noisy pairs
de-rigging
flash / no-flash photography
flash / no-flash photography
flash / no-flash photography
flash / no-flash photography
flash / no-flash photography
flash / no-flash photography
flash / no-flash photography
non-photorealistic camera
four pictures, four lighting directions, different shadows
non-photorealistic camera
Edge detection -> non-photorealistic rendering
Similar setup for photometric stereo
high dynamic range imaging
high dynamic range imaging
dynamic range: ratio between brightest and darkest value
what's the problem?
high dynamic range imaging
high dynamic range imaging
What do we vary?
- shutter speed: linear and reliable, can be noise (long exposures)
- aperture: change DoF, less orders of magnitude
- ISO: noisy, even less orders of magnitude
- filters: need to move the camera, good range, color shift
high dynamic range imaging
overview
- estimate camera response curve
- capture multiple low dynamic range (LDR) exposures
- fuse LDR images into 32 bit HDR image
- possibly convert to absolute radiance
- application specific use:
- image-based rendering lighting
- tone mapping
high dynamic range imaging
estimate curve response
high dynamic range imaging
estimate curve response
no need if we have RAW
use a color chart
high dynamic range imaging
estimate curve response
no need if we have RAW
use a color chart
high dynamic range imaging
don't have a color chart?
high dynamic range imaging
capture exposure, apply lookup table
high dynamic range imaging
individual exposure is radiance (X) * exposure time (t): $I_{lin_i} = t_i X$
high dynamic range imaging
radiance up to a scale... use a reference
high dynamic range imaging
Image Based Lighting
Light Probes
tone mapping
tone mapping
HDR data -> LDR displays...
tone mapping
sun overexposed
foreground too dark
tone mapping
gamma correction
colors are washed out!
tone mapping
gamma in intensity only
intensity details lost
tone mapping
compute gradients, scale them, integrate (Poisson eq.)
tone mapping
tone mapping
results are quite similar
Durand's looks a bit better
tone mapping
a lot of work in Tone Mapping
most looking for "tone reproduction" : perceptually based
inverse tone mapping
can we expand LDR to HDR?
[Masia et al] “Evaluation of reverse tone mapping through varying exposure conditions”
HDR Display
recap
- super-resolution (shifted stack)
- increase DoF (focal stack)
- extract depth (aperture/focus stack)
- Motion blurred (blurred + Low res)
- Low light photography (blurred + noisy)
- Relight - denoise (flash - no flash)
- NP Camera (4 flash camera)
- HDR (exposure stack)
- Tone mapping
today's lab (Lab 2)
Due: 9 November
- Focal Stack
- HDR
- Tone Mapping
- Whenever possible, send me emails with [COMPHO] Pleeeease
focal stack: use highest gradient
HDR: