today's schedule

• Camera, optics, sensors
• Camera Image processing pipeline
• Lab 1

the digital camera

camera, optics, sensors

let's design a camera

pinhole camera

dimensionality reduction

camera obscura

tracing drawings

problems with pinhole camera

lenses

thin lens formula $\frac{1}{f} = \frac{1}{S_1}+\frac{1}{S_2}$

aperture

aperture

aperture

focus

focus

depth of field

depth of field

bokeh

field of view - zoom

field of view vs viewpoint

field of view vs viewpoint

exposure

motion blur

aberrations

distortion

sensors

what's a pixel

what's a pixel

how to capture color

Only static images

how to capture color

Expensive

how to capture color

Noise

how to capture color

Interference: Lippmann

http://www.nobelprize.org/nobel_prizes/themes/physics/biedermann/index.html

most sensors

Spatial multiplexing

photon noise

gain noise

sensor noise

photons to RAW

image pipeline

RAW to JPEG

RAW to JPEG

Pipeline

bayer mosaic

Picture of sensor

RAW file

demosaicing

demosaicing

green channel

edge directed

edge directed

edge directed

naive

rest of channels?

bad color fringes

green based interpolation

• Interpolate green using e.g. edge-based
• For recorded red pixels compute R-G
• At empty pixels Interpolate R-G naively
• Add G
• Same for blue

all channels

denosing

gaussian filter

Output is blurred

bilateral filter

comparison

bilateral filter

much more about noise... maybe another day.

white balance

Von Kries adaptation

Multiply each channel by a gain factor

• R’=R*kr
• G’=G*kg
• B’=B*kb

how to find the coeficients

• white/grey target
• manual selection of white
• grey world assumption
• brightest pixel assumption

gamma correction

instead of light intensity $x$ Store $x^\gamma$

from 10-12 linear bits to 8

perceptually linear spacing: more precission black

standard 8 bit color space of most images sRGB roughly equivalent to $\gamma$=1/2.2

compresion JPEG

Joint Photographic Experts Group

compresion JPEG

• transform to YCbCr
• downsample chroma components Cb & Cr
• 4:4:4 – no downsampling
• 4:2:2 – reduction by factor 2 horizontally
• 4:2:0 – reduction by factor 2 both horizontally and vertically
• split into blocks of 8x8 pixels
• discrete cosine transform (DCT) of each block & component
• quantize coefficients
• entropy coding (run length encoding – lossless compression)

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

recap RAW to JPEG

deblurring

deblurring cross-channel prior

Instead of independent steps, all as one optimization problem

credits and references and aditional readings

These slides have been prepared with materials, slides, and discussions from the following.

• Stanford Digital Photography Course [Levoy]
• Reference(s) demosaicing [Li et al.]
• Reference(s) demosaicing [Gunturk et al.]
• Reference denoising [Baudes et al.]
• Bilateral Filter [Tomasi and Manduchi]
• Heide et al., “High-Quality Computational Imaging Through Simple Lenses”, ACM Trans. Graph. 2013
• Heide et al., Pulli, “FlexISP: A Flexible Camera Image Processing Framework”, ACM SIGGRAPH Asia 2014
• Capturing Linear Images

Due: 26 October

• dcraw
• Demosaicing
• Gamma encoding
• white balance
• Color Transformation and Median Filter

franchomelendez@cs.uni.wroc.pl