Computational Photography

Digital Photography

by dr. Francho Melendez

today's schedule

  • Camera, optics, sensors
  • Image processing pipeline
  • Lab 1

the digital camera

camera, optics, sensors

let's design a camera

pinhole camera

dimensionality reduction

camera obscura

tracing drawings

problems with pinhole camera

lenses

lensmaker's equation $\frac{1}{f} = \frac{1}{S_1}+\frac{1}{S_2}$

aperture

aperture


F-number N defined by N = f/D

aperture

focus

focus

depth of field

depth of field

bokeh

field of view - zoom

field of view vs viewpoint

field of view vs viewpoint

Large FOV, close
Small FOV, far

exposure

motion blur

aberrations

distortion

sensors

what's a pixel

what's a pixel

how to capture color

Only static images

how to capture color

Expensive

how to capture color

Noise

how to capture color

Interference: Lippmann

http://www.nobelprize.org/nobel_prizes/themes/physics/biedermann/index.html

most sensors

Spatial multiplexing

photon noise

gain noise

sensor noise

photons to RAW

recap

  • pinhole camera
  • lenses
  • aperture
  • focus
  • depth of field (DOF)
  • field of view (FOV)
  • exposure and motion blur
  • how to capture color
  • bayer patern
  • so... how photos get to RAW

references and further reading

  • London, Upton, Stone, “Photography”, Pearson, 11th edition, 2013
  • Stanford CS 178, “Digital Photography”, Course Notes
  • wikipedia

image pipeline

demosaic(k)ing, denoising, gamut mapping, compression

A.K.A. from RAW to JPEG

RAW to JPEG

RAW to JPEG

Pipeline

bayer mosaic

Picture of sensor

RAW file

demosaicing

demosaicing






green channel


edge directed


edge directed


edge directed


naive

rest of channels?


bad color fringes


green based interpolation

  • Interpolate green using e.g. edge-based
  • For recorded red pixels compute R-G
  • At empty pixels Interpolate R-G naively
  • Add G
  • Same for blue

all channels


denosing

gaussian filter

Output is blurred

bilateral filter

comparison

bilateral filter

much more about noise... maybe another day.

white balance

Von Kries adaptation

Multiply each channel by a gain factor

  • R’=R*kr
  • G’=G*kg
  • B’=B*kb

how to find the coeficients

  • white/grey target
  • manual selection of white
  • grey world assumption
  • brightest pixel assumption

gamma correction

instead of light intensity $x$ Store $x^\gamma$

from 10-12 linear bits to 8

perceptually linear spacing: more precission black

standard 8 bit color space of most images sRGB roughly equivalent to $\gamma$=2.2

compresion JPEG

Joint Photographic Experts Group

compresion JPEG

  • transform to YCbCr
  • downsample chroma components Cb & Cr
    • 4:4:4 – no downsampling
    • 4:2:2 – reduction by factor 2 horizontally
    • 4:2:0 – reduction by factor 2 both horizontally and vertically
  • split into blocks of 8x8 pixels
  • discrete cosine transform (DCT) of each block & component
  • quantize coefficients
  • entropy coding (run length encoding – lossless compression)

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

compresion JPEG

recap RAW to JPEG

deblurring

deblurring cross-channel prior

video

Instead of independent steps, all as one optimization problem

video

credits and references and aditional readings

These slides have been prepared with materials, slides, and discussions from the following.

today's lab

Due: 20 October

  • Demosaicing
  • Gamma encoding
  • Color Transformation and Median Filter
  • dcraw

announcements

http://franchomelendez.com/index.php/computational-photography/

http://franchomelendez.com/Uwr/teaching/COMPHO/Labs/Lab1.zip


franchomelendez@cs.uni.wroc.pl