lab 5

• difficulties?
• questions?
• Good Looking Results?

today's schedule

• Recap
• Blur
• Deblurring Blur
• Focal Sweep
• Coded Appertures
• Focus Deblurring
• Depth estimation
• Lab 5

recap

several llumination coding strategies

motion tracking

• motion capture, unique temporal code for LEDs
• e.g. phase space:
• 960 fps
• < 10 ms latency
• visible or IR LEDs

pulse systems: LIDAR

continues wave

comercial devices

light stage 3

photometric stereo

spatial coding

assisted stereo

blur

source of blur

let's take a picture

source of blur

in this case camera shake

image Formation

is a convolution

image Formation

Deblurring is a deconvolution

deconvolution: 2 types

old Problem

non-blind deconvolution

defocus blur

non-blind deconvolution

ill posed problem

blind deconvolution

ill posed problem

today we focus on defocus Blur

How can be restricted to be solved?

first let's go back to aperture

apertures revisited

aperture and PSF

aperture and PSF

aperture and PSF

aperture and PSF

aperture and PSF

aperture and PSF

• Depth-dependent PSF


• Circular apperture is not (well) invertible

aperture and PSF

• Depth-dependent PSF
• Engineer PSF to be depth invariant -> deconvolution is easier


• Circular apperture is not (well) invertible
• Engineer PSF to be broadband (flat Fourier magnitudes) -> well posed (Coded Appertures)
• Coded apperture for depth recovery
• Coded apperture based on perception

depth invariant PSF

change Focal Length with Time

focal sweep

focal sweep

focal sweep

Weiner deconvolution + denoising [Dabov et al.]

lattice focal lens

lattice focal lens

diffusion

coded apertures

avoid cero crossing

better invertibility

may include transparency

[Masia 2012]

optimize for perception

what do we do with depth?

depth and deblurring

aperture and PSF

can we extract depth from defocus and deblur?

ill posed

challenges

key contributions

[Levi et al. 2007]

defocus as local convolution

defocus as local convolution

overview

how to restrict the deconvolution problem?

defocus as local convolution

natural images

prior

what about depth?

second idea

PFS changes with Scale

why coded?

how?

1D intuition

aperture design

aperture design

all together

depth regularization

depth regularization

results

refocus

refocus

refocus

refocus

refocus

Fourier Transform

Fourier Transform

Low-pass Filter

Low-pass Filter

Low-pass Filter

Low-pass Filter

Low-pass Filter

High-pass Filter

Unsharp Masking

Band Pass Filter

Oriented Band-pass Filter

dual modulation

Image Downsampling and Upsampling

Image Downsampling and Upsampling

Image Downsampling and Upsamplingn

Image Downsampling and Upsampling

Image Downsampling and Upsampling

Aliasing

Aliasing

Lens as Optical Low-pass Filter

Lens as Optical Low-pass Filter

Lens as Optical Low-pass Filter

Lens as Optical Low-pass Filter

Deconvolution

Inverse Filtering

Inverse Filtering with Noise

Inverse Filtering with Noise

Deconvolution with Wiener Filtering

Deconvolution with Wiener Filtering

Deconvolution with Wiener Filtering

Deconvolution with Wiener Filtering

Convolution as Matrix Multiplication

Linear Least Squares

Linear Least Squares

Linear Least Squares

Total Variation

Total Variation

Total Variation

Total Variation

ADMM

ADMM

ADMM

ADMM

ADMM

ADMM

ADMM

dual modulation

ADMM

ADMM

ADMM

ADMM

Summary

• basic signal processing: sampling, aliasing, filtering
• implement matrix-free operations for Ax and A’x if efficient (e.g. multiplications and divisions in frequency space)
• split difficult problems (e.g., inverse problems with non- differentiable priors) into easier subproblems - ADMM

Due: 14 December

• Filtering in Frequency domain
• Inverse Filtering
• Weiner Deconvolution
• ADMM

Lecture 7

franchomelendez@cs.uni.wroc.pl

credits and references and aditional readings

These slides have been prepared with materials, slides, and discussions from the following.

• Gordon Wetzstein
• Anat Levin
• Rob Fergus
• Svetlana Lazebnik