# Computational Photography

## Computational Imaging II

by dr. Francho Melendez

## lab 2

• difficulties?
• questions?
• Did you try other averaging methods?
• Bilateral Filter?

## today's schedule

• Recap
• Panoramas,Mosaics
• Some cool papers
• Warping
• Homographies
• Lab 3

# recap

## RAW to JPEG

take 2 or more images and combine them to get a "better one"

## expand resolution

idea: capture multiple low-res (LR) images and fuse them into a single super-resolved (SR) image

## depth from focus

Confocal Stereo [Hasinoff and Kutulakos 2007]

## flash / no-flash photography

[Pettschnigg et al., 2004]

## flash / no-flash photography

[Eisemann, Durand 2004]

## tone mapping

[Durand and Dorsey, 2002]

# Mosaics

## expanding the field of view

• Compact Camera FOV = 50 x 35°
• Human FOV = 200 x 135°
• Panoramic Mosaic = 360 x 180°

## simple overview

• 4 correspondances
• reproject one on the other

### more complete overview

• Automatic correspondances: Feature detector
• Projection in a different space
• Blending

### a pencil of rays contains all views

same center of projection!

## image reprojection

do we need geometry of the two planes in respect to the eye?

## 2Dtransformation

but what kind of transformation?

## 2Dtransformation

but what kind of transformation?

Projective – mapping between any two projection planes with the same center of projection

• rectangle should map to arbitrary quadrilateral
• parallel lines aren’t
• but must preserve straight lines
• same as: project, rotate, reproject

called Homography

## homography

$\begin{bmatrix}wx'\\wy'\\w\end{bmatrix} = \begin{bmatrix}* & * & *\\* & * & *\\* & * & *\end{bmatrix} \begin{bmatrix}x\\y\\1\end{bmatrix}$

$p' = Hp$

# Some cool things with mosaics

## Interactive Photomontage

Let there be color - Large scale Texturing of 3D reconstructions [Michael Waechter et al. 2014]

## Transfusive Image Manipulation

Initialize with sift images, compute Wrap between images

# Warping

## 2D linear transformations

as 2D matrices (translation cannot be expressed as a 2D matrix)

## 2Dimage transformations

These transformations are a nested set of groups Closed under composition and inverse is a member

## warping

Given a coordinate transform (x’,y’) = T(x,y) and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(T(x,y))?

## forward VS backwards

Backward Mapping eliminate holes

Needs a invertible wrap function: Not always possible

## non-parametric warping

• Input correspondences at key feature points
• Define a triangular mesh over the points
• Same mesh in both images!
• Warp each triangle separately from source to destination (affine)

## morphing sequence

• Create an intermediate shape (by interpolation)
• Warp both images towards it
• Cross-dissolve the colors in the newly warped images
• $Image_{halfway} = (1-t)*Image_1 + t*image_2$

# homographies

### a pencil of rays contains all views

same center of projection!

## homography

$\begin{bmatrix}wx'\\wy'\\w\end{bmatrix} = \begin{bmatrix}* & * & *\\* & * & *\\* & * & *\end{bmatrix} \begin{bmatrix}x\\y\\1\end{bmatrix}$

$p' = Hp$

### planar scene (or far away)

PP3 is a projection plane of both centers of projection, so we are OK

## for a general scene?

### getting H

• Find the homography H given a set of p and p’ pairs
• How many correspondences are needed?
• Tricky to write H analytically, but we can solve for it!

### getting H

$\begin{bmatrix}wx'\\wy'\\w\end{bmatrix} = \begin{bmatrix}a & b & c\\d & e & f\\g & h & i\end{bmatrix} \begin{bmatrix}x\\y\\1\end{bmatrix}$

9 unknowns and $w$

$w$ is easy: $w = gy + hx + i$

Set up a system of linear equations:

$Ah = b$

where vector of unknowns $h = [a,b,c,d,e,f,g,h]^T$

Need at least 8 eqs, (setting $i=1$)

Solve for h. If overconstrained, solve using least-squares

$min\|Ah-b\|^2$

# blending

## today's lab

Due: 16th November

• Warping with Homographies
• Compute homographies
• Manual Mosaic
• Blending