Category

Camera Calibration / Multi-view Camera

# What

- Field localization from a single broadcast image
- Early work using Deep Learning to directly estimate a Homography Matrix

# How?

### Homography Optimization w/ Template

Given an homography estimate $h^i$ a registration error network is used to estimate the loss of the estimate $h^i$. Since the registration error network is differential, it can be used to optimize $h^i$.

### Initial Registration Network

A ResNet-18 Architecture with the classification head swapped out for a fc-layer with an output of 8 (elements of the homography matrix).

### Registration Error Network

ResNet-18 + Spectral Normalization +6-channel input (concat of input image& warped target template).

Trained from scratch.

### Data Augmentation

- Random crop (light, 90% ~ 100% of original remains)
- Random horizontal flips
- Shadow simulation (random 50% opacity black patches)
- Random Translation (~1/2 full width), rotation (0~45 deg) and scaling(0.5~2x)
- Gaussian blur with kernel size 9

# And?

Interesting idea but is probably very compute intensive since a single iteration requires DNN inference. However, the optimization idea can be modularized and used as a post processing step along with ideas.