# Optimizing Through Learned Errors for Accurate Sports Field Registration

Category
Camera Calibration / Multi-view Camera

# What

• Field localization from a single broadcast image
• Early work using Deep Learning to directly estimate a Homography Matrix

# How?

### Homography Optimization w/ Template

Given an homography estimate

a registration error network is used to estimate the loss of the estimate
. Since the registration error network is differential, it can be used to optimize
.

### Initial Registration Network

A ResNet-18 Architecture with the classification head swapped out for a fc-layer with an output of 8 (elements of the homography matrix).

### Registration Error Network

ResNet-18 + Spectral Normalization +6-channel input (concat of input image& warped target template).

Trained from scratch.

### Data Augmentation

• Random crop (light, 90% ~ 100% of original remains)
• Random horizontal flips
• Shadow simulation (random 50% opacity black patches)
• Random Translation (~1/2 full width), rotation (0~45 deg) and scaling(0.5~2x)
• Gaussian blur with kernel size 9

# And?

Interesting idea but is probably very compute intensive since a single iteration requires DNN inference. However, the optimization idea can be modularized and used as a post processing step along with ideas.