Atom Scott 🇬🇧
Atom Scott 🇬🇧

Optimizing Through Learned Errors for Accurate Sports Field Registration

Url
https://arxiv.org/pdf/1909.08034.pdf
Category
Camera Calibration / Multi-view Camera

What

  • Field localization from a single broadcast image
  • Early work using Deep Learning to directly estimate a Homography Matrix
image

How?

Homography Optimization w/ Template

Given an homography estimate a registration error network is used to estimate the loss of the estimate . Since the registration error network is differential, it can be used to optimize .

image

Initial Registration Network

A ResNet-18 Architecture with the classification head swapped out for a fc-layer with an output of 8 (elements of the homography matrix).

Registration Error Network

ResNet-18 + Spectral Normalization +6-channel input (concat of input image& warped target template).

Trained from scratch.

Data Augmentation

  • Random crop (light, 90% ~ 100% of original remains)
  • Random horizontal flips
  • Shadow simulation (random 50% opacity black patches)
  • Random Translation (~1/2 full width), rotation (0~45 deg) and scaling(0.5~2x)
  • Gaussian blur with kernel size 9

And?

Interesting idea but is probably very compute intensive since a single iteration requires DNN inference. However, the optimization idea can be modularized and used as a post processing step along with ideas.