Assignment 2 – Planar Augmented Reality
0. Introduction
In the previous assignment you were given a set of pre-computed 2D-3D correspondences
from which you had to compute the camera matrix. Once you had computed the camera
matrix you saw how it was possible to use it to project the original 3D points into the image
space of the camera.
In this assignment you will apply similar principles to compute the projection from a planar
target to a real-world image in real-time as was demonstrated in lectures. Specifically you
will use your web-cam to capture images of a checkerboard which you will track using
OpenCV and then overlay another image on top of the image which is aligned with the
checkerboard. In effect you will use plane-to-plane calibration to perform. planar based
augmented reality (AR).
For this assignment you should create single python file called camera.py containing all of
the functions you develop. Please ensure that your code is well commented.
If you have any questions or issues during the assignment, please feel free to post them to
the course forum on the moodle page.
1. Planar homographies
A. In this section you first will learn to use the openCV routines for calibrating a real camera
(e.g. the web cam on your machine). To do this you should follow the online tutorial
below:
http://opencv-python-
tutroals.readthedocs.org/en/latest/py_tutorials/py_calib3d/py_calibration/py_calibration.ht
ml
Having completed the tutorial you should be able to (i) compute the distortion (and
intrinsic) parameters for your camera, and, (ii) undistort images from your camera such
that the lines of the input checkerboard are straightened. Note that the result of this step
will be a set of calibration parameters for the camera that you can us in the next step.
That is the output of this step should be calibration parameters as opposed to code.
B. Using what you have learned from part A. take an image of the checkerboard and
undistort it. Now using the facilities provided by the OpenCV calib3d library generate a
set of correspondences between the checkerboard and the undistorted image. Using
these correspondences compute the 3x3 homography (see notes on planar camera
model) that maps points on the checkerboard to points on in the image. To start this part
of the assignment you should download the associated code from the moodle webpage.
C. Finally take a look at the WarpPerspective function in the image processing section of
opencv: ective
Using this function and an image of your choice, project the image such that it appears to
lie in the plane of the checkerboard.