Augmented Reality using Fiducial Markers

ArUco Fiducial Markers

Published in

Towards Data Science

3 min readMay 17, 2020

According to Wikipedia a fiducial marker “is an object placed in the field of view of an imaging system which appears in the image produced, for use as a point of reference or a measure. It may be either something placed into or on the imaging subject, or a mark or set of marks in the reticle of an optical instrument.” [1] Fiducial markers have been used in lots of different settings as measures. For example, if you were taking a picture of someone standing next to a ruler to see how tall they were, the ruler would be acting as a fiducial marker.

In 2014 a research paper was produced which proposed a fiducial marker system specially designed for camera pose estimation [2]. Camera pose estimation is a process of finding correspondences between points in the real environment and their 2D image projection. Using fiducial markers makes this process a lot easier. The markers that they proposed are known as ArUco markers. Two of the authors of that paper, Rafael Muñoz and Sergio Garrido, produced a module to work with them and this in turn was incorporated into OpenCV as the aruco module [3].

Detecting an ArUco marker with OpenCV and Python

In a previous article I mentioned Pyto [4] so I thought I would write a script that would find an ArUco marker and then draw a box around it, from a live camera feed on my iPhone. I used the following ArUco marker image:

I printed this image off on a piece of paper and placed it on my desk:

You’ll spot I decreased the transparency to 50% to save my black ink cartridge! It still works fine.

Here’s my script that detects the marker and then draws the box round it, together with the ID:

Here’s the script running with the ArUco Marker on my table:

Augmented Reality

Once I was able to detect the marker I thought it would be good to place an image on top of it that would maintain the right perspective as you moved the camera. In order to achieve this, we need to know the homography of the marker. We can then transform the image into that same homography and place it correctly on top of the marker.

You don’t need to understand the actual details of homography, homogeneous co-ordinates or matrices in order to be able to achieve this. But if you’re interested, check out the Wiki article references at the end [5][6].

To get the homography we call the OpenCV method findHomography. Once we have the homography we can then call the method warpPerspective which will do an affine transformation on the image to put it in the right perspective and place it correctly on top of the marker.

Here’s my script that does this:

And this is what it looks like as it’s running in Pyto: