We have made a computer visual system which is able to detect, recognize objects in a scene and mark them using a laser projector. Use case for this spatial augmented reality system can be various. For example, advertisement but also think of museums, galleries where the artifacts can be just placed in the exhibition space and additional information displayed next to them.

Model

We used YOLOv3 for object detection and augmented reality makers (ArUco) for marking the coordinate system to the scene.

System overview

Examples

This is the example of the scene captured by a camera and run through YoloV3 object detection network. We can see AruCo markers which are used as a relative reference to calculate the position of projected labels.

Also we use an additional deep neural network to detect the written text in the scene.

In the end, we calculate the absolute location of projected objects and text labels and project them via laser projector back to the scene. Pretty high precision, on the spot!

---