Vision for robotics

Computer vision is the science of extracting information from camera images, from points in the image to three-dimensional reconstruction of the observed scene. The wealth of information provided by a camera makes it a sensor of choice for autonomous robotics applications.

However, unlike other sensors whose measurement gives a direct distance, for example, the image has to be processed. This processing is designed to produce geometric measurements useful for the application. These geometric measurements in the image, leading to measurements in space, are at the heart of computer vision. The methods used are based on a common foundation, but differ according to the type of camera used (conventional or non-conventional).

The aim of this article is to give an overview of single-, two- and three-dimensional cameras, and to present how images from these cameras are geometrically modeled and processed for robotics applications. First, we review the classical elements of cameras, to better explain the unconventional nature of other types of cameras with great potential for robotics. Next, the geometry of some of these cameras is explained and formalized, before tackling the notion of visual primitive via the extraction and mapping of points of interest. Cameras, their geometry and visual primitives are then applied to robotics, with particular emphasis on unconventional cameras and mobile robotics.

Although this article mentions a few classic machine vision tools, sometimes associated with robotics, it focuses on the cutting edge of vision technologies for the booming robotics sector. In fact, three-dimensional or panoramic vision and mobile robotics represent the two main areas of innovation in this sector. Mobile robotics is clearly at the heart of technological innovation, with companion robots, personal assistance robots and robotic transport systems.

Industry is, of course, the traditional sector in which vision and robotics come together, because the environment can be controlled. But robots have also demonstrated their potential in land, air, sea and underwater applications, and are now being exploited in the vast fields of security, first aid, personal assistance and exploration.

Although this article focuses on innovative vision techniques for robotics, it is designed to help readers understand them, starting with the classic camera. Through examples, the reader is led to a full understanding of several techniques that can clearly be diverted to other fields of application.

The sheer number of vision methods and applications for industrial and mobile robotics precludes an exhaustive presentation of the field, both technically and methodologically. The few references...