Autonomous robot interaction and use of objects.

Abstract

This thesis is focused on the skills that must be performed by an autonomous robot to interact with objects in a complex environment. For a robot to manipulate and interact effectively with an object it must first learn the appearance of the object, determine the shape of the object, be able to recognise and localise the object in the environment and determine the physical properties of the object. We consider each of these skills in turn.

A new method of object recognition using local image feature matching has been developed that is more accurate than existing methods, as well as being more efficient in some circumstances. Next we developed a system that combines robot initiated object motion and long term image feature tracking to accurately extract object features from complex scene images. This allows a robot to learn to recognise previously unseen objects in the presence of clutter, noise and background motion. The object feature matching and segmentation methods are then combined with 3D reconstruction methods to determine the object’s shape. This is done by stitching together multiple views of the object.

Physical properties (weight, friction, centre of mass, etc) are important factors in determining how a robot can use an object. However, unlike shape and appearance, these may be impossible to determine by passive observation. We have developed a method in which the robot performs experiments on the object, and uses the outcomes to update its knowledge of the object’s physical properties. To perform the most informative experiments, a physics simulator is used to internally rehearse each experiment and its potential outcomes. The outcome of each experiment, after being performed on the physical object, is input back into the simulator forming a hypothesis-experiment-refinement loop. In this way the robot effectively learns the internal properties of an object. Finally, the robot uses this knowledge to plan and carry out a simple task in which the object is used as a tool.