Multi-Object, Multi-Sensor Detection And Tracking Of Pedestrians On A Mobile Robot

Abstract

Autonomous mobile robots will play an important role in human society in the near future. In particular, many autonomous mobile robots will perform long-term tasks in the same environments that humans reside, sometimes even interacting with people face-to-face. Examples of such robots include self-driving cars, domestic service robots, autonomous couriers, and robotic tour guides. Robots that perform these missions must plan based on accurate, robust information about the local dynamic state of the nearby pedestrians; this work explores the means of attaining dynamic scene information on person-scale mobile robots. From a multi-object tracking perspective, varying environment backgrounds, sensor noise, false detections (clutter), occlusion, and imperfect measurement extraction make inferring information about the dynamic scene difficult. Pedestrians also have certain quirks that pose additional challenges as trackable objects - they walk in groups, have difficult-to-predict motion, and vary in appearance (such as differing clothes and height). In this work, a suite of five different tracking "architectures" are developed and compared, leveraging sensor data from a 2D lidar sensor and an array of cameras with non-overlapping fields of view. In each architecture, the multi-sensor data are fused in a recursive Bayesian multi-object tracker, with certain differences in modeling representation and measurement pre-processing. These tracking algorithms are evaluated quantitatively in a set of real-world experiments, and the overall tracking performance of each architecture is evaluated with a set of rigorous multiobject metric scores. It is found that the high recall rate of the lidar detection complements the higher-precision but lower-recall visual detection, and that some simple pre-processing steps are of benefit in high-clutter scenarios.