Loop Closure Detection

Visual loop closure detection and application to topological mapping

INTRODUCTION

Loop-closure detection is crucial for enhancing the robustness of both topological (see for example Mark Cummins work) and metrical SLAM algorithms (see for example the page of Andrew Davison). This problem consists in detecting when the robot has returned to a past location after having discovered new terrain for a while.

Such detection makes it possible to increase the precision of the actual pose estimate. Recognizing previously mapped locations can also be relevant for addressing the global localization problem, or even for recovering from a kidnapping (when a robot is moved by something it does not control). Hence, solving the loop-closure detection problem not only improves SLAM performances, but it enables additional capabilities to mobile robots.

We developped a vision-based loop-closure detection algorithm with applications to topological and metrical SLAM. Our method relies on Bayesian filtering for loop-closure probability computation, with images encoded according to the incremental bags of visual words scheme (see our indoor navigation project). The overall complexity of the designed solution scales linearly with the number of places, making it possible to detect loop-closures in real-time conditions.

These video illustrate the approach indoor and outdoor (more videos on Adrien web page):

Visual loop closure detection - indoor

Outdoor loop closure detection

The next pictures shows the loop detection results in indoor environment. The approach has been validated outdoor also, see our ICRA 08 article for details.

Loop closure detection results

Images from the top corridor (top row) and bottom corridor (bottom row) joining the New-York and London elevators on the plan: note the strong similarities between these images

It is important to notice that in spite of the strong perceptual aliasing in the environment (i.e. several distinct places look similar), no false positive detections were made (i.e. when the algorithm claims that a loop-closure has been detected whereas none occured), and most of the time the camera was in an already visited place, a correct loop-closure has been found.

TOPOLOGICAL SLAM APPLICATION

Our approach has been applied to build topological maps incrementally and in real time. The figure illustrate an environment on the left and the constructed map on the right, with colors used to show the correspondance:

We also added metric information coming from the odometry of a robot and used a relaxation algorithm to correct odometry errors when loop-closure are detected.

Topological mapping using vision and odometry

METRICAL SLAM APPLICATION

The approach has also been applied to metrical visual SLAM pose correction. Starting from Andrew Davison MonoSLAM, our loop-closure detection algorithm is used to detect robot kidnapping and provide an initial pose estimate to reset the SLAM algorithm after kidnapping.

EKF SLAM kidnapping - relocalization

DATASETS

Here are the datasets used in our experiments:

• Indoor dataset
• Outdoor dataset
• Camera calibration parameters

Each archive contains the following:

• Images: This is a directory containing images numbered sequentially in order of collection. The camera (a simple camcorder, whose parameters are given below) was held at hand and approximatively pointed in the direction of the displacement. Images were acquired at 1Hz while the person holding the camera was moving at average walking speed (i.e. approximatively 5km/h). Images are of size 240x192 and in the ppm format.
• XXXGroundTruth.mat - XXXGroundTruth.txt: These files correspond to the ground truth correspondence matrix (in Matlab and text formats) that gives image-to-image correspondences: each entry (i,j) of the correspondence matrix is 1 if image i and image j were determined to be taken at the same place, and 0 otherwise. Image-to-image matches have been labeled at hand, so the obtained correspondences are approximate and are still subject to visual inspection if one would use these ground truth files.

PUBLICATIONS

Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A.	Visual topological SLAM and global localization. Proceedings of the International Conference on Robotics and Automation (ICRA). 2009.
Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A.	A Fast and Incremental Method for Loop-Closure Detection Using Bags of Visual Words. IEEE Transactions On Robotics, Special Issue on Visual SLAM. 2008.
Angeli, A.	Détection visuelle de fermeture de boucle et applications à la localisation et cartographie simultanées. 2008.
Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A.	Incremental vision-based topological SLAM. Proceedings of the 2008 IEEE International Conference on Intelligent Robots and Systems (IROS 2008). 2008.
Angeli, A., Filliat, D., Doncieux, S., & Meyer, J.-A.	Real-Time Visual Loop-Closure Detection. Proceedings of the International Conference on Robotics and Automation (ICRA). 2008.

 

PARTICIPANTS

>> David FILLIAT is the project leader.

>> Adrien Angeli worked on this project during his PhD on loop closure detection (in collaboration with the AnimatLab, project ROBUR).

>> Stéphane Bazeille is working on this project during his post-doc.

>> Nicolas Beaufort and Zhenjiang Ni worked on this project during their intership.