Session: 16-01-01: Poster Session: NSF-Funded Research (Grad & Undergrad)

Paper Number: 99904

99904 - Ssa-Cv-Loc: Towards Robust Multi-Agent Localization and Place Recognition in Uncertain Environments via Semantic World Understanding 

Multiagent systems consisting of unmanned mobile robots capable of unsupervised collaborative task completion have gained a lot of interest from industry in the context of large-scale manufacturing in light of the recent acute skilled labor shortage caused by the COVID-19 pandemic. These systems capable of Simultaneous Localization and Mapping (SLAM) need to be safe for human operators while reliably performing various tasks in uncertain and dynamic workspaces. To achieve these functionalities, Collaborative Visual Simultaneous Localization and Mapping (CV-SLAM) frameworks based on the feature-based, keyframe-centric photogrammetric techniques, have seen wide adoption. Comparative with classical SLAM approaches,  CV-SLAM posits to be more suitable for real-world applications such as the creation of large-scale metric-semantic global maps, shared situational awareness, or human-robot collaboration in a wide variety of indoor and outdoor environments.  However, most CV-SLAM frameworks define collaboration as sharing agents’ keyframe data for fusing local maps into a geometrically-consistent global map with the infusion of human-readable labels (semantics) as an afterthought.  

Some of the most common CV-SLAM algorithms use the Bag-of-Words (BoW) technique. These approaches have two major limitations. First, most of the works available assume the scenes are mostly static, and secondly, the semantic information is limited from the visual words generated. Thus, most CV-SLAM have limited performance in dynamic environments.  

Leveraging the current advancements in Machine Learning techniques for image segmentation and classification, in this work we propose a novel centralized localization algorithm that maximizes the utilization of semantic data for improving navigation for collaborative robotic systems.  

Furthermore, we also propose a novel place recognition algorithm based on a novel descriptor called Semantic Spatial Array (SSA) to overcome the limitation of BoW algorithm. In this approach, we utilize semantic knowledge extracted from visual observations, in creating 1D dynamic arrays that uniquely characterize a physical space (spatial description) observed in keyframes. The novel spatial descriptor is generated on-the-fly, requires no pretrained model, no additional data other than semantics and low-level data (pose, 3D points etc.), and linearly scales to the number of agents. Through a newly developed real-world dataset designed specifically for uncertain multiagent robotics scenarios, in this poster, we will demonstrate the robustness of the novel algorithms, described in the previous paragraph. Furthermore, we will compare their performances in localization and local mapping modules per agent, against ORB-SLAM3, one of the top-performing single-agent full SLAM pipelines. We also aim to demonstrate the ability of a robotic agent to recognize places visited by other robotic agents without requiring them to meet beforehand for exchanging data. This project is funded by NSF under the grant NSF #2024795 and for the benefit of the community, we will make the new dataset and all relevant source codes, open-source and publicly available.  

Presenting Author: Azmyin Md Kamal Louisiana State University - Baton Rouge

Presenting Author Biography: Ph.D. student in ME, Louisiana State University. M.S in Mechanical Engineering from the University of Louisiana at Lafayette. IEEE Student Member in Robotics and Automation Society. Interested in multiagent collaborative robotics, computer vision, and AI

Authors:

Azmyin Md Kamal Louisiana State University - Baton Rouge
Corina Barbalata Louisiana State University - Baton Rouge