Session: 08-11-01: Mobile Robots and Unmanned Ground Vehicles I

Paper Number: 165624

A Standard Framework for Testing and Benchmarking of 2D RRT Path Planner 

Path-planning algorithms play a crucial role in a wide range of applications, including autonomous driving, robotics, and industrial automation. These algorithms are fundamental for enabling autonomous agents to navigate efficiently through environments while avoiding obstacles and optimizing travel time. Over the years, researchers have developed numerous algorithms, such as A*, Dijkstra, and the Rapidly Exploring Random Tree (RRT) family, including RRT*, RRT-Connect, Lazy RRT, Informed RRT*, and RRT*-Smart. Each of these algorithms offers unique advantages and trade-offs, making them more or less suitable for different scenarios. Some prioritize computational efficiency, while others focus on finding optimal or near-optimal paths. The diversity of algorithmic approaches has led to the development of various benchmarking frameworks, such as Bench-MR, OMPL (Open Motion Planning Library), and PathBench, which provide tools for evaluating algorithm performance. However, despite the existence of these benchmarking tools, a standardized reference set of maps or unified evaluation metrics remains lacking. This inconsistency makes it difficult to fairly compare the performance of different path-planning algorithms across diverse scenarios. To address this issue, researchers have designed a variety of test environments, including large obstacle scenarios, small dense obstacle scenarios, maze-like structures, narrow passageways, and real-world-inspired layouts. However, these test environments are often custom-designed to highlight the strengths of specific algorithms, leading to biased performance assessments. While such tailored evaluations provide valuable insights in controlled settings, they do not always reflect the algorithm's robustness and generalizability across different conditions. A standardized benchmarking framework with a well-defined set of test scenarios would facilitate a more comprehensive and fair comparison of path-planning algorithms.

This paper proposes a standardized benchmarking framework specifically designed for evaluating and comparing path-planning algorithms in 2D static environments. The framework introduces a structured set of 33 benchmark maps categorized into 11 distinct environment types: Macro Obstacles Scenario, Micro Obstacles Scenario, Multi-Obstacle Indoor Scenario, Circular Maze Scenario, Narrow Path Corridor Scenario, Indoor Room Scenario, Serpentine Corridor Scenario, Enclosed Starting Scenario, Enclosed Goal Scenario, Narrow Gate Passage Scenario, and Branching Path Scenario. Each scenario is further divided into three levels of complexity—simple, medium, and difficult—to assess algorithm performance under varying conditions. To demonstrate the utility of this framework, five widely used path-planning algorithms—RRT, RRT*, Informed RRT*, RRT*-Smart, and Lazy RRT—are tested and evaluated based on key performance metrics, including time efficiency, path quality, and computational cost. The results reveal distinct performance characteristics among the tested algorithms, highlighting trade-offs between exploration speed, path optimization, and computational efficiency across different environments. By establishing a standardized evaluation methodology, this study contributes a reliable and structured benchmarking framework that facilitates fair comparisons of path-planning algorithms. This approach not only provides valuable insights for selecting the most suitable algorithm for specific applications but also lays the groundwork for future research, including potential extensions to dynamic and higher-dimensional environments.

Presenting Author: Qiuhao Ma Southern Illinois University Edwardsville

Presenting Author Biography: Qiuhao Ma is a graduate student in Mechanical Engineering at Southern Illinois University Edwardsville (SIUE), with expertise in robotics, path planning, and machine learning. He has a strong background in developing innovative robotic systems and frameworks, with proficiency in MATLAB, Python, C++, ROS, and YOLOv5.
He holds a Bachelor's degree in Mechatronics and Robotics Engineering from SIUE and previously studied Robotics Engineering at Changshu Institute of Technology. He has been recognized for academic excellence, graduating Cum Laude and receiving multiple awards, including the Outstanding Senior Assignment Award and the First Prize Scholarship at Changshu Institute of Technology.

Authors:

Qiuhao Ma Southern Illinois University Edwardsville
Yuchen Xia Southern Illinois University Edwardsville
Jiayi Yuan Southern Illinois University Edwardsville
Haibo Zhao Southern Illinois University Edwardsville
Shihao Zhao Southern Illinois University Edwardsville
Mingshao Zhang Southern Illinois University Edwardsville