Session: 07-10-02: Medical Robotics, Rehabilitation, and Surgery II

Paper Number: 168097

Reinforcement Learning-Based Model Predictive Control for a Soft Robotic Assistive Bionic Joint Knee Brace 

Introduction

Knee osteoarthritis (KOA) affects over 30 million adults in the U.S., leading to chronic pain and mobility limitations. Traditional treatments, such as surgery, are costly and invasive, while existing lower limb exoskeletons often lack adaptability or are overly complex and expensive. Soft robotic systems, particularly those using fluidic muscle actuators, offer a promising alternative due to their intrinsic compliance and ability to mimic natural muscle engagement. However, challenges such as nonlinear actuator dynamics and joint misalignment hinder their effectiveness. This work introduces a Reinforcement Learning-Based Model Predictive Control (RL-MPC) framework for a soft robotic Assistive Bionic Joint (ABJ) Knee Brace. The system integrates neuromuscular modeling, gravity-balancing features, and data-driven adaptive control to ensure natural gait alignment and provide personalized assistance. By leveraging 3D motion analysis in simulation, live EMG sensors for data-driven modeling, and center of mass trajectory for balancing, the system detects user intention and dynamically adjusts assistance levels through the RL-MPC controller. This approach aims to improve rehabilitation outcomes, offering a cost-effective and user-friendly solution for individuals with knee diseases and other lower limb disabilities.

Contribution of the work

This work presents a Reinforcement Learning-Based Model Predictive Control (RL-MPC) framework for a Soft Robotic Assistive Bionic Knee Brace, combining MPC’s predictive power with RL’s adaptability to handle nonlinear soft robot dynamics. The system integrates fluidic muscle actuators, neuromuscular modeling, and gravity balancing to optimize gait assistance. A data-driven algorithm uses 3D motion analysis, EMG sensors, and Functional near-infrared spectroscopy (fNIRS) to detect user intent and monitor cortical activation, enhancing motor recovery insights. RL-MPC enables adaptive, assist-as-needed (AAN) support, mimicking natural movement for improved rehabilitation. fNIRS is integrated to measure cortical activation during walking tasks, providing insights into the neural mechanisms driving motor recovery. Experiments with healthy and unhealthy users validate its ability to guide trajectories while maintaining compliance, offering a user-focused solution surpassing traditional exoskeletons in gait imitation and rehabilitation efficacy.

Methodology

The soft robotic ABJ knee brace utilizes fluidic muscle actuators to deliver compliant and natural assistance during walking. A bilateral control system enables real-time interaction between the brace and the user, ensuring responsive support based on the user’s movements. An intuitive user interface allows therapists and patients to monitor and control the rehabilitation process seamlessly.

To predict motion intentions, a deep learning-based path classification system is trained on datasets of lower limb movements. fNIRS data is combined with motion intention predictions and robotic performance metrics into a unified framework for comprehensive analysis. Collected data is used to optimize the deep learning model, control algorithms, and fNIRS integration, ensuring the system remains adaptive, precise, and user-friendly. The system analyzes gait patterns to predict user intentions, enabling the brace to provide personalized assistance along the intended movement trajectory.

The brace provides real-time assistance to ensure smooth and natural gait patterns while minimizing strain on the user. Key metrics such as compressed pressure for fluidic muscles, joint angle changes, and center of mass balancing are monitored to evaluate the effectiveness of the assistance during walking. These metrics are used to dynamically adjust the support provided by the brace, ensuring optimal assistance throughout the rehabilitation process.

Preliminary results and conclusions

The integration of Reinforcement Learning (RL) and Model Predictive Control (MPC) into the Assistive Bionic Joint (ABJ) Knee Brace shows promising results. Preliminary experiments used EMG signals with a CNN-trained model to predict human intention, achieving below 5% RMSE error. The MPC provided precise adaptive assistance, while RL optimized input pressure estimation, enabling smooth assist-as-needed (AAN) movements and reduced muscle activity. Current findings are based on a single subject, requiring expanded data collection for model refinement. Future work will focus on diverse data, fNIRS integration, and comprehensive validation to enhance system performance. The RL-MPC framework demonstrates significant potential for improving rehabilitation efficacy in assistive bionic knee braces.

 

 

Presenting Author: Jordan O'connor San Jose State University

Presenting Author Biography: Jordan O'Connor is a Master student and Research Assistant in mechanical engineering department at San Jose State University.

Authors:

Jordan O'connor San Jose State University
Kiet Duong San Jose State University
Van Le San Jose State University
Adrian Joseph Lucas San Jose State University
Syed Zaidi IntelliScience Training Institute
Vimal Viswanathan San Jose State University
Li Jin San Jose State University
Yi Yuan San Jose State University
Lin Jiang San Jose State University