Session: 07-10-03: Medical Robotics, Rehabilitation, and Surgery III

Paper Number: 167257

Enhancing Upper Limb Rehabilitation With fNIRS-Measured Brain Activity and Human Robot Interaction 

Introduction

Robotic rehabilitation is transforming occupational therapy by providing precise, adaptive, and data-driven solutions for upper limb impairments. Traditional methods rely on repetitive exercises, but robotics enhance therapy with real-time assistance, progress monitoring, and intention-driven support tailored to patient needs. Advances in neuroimaging and machine learning offer insights into neural activity and functional connectivity, yet challenges persist in refining adaptive control, intention recognition, and clinical feasibility.  

This study aims to develop a bilateral controller and interactive interface for human-robot interaction using a Franka Emika 7-DOF robot. A deep learning-based path classification system will predict motion intentions, enabling the robot to guide patients through personalized exercises, from simple tasks to complex routines. Efficacy will be assessed using torque, muscle strength, and trajectory smoothness metrics. Simultaneously, fNIRS will monitor cortical activation and functional connectivity, providing insights into neural rehabilitation mechanisms. By integrating motion intention prediction with neural activity analysis, this work aims to create a holistic, adaptive framework for effective upper limb rehabilitation.

Contribution of the work

This work develops an intelligent robotic rehabilitation system integrating multimodal sensorimotor feedback, machine learning, and human-robot interaction to enhance upper limb therapy. By combining advanced robotics, deep learning, and neuroimaging, the system provides adaptive, intention-driven assistance tailored to individual patient needs. A bilateral controller and user interface for the Franka Emika 7-DOF robot enable personalized therapeutic exercises, while fNIRS monitors cortical activation to offer insights into neural rehabilitation mechanisms. This multimodal approach improves precision, adaptability, and clinical feasibility, creating a patient-centered framework for effective motor recovery.

Methodology

The Franka Emika 7-DOF robotic arm serves as the core platform, offering precise and adaptive assistance for therapeutic exercises. A bilateral controller enables real-time interaction and sensory feedback between the robot and patient, ensuring responsive support, while an intuitive user interface allows therapists and patients to monitor and control the rehabilitation process seamlessly. To predict motion intentions, a deep learning-based path classification system is trained on datasets of upper limb movements aligned with Wolf Motor Function Tests (WMFT) criteria. By analyzing movement patterns, the system predicts patient intentions, enabling the robot to guide individuals through personalized exercises along their intended paths. The system’s accuracy and adaptability are validated through iterative testing, ensuring robust performance across diverse rehabilitation tasks.

The robot provides real-time assistance, ensuring smooth and accurate movement trajectories while minimizing strain. Key metrics such as torque, muscle strength, and trajectory smoothness are monitored to evaluate exercise efficacy and track patient progress. Simultaneously, functional near-infrared spectroscopy (fNIRS) is integrated to measure cortical activation and assess functional connectivity during tasks, offering insights into the neural mechanisms driving rehabilitation.

Data from motion intention predictions, robotic performance, and neural activity are combined into a unified framework for comprehensive analysis. This multimodal approach evaluates the system’s effectiveness in improving motor function and neural recovery. Collected data is used to optimize the deep learning model, robotic control algorithms, and fNIRS integration, ensuring the system remains adaptive, precise, and user-friendly. This iterative refinement paves the way for broader clinical application, enhancing rehabilitation outcomes for individuals with upper limb impairments.

Preliminary results and conclusions

A custom wrist-mounted end-effector supports participants' wrists and measures torque during therapeutic tasks, enabling precise monitoring of movement quality and progress. Initial tests show the deep learning-based system predicts motion intentions with 79% accuracy, allowing the Franka robot to guide patients adaptively. These preliminary results highlighted the potential of the system to deliver personalized, data-driven rehabilitation, paving the way for improved outcomes in upper limb motor recovery. Future work will expand the dataset for improved prediction accuracy and integrate multimodal sensors, including fNIRS, to enhance adaptability and functional connectivity analysis, advancing upper limb rehabilitation outcomes.

Presenting Author: Haider Allawi San Jose State University

Presenting Author Biography: Haider Allawi is the Graduate Student and the Research Assistant in BioRob lab at San Jose State University.

Authors:

Haider Allawi San Jose State University
Justin Catalano San Jose State University
Tyler Yee San Jose State University
Emily Wang San Jose State University
Zachariah Holder San Jose State University
Daniel Sam San Jose State University
Yue Luo San Jose State University
Yi Yuan San Jose State University
Megan Chang San Jose State University
Armin Moghadam San Jose State University
Lin Jiang San Jose State University