Session: 07-10-03: Medical Robotics, Rehabilitation, and Surgery III

Paper Number: 164703

EMG Signal Hand Gesture Classification Using a CNN-Transformer Model and Transfer Learning 

Motivation

Electromyography (EMG)-based hand gesture classification plays a vital role in advancing myoelectric prosthetics, human-computer interaction, and rehabilitation. The ability to classify hand gestures using machine learning with EMG signals is crucial for improving prosthetic control, making devices more responsive. Traditional machine learning approaches, including Support Vector Machines (SVM), Random Forest, and Linear Discriminant Analysis (LDA), have been used in EMG classification but often lack adaptability when applied to new users and user-conditions. Deep learning approaches, particularly Convolutional Neural Networks (CNNs), have improved classification performance by capturing spatial features in EMG signals. However, CNNs struggle to model temporal dependencies, which are essential for distinguishing between complex hand gestures. Transformers, which leverage self-attention mechanisms, have demonstrated exceptional performance in time-series classification tasks of which EMG signals are a representative example. This study aimed to integrate CNNs and Transformers to enhance spatial-temporal feature extraction while leveraging transfer learning to improve model adaptability across individuals.

Methodology

This research utilizes an EMG dataset from Ninapro comprising 80 subjects performing 18 hand gestures. A structured preprocessing pipeline, including normalization, filtering, and windowing segmentation, was applied to prepare and label data. The CNN-Transformer model was trained under multiple scenarios, including general model training on 80 subjects, validation on trained subjects performing untrained movements, and evaluation on entirely unseen subjects. To improve adaptability to new patterns, transfer learning was introduced, where the pre-trained model was fine-tuned using both partial and full movement data from new individuals to compare the learning effectiveness. The model's performance was evaluated using metrics such as accuracy, F1-score, precision, recall, and confusion matrices. A comparative study was conducted to evaluate the benefits of integrating CNN and Transformer architectures, analyzing the contributions of CNN-based spatial feature extraction and Transformer-based temporal modeling.

Results

Experimental results demonstrated that the CNN-Transformer model outperforms traditional CNN-based approaches, achieving higher classification accuracy with less computational cost in recognizing EMG signals. The Transformer component enhances temporal feature learning, leading to superior generalization across different subjects. The inclusion of transfer learning further improves performance, particularly when the model is applied to new individuals or gestures with limited training data.

Contribution

This research introduces a CNN-Transformer model that integrates CNNs for spatial feature extraction and Transformers for modeling temporal dependencies. The proposed approach differs from traditional CNN-based models by incorporating self-attention mechanisms, which enhance the ability to recognize gestures by focusing on relevant signal patterns over time. Additionally, transfer learning was implemented to improve generalization across new users or patterns, addressing the challenge of inter-subject variability in EMG signals. The study provides a novel and simple deep learning framework that enhances classification accuracy and robustness while ensuring adaptability for real-world applications in myoelectric prosthetics. The effectiveness of the CNN-Transformer model was evaluated through comparative analysis with traditional CNN-based models, highlighting the advantages of integrating Transformers. Furthermore, fine-tuning strategies with both partial and full movement data were explored to assess the impact of personalization on model performance. The study also examined how the combination of CNN and Transformer components contribute to improved classification outcomes, providing insights into model generalizability, which is more closely to real-world and daily use cases and applications.

 

Conclusions

The study highlights the effectiveness of combining CNNs and Transformers for EMG classification, particularly when integrated with Transfer Learning to enhance adaptability across 80 participants. The proposed CNN-Transformer hybrid model provides a more generalizable approach for prosthetic control systems, making them more responsive to variations. These findings have implications for the development of next-generation prosthetic devices, improving functionality and usability for individuals with limb loss. 

Presenting Author: Linghui Meng University of Canterbury

Presenting Author Biography: Jeff Meng is from Tianjin, China, and moved to Christchurch, New Zealand in 2017 to complete his undergraduate studies at the University of Canterbury. There, his multicultural background fueled his passion for engineering, blending creativity with technical skills. Jeff graduated in 2021 with a Bachelor of Engineering with Honors in Mechanical Engineering and a minor in Biomedical Engineering, achieving Second Class Honors (Division One). Jeff has a strong interest in prosthetics and blending electromyography and electroencephalography with artificial intelligence and neural networks to enhance their performance. He returned to the University of Canterbury in late 2022 to work toward his Ph.D. in order to conduct research in this field.
Outside his studies, Jeff has a strong passion for sports, enjoying boxing, badminton, and rigorous gym workouts, where pushing beyond his limits brings him great satisfaction. He also has a deep interest in cars, dedicating time to their upkeep and enjoying the hands-on experience of routine maintenance.

Authors:

Linghui Meng University of Canterbury
James Atlas University of Canterbury
Deborah Munro Univerisity of Canterbury