Session: 03-15-02: Smart Manufacturing and Robotics for the Future II

Paper Number: 166814

Digital Twin Synchronization for Real-Time Process Control Using Sim-Rl in Smart Additive Manufacturing 

Recent advancements in digital twin technology have opened new avenues for smart manufacturing. These smart manufacturing systems are increasingly relying on adaptive control mechanisms to optimize complex processes to reduce production wastage. This research presents a cutting-edge approach integrating Soft Actor-Critic (SAC) reinforcement learning with digital twin technology to enable real-time control in robotic additive manufacturing. We pioneered our methodology using a ViperX 300s robot arm, implementing two distinct environments: Task 1: static target reaching and Task 2: dynamic target following to mimic the defect correction in real practice. The system architecture combines Unity game engine’s simulation environment with ROS2 for seamless innovative digital twin synchronization. We implemented hierarchical reward structure to address common reinforcement learning challenges including local minima avoidance, convergence acceleration, and training stability, while leveraging transfer learning to efficiently adapt trained models across different task training. This work advances the integration of reinforcement learning with realistic digital twins for industrial & manufacturing robotics applications, providing a framework for enhanced adaptive real-time control for smart additive manufacturing processes.

This study introduces a novel digital twin framework using the ViperX 300s 6DoF robot arm. A custom Unity virtual environment was established from scratch, integrating ViperX 300s via the URDF-Importer plugin. Due to the absence of ROS2 packages for synchronizing Unity with the physical ViperX 300s robot arm, a new package was created to relay joint data from Unity to control the real arm using forward kinematics. To the authors’ knowledge, this is the first ROS2-based digital twin connection between Unity and robot arm that achieved a minimal 20 ms lag. Training efficiency of SAC was enhanced through Hierarchical Reward Structure, and Transfer Learning. The trained reinforcement learning agent produced two models for defect mitigation in WAAM: one for surface defect filling and another for large cavity repair.

The results highlight the stability of transfer learning, ensuring a smooth transition from Task 1 SAC to Task 2 SAC with transfer learning (SAC TL) while maintaining consistent performance across tasks crucial for adaptive control applications. SAC TL outperformed SAC without transfer learning, achieving a higher cumulative reward, faster convergence, and better stability. Faster convergence and higher reward attainment indicate improved adaptation to task variations, essential for dynamic robotic systems and industrial applications as time is the most valuable asset. SAC TL showed lower value loss, faster policy stabilization, and higher entropy, ensuring efficient learning. Testing confirmed stable end-effector tracking, highlighting transfer learning’s role in faster learning and better policy generalization.

Deployed the developed framework, reinforcement learning models adapted to real-time defects, overcoming material inconsistencies through Unity-robot synchronization. Two key reinforcement learning tasks are static target reaching for precise defect correction and dynamic target following for real-time trajectory planning for enhanced defect handling, minimizing materials waste and post-processing. Unity-enabled training accelerated learning and optimized RL model transfer to real-world AM, improving manufacturing efficiency and part quality. This approach outperforms traditional control methods, ensuring adaptive, autonomous defect mitigation in AM.

Presenting Author: Sen Liu University of Louisiana at Lafayette

Presenting Author Biography: Sen Liu, is an Assistant Professor at the Department of Mechanical Engineering, University of Louisiana at Lafayette. He was a postdoc scholar at Stanford University, and received PhD degree from Colorado School of Mines. His research interests include metal additive manufacturing, machine learning, robotics for manufacturing, in-situ monitoring and control for manufacturing processes.

Authors:

Sen Liu University of Louisiana at Lafayette
M Matsive Ali University of Louisiana at Lafayette