Session: 03-12-01: Digital Manufacturing Process Simulation and Validation

Paper Number: 113240

113240 - Digital Twin Based Learning From Demonstration System for Industrial Robots 

Learning from Demonstration (LfD) is an approach to robot programming where the machine aims to replicate the task presented by a human without being explicitly programmed to execute this task. The LfD method has been tested to be an effective way to create complex robot routines even for users without coding skills, because it allows the user to directly demonstrate the execution of the task and let the machine itself infer the sequence of actions required to replicate it. On the other hand, such approach requires the system to have a thorough understanding of the environment where the task is being presented. As a result, most of the modern LfD implementations heavily rely on sensors and machine vision for the robot to be able to capture the state of the surrounding world and the task being demonstrated. Such dependence on often costly hardware inhibits the spread of otherwise effective and user-friendly LfD approach among the actors with less financial resources, such as small laboratories and businesses, as well as hobby roboticists.

Our research paper presents an alternative approach to LfD with a system based on a fully simulated 3D environment, which intends to breach the mentioned hardware requirement gap using the power of simulation. In this work, we demonstrate how executing LfD process entirely in a simulation brings multiple potential benefits to the table. Firstly, it can eliminate the need for hardware sensors on the robot, as the information about its surroundings is already contained within the digital 3D environment. Secondly, simulation serves as a unified medium for recording demonstrated tasks and can facilitate sharing of the produced solutions between different types of robotic cells with minimal to no reconfiguration. Finally, the presented work demonstrates how a Virtual Reality interface can be employed to let the operator interact with the digital environment in a natural way when recording task demonstrations for the robot.

The system is built on top of commonly available software components with the intention of making it easy to reproduce and extend with new capabilities. User interface for executing LfD is implemented in Unity engine, which has gained popularity in both scientific and engineering community as a robust tool for building interactive process visualizations. Robot control and simulation is handled by Robot Operating System (ROS) with the help of MoveIt motion planning framework, and utilizes the ROS-Industrial robot description standard for easier integration with new industrial robot models. The code of the LfD system presented in this work is made available available as an open-source project.

Presenting Author: Yevhen Bondarenko Tallinn University of Technology

Presenting Author Biography: Yevhen Bondarenko, is PhD candidate in the Department of Mechanical and Industrial Engineering at Tallinn University of Technology. He has a background in the development of digital twins of various industrial systems and application of virtual reality interfaces for control of the said systems. The current focus of his PhD research is the creation of a human-centric framework for immersive applications for controlling digital twins in collaborative virtual environments.

Authors:

Yevhen Bondarenko Tallinn University of Technology
Simone Luca Pizzagalli Tallinn University of Technology
Vladimir Kuts Tallinn University of Technology
Eduard Petlenkov Tallinn University of Technology
Tauno Otto Tallinn University of Technology