Session: 16-01-01: Poster Session: NSF-Funded Research (Grad & Undergrad)

Paper Number: 98529

98529 - Learner: System Architecture of an Upper Body Haptic Interface for High-Resolution Interaction With Virtual Environments 

The “Learning Environments with Augmentation and Robotics for Next-gen Emergency Responders” (LEARNER) System focuses on developing an accessible, immersive training platform for accelerated expertise development of emergency responders. Current training for disaster scenarios is inadequate, where human augmentation technologies such as augmented or virtual reality can render virtual hazardous environments and future crises. To improve professional training for emergency responders, we present a system architecture of a robotic platform which enables a human pilot to physically and visually experience a virtual environment. Virtual Reality (VR) technology can visually render virtual environments with high resolutions, but the immersion is broken once the user tries to physically interact with virtual objects. Haptic feedback refers to devices which emulate those virtual interaction forces, delivering the feeling of grasping a virtual ball or pushing a box. Currently, there are very few haptic devices which can deliver large magnitude force simulation, where many devices focus on the high-precision feedback to the hands or fingers. This work presents a strategy for delivering large magnitude forces at the arms for representing a wide variety of contact scenarios and enhance the overall experience.

The robotic platform utilizes several key hardware and software components to generate high resolution contact forces to the user for a realistic and immersive experience. An upper body powered exoskeleton (Make: Sarcos Robotics) comprised of two 5 degrees of freedom (DoF) arms provides the haptic contact to user handles at the end effectors. The interaction forces between the user and the robot are measured through 6 DoF force/torque sensors in the user handles and the joint positions are measured through absolute encoders at each DoF. The exoskeleton is controlled using the Open Robotics Software and Simulation Construction Set software packages both developed by the Institute for Human and Machine Cognition (IHMC). These packages contain simulation and runtime tools for modeling robots and developing controllers for real-time applications. The robot controller algorithm uses a Whole-Body Control (WBC) strategy which solves multiple, task-based control objectives using a Quadratic Program (QP) to determine safe overall behavior for the robot. The VR development platform, Unity, is utilized to build the virtual environment, compute virtual dynamics, and generate various tasks with multiple contact scenarios. The OpenXR software package for Unity, contains the middleware for communicating with the HTC Vive Pro which provides the visual feedback to the user while also measuring head-mounted display (HMD) position, orientation, and eye tracking for studying user gaze and attention. This robotic platform has been found to be effective in realizing haptic feedback for static and dynamic contact and transparent during no contact scenarios. Smooth transitions between contact scenarios are accomplished with minimal tuning in the controller parameters, demonstrating the flexibility and effectiveness of the WBC control strategy.

As a major component of LEARNER’s overall vision, this robotic platform gives first responders the ability to experience the unstructured and unpredictable environments of a disaster emergency physically and visually. Emergency responders are responsible for solving critical tasks under extreme duress in complex, dangerous, and dynamic environments. The haptic interface emulates the interactions with the virtual environment for static, dynamic, and no contact scenarios to create a realistic, immersive experience. Furthermore, this novel platform may be utilized for the gaming technologies, robot teleoperation, and personalized trainings and product development in industries.

Presenting Author: Connor Herron Virginia Polytechnic Institute and State University (Virginia Tech)

Presenting Author Biography: N/A

Authors:

Connor Herron Virginia Polytechnic Institute and State University (Virginia Tech)
Benjamin Beiter Virginia Polytechnic Institute and State University (Virginia Tech)
Bhaben Kalita Virginia Polytechnic Institute and State University (Virginia Tech)
Alexander Leonessa Virginia Polytechnic Institute and State University (Virginia Tech)