Session: 18-01-02: AI Implementation in Industry - I

Paper Number: 149935

149935 - Closing the Skills Gap With Generative Ai, Xr and Learning Engineering 

Skills gap is a major problem in advanced manufacturing and many other industries. The problem is further exacerbated by shortage of skilled instructors, rapid technological advances and an aging workforce approaching retirement. Here, we review the landscape of AI, XR and Learning Engineering to outline a promising approach to addressing workforce training. 

Generative AI (Gen-AI) technologies have developed rapidly on both research and commercialization fronts in recent years, as seen from the rapid launch of increasingly powerful models from OpenAI, Google, Nvidia, and many other smaller startups. Concurrently, VR/MR hardware industry has produced powerful devices at affordable prices. Combining these two technologies can advance the price-performance of both the stimulus-generation and response-evaluation aspects of training and assessment, ultimately realizing the promise of personalized and adaptive learning at scale and delivering outcomes approaching 1-1 expert-led tutoring and apprenticeship. However, a critical barrier is the high cost of code based immersive content creation which requires a multidisciplinary skilled team of coders, artists and game designers to create and maintain training content. 

We present case studies showing that co-design and Gen-AI powered no-code tools are a winning combination for reducing the time, cost and skill required for high quality XR content creation and delivery. By enabling non-technical instructional designers and subject matter experts to rapidly create, edit, share and deliver quality XR content, it becomes possible to democratize high quality workforce training, while also delivering high return on investment (ROI), rapid time to value (TTV)) and functional (engagement, learning, retention and transfer) benefits. 

 Based on a review of the end to end needs of the training industry and the design patterns of no-code tools, we list a core set of key features desired in a general-purpose no-code tool for immersive interactive VR/AR simulations for training: 

Drag-and-Drop interface that allows users to create simulations by simply dragging and dropping pre-built components and assets which would eliminate the need for coding knowledge

Asset library of pre-built 3D models, environments, audio clips, and other assets that users can choose from to build their simulations. The asset library should be diverse and customizable to accommodate various training scenarios including defense, manufacturing, healthcare and education.

Simulation authoring to enable users to author interactive scenarios by defining the behavior of objects, characters, and environments using triggers, actions, animations, and interactions via visual interface.

Cross platform support so that authored content can run on a wide range of devices, such as Oculus Rift, HTC Vive, Microsoft HoloLens, and mobile AR platforms like ARKit and ARCore. 

Real-time editing and preview to allow users to make changes to the simulation and see the results immediately, which would enable iterative design and rapid prototyping.

Collaboration and sharing to allow multiple users to work together on the same simulation similar to Google Docs.

Analytics and tracking to gather data on trainee interactions and performance within the simulation so that trainers and organizations can track progress, identify areas for improvement, and make data-driven decisions regarding the effectiveness of the training and conduct learning engineering to optimize the learners’ experiences.

Integrations with other systems and tools commonly used in training, such as learning management systems (LMS), 3D CAD and other software, marketplaces etc., which would allow seamless integration of the simulations into existing training workflows.

Scalability and performance to support complex simulations with a large number of objects and interactions which also optimizes performance to ensure smooth and immersive experiences for trainees regardless of the variation in client device capabilities.

In this presentation, we will outline high level architectures for no-code tools that serve common use cases in training such as rapid scenario creation, automated response scoring and feedback, and adaptive dialogue with AI agents. We will also present results from studies of efficiency of creation, AI-powered training, cross-platform deployment, co-design with subject matter experts and community members and creation by students to highlight the wide ranging impact of broader participation and collaboration on content quality, agility, learner experience and outcomes. Through these examples, we aim to demonstrate the opportunities afforded by co-design and no-code tools for delivering the same kind of price-performance improvements in XR content that we have witnessed in XR hardware and AI models.

Presenting Author: Rajesh Jha SimInsights Inc

Presenting Author Biography: Rajesh Jha is the Founder and CEO of SimInsights Inc, a California company focused on applying AI and XR technologies to enable non-technical users to author, publish and evaluate immersive, intelligent and interactive digital twins for skill training, credentialing and other applications. SimInsights offers two products - HyperSkill, a no-code platform, and Skillful, a content collection for career exploration. The no-code approach delivers up to 100 times improvements in cost, time and risk compared to code based alternatives. Prior to SimInsights, Raj was Program Manager at Altair for 3D modeling, simulation and optimization and shipped multiple enterprise products. He holds a BS and MS degrees in Mechanical Engineering from IITBHU and Ohio State University and an MBA from UCLA Anderson School of Management.

Authors: