Session: 16-01-01: Government Agency Student Poster Competition

Paper Number: 150585

150585 - Team Situation Awareness Between Human-Ai Teaming to Enable Adhd Workforce Participation in the Future Construction Site 

While people with attention-deficit/hyperactivity disorder (ADHD) have been marginalized in the construction workplace due to potentially higher risks of injuries, their unique talents could be leveraged using an ecosystem of co-bots driven by artificial intelligence (AI). For humans and machines to become true teammates—and correlatively, for technology to extend occupational opportunities to people with such neurodiversity—intelligent machines must assess, adapt, and respond to both workers and their environment. Such agility requires a reciprocal teaming capability wherein workers can engage their AI counterparts as more than tools, and AI systems can collaborate with workers seamlessly by predicting their behaviors.

To lay the necessary foundations for building this human-AI teaming workspace for construction workers with neurodiversity, this proof-of-concept project translates non‐invasive biomechanical (e.g., posture, gait) and psychophysiological metrics (e.g., brain signals, eye movements, etc.) into information. Personalized AI-based training systems can assess, model, and leverage to predict workers’ behaviors for improved worker‐machine teaming. To achieve this goal, this project: (1) Uses mixed-reality simulated future jobsite settings and biomechanical/psychophysiological metrics to understand ADHD workers-technology interactions during human-machine collaborative tasks; (2) Develops and test AI algorithms to automatically capture interactions and performance, and to select and provide optimal real-time feedback interventions in a timely manner to prevent injuries; (3) Examines the negative impacts of wearable technologies and AI, including privacy, security, integrity, usability, and other ethical issues within teaming context; (4) Determines and measure the social and economic impacts of the proposed worker-AI teaming system to enable diverse workforce participation; and (5) Produces an AI-informed training teammate to promote active, cooperative learning and to demonstrate the feasibility of human-AI teaming technology.

Data collection takes place in a multi-sensor immersive mixed-reality environment consisting of a virtual projection of the environment, passive haptics simulating future construction sites, and environmental modalities that capture their realistic responses to their AI-teammates and examine human performance measures. One of the factors that is considered in this project is Team situation awareness (TSA). TSA is the shared understanding of the situation knowledge among team members, which is crucial to incorporating ADHD workers into complex work environments for safety reasons. Participants were asked to cooperate and coordinate with their human or co-root teammates under different conditions such as information overload, time pressure, and visualized warning. TSA scores are collected through SAGAT queries about tasks and environment, which reflect participants’ understandings of task performance and safety issues. psychophysiological metrics are also collected using wearable sensors such as fNIRS and smart watches. The collected data then is used to develop a multi-dimensional predictive model to prevent potential incidents.

The analysis resulted in computational classification algorithms that can measure, track, and predict ADHD workers’ safety performance in the worker-machine teaming context, the project uses the experiments’ multi-modal heterogeneous sensor data to transform the current state of knowledge around diverse workforce development by characterizing skill acquisition, cognitive-process failures, and at-risk behavior. Such knowledge will enable personalizing feedback loops to optimize training goals down to individual users across industries.

Presenting Author: Ching-Yu Cheng Industrial Engineering

Presenting Author Biography: Graduate student of Purdue Industrial Engineering.
Research Interest: Construction Management, Human Factor, Situation Awareness

Authors:

Behzad Esmaeili Industrial Engineering
Ching-Yu Cheng Industrial Engineering