Session: 14-03-01: Reliability and Safety in Industrial Automation Systems

Paper Number: 71261

Start Time: Monday, 06:20 PM

71261 - An Integrative and Transdisciplinary Approach for a Human-Centered Design of AI-Based Work Systems 

A furniture manufacturer is adding a fleet of automated guided vehicles (AGV) to the forklift workforce in its new production halls. The master controller applies artificial intelligence (AI) based algorithms to find optimal routes for the AGVs as well as for the forklift drivers. After a few weeks, it becomes apparent that the master controller assigns more trips to forklift driver A than to his colleague driver B. Apparently, A is faster. This inequality increases even further as the year progresses. Driver A completes her assignments ever faster, because otherwise she would not be able to keep up. A becomes inattentive at some point, and is increasingly exhausted (following scenario descriptions in [1])

With the further advancement of powerful algorithms based on Artificial intelligence (AI), their application is spreading in the world of work analogously to the consumer world. Due to their intrinsic ability to deal with large and heterogeneous data, these algorithms are predestined to be applied as control functions on all levels of the automation hierarchy. As such, AI has the potential to have a large impact on various occupational safety and health aspects as well as ethical aspects located therein.
Not only risks of physical and environmental harm but also ergonomic and psychological hazards have to be analyzed, localized and finally be avoided by appropriate measures. Moreover, an early identification of possible tendencies of a software to contribute to the development of certain moral values and norms (embedded values) facilitates an adaption of ethical criteria to the specific deployment context, deployment task, and users. In many areas of application, there is, furthermore, a growing need to harmonize not only entrepreneurial and employee-related interests, but also (especially) the latter with those of clients, customers or patients, and, indirectly, society in a broader sense. Ideally, all this is done already during the design of a technical system allowing for an iterative process of risk assessment and application of measures. Even though the methodological portfolio is large in the respective disciplines and numerous standards of technology design exist, today's engineering processes are far from integrating all these aspects in their design approach.

Usually, in complex technical systems with pronounced human-machine-interactions (HMI), psychological concerns are only investigated before or while system deployment. Ethical concerns are to date not systematically covered by occupational health and safety measures at all. However, AI components optimize strategies to reach predefined goals relevant to the work system and dynamically adapt their behavior to these. With this, in many areas of application AI-based systems require an assessment of potential relationships with aspects of health, safety and ethical values prior to the programming of the algorithms, viz. particularly when human-machine interaction is collaborative or cooperative and when the AI components are applied in a master control functionality.

In a discussion paper, we propose and discuss requirements and possible impediments for the development of an engineering approach that allows for a combination of technical risk assessment with psychological hazard analysis and an alignment with ethical criteria in the design phase of AI-based systems. In order to limit conceptual blur and to be able to integrate psychological and ethical requirements directly into the model based engineering process, the requirements and their respective relations need to be formalized and thus be made available for the engineering process. Without a doubt, this necessitates a transdisciplinary paradigm involving technical, psychological and ethical expertise as well as the development of a common language.
Possibly, such an approach could pave the way towards an online-assessment of systems during operation and therefore overcome the new challenges posed by AI-based systems, which significantly influence human-machine-interaction in a dynamic and probabilistic way.

Keywords: risk assessment, psychological hazard analysis, ethical evaluation, model based system engineering

Reference

[1] Adler, R., Heidrich, J., Jöckel, L., Kläs, M. (2020), KI-Systeme in der Produktionsautomatisierung, Gesellschaft für Informatik, Projekt „Exam AI – KI Testing & Auditing“, https//testings-ai.gi.de.

Presenting Author: Larissa Schlicht German Federal Institute for Occupational Safety and Health

Authors:

Larissa Schlicht German Federal Institute for Occupational Safety and Health
Marlen Melzer German Federal Institute for Occupational Safety and Health
Ulrike Rösler German Federal Institute for Occupational Safety and Health
Stefan Voß German Federal Institute for Occupational Safety and Health
Silvia Vock German Federal Institute for Occupational Safety and Health