Session: 02-07-01: IoT and Digital Twins

Paper Number: 145378

145378 - Toward Digital Twin of the Design Process - an Attempt to Capture Design Rationale From Spoken Words 

As products increase their scale and complexity, communication has become much more difficult. due to the increased number of people involved in the development. Furthermore, current design trends such as open innovation and the globalization of product development demand rapid rebuilding of the design team which may hinder the establishment of stable communications. Though, the drawings, models, documents and other related data are shared among design teams, design rationale, such as the reasons behind the determination of design, is rarely expressed in these data. When the design team was stable for certain years, designers could presume the design rationales as the contexts were shared. However, in regards to the teams that would not last for long, design data would not convey the design rationales properly. Lack of design rationales in shared data is crucial as the design rationales play an important role when coordinating the design among the design team. Despite their importance, design rationales are not explicitly expressed in CAD data, simulation models or alike. They are rarely described even in documents such as design reports. The primary reason is that it is too time-consuming to describe design rationale while designing a product. Therefore, a tool to capture design rationales with low load is required.

This paper proposes a preliminary study on automatic extraction of design rationales from conversations while designing a product. This attempt aims to establish a digital twin of a design process, which contains not only inputs and outputs of processes but also design rationales. To reuse the data or information captured from design processes in the future process, design rationale is indispensable. Like other digital twins, sensors are installed to design processes. This study employed cameras, microphones and smart-glasses as sensors to detect when design rationales are spoken.

The proposed automatic extraction method of design rationale detects where the design rationale is spoken based on multiple clues, which are important words, behaviours and physiological reactions. Design rationale should have certain importance among sentences in the conversations. Thus, TF-IDF, a natural language processing technique, is employed to extract design rationale from the conversations. In addition, behaviours and physiological reactions are monitored. To detect characteristic behaviours, OpenPose is applied to recorded video data. As for physiological reactions, blinks are recorded by smart-glasses as they reflect phycological status. To decide whether design rationale is spoken in each time periods, an evaluation function, which reflects trustworthiness of each indicator (e.g. importance of words, lean back, blink interval, etc.) is proposed.

To verify the validity of the proposed method, design experiments were conducted. Design experiments were conducted by pairs of designers so that design rationales are spoken. The examinees were 4th year university students and graduate students who have learnt the operation of Autodesk Fusion 360 (3D-CAD software) in several projects. Scissors were selected as a design subject considering difficulty and complexity. Examinees were asked to design scissors to cut a rolled cloth or a thick magazine. Detected design rationales were tested against correct data, which were manually extracted from the experiments with a certain rule. As a result, one of the tests showed that 81% of design rationales could be extracted while 48% of where design rationales were not spoken was also extracted.

This paper shows certain possibility of extracting design rationales by monitoring spoken words, behaviours and physiological reactions. However, as the proposed method assumes ideal conditions to detect design rationales (e.g. settings to let examinees speak out design rationales, design process is fully video recorded, etc.) several issues such as privacy needs to be addressed.

Presenting Author: Kazuya Oizumi Tokyo Denki University

Presenting Author Biography: Kazuya is a associate professor at the Department of Design Engineering , Tokyo Denki University. He got Ph.D. degree in engineering at the University of Tokyo in 2013. He was assistant professor at Department of Systems Innovation, the University of Tokyo. He moved to Tokyo Denki University in 2020. He researches in engineering design and service design. His interests are in integrated modeling of design objectives and design activities, in particular how to obtain insightful information for innovation and management.

Authors:

Kazuya Oizumi Tokyo Denki University
Ryo Nakazawa Tokyo Electric Power Company Holdings, Inc
Yuya Oda Tokyo Denki University