Session: 03-16-01: Securing Advanced Manufacturing: Cybersecurity and Edge Computing for Industrial IoT

Paper Number: 166075

Dynamark: A Reinforcement Learning Framework for Dynamic Watermarking in Industrial Machine Tool Controllers 

Cybersecurity threats targeting industrial machine tool controllers (MTCs) pose severe risks to modern manufacturing systems. These controllers are inherently vulnerable due to their reliance on networked communications, physical limitations, and insufficient built-in security measures. As a result, MTCs become attractive targets for various cyberattacks that disrupt operations, degrade product quality, and compromise safety. Among these, replay attacks are particularly detrimental. In such attacks, adversaries record valid control signals and later reintroduce them into the system, causing controllers to process outdated or manipulated commands. This deceptive tactic bypasses traditional anomaly detection methods and can lead to unexpected system behavior, operational downtimes, and hazardous conditions.

To mitigate these threats, researchers have explored proactive cybersecurity techniques, with watermarking emerging as a promising approach. Watermarking involves injecting controlled perturbations into communication signals, thereby embedding a verifiable signature into system operations. This signature facilitates real-time integrity checks, enabling rapid identification of replay attacks. However, conventional watermarking methods have limitations. They often struggle to balance detection accuracy with operational efficiency and may rely on static parameters that do not account for the dynamic nature of industrial environments. Such inflexibility can result in either missed detections or unnecessary system disturbances.

Addressing these challenges, we propose DynaMark, a novel framework that leverages reinforcement learning to implement dynamic watermarking specifically designed for MTCs. DynaMark distinguishes itself by adaptively modifying the watermarking variance in real time. Its reinforcement learning component continuously monitors system performance and environmental conditions, then adjusts watermark parameters to establish the trade-off between detection accuracy and control performance. Under normal operating conditions, DynaMark maintains system stability and energy efficiency while ensuring that replay attacks are swiftly detected when they occur. This dynamic adaptation makes DynaMark a robust, scalable, and intelligent solution for industrial cybersecurity.

The effectiveness of DynaMark was evaluated within a digital twin environment that replicates the complex behavior of real-world MTC systems. Our experiments demonstrate that DynaMark outperforms traditional constant-variance watermarking approaches. During normal operations, the framework preserves system stability and minimizes energy overhead. In scenarios involving replay attacks, DynaMark rapidly identifies the intrusion, thereby reducing potential operational disruptions and preventing compromised product quality. The experimental results underscore the capability of DynaMark to adapt to evolving threats while maintaining optimal system performance. DynaMark represents a significant advancement in the cybersecurity of industrial control systems. By integrating reinforcement learning with dynamic watermarking, our framework effectively addresses the vulnerabilities inherent in MTCs and provides a proactive solution against replay attacks.

Presenting Author: Navid Aftabi University of Washington

Presenting Author Biography: Navid Aftabi received his B.Sc. degree in Computer Engineering from University of Tabriz, Tabriz, Iran, in 2015. He earned an M.Sc. in Industrial Engineering from Sharif University of Technology, Tehran, Iran, in 2019, and another M.Sc. degree in Industrial Engineering from Sabanci University, Istanbul, Turkey, in 2022. He is currently a Ph.D. student in the Department of Industrial & Systems Engineering at University Washington. His research leverages AI, operations research, and statistical modeling to create efficient, resilient solutions for complex systems. He focuses on data-driven and OR-based frameworks for resiliency, anomaly detection, and recovery in cyber-physical and other complex systems. He is a member of Society of Manufacturing Engineers (SME), Institute for Industrial and Systems Engineers (IISE), and Institute for Operations Research and the Management Sciences (INFORMS).

Authors:

Navid Aftabi University of Washington
Abhishek Hanchate Texas A&M University
Satish Bukkapatnam Texas A&M University
Dan Li University of Washington