Session: 03-16-01: Securing Advanced Manufacturing: Cybersecurity and Edge Computing for Industrial IoT

Paper Number: 165962

Practically Leveraging LLMs for Manufacturing Cybersecurity 

Introduction and Motivation:
Cybersecurity in advanced manufacturing is increasingly critical as industrial control systems (ICS) face rising threats and a shortage of skilled security professionals. Large Language Models (LLMs), particularly multi-modal LLMs, offer a potential solution by analyzing complex manufacturing documentation to identify weaknesses. This study explores the feasibility of using LLMs to assess cybersecurity risks in manufacturing, leveraging their ability to process unstructured and structured data. Our goal is to bridge the ICS cybersecurity knowledge gap and provide AI-driven insights that support manufacturers in securing industrial environments.

Contribution to Science and Engineering:
This research contributes to both AI and industrial cybersecurity by demonstrating how LLMs can assist in automated security analysis of manufacturing systems. By processing multi-modal data—textual descriptions, network diagrams, and technical reports—LLMs can augment cybersecurity assessments in ICS environments. Our findings highlight the strengths and limitations of AI-augmented cybersecurity workflows, offering a roadmap for integrating LLMs into industrial security frameworks.

Methodology:
We analyzed a 100-page Digital Thread handbook from an advanced manufacturing facility using a mix of private and public LLMs. Private models were tested for security reasons but lacked domain-specific knowledge, leading us to integrate Retrieval-Augmented Generation (RAG) with public LLMs. While this improved text analysis, challenges remained in processing tables, diagrams, and multimodal content. Additional techniques, such as leveraging vision-enabled models and document accessibility tools, were explored to enhance LLM analysis.

Challenges and Observations:

·  Model Limitations & Validation: The LLMs sometimes hallucinated details or gave overly general answers, underscoring the need for careful validation of results​. Domain-specific queries (e.g. about machinery or protocols) required expert-informed prompts to get useful answers, highlighting that the finer points of ICS security require context beyond the LLM’s base training. Ensuring output accuracy remains difficult without a human in the loop.

·  Data Modality and Context: Handling heterogenous industrial data proved problematic. The LLMs excelled with narrative text but had trouble with tables or images. In one instance, the vision-capable model fixated on irrelevant details in a facility diagram (navigating our efforts to “counting toilets” in a floorplan) instead of extracting network diagrams, illustrating the gap in multi-modal understanding​. Such observations confirm that current LLM tools may silently omit non-textual information, leading to incomplete analysis of ICS documentation​.

·  Security Risks of LLM Integration: Introducing LLMs into ICS workflows brings new security considerations. We had to ensure sensitive facility information was protected when using cloud-based models (prompting a pivot to private models initially). Moreover, the threat of prompt injection and insecure output handling is real – a crafty input or blindly trusted LLM output could result in unauthorized actions or vulnerabilities​. Similarly, training data poisoning is a concern: if an adversary’s data influences the model, it might skew or mislead analysis​. These risks, along with issues like model backdoors or overreliance on AI, mean robust safeguards and oversight are necessary when leveraging LLMs for cybersecurity.

Preliminary Results and Conclusions:

Our results suggest that LLMs can serve as junior cybersecurity analysts, effectively extracting relevant information from lengthy documentation. The models correctly identified manufacturing equipment, analyzed potential ransomware attack vectors, and recommended standard mitigation measures. However, they struggled with facility-specific nuances and overlooked key details in diagrams and contextual analysis, emphasizing the need for human-in-the-loop validation.

This human-in-the-loop validation must be well-defined to ensure that LLMs enhance cybersecurity efforts rather than introduce additional risks. Without clear validation frameworks, there is a risk of over-reliance on AI-generated insights, which could lead to misinterpretation of weaknesses or missed security threats. Establishing structured oversight mechanisms—such as expert verification of AI-generated recommendations, cross-referencing outputs with trusted cybersecurity databases, and implementing strict access controls—will be essential for safely and effectively deploying LLMs in manufacturing cybersecurity. 

Presenting Author: Matthew Luallen CyManII

Presenting Author Biography: Matthew E. Luallen is the Lead Research Scientist for Education Translation at the Information Trust Institute at the University of Illinois where he coordinates and conducts research that addresses securing the nation’s critical infrastructure. Luallen served as a Co-Founder of CYBATI, where he led the company in further developing and expanding training services to enhance the understanding of, and provide protection from, cyber-physical threats. He also served as a Co-Founder of Dragos Security co-developing CyberLens™ for operational technology device and communications discovery and analysis. He was a Co-Founder of Encari, a NERC CIP cybersecurity consulting firm helping the US and Canadian power grid defend strategic assets from cyber-physical attacks. He was also an Information Security Network Engineer and Architect at Argonne National Laboratory.

Authors:

Curtis Taylor CyManII
Monica Akbar CyManII
Gabriela Ciocarlie CyManII
Matthew Luallen CyManII