Session: 09-17-01: AI for Energy

Paper Number: 167254

STA-DRL: A Spatial-Temporal Attention Deep Reinforcement Learning Framework for HVAC Optimization Using Python-Based Simulation 

Heating, Ventilation, and Air Conditioning (HVAC) systems are critical for maintaining comfortable indoor environments, but they are also among the largest consumers of energy in buildings. Traditional HVAC controllers often operate inefficiently because they cannot adapt to dynamic conditions such as changing weather, occupancy patterns, and fluctuating electricity prices. To address this challenge, we present STA-DRL (Spatial-Temporal Attention Deep Reinforcement Learning), a novel AI-based framework designed to optimize HVAC performance in multi-zone buildings. STA-DRL leverages advanced deep learning techniques to analyze both spatial heat distribution across building zones and temporal patterns such as occupancy and energy costs, enabling real-time adjustments to HVAC setpoints for improved energy efficiency and comfort.

The key innovation of STA-DRL lies in its hybrid architecture, which combines convolutional neural networks (CNNs) to model room-to-room heat transfer and long short-term memory (LSTM) networks to capture time-dependent factors like outdoor temperature and electricity pricing. Additionally, an attention mechanism is used to prioritize zones based on occupancy and peak pricing periods, ensuring that energy is used efficiently where it is needed most. This approach not only reduces energy consumption but also maintains occupant comfort by minimizing temperature deviations.

We evaluated STA-DRL in a simulated multi-zone building environment and compared its performance to traditional rule-based controllers. The results demonstrate significant improvements: STA-DRL achieved 22% lower energy costs while keeping comfort violations to just 4.1%, outperforming baseline methods. The system’s interpretability is enhanced through visualizations of heat matrices and attention weights, providing clear insights into how it prioritizes zones and adapts to dynamic conditions. These visualizations make it easier for building managers to understand and trust the system’s decisions.

A standout feature of STA-DRL is its adaptive reward function, which dynamically balances energy cost, thermal comfort, and peak demand reduction. This flexibility makes the system highly suitable for real-world deployment in commercial buildings, where conditions can vary widely. By reducing energy consumption and operational costs without compromising comfort, STA-DRL offers a scalable and practical solution for sustainable building automation.

The implications of this work are significant for the future of smart infrastructure. As buildings become more connected and data-driven, solutions like STA-DRL will play a crucial role in reducing energy waste and lowering carbon footprints. The framework’s ability to adapt to real-time conditions and prioritize occupant comfort makes it a valuable tool for building managers and energy policymakers alike.

In addition to its technical contributions, this work also highlights the potential of AI-driven approaches to address real-world challenges in energy management. By combining spatial and temporal modeling with reinforcement learning, STA-DRL sets a new standard for intelligent HVAC control. Future work will focus on deploying STA-DRL in real-world settings, further optimizing its performance under varying conditions, and exploring its integration with other smart building systems.

Presenting Author: Abdullahi Elnaiem North Carolina A&T State University

Presenting Author Biography: Abdullahi Elnaiem, Ph.D. Candidate in Mechanical Engineering, North Carolina A&T State University
Abdullahi Elnaiem specializes in the application of machine learning and artificial intelligence in robotics, automation, and control systems. His research focuses on developing innovative automated solutions to improve industrial efficiency. Abdullahi employs advanced tools like TensorFlow, PyTorch, and ROS to bridge theoretical research with practical implementations.

Authors:

Abdullahi Elnaiem North Carolina A&T State University
Amanuel Tereda North Carolina A&T State University
MA Muktadir North Carolina A&T State University
Sun Yi North Carolina A&T State University