As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.
ACM e-Energy
Power Grid Behavioral Patterns and Risks of Generalization in Applied Machine Learning
Shimiao Li, Ján Drgoňa, Shrirang Abhyankar, and 1 more author
In Companion Proceedings of the 14th ACM International Conference on Future Energy Systems, Jun 2023
Recent years have seen a rich literature of data-driven approaches designed for power grid applications. However, insufficient consideration of domain knowledge can impose a high risk to the practicality of the methods. Specifically, ignoring the grid-specific spatiotemporal patterns (in load, generation, and topology, etc.) can lead to outputting infeasible, unrealizable, or completely meaningless predictions on new inputs. To address this concern, this paper investigates real-world operational data to provide insights into power grid behavioral patterns, including the time-varying topology, load, and generation, as well as the spatial differences (in peak hours, diverse styles) between individual loads and generations. Then based on these observations, we evaluate the generalization risks in some existing ML works caused by ignoring these grid-specific patterns in model design and training.
Constructing Neural Network Based Models for Simulating Dynamical Systems
Christian Legaard, Thomas Schranz, Gerald Schweiger, and 6 more authors
Dynamical systems see widespread use in natural sciences like physics, biology, and chemistry, as well as engineering disciplines such as circuit analysis, computational fluid dynamics, and control. For simple systems, the differential equations governing the dynamics can be derived by applying fundamental physical laws. However, for more complex systems, this approach becomes exceedingly difficult. Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. In recent years, there has been an increased interest in applying data-driven modeling techniques to solve a wide range of problems in physics and engineering. This article provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome. Based on the reviewed literature and identified challenges, we provide a discussion on promising research areas.
Structural inference of networked dynamical systems with universal differential equations
J. Koch, Z. Chen, A. Tuor, and 2 more authors
Chaos: An Interdisciplinary Journal of Nonlinear Science, Feb 2023
Networked dynamical systems are common throughout science in engineering; e.g., biological networks, reaction networks, power systems, and the like. For many such systems, nonlinearity drives populations of identical (or near-identical) units to exhibit a wide range of nontrivial behaviors, such as the emergence of coherent structures (e.g., waves and patterns) or otherwise notable dynamics (e.g., synchrony and chaos). In this work, we seek to infer (i) the intrinsic physics of a base unit of a population, (ii) the underlying graphical structure shared between units, and (iii) the coupling physics of a given networked dynamical system given observations of nodal states. These tasks are formulated around the notion of the Universal Differential Equation, whereby unknown dynamical systems can be approximated with neural networks, mathematical terms known a priori (albeit with unknown parameterizations), or combinations of the two. We demonstrate the value of these inference tasks by investigating not only future state predictions but also the inference of system behavior on varied network topologies. The effectiveness and utility of these methods are shown with their application to canonical networked nonlinear coupled oscillators.
Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles
Wenceslao Shaw Cortez, Soumya Vasisht, Aaron Tuor, and 3 more authors
IFAC-PapersOnLine, Feb 2023
12th IFAC Symposium on Nonlinear Control Systems NOLCOS 2022
Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models require a large number of observations and lack operational guarantees for safety-critical systems. Data-driven models leveraging available partially characterized dynamics have potential to provide reliable systems models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work we explore this middle-ground between expert-modeled and pure data-driven modeling. We present control-oriented parametric models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations to construct data-driven blackbox and graybox representations of the AUV dynamics. In addition, we explore a hybrid formulation that explicitly models the residual error related to imperfect graybox models. We compare the prediction performance of the learned models for different distributions of initial conditions and control inputs to assess their suitability for control.
Homotopy Learning of Parametric Solutions to Constrained Optimization Problems
Shimiao Li, Ján Drgoňa, Aaron R Tuor, and 2 more authors
We present differentiable predictive control (DPC) as a deep learning-based alternative to the explicit model predictive control (MPC) for unknown nonlinear systems. In the DPC framework, a neural state-space model is learned from time-series measurements of the system dynamics. The neural control policy is then optimized via stochastic gradient descent approach by differentiating the MPC loss function through the closed-loop system dynamics model. The proposed DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constraints. In embedded implementation using a Raspberry-Pi platform, we experimentally demonstrate that it is possible to train constrained control policies purely based on the measurements of the unknown nonlinear system. We compare the control performance of the DPC method against explicit MPC and report efficiency gains in online computational demands, memory requirements, policy complexity, and construction time. In particular, we show that our method scales linearly compared to exponential scalability of the explicit MPC solved via multiparametric programming.
Data-driven Stabilization of Discrete-time Control-affine Nonlinear Systems: A Koopman Operator Approach
Subhrajit Sinha, Sai Pushpak Nandanoori, Ján Drgona, and 1 more author
In 2022 European Control Conference (ECC), Aug 2022
The availability of computational power, and a wealth of data from sensors have boosted the development of model-based predictive control for smart and effective control of advanced buildings in the last decade. More recently occupant-behavior models have been developed for including people in the building control loops. However, while important objectives of scientific research are reproducibility and replicability of results, not all information is available from published documents. Therefore, the aim of this paper is to propose a guideline for a thorough and standardized occupant-behavior model documentation. For that purpose, the literature screening for the existing occupant behavior models in building control was conducted, and the occupant behavior modeling processes were studied to extract practices and gaps for each of the following phases: problem statement, data collection, and preprocessing, model development, model evaluation, and model implementation. The literature screening pointed out that the current state-of-the-art on model documentation shows little unification, which poses a particular burden for the model application and replication in field studies. In addition to the standardized model documentation, this work presented a model-evaluation schema that enabled benchmarking of different models in field settings as well as the recommendations on how OB models are integrated with the building system.
Dissipative Deep Neural Dynamical Systems
Ján Drgoňa, Aaron Tuor, Soumya Vasisht, and 1 more author
The problem of synthesizing stochastic explicit model predictive control policies is known to be quickly intractable even for systems of modest complexity when using classical control-theoretic methods. To address this challenge, we present a scalable alternative called stochastic parametric differentiable predictive control (SP-DPC) for unsupervised learning of neural control policies governing stochastic linear systems subject to nonlinear chance constraints. SP-DPC is formulated as a deterministic approximation to the stochastic parametric constrained optimal control problem. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem’s value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the SP-DPC problem’s value function is back propagated through the closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy which allows for direct model-based policy optimization. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.
Learning Constrained Adaptive Differentiable Predictive Control Policies With Guarantees
Deciding on a suitable algorithm for energy demand prediction in a building is non-trivial and depends on the availability of data. In this paper we compare four machine learning models, commonly found in the literature, in terms of their generalization performance and in terms of how using different sets of input features affects accuracy. This is tested on a data set where consumption patterns differ significantly between training and evaluation because of the Covid-19 pandemic. We provide a hands-on guide and supply a Python framework for building operators to adapt and use in their applications.
Physics-constrained deep learning of multi-zone building thermal dynamics
Ján Drgoňa, Aaron R. Tuor, Vikas Chandan, and 1 more author
We present a physics-constrained deep learning method to develop control-oriented models of building thermal dynamics. The proposed method uses systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our method incorporates structural prior knowledge from traditional physics-based building modeling into the architecture of the deep neural network model. Further, we also use penalty methods to provide inequality constraints, thereby bounding predictions within physically realistic and safe operating ranges. We observe that stable eigenvalues accurately characterize the dissipativeness of the system, and use a constrained matrix parameterization based on the Perron-Frobenius theorem to bound the dominant eigenvalues of the building thermal model parameter matrices. We demonstrate the effectiveness and physical interpretability of the proposed data-driven modeling approach on a real-world dataset obtained from an office building with 20 thermal zones. The proposed data-driven method can learn interpretable dynamical models that achieve high accuracy and generalization over long-term prediction horizons. We show that using only 10 days’ measurements for training, our method is capable of generalizing over 20 consecutive days. We demonstrate that the proposed modeling methodology is achieving state-of-the-art performance by significantly improving the accuracy and generalization compared to classical system identification methods and prior advanced methods reported in the literature. compared to prior state-of-the-art methods reported in the literature.
L4DC
Automating Discovery of Physics-Informed Neural State Space Models via Learning and Evolution
Elliott Skomski, Ján Drgoňa, and Aaron Tuor
In Proceedings of the 3rd Conference on Learning for Dynamics and Control, 07 – 08 june 2021