SciML for buildings

Developing differentiable predictive control (DPC) as a market-ready technology for energy-efficient building control.

Abstract

At PNNL, I have led the development and demonstration of differentiable predictive control (DPC), a state-of-the-art advanced control method, that enables rapid design and scalable deployment of predictive controls for building energy systems. DPC provides benefits of model predictive control (MPC) at costs and complexity of traditional controls allowing for improved control and predictions which result in reduced energy use, demand flexibility and improved air quality. Under this project, my PNNL team collaborates with industry partners to integrate DPC with existing market-available control products to develop affordable, high-performance, and scalable predictive control solutions that can be deployed in new and existing buildings to reduce energy, peak demand, greenhouse gas emissions, and improve occupant comfort. The project will demonstrate the DPC technology’s flexibility in design and deployment to industry use cases and underlying building energy systems without substantial, bespoke rework. This applied research is directed by industry objectives to bring to the market this new capability, and enable its broad deployment. The resulting buildings control technologies and deployment enabling software will significantly advance BTO’s goals to reduce emissions in US buildings.

Overview of Differentiable Predictive Control (DPC) methodology for real-world building control.

Modeling building thermal dynamics with SciML

Energy-efficient buildings are one of the top priorities to sustainably address the global energy demands and reduction of the CO2 emissions. It has been demonstrated that advanced building control, like model predictive control (MPC) or deep reinforcement learning (DRL), can notably reduce the energy use and mitigate greenhouse gas emissions. However, in order for these advanced control methods to work in practice, they heavily rely on accurate prediction models of building thermal dynamics.

To address this, our team has developed novel scientific machine learning (SciML) method for control-oriented modeling of building thermal dynamics. Specifically, our method incorporates structural prior knowledge from traditional physics-based building modeling into the architecture of the deep neural network model. Further, we also use penalty methods to enforce inequality constraints, thereby bounding predictions within physically realistic and safe operating ranges. We show that using only 10 days’ measurements for training, our method is capable of generalizing over 20 consecutive days. We demonstrate that the proposed modeling methodology is achieving state-of-the-art performance by significantly improving the accuracy and generalization compared to classical system identification methods and prior advanced methods reported in the literature. compared to prior state-of-the-art methods reported in the literature. For more information read our paper.

Generic structure of physics-inspired neural dynamics model architecture. Weights of individual neural blocks are parametrized by linear maps with constrained eigenvalues to enforce the energy dissipativity property of the modeled system.

Differentiable predictive control of building energy systems

Despite intensive research efforts, the practical applications of advanced control methods such as MPC and DRL are still in the early stages. One of these challenges is the complexity of designing and deploying the optimal control policies in the current predominantly rule-based building automation systems.

To address this limitation, our team has developed novel methodology called differentiable predictive control (DPC) for learning constrained control policies for dynamical systems subject to operational and safety constraints. DPC presents a novel data-driven policy optimization algorithm that combines the benefits of both worlds, data-driven adaptive nature of DRL, and model-based performance and safety guarantees of MPC. We have demonstrated the proposed DPC method in a high fidelity simulation environment using learned model of multi-zone building thermal dynamics. For more information read our paper.

Conceptual overview of Differentiable Predictive Control (DPC) methodology for buidling control. Step 1: physics-constrained system identification of the building thermal dynamics. Step 2: Learning neural control policies in closed-loop simulations with differentiable system dynamics model and MPC constraints.

High fidelity simulation platform of building energy systems

Development of new building HVAC control algorithms has grown due to needs for energy efficiency and operational flexibility. However, case studies demonstrating new algorithms are largely individualized, making algorithm performance difficult to compare directly. In addition, the effort and expertise required to implement case studies in real or simulated buildings limits rapid prototyping potential. To address this issue, our team has been co-developing a high fidelity building simulation framework for performance benchmarking of modern control methods. This multi-institution project called Building Optimization Testing Framework (BOPTEST) was recognized with the best paper award issued by the Journal of Building Performance Simulation. For more information visit the project website.

Conceptual overview of the BOPTEST framework designed to facilitate the performance evaluation and benchmarking of advanced building control strategies.

Open-source Software

The technology developed under this project is being open-sourced as part of the Neuromancer Scientific Machine Learning (SciML) library developed by our team at PNNL.

PNNL Team

Draguna Vrabie (PI)
Ján Drgoňa (Co-PI)
Aaron Tuor (Co-PI)
James Koch
Madelyn Shapiro
Wenceslao Shaw Cortez
Soumya Vasisht
Rahul Birmiwal
Himanshu Sharma
Brian Hutchinson (WWU Professor)
Seth Briney (WWU student)
Diego Llanes (WWU student)
Harry Qiang (WWU student)

Industrial Partners

Acknowledgements

This project is supported through the U.S. Department of Energy (DOE), Energy Efficiency and Renewable Energy, Building Technologies Office (BTO) under the “Advancing Market-Ready Building Energy Management by Cost-Effective Differentiable Predictive Control” and the “Dynamic decarbonization through autonomous physics-centric deep learning and optimization of building operations” projects. PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE) by Battelle Memorial Institute under Contract No. DE-AC05-76RL0-1830.

Part of this work emerged from the IBPSA Project 1, an international project conducted under the umbrella of the International Building Performance Simulation Association (IBPSA). Project 1 develops and demonstrates a BIM/GIS and Modelica Framework for building and community energy system design and operation. This research was partially supported by the Assistant Secretary for Energy Efficiency and Renewable Energy, Office of Building Technologies of the U.S. Department of Energy, under Contracts nos. DE-AC05-76RL01830, and DE-AC02-05CH11231.

References

2023

ACC

Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems

Truong X. Nghiem, Ján Drgoňa, Colin Jones, and 10 more authors

In 2023 American Control Conference (ACC), 07 – 08 june 2023

Bib HTML

@inproceedings{10155901,
  author = {Nghiem, Truong X. and Drgoňa, Ján and Jones, Colin and Nagy, Zoltan and Schwan, Roland and Dey, Biswadip and Chakrabarty, Ankush and Di Cairano, Stefano and Paulson, Joel A. and Carron, Andrea and Zeilinger, Melanie N. and Shaw Cortez, Wenceslao and Vrabie, Draguna L.},
  booktitle = {2023 American Control Conference (ACC)},
  title = {Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems},
  year = {2023},
  volume = {},
  number = {},
  pages = {3735-3750},
  doi = {10.23919/ACC55779.2023.10155901}
}

ACC

Data-Driven Control: Theory and Applications

Damoon Soudbakhsh, Anuradha M. Annaswamy, Yan Wang, and 6 more authors

In 2023 American Control Conference (ACC), 07 – 08 june 2023

Bib HTML

@inproceedings{10156081,
  author = {Soudbakhsh, Damoon and Annaswamy, Anuradha M. and Wang, Yan and Brunton, Steven L. and Gaudio, Joseph and Hussain, Heather and Vrabie, Draguna and Drgoňa, Ján and Filev, Dimitar},
  booktitle = {2023 American Control Conference (ACC)},
  title = {Data-Driven Control: Theory and Applications},
  year = {2023},
  volume = {},
  number = {},
  pages = {1922-1939},
  doi = {10.23919/ACC55779.2023.10156081}
}

BnE
Ten questions concerning reinforcement learning for building energy management

Zoltan Nagy, Gregor Henze, Sourav Dey, and 22 more authors

Building and Environment, 07 – 08 june 2023

Abs Bib HTML

As buildings account for approximately 40% of global energy consumption and associated greenhouse gas emissions, their role in decarbonizing the power grid is crucial. The increased integration of variable energy sources, such as renewables, introduces uncertainties and unprecedented flexibilities, necessitating buildings to adapt their energy demand to enhance grid resiliency. Consequently, buildings must transition from passive energy consumers to active grid assets, providing demand flexibility and energy elasticity while maintaining occupant comfort and health. This fundamental shift demands advanced optimal control methods to manage escalating energy demand and avert power outages. Reinforcement learning (RL) emerges as a promising method to address these challenges. In this paper, we explore ten questions related to the application of RL in buildings, specifically targeting flexible energy management. We consider the growing availability of data, advancements in machine learning algorithms, open-source tools, and the practical deployment aspects associated with software and hardware requirements. Our objective is to deliver a comprehensive introduction to RL, present an overview of existing research and accomplishments, underscore the challenges and opportunities, and propose potential future research directions to expedite the adoption of RL for building energy management.
@article{NAGY2023110435, title = {Ten questions concerning reinforcement learning for building energy management}, journal = {Building and Environment}, volume = {241}, pages = {110435}, year = {2023}, issn = {0360-1323}, doi = {https://doi.org/10.1016/j.buildenv.2023.110435}, url = {https://www.sciencedirect.com/science/article/pii/S0360132323004626}, author = {Nagy, Zoltan and Henze, Gregor and Dey, Sourav and Arroyo, Javier and Helsen, Lieve and Zhang, Xiangyu and Chen, Bingqing and Amasyali, Kadir and Kurte, Kuldeep and Zamzam, Ahmed and Zandi, Helia and Drgoňa, Ján and Quintana, Matias and McCullogh, Steven and Park, June Young and Li, Han and Hong, Tianzhen and Brandi, Silvio and Pinto, Giuseppe and Capozzoli, Alfonso and Vrabie, Draguna and Bergés, Mario and Nweye, Kingsley and Marzullo, Thibault and Bernstein, Andrey}, keywords = {Open AI Gym} }

2022

Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systems

Ján Drgoňa, Karol Kiš, Aaron Tuor, and 2 more authors

Journal of Process Control, Aug 2022

Abs Bib HTML Code

We present differentiable predictive control (DPC) as a deep learning-based alternative to the explicit model predictive control (MPC) for unknown nonlinear systems. In the DPC framework, a neural state-space model is learned from time-series measurements of the system dynamics. The neural control policy is then optimized via stochastic gradient descent approach by differentiating the MPC loss function through the closed-loop system dynamics model. The proposed DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constraints. In embedded implementation using a Raspberry-Pi platform, we experimentally demonstrate that it is possible to train constrained control policies purely based on the measurements of the unknown nonlinear system. We compare the control performance of the DPC method against explicit MPC and report efficiency gains in online computational demands, memory requirements, policy complexity, and construction time. In particular, we show that our method scales linearly compared to exponential scalability of the explicit MPC solved via multiparametric programming.
@article{DRGONA202280, title = {Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systems}, journal = {Journal of Process Control}, volume = {116}, pages = {80-92}, year = {2022}, month = aug, issn = {0959-1524}, doi = {https://doi.org/10.1016/j.jprocont.2022.06.001}, url = {https://www.sciencedirect.com/science/article/pii/S0959152422000981}, author = {Drgoňa, Ján and Kiš, Karol and Tuor, Aaron and Vrabie, Draguna and Klaučo, Martin}, keywords = {Differentiable predictive control, Model predictive control, Neural state space models, Data-driven differentiable optimization, Deep learning} }
BnE
A guideline to document occupant behavior models for advanced building controls

Bing Dong, Romana Markovic, Salvatore Carlucci, and 20 more authors

Building and Environment, Aug 2022

Abs Bib HTML

The availability of computational power, and a wealth of data from sensors have boosted the development of model-based predictive control for smart and effective control of advanced buildings in the last decade. More recently occupant-behavior models have been developed for including people in the building control loops. However, while important objectives of scientific research are reproducibility and replicability of results, not all information is available from published documents. Therefore, the aim of this paper is to propose a guideline for a thorough and standardized occupant-behavior model documentation. For that purpose, the literature screening for the existing occupant behavior models in building control was conducted, and the occupant behavior modeling processes were studied to extract practices and gaps for each of the following phases: problem statement, data collection, and preprocessing, model development, model evaluation, and model implementation. The literature screening pointed out that the current state-of-the-art on model documentation shows little unification, which poses a particular burden for the model application and replication in field studies. In addition to the standardized model documentation, this work presented a model-evaluation schema that enabled benchmarking of different models in field settings as well as the recommendations on how OB models are integrated with the building system.
@article{DONG2022109195, title = {A guideline to document occupant behavior models for advanced building controls}, journal = {Building and Environment}, volume = {219}, pages = {109195}, year = {2022}, issn = {0360-1323}, doi = {https://doi.org/10.1016/j.buildenv.2022.109195}, url = {https://www.sciencedirect.com/science/article/pii/S0360132322004310}, author = {Dong, Bing and Markovic, Romana and Carlucci, Salvatore and Liu, Yapan and Wagner, Andreas and Liguori, Antonio and {van Treeck}, Christoph and Oleynikov, Dmitry and Azar, Elie and Fajilla, Gianmarco and Drgoňa, Ján and Kim, Joyce and Vellei, Marika and {De Simone}, Marilena and Shamsaiee, Masood and Bavaresco, Mateus and Favero, Matteo and Kjaergaard, Mikkel and Osman, Mohamed and Frahm, Moritz and Dabirian, Sanam and Yan, Da and Kang, Xuyuan}, keywords = {Guideline documentation, Occupant behavior model, Building control, Model Predictive control (MPC)} }

Learning Constrained Adaptive Differentiable Predictive Control Policies With Guarantees

Ján Drgoňa, Aaron Tuor, and Draguna Vrabie

Aug 2022

arXiv Bib Code

@misc{drgona2022learning,
  title = {Learning Constrained Adaptive Differentiable Predictive Control Policies With Guarantees},
  author = {Drgoňa, Ján and Tuor, Aaron and Vrabie, Draguna},
  year = {2022},
  eprint = {2004.11184},
  archiveprefix = {arXiv},
  primaryclass = {eess.SY}
}

2021

Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings

Journal of Building Performance Simulation, Aug 2021

Bib HTML Code

@article{Blum2021,
  author = {},
  title = {Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings},
  journal = {Journal of Building Performance Simulation},
  volume = {14},
  number = {5},
  pages = {586-610},
  year = {2021},
  publisher = {Taylor & Francis},
  doi = {10.1080/19401493.2021.1986574},
  url = {https://doi.org/10.1080/19401493.2021.1986574},
  eprint = {https://doi.org/10.1080/19401493.2021.1986574}
}

BS
Energy prediction under changed demand conditions: robust machine learning models and input feature combinations

Thomas Schranz, Johannes Exenberger, Christian Møldrup Legaard, and 2 more authors

In Proceedings of Building Simulation 2021: 17th Conference of IBPSA, Sep 2021

Abs Bib HTML

Deciding on a suitable algorithm for energy demand prediction in a building is non-trivial and depends on the availability of data. In this paper we compare four machine learning models, commonly found in the literature, in terms of their generalization performance and in terms of how using different sets of input features affects accuracy. This is tested on a data set where consumption patterns differ significantly between training and evaluation because of the Covid-19 pandemic. We provide a hands-on guide and supply a Python framework for building operators to adapt and use in their applications.
@inproceedings{bs2021_30806, doi = {https://doi.org/10.26868/25222708.2021.30806}, url = {https://publications.ibpsa.org/conference/paper/?id=bs2021_30806}, year = {2021}, month = sep, publisher = {IBPSA}, author = {Schranz, Thomas and Exenberger, Johannes and Legaard, Christian Møldrup and Drgona, Ján and Schweiger, Gerald}, title = {Energy prediction under changed demand conditions: robust machine learning models and input feature combinations}, booktitle = {Proceedings of Building Simulation 2021: 17th Conference of IBPSA}, volume = {17}, isbn = {978-1-7750520-2-9}, address = {Bruges, Belgium}, series = {Building Simulation}, pages = {3268--3275}, issn = {2522-2708}, organisation = {IBPSA}, editors = {} }
Physics-constrained deep learning of multi-zone building thermal dynamics

Ján Drgoňa, Aaron R. Tuor, Vikas Chandan, and 1 more author

Energy and Buildings, Sep 2021

Abs Bib HTML Code

We present a physics-constrained deep learning method to develop control-oriented models of building thermal dynamics. The proposed method uses systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our method incorporates structural prior knowledge from traditional physics-based building modeling into the architecture of the deep neural network model. Further, we also use penalty methods to provide inequality constraints, thereby bounding predictions within physically realistic and safe operating ranges. We observe that stable eigenvalues accurately characterize the dissipativeness of the system, and use a constrained matrix parameterization based on the Perron-Frobenius theorem to bound the dominant eigenvalues of the building thermal model parameter matrices. We demonstrate the effectiveness and physical interpretability of the proposed data-driven modeling approach on a real-world dataset obtained from an office building with 20 thermal zones. The proposed data-driven method can learn interpretable dynamical models that achieve high accuracy and generalization over long-term prediction horizons. We show that using only 10 days’ measurements for training, our method is capable of generalizing over 20 consecutive days. We demonstrate that the proposed modeling methodology is achieving state-of-the-art performance by significantly improving the accuracy and generalization compared to classical system identification methods and prior advanced methods reported in the literature. compared to prior state-of-the-art methods reported in the literature.
@article{DRGONA2021110992, title = {Physics-constrained deep learning of multi-zone building thermal dynamics}, journal = {Energy and Buildings}, volume = {243}, pages = {110992}, year = {2021}, issn = {0378-7788}, doi = {https://doi.org/10.1016/j.enbuild.2021.110992}, url = {https://www.sciencedirect.com/science/article/pii/S0378778821002760}, author = {Drgoňa, Ján and Tuor, Aaron R. and Chandan, Vikas and Vrabie, Draguna L.}, keywords = {System identification, Physics-constrained deep learning, Building thermal dynamics, Control-oriented modeling} }

Deep Learning Explicit Differentiable Predictive Control Laws for Buildings

Ján Drgoňa, Aaron Tuor, Elliott Skomski, and 2 more authors

IFAC-PapersOnLine, Sep 2021

7th IFAC Conference on Nonlinear Model Predictive Control NMPC 2021

Bib HTML Code

@article{DRGONA202114,
  title = {Deep Learning Explicit Differentiable Predictive Control Laws for Buildings},
  journal = {IFAC-PapersOnLine},
  volume = {54},
  number = {6},
  pages = {14-19},
  year = {2021},
  note = {7th IFAC Conference on Nonlinear Model Predictive Control NMPC 2021},
  issn = {2405-8963},
  doi = {https://doi.org/10.1016/j.ifacol.2021.08.518},
  url = {https://www.sciencedirect.com/science/article/pii/S2405896321012933},
  author = {Drgoňa, Ján and Tuor, Aaron and Skomski, Elliott and Vasisht, Soumya and Vrabie, Draguna},
  keywords = {constrained deep learning, neural state space models, system identification, differentiable predictive control, explicit nonlinear MPC},
  abs = {We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from the observed system’s dynamics, and the neural control law is optimized offline by leveraging the differentiable closed-loop system model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law’s parameters directly by backpropagating economic MPC loss through the learned system model. The control performance of the proposed DPC method is demonstrated in simulation using learned model of multi-zone building thermal dynamics.}
}

2020

Cutting the Deployment Costs of Physics-Based MPC in Buildings by Simulation-Based Imitation Learning

Ján Drgona, Lieve Helsen, and Draguna L. Vrabie

Oct 2020

Bib HTML Code

@proceedings{DrgonaDSCC2020,
  author = {Drgona, Ján and Helsen, Lieve and Vrabie, Draguna L.},
  title = {{Cutting the Deployment Costs of Physics-Based MPC in Buildings by Simulation-Based Imitation Learning}},
  volume = {Volume 1: Adaptive/Intelligent Sys. Control; Driver Assistance/Autonomous Tech.; Control Design Methods; Nonlinear Control; Robotics; Assistive/Rehabilitation Devices; Biomedical/Neural Systems; Building Energy Systems; Connected Vehicle Systems; Control/Estimation of Energy Systems; Control Apps.; Smart Buildings/Microgrids; Education; Human-Robot Systems; Soft Mechatronics/Robotic Components/Systems; Energy/Power Systems; Energy Storage; Estimation/Identification; Vehicle Efficiency/Emissions},
  series = {Dynamic Systems and Control Conference},
  pages = {V001T09A001},
  year = {2020},
  month = oct,
  abs = {{It has been shown that model predictive control (MPC) is a promising solution for energy-efficient building operations. However, the deployment of MPC in a large portion of the building stock has not been possible partially because of high installation costs. Every building is unique and requires a tailored MPC solution. The best performing solutions are often based on physics-based modeling, which is, however, computationally expensive and requires dedicated software. A promising direction that tackles this problem is to train a neural network-based optimal control policy to imitate the behavior of physics-based MPC from the simulation data generated offline. The neural networks give control actions that closely approximate those produced by physics-based MPC, but with a fraction of the computational and memory requirements and without the need for licensed software. The main advantage of the proposed approach stems from simple evaluation at execution time, leading to low computational foot-prints and easy deployment on embedded HW platforms. In the case study, we present the energy savings potential of physics-based MPC applied to an office building in Belgium. We demonstrate how neural network approximators can be used to cut the implementation and maintenance costs of MPC deployment without compromising performance. We also critically assess the presented approach by pointing out the remaining challenges and open research questions.}},
  doi = {10.1115/DSCC2020-3118},
  url = {https://doi.org/10.1115/DSCC2020-3118},
  eprint = {https://asmedigitalcollection.asme.org/DSCC/proceedings-pdf/DSCC2020/84270/V001T09A001/6622197/v001t09a001-dscc2020-3118.pdf}
}

All you need to know about model predictive control for buildings

Ján Drgoňa, Javier Arroyo, Iago Cupeiro Figueroa, and 8 more authors

Annual Reviews in Control, Sep 2020

Bib HTML

@article{DRGONA2020190,
  title = {All you need to know about model predictive control for buildings},
  journal = {Annual Reviews in Control},
  volume = {50},
  month = sep,
  pages = {190-232},
  year = {2020},
  issn = {1367-5788},
  doi = {https://doi.org/10.1016/j.arcontrol.2020.09.001},
  url = {https://www.sciencedirect.com/science/article/pii/S1367578820300584},
  author = {Drgoňa, Ján and Arroyo, Javier and {Cupeiro Figueroa}, Iago and Blum, David and Arendt, Krzysztof and Kim, Donghun and Ollé, Enric Perarnau and Oravec, Juraj and Wetter, Michael and Vrabie, Draguna L. and Helsen, Lieve},
  keywords = {Model predictive control, Building climate control, MPC formulation, MPC software tools, MPC implementation},
  abs = {It has been proven that advanced building control, like model predictive control (MPC), can notably reduce the energy use and mitigate greenhouse gas emissions. However, despite intensive research efforts, the practical applications are still in the early stages. There is a growing need for multidisciplinary education on advanced control methods in the built environment to be accessible for a broad range of researchers and practitioners with different engineering backgrounds. This paper provides a unified framework for model predictive building control technology with focus on the real-world applications. From a theoretical point of view, this paper presents an overview of MPC formulations for building control, modeling paradigms and model types, together with algorithms necessary for real-life implementation. The paper categorizes the most notable MPC problem classes, links them with corresponding solution techniques, and provides an overview of methods for mitigation of the uncertainties for increased performance and robustness of MPC. From a practical point of view, this paper delivers an elaborate classification of the most important modeling, co-simulation, optimal control design, and optimization techniques, tools, and solvers suitable to tackle the MPC problems in the context of building climate control. On top of this, the paper presents the essential components of a practical implementation of MPC such as different control architectures and nuances of communication infrastructures within supervisory control and data acquisition (SCADA) systems. The paper draws practical guidelines with a generic workflow for implementation of MPC in real buildings aimed for contemporary adopters of this technology. Finally, the importance of standardized performance assessment and methodology for comparison of different building control algorithms is discussed.}
}