In this project we developed a range of scientific machine learning (SciML) methods for data-driven surrogate modeling and control of robotics platforms with embedded HW devices. Specifically, we employ differentiable programming that allow us to systematically combine physics-based models with black-box function approximators to obtain physics-informed dynamical system models of complex robotics platforms. The differentiability of such models allows us to use them in differentiable predictive control (DPC) method developed by our team. In DPC, we leverage the system dynamics surrogates as generative models to train neural control policies in closed-loop simulations. This offline model-based policy optimization approach is inspired by traditional model predictive control (MPC) and allows for systematic handling of state and input constraints for guaranteed performance and safety. For the implementation of these methods, we leverage our Pytorch-based SciML library Neuromancer developed by our team at PNNL. For more details, see the associated project.
Differentiable predictive control deployment on embedded HW devices
Method: We are concerned with the problem of deploying complex nonlinear control policies on HW devices with limited computational and memory resources. To address this issue, we present DPC as a deep learning-based alternative to the explicit MPC. In DPC, we trained a neural state-space model from time-series measurements of the real system and used it to train the neural control policy via stochastic gradient descent by differentiating the MPC loss function through the closed-loop system dynamics model. The DPC method learns model-based control policies with state and input constraints, while supporting time-varying references and constraints.
Embedded HW deployment: In this project, we collaborated with a team at the Slovak University of Technology, in Bratislava, who deployed the trained SciML models and DPC policies on embedded HW platform. Specifically, we demonstrated sim2real transfer and deploy the trained policy on a Raspberry-Pi platform. We experimentally demonstrate that it is possible to train constrained control policies purely based on the measurements of the unknown nonlinear system. We compare the control performance of the DPC method against explicit MPC and report efficiency gains in online computational demands, memory requirements, policy complexity, and construction time. For more details, see our paper.
Differentiable programming for modeling autonomous underwater vehicles
Method: Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models require a large number of observations and lack operational guarantees for safety-critical systems. SciML models leveraging prior knowledge of the system dynamics have potential to provide reliable models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work, present control-oriented parametric SciML models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations (UDEs) to construct data-driven control-oriented blackbox and graybox representations of the AUV dynamics. For more details, see our paper.
Real-world data: In this project, we collaborated with University of Washington’s Applied Physics Laboratory (UW APL) team who collected experimental data using their IVER AUV platform. The collected datasets have been used to train and validate the SciML surrogate models developed by our team.
- Soumya Vasisht
- Wenceslao Shaw Cortez
- Aaron Tuor
- James Koch
- Draguna Vrabie
- Peter Brodsky (UW APL)
- Greg Okopal (UW APL)
- Joseph Sammartino (UW APL)
- Ján Drgoňa
Differentiable predictive control of a quadcopter
In this project we develop DPC method with safety guarantees to train neural policies to control a 17 dimensional quadcopter system. The quadrotor has 3 simulations, one differentiable model in PyTorch to allow for DPC policy optimization, one in CasADi to allow for MPC, and one in Mujoco to allow for a validation environment.
This is an active project by John Viljoen. Stay tuned for the updates.
This research was supported by the Mathematics for Artificial Reasoning in Science (MARS) initiative via the Laboratory Directed Research and Development (LDRD) investments at Pacific Northwest National Laboratory (PNNL). PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE) by Battelle Memorial Institute under Contract No. DE-AC05-76RL0-1830.
- Domain-aware Control-oriented Neural Models for Autonomous Underwater VehiclesIFAC-PapersOnLine, Feb 202312th IFAC Symposium on Nonlinear Control Systems NOLCOS 2022
- Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear systemsJournal of Process Control, Aug 2022