Earlier Version
Learning control policies for stochastic systems with reach-avoid guarantees
Zikelic D, Lechner M, Henzinger TA, Chatterjee K. Learning control policies for stochastic systems with reach-avoid guarantees. arXiv, 10.48550/ARXIV.2210.05308.
Download (ext.)
https://arxiv.org/abs/2210.05308
[Preprint]
Preprint
| Submitted
| English
Author
Corresponding author has ISTA affiliation
Department
Grant
Abstract
We study the problem of learning controllers for discrete-time non-linear stochastic dynamical systems with formal reach-avoid guarantees. This work presents the first method for providing formal reach-avoid guarantees, which combine and generalize stability and safety guarantees, with a tolerable probability threshold $p\in[0,1]$ over the infinite time horizon. Our method leverages advances in machine learning literature and it represents formal certificates as neural networks. In particular, we learn a certificate in the form of a reach-avoid supermartingale (RASM), a novel notion that we introduce in this work. Our RASMs provide reachability and avoidance guarantees by imposing constraints on what can be viewed as a stochastic extension of level sets of Lyapunov functions for deterministic systems. Our approach solves several important problems -- it can be used to learn a control policy from scratch, to verify a reach-avoid specification for a fixed control policy, or to fine-tune a pre-trained policy if it does not satisfy the reach-avoid specification. We validate our approach on $3$ stochastic non-linear reinforcement learning tasks.
Publishing Year
Date Published
2022-11-29
Journal Title
arXiv
IST-REx-ID
Cite this
Zikelic D, Lechner M, Henzinger TA, Chatterjee K. Learning control policies for stochastic systems with reach-avoid guarantees. arXiv. doi:10.48550/ARXIV.2210.05308
Zikelic, D., Lechner, M., Henzinger, T. A., & Chatterjee, K. (n.d.). Learning control policies for stochastic systems with reach-avoid guarantees. arXiv. https://doi.org/10.48550/ARXIV.2210.05308
Zikelic, Dorde, Mathias Lechner, Thomas A Henzinger, and Krishnendu Chatterjee. “Learning Control Policies for Stochastic Systems with Reach-Avoid Guarantees.” ArXiv, n.d. https://doi.org/10.48550/ARXIV.2210.05308.
D. Zikelic, M. Lechner, T. A. Henzinger, and K. Chatterjee, “Learning control policies for stochastic systems with reach-avoid guarantees,” arXiv. .
Zikelic D, Lechner M, Henzinger TA, Chatterjee K. Learning control policies for stochastic systems with reach-avoid guarantees. arXiv, 10.48550/ARXIV.2210.05308.
Zikelic, Dorde, et al. “Learning Control Policies for Stochastic Systems with Reach-Avoid Guarantees.” ArXiv, doi:10.48550/ARXIV.2210.05308.
All files available under the following license(s):
Creative Commons Attribution-ShareAlike 4.0 International Public License (CC BY-SA 4.0):
Link(s) to Main File(s)
Access Level
Open Access
Material in ISTA:
Dissertation containing ISTA record
Later Version
Export
Marked PublicationsOpen Data ISTA Research Explorer
Sources
arXiv 2210.05308