Stopping criteria for value iteration on stochastic games with quantitative objectives

Kretinsky J, Meggendorfer T, Weininger M. 2023. Stopping criteria for value iteration on stochastic games with quantitative objectives. 38th Annual ACM/IEEE Symposium on Logic in Computer Science. LICS: Symposium on Logic in Computer Science vol. 2023.

Download (ext.)

Conference Paper | Published | English

Scopus indexed
Department
Abstract
A classic solution technique for Markov decision processes (MDP) and stochastic games (SG) is value iteration (VI). Due to its good practical performance, this approximative approach is typically preferred over exact techniques, even though no practical bounds on the imprecision of the result could be given until recently. As a consequence, even the most used model checkers could return arbitrarily wrong results. Over the past decade, different works derived stopping criteria, indicating when the precision reaches the desired level, for various settings, in particular MDP with reachability, total reward, and mean payoff, and SG with reachability.In this paper, we provide the first stopping criteria for VI on SG with total reward and mean payoff, yielding the first anytime algorithms in these settings. To this end, we provide the solution in two flavours: First through a reduction to the MDP case and second directly on SG. The former is simpler and automatically utilizes any advances on MDP. The latter allows for more local computations, heading towards better practical efficiency.Our solution unifies the previously mentioned approaches for MDP and SG and their underlying ideas. To achieve this, we isolate objective-specific subroutines as well as identify objective-independent concepts. These structural concepts, while surprisingly simple, form the very essence of the unified solution.
Publishing Year
Date Published
2023-07-01
Proceedings Title
38th Annual ACM/IEEE Symposium on Logic in Computer Science
Acknowledgement
This research was funded in part by DFG projects 383882557 “SUV” and 427755713 “GOPro”.
Volume
2023
Conference
LICS: Symposium on Logic in Computer Science
Conference Location
Boston, MA, United States
Conference Date
2023-06-26 – 2023-06-29
ISSN
IST-REx-ID

Cite this

Kretinsky J, Meggendorfer T, Weininger M. Stopping criteria for value iteration on stochastic games with quantitative objectives. In: 38th Annual ACM/IEEE Symposium on Logic in Computer Science. Vol 2023. Institute of Electrical and Electronics Engineers; 2023. doi:10.1109/LICS56636.2023.10175771
Kretinsky, J., Meggendorfer, T., & Weininger, M. (2023). Stopping criteria for value iteration on stochastic games with quantitative objectives. In 38th Annual ACM/IEEE Symposium on Logic in Computer Science (Vol. 2023). Boston, MA, United States: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/LICS56636.2023.10175771
Kretinsky, Jan, Tobias Meggendorfer, and Maximilian Weininger. “Stopping Criteria for Value Iteration on Stochastic Games with Quantitative Objectives.” In 38th Annual ACM/IEEE Symposium on Logic in Computer Science, Vol. 2023. Institute of Electrical and Electronics Engineers, 2023. https://doi.org/10.1109/LICS56636.2023.10175771.
J. Kretinsky, T. Meggendorfer, and M. Weininger, “Stopping criteria for value iteration on stochastic games with quantitative objectives,” in 38th Annual ACM/IEEE Symposium on Logic in Computer Science, Boston, MA, United States, 2023, vol. 2023.
Kretinsky J, Meggendorfer T, Weininger M. 2023. Stopping criteria for value iteration on stochastic games with quantitative objectives. 38th Annual ACM/IEEE Symposium on Logic in Computer Science. LICS: Symposium on Logic in Computer Science vol. 2023.
Kretinsky, Jan, et al. “Stopping Criteria for Value Iteration on Stochastic Games with Quantitative Objectives.” 38th Annual ACM/IEEE Symposium on Logic in Computer Science, vol. 2023, Institute of Electrical and Electronics Engineers, 2023, doi:10.1109/LICS56636.2023.10175771.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data ISTA Research Explorer

Web of Science

View record in Web of Science®

Sources

arXiv 2304.09930

Search this title in

Google Scholar
ISBN Search