{"acknowledgement":"This work was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 10103441, the ERC Starting Grant DEUCE (101077178) and the DFG through the Cluster of Excellence EXC 2050/1 (CeTI, project ID 390696704, as part of Germany’s Excellence Strategy) and the DFG grant 389792660 as part of TRR 248 (see https://perspicuous-computing.science).","ec_funded":1,"publisher":"Springer Nature","intvolume":"     16143","conference":{"end_date":"2025-08-28","location":"Aarhus, Denmark","name":"QEST-FORMATS: International Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems","start_date":"2025-08-26"},"volume":16143,"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","publication_status":"published","project":[{"call_identifier":"H2020","name":"IST-BRIDGE: International postdoctoral program","grant_number":"101034413","_id":"fc2ed2f7-9c52-11eb-aca3-c01059dda49c"}],"publication_identifier":{"isbn":["9783032057914"],"eissn":["1611-3349"],"issn":["0302-9743"]},"doi":"10.1007/978-3-032-05792-1_11","article_processing_charge":"No","_id":"20610","type":"conference","date_created":"2025-11-09T23:01:34Z","date_published":"2025-10-02T00:00:00Z","day":"02","oa_version":"None","quality_controlled":"1","alternative_title":["LNCS"],"author":[{"last_name":"Meggendorfer","id":"b21b0c15-30a2-11eb-80dc-f13ca25802e1","orcid":"0000-0002-1712-2165","full_name":"Meggendorfer, Tobias","first_name":"Tobias"},{"first_name":"Maximilian","id":"02ab0197-cc70-11ed-ab61-918e71f56881","last_name":"Weininger","orcid":"0000-0002-0163-2152","full_name":"Weininger, Maximilian"},{"full_name":"Wienhöft, Patrick","last_name":"Wienhöft","first_name":"Patrick"}],"abstract":[{"text":"Markov decision processes (MDPs) are a fundamental model of decision making which exhibit non-deterministic choice as well as probabilistic uncertainty. Traditionally, verification assumes exact knowledge of the probabilities that govern the behaviour of an MDP. However, this assumption often is unrealistic, e.g. when modelling cyber-physical systems or biological processes. There, we can employ statistical model checking (SMC) to obtain an estimate of the MDP’s value (e.g. the maximal probability of reaching a goal state) that is close to the true value with high confidence (probably approximately correct). Model-based SMC algorithms sample the MDP and build a model of it by estimating all transition probabilities, essentially for every transition answering the question: “What are the odds?” However, so far the statistical methods employed by state-of-the-art SMC verification algorithms are quite naive or even compromise the correctness guarantees.\r\n\r\nOur first contribution is to survey, categorize, and analyse statistical methods, identifying those few that are most efficient and that provide suitable guarantees for the verification setting. Secondly, we propose improvements that exploit structural knowledge of the MDP. Both contributions generalize to many types of problem statements as they are largely independent of the setting. Moreover, our experimental evaluation shows that they lead to significant gains, reducing the number of samples that an SMC algorithm has to collect by up to two orders of magnitude.","lang":"eng"}],"title":"What are the odds? Improving statistical model checking of Markov decision processes","date_updated":"2025-11-10T08:06:27Z","page":"195-218","publication":"Second International Joint Conference on QEST+FORMATS","department":[{"_id":"KrCh"}],"month":"10","year":"2025","citation":{"apa":"Meggendorfer, T., Weininger, M., &#38; Wienhöft, P. (2025). What are the odds? Improving statistical model checking of Markov decision processes. In <i>Second International Joint Conference on QEST+FORMATS</i> (Vol. 16143, pp. 195–218). Aarhus, Denmark: Springer Nature. <a href=\"https://doi.org/10.1007/978-3-032-05792-1_11\">https://doi.org/10.1007/978-3-032-05792-1_11</a>","chicago":"Meggendorfer, Tobias, Maximilian Weininger, and Patrick Wienhöft. “What Are the Odds? Improving Statistical Model Checking of Markov Decision Processes.” In <i>Second International Joint Conference on QEST+FORMATS</i>, 16143:195–218. Springer Nature, 2025. <a href=\"https://doi.org/10.1007/978-3-032-05792-1_11\">https://doi.org/10.1007/978-3-032-05792-1_11</a>.","mla":"Meggendorfer, Tobias, et al. “What Are the Odds? Improving Statistical Model Checking of Markov Decision Processes.” <i>Second International Joint Conference on QEST+FORMATS</i>, vol. 16143, Springer Nature, 2025, pp. 195–218, doi:<a href=\"https://doi.org/10.1007/978-3-032-05792-1_11\">10.1007/978-3-032-05792-1_11</a>.","ama":"Meggendorfer T, Weininger M, Wienhöft P. What are the odds? Improving statistical model checking of Markov decision processes. In: <i>Second International Joint Conference on QEST+FORMATS</i>. Vol 16143. Springer Nature; 2025:195-218. doi:<a href=\"https://doi.org/10.1007/978-3-032-05792-1_11\">10.1007/978-3-032-05792-1_11</a>","short":"T. Meggendorfer, M. Weininger, P. Wienhöft, in:, Second International Joint Conference on QEST+FORMATS, Springer Nature, 2025, pp. 195–218.","ieee":"T. Meggendorfer, M. Weininger, and P. Wienhöft, “What are the odds? Improving statistical model checking of Markov decision processes,” in <i>Second International Joint Conference on QEST+FORMATS</i>, Aarhus, Denmark, 2025, vol. 16143, pp. 195–218.","ista":"Meggendorfer T, Weininger M, Wienhöft P. 2025. What are the odds? Improving statistical model checking of Markov decision processes. Second International Joint Conference on QEST+FORMATS. QEST-FORMATS: International Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems, LNCS, vol. 16143, 195–218."},"language":[{"iso":"eng"}],"status":"public","scopus_import":"1"}