{"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","OA_place":"publisher","oa_version":"Published Version","file":[{"file_size":416814,"creator":"dernst","access_level":"open_access","date_updated":"2024-12-09T08:38:48Z","file_id":"18633","checksum":"b3315c74ce18ce0a30ed33d8c9972992","date_created":"2024-12-09T08:38:48Z","file_name":"2024_LMCS_Chatterjee.pdf","content_type":"application/pdf","success":1,"relation":"main_file"}],"_id":"18630","issue":"4","article_processing_charge":"Yes","citation":{"chicago":"Chatterjee, Krishnendu, and Laurent Doyen. “Stochastic Processes with Expected Stopping Time.” <i>Logical Methods in Computer Science</i>. EPI Sciences, 2024. <a href=\"https://doi.org/10.46298/lmcs-20(4:11)2024\">https://doi.org/10.46298/lmcs-20(4:11)2024</a>.","apa":"Chatterjee, K., &#38; Doyen, L. (2024). Stochastic processes with expected stopping time. <i>Logical Methods in Computer Science</i>. EPI Sciences. <a href=\"https://doi.org/10.46298/lmcs-20(4:11)2024\">https://doi.org/10.46298/lmcs-20(4:11)2024</a>","mla":"Chatterjee, Krishnendu, and Laurent Doyen. “Stochastic Processes with Expected Stopping Time.” <i>Logical Methods in Computer Science</i>, vol. 20, no. 4, EPI Sciences, 2024, p. 11:1-11:34, doi:<a href=\"https://doi.org/10.46298/lmcs-20(4:11)2024\">10.46298/lmcs-20(4:11)2024</a>.","ama":"Chatterjee K, Doyen L. Stochastic processes with expected stopping time. <i>Logical Methods in Computer Science</i>. 2024;20(4):11:1-11:34. doi:<a href=\"https://doi.org/10.46298/lmcs-20(4:11)2024\">10.46298/lmcs-20(4:11)2024</a>","ieee":"K. Chatterjee and L. Doyen, “Stochastic processes with expected stopping time,” <i>Logical Methods in Computer Science</i>, vol. 20, no. 4. EPI Sciences, p. 11:1-11:34, 2024.","short":"K. Chatterjee, L. Doyen, Logical Methods in Computer Science 20 (2024) 11:1-11:34.","ista":"Chatterjee K, Doyen L. 2024. Stochastic processes with expected stopping time. Logical Methods in Computer Science. 20(4), 11:1-11:34."},"arxiv":1,"article_type":"original","license":"https://creativecommons.org/licenses/by/4.0/","scopus_import":"1","ec_funded":1,"date_updated":"2024-12-09T08:42:58Z","date_created":"2024-12-08T23:01:56Z","type":"journal_article","has_accepted_license":"1","publication_identifier":{"eissn":["1860-5974"]},"author":[{"id":"2E5DCA20-F248-11E8-B48F-1D18A9856A87","last_name":"Chatterjee","full_name":"Chatterjee, Krishnendu","orcid":"0000-0002-4561-241X","first_name":"Krishnendu"},{"last_name":"Doyen","first_name":"Laurent","full_name":"Doyen, Laurent"}],"page":"11:1-11:34","corr_author":"1","publisher":"EPI Sciences","title":"Stochastic processes with expected stopping time","date_published":"2024-11-12T00:00:00Z","alternative_title":["LMCS"],"publication":"Logical Methods in Computer Science","project":[{"name":"Formal Methods for Stochastic Models: Algorithms and Applications","grant_number":"863818","call_identifier":"H2020","_id":"0599E47C-7A3F-11EA-A408-12923DDC885E"}],"publication_status":"published","DOAJ_listed":"1","language":[{"iso":"eng"}],"day":"12","department":[{"_id":"KrCh"}],"status":"public","file_date_updated":"2024-12-09T08:38:48Z","abstract":[{"text":"Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical optimization criterion is the maximal expected total reward where the MDP stops after T steps, which can be computed by a simple dynamic programming algorithm. We consider a natural generalization of the problem where the stopping times can be chosen according to a probability distribution, such that the expected stopping time is T, to optimize the expected total reward. Quite surprisingly we establish inter-reducibility of the expected stopping-time problem for Markov chains with the Positivity problem (which is related to the well-known Skolem problem), for which establishing either decidability or undecidability would be a major breakthrough. Given the hardness of the exact problem, we consider the approximate version of the problem: we show that it can be solved in exponential time for Markov chains and in exponential space for MDPs.","lang":"eng"}],"quality_controlled":"1","intvolume":"        20","oa":1,"related_material":{"record":[{"status":"public","relation":"earlier_version","id":"10004"}]},"ddc":["000"],"doi":"10.46298/lmcs-20(4:11)2024","acknowledgement":"The authors are grateful to the anonymous reviewers of LICS 2021 and of a previous version of this paper for insightful comments that helped improving the presentation. The research presented in this paper was partially supported by the grant ERC CoG 863818 (ForM-SMArt).","tmp":{"name":"Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)","legal_code_url":"https://creativecommons.org/licenses/by/4.0/legalcode","image":"/images/cc_by.png","short":"CC BY (4.0)"},"month":"11","OA_type":"gold","volume":20,"year":"2024","external_id":{"arxiv":["2104.07278"]}}