{"arxiv":1,"corr_author":"1","type":"conference","title":"Strategy complexity of finite-horizon Markov decision processes and simple stochastic games","OA_type":"green","oa":1,"status":"public","OA_place":"repository","volume":7721,"author":[{"id":"2E5DCA20-F248-11E8-B48F-1D18A9856A87","full_name":"Chatterjee, Krishnendu","first_name":"Krishnendu","last_name":"Chatterjee","orcid":"0000-0002-4561-241X"},{"id":"3B699956-F248-11E8-B48F-1D18A9856A87","full_name":"Ibsen-Jensen, Rasmus","first_name":"Rasmus","orcid":"0000-0003-4783-0389","last_name":"Ibsen-Jensen"}],"project":[{"_id":"2584A770-B435-11E9-9278-68D0E5697425","call_identifier":"FWF","name":"Modern Graph Algorithmic Techniques in Formal Verification","grant_number":"P 23499-N23"},{"_id":"25863FF4-B435-11E9-9278-68D0E5697425","call_identifier":"FWF","grant_number":"S11407","name":"Game Theory"},{"grant_number":"279307","name":"Quantitative Graph Games: Theory and Applications","call_identifier":"FP7","_id":"2581B60A-B435-11E9-9278-68D0E5697425"}],"date_updated":"2025-09-23T09:24:13Z","day":"17","doi":"10.1007/978-3-642-36046-6_11","publisher":"Springer Nature","publication_status":"published","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","scopus_import":"1","date_created":"2025-07-10T14:08:49Z","date_published":"2013-01-17T00:00:00Z","quality_controlled":"1","acknowledgement":"Work of the second author supported by the Sino-Danish Center for the Theory of Interactive Computation, funded by the Danish National Research Foundation and the National Science Foundation of China (under the grant 61061130540). The second author acknowledge support from the Center for research in the Foundations of Electronic Markets (CFEM), supported by the Danish Strategic Research Council. The first author was supported by FWF Grant No P 23499-N23, FWF NFN Grant No S11407-N23 (RiSE), ERC Start grant (279307: Graph Games), and Microsoft faculty fellows award.","publication":"Mathematical and Engineering Methods in Computer Science","oa_version":"Preprint","external_id":{"arxiv":["1209.3617"]},"department":[{"_id":"KrCh"}],"ec_funded":1,"article_processing_charge":"No","abstract":[{"lang":"eng","text":"Markov decision processes (MDPs) and simple stochastic games (SSGs) provide a rich mathematical framework to study many important problems related to probabilistic systems. MDPs and SSGs with finite-horizon objectives, where the goal is to maximize the probability to reach a target state in a given finite time, is a classical and well-studied problem. In this work we consider the strategy complexity of finite-horizon MDPs and SSGs. We show that for all ε > 0, the natural class of counter-based strategies require at most log log/(1/ε) + n + 1 memory states, and memory of size Omega(log log(1/ε) + n) is required, for ε-optimality, where n is the number of states of the MDP (resp. SSG). Thus our bounds are asymptotically optimal. We then study the periodic property of optimal strategies, and show a sub-exponential lower bound on the period for optimal strategies."}],"_id":"19995","year":"2013","publication_identifier":{"issn":["0302-9743"],"isbn":["9783642360442"],"eissn":["1611-3349"],"eisbn":["9783642360466"]},"conference":{"end_date":"2012-10-28","name":"MEMICS: Mathematical and Engineering Methods in Computer Science","location":"Znojmo, Czech Republic","start_date":"2012-10-25"},"language":[{"iso":"eng"}],"citation":{"apa":"Chatterjee, K., &#38; Ibsen-Jensen, R. (2013). Strategy complexity of finite-horizon Markov decision processes and simple stochastic games. In <i>Mathematical and Engineering Methods in Computer Science</i> (Vol. 7721, pp. 106–117). Znojmo, Czech Republic: Springer Nature. <a href=\"https://doi.org/10.1007/978-3-642-36046-6_11\">https://doi.org/10.1007/978-3-642-36046-6_11</a>","ieee":"K. Chatterjee and R. Ibsen-Jensen, “Strategy complexity of finite-horizon Markov decision processes and simple stochastic games,” in <i>Mathematical and Engineering Methods in Computer Science</i>, Znojmo, Czech Republic, 2013, vol. 7721, pp. 106–117.","ama":"Chatterjee K, Ibsen-Jensen R. Strategy complexity of finite-horizon Markov decision processes and simple stochastic games. In: <i>Mathematical and Engineering Methods in Computer Science</i>. Vol 7721. Springer Nature; 2013:106-117. doi:<a href=\"https://doi.org/10.1007/978-3-642-36046-6_11\">10.1007/978-3-642-36046-6_11</a>","short":"K. Chatterjee, R. Ibsen-Jensen, in:, Mathematical and Engineering Methods in Computer Science, Springer Nature, 2013, pp. 106–117.","mla":"Chatterjee, Krishnendu, and Rasmus Ibsen-Jensen. “Strategy Complexity of Finite-Horizon Markov Decision Processes and Simple Stochastic Games.” <i>Mathematical and Engineering Methods in Computer Science</i>, vol. 7721, Springer Nature, 2013, pp. 106–17, doi:<a href=\"https://doi.org/10.1007/978-3-642-36046-6_11\">10.1007/978-3-642-36046-6_11</a>.","chicago":"Chatterjee, Krishnendu, and Rasmus Ibsen-Jensen. “Strategy Complexity of Finite-Horizon Markov Decision Processes and Simple Stochastic Games.” In <i>Mathematical and Engineering Methods in Computer Science</i>, 7721:106–17. Springer Nature, 2013. <a href=\"https://doi.org/10.1007/978-3-642-36046-6_11\">https://doi.org/10.1007/978-3-642-36046-6_11</a>.","ista":"Chatterjee K, Ibsen-Jensen R. 2013. Strategy complexity of finite-horizon Markov decision processes and simple stochastic games. Mathematical and Engineering Methods in Computer Science. MEMICS: Mathematical and Engineering Methods in Computer Science, LNCS, vol. 7721, 106–117."},"main_file_link":[{"url":"https://doi.org/10.48550/arXiv.1209.3617","open_access":"1"}],"alternative_title":["LNCS"],"page":"106-117","month":"01","intvolume":"      7721"}