---
OA_place: publisher
OA_type: hybrid
_id: '21661'
abstract:
- lang: eng
text: Model checking undiscounted reachability and expected-reward properties on
Markov decision processes (MDPs) are key for the verification of systems that
act under uncertainty. Popular algorithms are policy iteration and variants of
value iteration; in tool competitions, most participants rely on the latter. These
algorithms generally need worst-case exponential time. However, the problem can
equally be formulated as a linear programme, solvable in polynomial time. In this
paper, we give a detailed overview of today’s state-of-the-art algorithms for
MDP model checking with a focus on performance and correctness. We highlight their
fundamental differences, and describe various optimizations and implementation
variants. We experimentally compare floating-point and exact-arithmetic implementations
of all algorithms on three benchmark sets using two probabilistic model checkers.
Our results show that (optimistic) value iteration is a sensible default, but
other algorithms are preferable in specific settings. This paper thereby provides
a guide for MDP verification practitioners—tool builders and users alike.
acknowledgement: "This research was funded by the European Union’s Horizon 2020 research
and innovation programme under Marie Skłodowska-Curie grant agreements 101008233
(MISSION)\r\nand 101034413 (IST-BRIDGE), by the Interreg North Sea project STORM_SAFE,
by a KI-Starter grant from the Ministerium für Kultur und Wissenschaft NRW, by NWO
VENI grant no. 639.021.754, and by NWO VIDI grant VI.Vidi.223.110 (TruSTy). Experiments
were performed with computing resources granted by RWTH Aachen University under
project rwth1632."
article_processing_charge: Yes (in subscription journal)
article_type: original
author:
- first_name: Arnd
full_name: Hartmanns, Arnd
last_name: Hartmanns
- first_name: Sebastian
full_name: Junges, Sebastian
last_name: Junges
- first_name: Tim
full_name: Quatmann, Tim
last_name: Quatmann
- first_name: Maximilian
full_name: Weininger, Maximilian
id: 02ab0197-cc70-11ed-ab61-918e71f56881
last_name: Weininger
orcid: 0000-0002-0163-2152
citation:
ama: Hartmanns A, Junges S, Quatmann T, Weininger M. The revised practitioner’s
guide to MDP model checking algorithms. International Journal on Software Tools
for Technology Transfer. 2026. doi:10.1007/s10009-026-00848-y
apa: Hartmanns, A., Junges, S., Quatmann, T., & Weininger, M. (2026). The revised
practitioner’s guide to MDP model checking algorithms. International Journal
on Software Tools for Technology Transfer. Springer Nature. https://doi.org/10.1007/s10009-026-00848-y
chicago: Hartmanns, Arnd, Sebastian Junges, Tim Quatmann, and Maximilian Weininger.
“The Revised Practitioner’s Guide to MDP Model Checking Algorithms.” International
Journal on Software Tools for Technology Transfer. Springer Nature, 2026.
https://doi.org/10.1007/s10009-026-00848-y.
ieee: A. Hartmanns, S. Junges, T. Quatmann, and M. Weininger, “The revised practitioner’s
guide to MDP model checking algorithms,” International Journal on Software
Tools for Technology Transfer. Springer Nature, 2026.
ista: Hartmanns A, Junges S, Quatmann T, Weininger M. 2026. The revised practitioner’s
guide to MDP model checking algorithms. International Journal on Software Tools
for Technology Transfer.
mla: Hartmanns, Arnd, et al. “The Revised Practitioner’s Guide to MDP Model Checking
Algorithms.” International Journal on Software Tools for Technology Transfer,
Springer Nature, 2026, doi:10.1007/s10009-026-00848-y.
short: A. Hartmanns, S. Junges, T. Quatmann, M. Weininger, International Journal
on Software Tools for Technology Transfer (2026).
date_created: 2026-04-05T22:01:32Z
date_published: 2026-03-09T00:00:00Z
date_updated: 2026-04-07T09:52:54Z
day: '09'
ddc:
- '000'
department:
- _id: KrCh
doi: 10.1007/s10009-026-00848-y
ec_funded: 1
has_accepted_license: '1'
keyword:
- Quantitative model checking
- Markov decision process
- Linear programming
- Value iteration
- Policy iteration
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.1007/s10009-026-00848-y
month: '03'
oa: 1
oa_version: Published Version
project:
- _id: fc2ed2f7-9c52-11eb-aca3-c01059dda49c
call_identifier: H2020
grant_number: '101034413'
name: 'IST-BRIDGE: International postdoctoral program'
publication: International Journal on Software Tools for Technology Transfer
publication_identifier:
eissn:
- 1433-2787
issn:
- 1433-2779
publication_status: epub_ahead
publisher: Springer Nature
quality_controlled: '1'
related_material:
record:
- id: '21668'
relation: software
status: public
scopus_import: '1'
status: public
title: The revised practitioner’s guide to MDP model checking algorithms
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2026'
...
---
_id: '13234'
abstract:
- lang: eng
text: Neural-network classifiers achieve high accuracy when predicting the class
of an input that they were trained to identify. Maintaining this accuracy in dynamic
environments, where inputs frequently fall outside the fixed set of initially
known classes, remains a challenge. We consider the problem of monitoring the
classification decisions of neural networks in the presence of novel classes.
For this purpose, we generalize our recently proposed abstraction-based monitor
from binary output to real-valued quantitative output. This quantitative output
enables new applications, two of which we investigate in the paper. As our first
application, we introduce an algorithmic framework for active monitoring of a
neural network, which allows us to learn new classes dynamically and yet maintain
high monitoring performance. As our second application, we present an offline
procedure to retrain the neural network to improve the monitor’s detection performance
without deteriorating the network’s classification accuracy. Our experimental
evaluation demonstrates both the benefits of our active monitoring framework in
dynamic scenarios and the effectiveness of the retraining procedure.
acknowledgement: This work was supported in part by the ERC-2020-AdG 101020093, by
DIREC - Digital Research Centre Denmark, and by the Villum Investigator Grant S4OS.
article_processing_charge: Yes (in subscription journal)
article_type: original
arxiv: 1
author:
- first_name: Konstantin
full_name: Kueffner, Konstantin
id: 8121a2d0-dc85-11ea-9058-af578f3b4515
last_name: Kueffner
orcid: 0000-0001-8974-2542
- first_name: Anna
full_name: Lukina, Anna
id: CBA4D1A8-0FE8-11E9-BDE6-07BFE5697425
last_name: Lukina
- first_name: Christian
full_name: Schilling, Christian
id: 3A2F4DCE-F248-11E8-B48F-1D18A9856A87
last_name: Schilling
orcid: 0000-0003-3658-1065
- first_name: Thomas A
full_name: Henzinger, Thomas A
id: 40876CD8-F248-11E8-B48F-1D18A9856A87
last_name: Henzinger
orcid: 0000-0002-2985-7724
citation:
ama: 'Kueffner K, Lukina A, Schilling C, Henzinger TA. Into the unknown: Active
monitoring of neural networks (extended version). International Journal on
Software Tools for Technology Transfer. 2023;25:575-592. doi:10.1007/s10009-023-00711-4'
apa: 'Kueffner, K., Lukina, A., Schilling, C., & Henzinger, T. A. (2023). Into
the unknown: Active monitoring of neural networks (extended version). International
Journal on Software Tools for Technology Transfer. Springer Nature. https://doi.org/10.1007/s10009-023-00711-4'
chicago: 'Kueffner, Konstantin, Anna Lukina, Christian Schilling, and Thomas A Henzinger.
“Into the Unknown: Active Monitoring of Neural Networks (Extended Version).” International
Journal on Software Tools for Technology Transfer. Springer Nature, 2023.
https://doi.org/10.1007/s10009-023-00711-4.'
ieee: 'K. Kueffner, A. Lukina, C. Schilling, and T. A. Henzinger, “Into the unknown:
Active monitoring of neural networks (extended version),” International Journal
on Software Tools for Technology Transfer, vol. 25. Springer Nature, pp. 575–592,
2023.'
ista: 'Kueffner K, Lukina A, Schilling C, Henzinger TA. 2023. Into the unknown:
Active monitoring of neural networks (extended version). International Journal
on Software Tools for Technology Transfer. 25, 575–592.'
mla: 'Kueffner, Konstantin, et al. “Into the Unknown: Active Monitoring of Neural
Networks (Extended Version).” International Journal on Software Tools for Technology
Transfer, vol. 25, Springer Nature, 2023, pp. 575–92, doi:10.1007/s10009-023-00711-4.'
short: K. Kueffner, A. Lukina, C. Schilling, T.A. Henzinger, International Journal
on Software Tools for Technology Transfer 25 (2023) 575–592.
corr_author: '1'
date_created: 2023-07-16T22:01:11Z
date_published: 2023-08-01T00:00:00Z
date_updated: 2025-04-15T06:55:00Z
day: '01'
ddc:
- '000'
department:
- _id: ToHe
doi: 10.1007/s10009-023-00711-4
ec_funded: 1
external_id:
arxiv:
- '2009.06429'
isi:
- '001020160000001'
file:
- access_level: open_access
checksum: 3c4b347f39412a76872f9a6f30101f94
content_type: application/pdf
creator: dernst
date_created: 2024-01-30T12:06:07Z
date_updated: 2024-01-30T12:06:07Z
file_id: '14903'
file_name: 2023_JourSoftwareTools_Kueffner.pdf
file_size: 13387667
relation: main_file
success: 1
file_date_updated: 2024-01-30T12:06:07Z
has_accepted_license: '1'
intvolume: ' 25'
isi: 1
language:
- iso: eng
month: '08'
oa: 1
oa_version: Published Version
page: 575-592
project:
- _id: 62781420-2b32-11ec-9570-8d9b63373d4d
call_identifier: H2020
grant_number: '101020093'
name: Vigilant Algorithmic Monitoring of Software
publication: International Journal on Software Tools for Technology Transfer
publication_identifier:
eissn:
- 1433-2787
issn:
- 1433-2779
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
related_material:
record:
- id: '10206'
relation: shorter_version
status: public
scopus_import: '1'
status: public
title: 'Into the unknown: Active monitoring of neural networks (extended version)'
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 25
year: '2023'
...
---
_id: '10861'
abstract:
- lang: eng
text: We introduce in this paper AMT2.0, a tool for qualitative and quantitative
analysis of hybrid continuous and Boolean signals that combine numerical values
and discrete events. The evaluation of the signals is based on rich temporal specifications
expressed in extended signal temporal logic, which integrates timed regular expressions
within signal temporal logic. The tool features qualitative monitoring (property
satisfaction checking), trace diagnostics for explaining and justifying property
violations and specification-driven measurement of quantitative features of the
signal. We demonstrate the tool functionality on several running examples and
case studies, and evaluate its performance.
article_processing_charge: No
article_type: original
author:
- first_name: Dejan
full_name: Nickovic, Dejan
id: 41BCEE5C-F248-11E8-B48F-1D18A9856A87
last_name: Nickovic
- first_name: Olivier
full_name: Lebeltel, Olivier
last_name: Lebeltel
- first_name: Oded
full_name: Maler, Oded
last_name: Maler
- first_name: Thomas
full_name: Ferrere, Thomas
id: 40960E6E-F248-11E8-B48F-1D18A9856A87
last_name: Ferrere
orcid: 0000-0001-5199-3143
- first_name: Dogan
full_name: Ulus, Dogan
last_name: Ulus
citation:
ama: 'Nickovic D, Lebeltel O, Maler O, Ferrere T, Ulus D. AMT 2.0: Qualitative and
quantitative trace analysis with extended signal temporal logic. International
Journal on Software Tools for Technology Transfer. 2020;22(6):741-758. doi:10.1007/s10009-020-00582-z'
apa: 'Nickovic, D., Lebeltel, O., Maler, O., Ferrere, T., & Ulus, D. (2020).
AMT 2.0: Qualitative and quantitative trace analysis with extended signal temporal
logic. International Journal on Software Tools for Technology Transfer.
Springer Nature. https://doi.org/10.1007/s10009-020-00582-z'
chicago: 'Nickovic, Dejan, Olivier Lebeltel, Oded Maler, Thomas Ferrere, and Dogan
Ulus. “AMT 2.0: Qualitative and Quantitative Trace Analysis with Extended Signal
Temporal Logic.” International Journal on Software Tools for Technology Transfer.
Springer Nature, 2020. https://doi.org/10.1007/s10009-020-00582-z.'
ieee: 'D. Nickovic, O. Lebeltel, O. Maler, T. Ferrere, and D. Ulus, “AMT 2.0: Qualitative
and quantitative trace analysis with extended signal temporal logic,” International
Journal on Software Tools for Technology Transfer, vol. 22, no. 6. Springer
Nature, pp. 741–758, 2020.'
ista: 'Nickovic D, Lebeltel O, Maler O, Ferrere T, Ulus D. 2020. AMT 2.0: Qualitative
and quantitative trace analysis with extended signal temporal logic. International
Journal on Software Tools for Technology Transfer. 22(6), 741–758.'
mla: 'Nickovic, Dejan, et al. “AMT 2.0: Qualitative and Quantitative Trace Analysis
with Extended Signal Temporal Logic.” International Journal on Software Tools
for Technology Transfer, vol. 22, no. 6, Springer Nature, 2020, pp. 741–58,
doi:10.1007/s10009-020-00582-z.'
short: D. Nickovic, O. Lebeltel, O. Maler, T. Ferrere, D. Ulus, International Journal
on Software Tools for Technology Transfer 22 (2020) 741–758.
date_created: 2022-03-18T10:10:53Z
date_published: 2020-08-03T00:00:00Z
date_updated: 2024-10-09T20:58:18Z
day: '03'
department:
- _id: ToHe
doi: 10.1007/s10009-020-00582-z
external_id:
isi:
- '000555398600001'
intvolume: ' 22'
isi: 1
issue: '6'
keyword:
- Information Systems
- Software
language:
- iso: eng
month: '08'
oa_version: None
page: 741-758
publication: International Journal on Software Tools for Technology Transfer
publication_identifier:
eissn:
- 1433-2787
issn:
- 1433-2779
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
related_material:
record:
- id: '299'
relation: earlier_version
status: public
scopus_import: '1'
status: public
title: 'AMT 2.0: Qualitative and quantitative trace analysis with extended signal
temporal logic'
type: journal_article
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 22
year: '2020'
...