Shielded reinforcement learning for industrial applications: A systematic literature survey

The safety of Reinforcement Learning (RL)-based controllers has become a prominent research area in recent years, with various approaches being proposed to address this critical issue. Runtime Safety Assurance (RSA) methods for RL, such as Shielded RL, provide formal safety guarantees by preventing agents from taking unsafe actions and suggesting safe alternatives when necessary. However, previous surveys and reviews on RSA for RL have not thoroughly analysed the challenges and applications within the industrial sector. This study builds on existing state-of-the-art research on Shielded RL methods, emphasising its contributions to industrial applications and offering a domain-specific categorisation. This categorisation highlights the primary industrial domains utilising Shielded RL, detailing the optimised functions achieved by RL and the safety functions ensured by the shield. Additionally, the study presents a categorisation based on environmental features, enabling readers to assess the complexity of the problems addressed by the techniques studied. The shield’s attributes are analysed for each work, identifying key trends in their application, including their adaptability to new scenarios. Finally, a basic categorisation model for Shielded RL approaches, grounded in industrial safety standards, is introduced. This model serves as a baseline for future studies aiming to evaluate the maturity level of the works reviewed.