Publications

Authors:

Castellini, A.; Bianchi, F.; Zorzi, E.; Simao, T. D.; Farinelli, A.; Spaan, M. T. J.

Title:

Scalable Safe Policy Improvement via Monte Carlo Tree Search

Year:

2023

Type of item:

Contributo in atti di convegno

Tipologia ANVUR:

Contributo in Atti di convegno

Language:

Inglese

Name of journal:

PROCEEDINGS OF MACHINE LEARNING RESEARCH

ISSN of journal:

2640-3498

Congresso:

International Conference on Machine Learning

Place:

Hawaii, USA

Period:

23-29 July 2023

:

PMLR

Publisher:

PMLR

Page numbers:

3732-3756

Keyword:

Safe policy improvement, Monte Carlo Tree Search, Scalability, SPIBB

Short description of contents:

Algorithms for safely improving policies are important to deploy reinforcement learning approaches in real-world scenarios. In this work, we propose an algorithm, called MCTS-SPIBB, that computes safe policy improvement online using a Monte Carlo Tree Search based strategy. We theoretically prove that the policy generated by MCTS-SPIBB converges, as the number of simulations grows, to the optimal safely improved policy generated by Safe Policy Improvement with Baseline Bootstrapping (SPIBB), a popular algorithm based on policy iteration. Moreover, our empirical analysis performed on three standard benchmark domains shows that MCTS-SPIBB scales to significantly larger problems than SPIBB because it computes the policy online and locally, i.e., only in the states actually visited by the agent.

Web page:

https://proceedings.mlr.press/v202/castellini23a/castellini23a.pdf

Product ID:

144314

Handle IRIS:

11562/1113706

Last Modified:

February 14, 2025

Bibliographic citation:

Castellini, A.; Bianchi, F.; Zorzi, E.; Simao, T. D.; Farinelli, A.; Spaan, M. T. J., Scalable Safe Policy Improvement via Monte Carlo Tree Search in «PROCEEDINGS OF MACHINE LEARNING RESEARCH» PMLR  in Proceedings of the 40 th International Conference on Machine Learning, Honolulu, Hawaii, USA , PMLR , Proceedings of "International Conference on Machine Learning" , Hawaii, USA , 23-29 July 2023 , 2023 , pp. 3732-3756

Consulta la scheda completa presente nel repository istituzionale della Ricerca di Ateneo