Intelligent Heating Control based on Reinforcement Learning Techniques

Starting date
October 1, 2020
Duration (months)
Computer Science, Department of Engineering for Innovation Medicine
Managers or local contacts
Farinelli Alessandro

The core idea of RL-HEAT is to investigate the use of advanced Artificial Intelligence (AI) techniques (specifically Safe Reinforcement Learning) for Cyber-Physical Systems focusing specifically on heating systems.

The recent availability of mature technology for Cyber-Physical Systems (CPS) and the impressive progress in Artificial Intelligence make intelligent CPS a critical asset in several application scenarios, ranging from energy management in the smart grid, to heating and cooling in smart buildings and to advanced manufacturing solutions for Industry 4.0. In particular, the application of Intelligent CPS for heating and thermal energy management is a key point to achieve a sustainable use of energy, reduce the carbon footprint of human activities and increase the productivity of this industrial sector.

A crucial element for intelligent CPS to be successfully deployed in such sector is the ability to adapt their behaviours to changes in the operational environments, and a widely used approach in the area of Artificial Intelligence to realize this is Reinforcement Learning. In the RL-HEAT project we will focus on Reinforcement Learning approaches for intelligent CPS, and specifically on Safe RL methods. Safe RL, aims at adapting the behaviour of the system so to guarantee (with high probability) an improvement in the performance.

A key goal for RL-HEAT is to apply Safe RL to perform adaptive control for an intelligent boiler. Our aim is to properly model the control problem for the intelligent boiler so to devise and apply novel Safe RL techniques. The RL-HEAT project will focus on model-based approaches to RL (e.g., Partially Observable Markov Decision Processes (POMDPs)) and will investigate recent, approximate solution techniques (e.g., the POMCP algorithm). Within this context we will consider the specific challenge of inserting a-priori knowledge inside the model by leveraging on ad-hoc solutions already applied by the industrial partner to perform adaptive control. The empirical evaluation will be performed on a prototype intelligent boiler provided by the industrial partner.


Giordano controls s.p.a.
Funds: assigned and managed by the department

Project participants

Alberto Castellini
Temporary Assistant Professor
Alessandro Farinelli
Full Professor
Eros Ghignoni
Riccardo Muradore
Associate Professor
Francesco Trotti
PhD student
Edoardo Zorzi
Scholarship holder
Research areas involved in the project
Sistemi intelligenti
Artificial intelligence
Sistemi ciberfisici
Embedded and cyber-physical systems


Research facilities