Integration and Evaluation of Deep Reinforcement Learning Controller in a Building Co-Simulation Environment

Ahmed Amrani (1) , Rim Kaddah (1) , Jean-Philippe Tavella (2) , Mathieu Schumann (2)

1 IRT SystemX
2 EDF Labs

Deep Reinforcement Learning (DRL) is a promising Artificial Intelligence (AI) approach for buildings heating control. However, DRL controllers require dynamic simulation involving heterogeneous physical models (building, power-grid, etc.). Co-simulation allows the inter-operation between heterogeneous components when exported following the Functional Mock-Up Interface (FMI) standard. Controllers based on DRL can be implemented using various languages but are mostly based on Python libraries like Tensorflow. Their integration into co-simulation environments requires their exportation as Functional Mockup Units (FMUs). Thus, language-specific FMI-compliant export tools are needed for every programming language, specific library or platform used. This process is costly in effort and time and results in large FMUs. This paper proposes a novel method that simplifies AI-based controller integration regardless of the language or platform in a co-simulation environment and apply our methodology to assess a DRL controller. For the first objective, we use existing FMI export tools to create an FMU having the same input and output parameters as the controller. In the proposed architecture, the FMU acts as a proxy whose objective is to communicate with an external DRL controller deployed on a local or remote machine. We propose an application of this generic architecture for heating control in a house using DACCOSIM NG co-simulation environment. Through this architecture, the deployed DRL-based controller is connected to a house energy model, which includes weather conditions and heating. We show that our proposed controller is capable to learn the system dynamics and keep temperature within 1 degree of setpoint 93% of the time.

Mots clés: Deep Reinforcement Learning; Co-Simulation; Building; Heating control

Domaines: Intelligence artificielle [cs.AI]; Apprentissage [cs.LG]; Modélisation et simulation

Ahmed Amrani, Rim Kaddah, Jean-Philippe Tavella, Mathieu Schumann. Integration and Evaluation of Deep Reinforcement Learning Controller in a Building Co-Simulation Environment. Building Simulation and Optimization (BSO-V 2020), Sep 2020, Loughborough (virtual conference), United Kingdom. ⟨hal-02957010⟩