Reinforcement Learning-based Energy Flow Management

  • Pariya Kiani

Student thesis: Master's Thesis

Abstract

This thesis explores the application of Reinforcement Learning (RL) to optimize energy flow in photovoltaic-battery storage systems, which are essential for modern sustainable energy management. In this research, a Temporal Difference (TD)-based RL agent is tasked with controlling battery power, interacting with a simulated environment built in Simulink. The problem centers around effectively distributing energy between photovoltaic generation, battery storage, and the grid.
The primary goal of this work is to minimize energy costs and maximizing self-consumption. Unlike traditional rule-based approaches, this algorithm controls the agent’s experience over time to adapt to the specific dynamics of the environment without relying on forecasts or future data. The agent learns by adjusting its actions over time and updating its memory in each episode to improve future decision-making. The implemented algorithm uses a Q-learning framework with eligibility traces to reinforce learning and improve the agent's memory of past actions and their impact on future outcomes. A combination of epsilon-greedy and phased learning strategies, along with parameter tuning, was employed to enhance optimization process.
The learning process was validated by performing tests on unseen data to evaluate the robustness and adaptability of the algorithm across different seasons and varying environmental conditions. Results indicated that the RL-based method outperformed OCO in most cases, particularly when trained on winter data, with improvements of up to 54% during short-term testing. However, in some test scenarios, the performance was almost the same as the OCO, highlighting challenges such as overfitting to seasonal data and the need for further refinement.
The findings of this study underscore the potential of RL in energy management applications, demonstrating its ability to dynamically learn and optimize complex energy systems.
Date of AwardSept 2024
Original languageEnglish
SupervisorKathrin Maria Kefer (Supervisor)

Cite this

'