Energy Pricing Mechanisms Among Intelligent V2G Station Operators in a Shared Reinforcement Learning Environment

Abstract

Today’s energy market is a complex negotiation between producers and consumers consisting of a series of auctions on different timeframes. In such an environment, wholesale prices are shaped by the supply-demand relationship resulting in increased pricing during high-demand hours and vice versa. The capability of electric cars to store energy and feed it back to the grid enables two-way charging stations (Vehicle to Grid garages, V2G) to schedule the charging of their cars in a way that takes advantage of price fluctuation to optimize the final charging cost. In addition, the possibility of multiple garages to cooperate by mutually satisfying their needs (demand and complementary supply of energy from different garages) without occupying capacity in the main grid allows even more flexibility. However, cooperation of garages is not granted, as the individual interest may oppose the collective interest and a single garage does not have the complete picture of the charging state of the rest, therefore, even if it had the rationality to act unselfishly, it doesn’t have the necessary information. In this project, a multi-DDQN-agent reinforcement learning environment is implemented where each agent manages a single V2G-garage. The role of the intelligent agent is to optimize the transaction policy of the garage on a self-interest basis. We consider that garages are interconnected within a subgrid, transact with a central manager who in turn aggregates their demands, transacts with energy providers on the main grid, and finally, distributes the cost to the agents according to a pricing policy. The purpose of this project is to study pricing mechanisms which incentivize intelligent agents to learn to cooperate in order to optimize the total cost of satisfying energy needs of V2G garages.