Applications of Game Theory to Design and Operation of Modern Power Systems: A Comprehensive Review

Abstract

In this work, we review papers that employ game theoretic tools to study the operation and design of modern electric grids. We consider four topics in this context: energy trading, energy balancing, grid planning, and system reliability, and we demonstrate the advantages of using game-theoretic approaches for analyzing complex interactions among independent players. The results and conclusions provide insights regarding many aspects of design and operation, such as efficient methodologies for expansion planning, cyber-security, and frequency stability, or fair-benefit allocation among players. A central conclusion is that modeling the system from the perspective of one entity with unlimited information and control span is often impractical, so correct modeling of the selfish behavior of independent players may be critical for the development of future power systems. Another conclusion is that correct usage of incentives by appropriate regulation or sophisticated pricing mechanisms may improve the social welfare, and, in several cases, the results obtained are as good as those obtained by central planning. Using an extensive content analysis, we point to several trends in the current research and attempt to identify the research directions that are currently at the focus of the community.

1. Introduction

Recently, there has been a wide interest in game-theoretic approaches for studying modern electric grids, which are going through a transformative change due to the penetration of new disruptive technologies. This transformation, which is sometimes referred to as ‘The Energy Transition’, mainly refers to two major changes: the increasing integration of renewable energy sources and the increasing use of electric vehicles [1,2]. Legacy electric grids were not planned with these technological innovations in mind, so they have to significantly change in order to support them [3,4,5]. Furthermore, these technologies can be purchased and managed by any citizen; hence, the way they will develop depends on the opinions of many people. For example, they may be integrated in a centralized manner by big companies or crowd funding, or in a distributed small scale manner in residential areas. As a result, the private sector is becoming a key player in the energy market, and its influence on the development of electrical grids is gradually growing. Consequently, electric utilities and governments that have worked in a centralized manner for over a century have to fundamentally change the ways they plan and operate the electric grid, and they need to find new ways to predict and control the behavior of multiple entities operating within a single system.

The research community currently understand quite well the behavior of each player in the power system, be it a grid operator, a consumer, an energy source, or a storage system. Yet, recent studies show that the interactions among these players are at least as important as the individual behavior of each of them [6,7,8]. The current approach of most studies is to forecast the development of power systems by solving optimization problems from the perspective of one entity with unlimited knowledge and control span [9,10,11,12,13]. While this approach might have been relevant for many years, it is gradually becoming unrealistic due to the decentralization and deregulation of the energy market. A major challenge is therefore to predict the development of a power system, taking into account the different objectives of the many players. This can be done based on game theory, which studies strategic interaction among rational players.

The seminal results of game theory were established in the 1950s, whereas the first application of game theory for studying power systems can be found in the 1970s [14,15,16]. Game theory can be divided into two branches: non-cooperative game theory and cooperative game theory [17]. Additional classification can be made in accordance with certain properties, such as the number of stages (static or dynamic games), the structure of information (perfect or imperfect, complete or incomplete), type of strategies (pure or mixed), and so forth [18]. Figure 1 shows the number of publications in applications of game theory in the fields of energy markets and power systems. The search was done in Scopus, which is Elsevier’s abstracts and citations database, and was divided to three different topics: the energy market, power systems management and control, and power systems planning.

As can be seen in Figure 1, the study of energy markets and power systems from a game theoretic perspective started gaining popularity in the early 2000s. This can be attributed to two major developments in the 1990s: the first was the establishment of deregulated networks and free markets in the energy sector, and the second was the introduction of new technologies, such as distributed energy sources, energy storage devices, and controllable loads. Due to these developments, multiple players with different interdependent objectives currently participate in the energy market and influence the development of electrical grids. Consequently, game theory is a natural choice to study decision making in modern power systems.

Few papers review how game theory may be used in the context of power systems. A comprehensive review of game theoretic approaches for solving power system problems is provided in Reference [19]. The reviewed studies are divided according to the type of game they use: non-cooperative (both static and dynamic), cooperative or evolutionary. Moreover, a summary of different objective functions, methods and case studies is presented. Based on this summary, the authors suggest several topics for future research, such as interaction among electric vehicles and storage devices for energy balancing, fairness aspects in fully-competitive distributed energy trading scenarios, and the impact of renewable energy sources on demand side management, when dynamic pricing techniques are utilized. Over the years many of the suggestions for future research were indeed realized, as we demonstrate in this study. In addition, Reference [20,21] survey studies that employ game theory to investigate smart-grid related problems. Reference [20] surveys a variety of smart-grid related topics, such as demand side management, plug-in hybrid electric vehicles, and smart-grid security, whereas Reference [21] focuses on studies that use non-cooperative game theory to study energy trading. Reference [21] also provides an overview of smart-grid architectures. The similarities and differences among the studies are presented, focusing on the type and number of players, the players’ incentives, and the method used, e.g., analytic versus numeric. Moreover, studies that use game theoretic approaches to research micro-grids are surveyed in Reference [22,23]. Reference [22] focuses on cooperation among micro-grids, and presents the advantages of using game theory to study this field. Reference [23] provides an overview of game-theoretic methods for three problems in the context of micro-grids: micro-grid systems (focusing on energy exchange and control), demand side management, and smart-grid communications. Reference [24] surveys works that use game theory for studying demand-response mechanisms. Here the studies are divided according to the type of game: (1) non-cooperative, (2) cooperative, (3) evolutionary, (4) Stackelberg, and (5) Bayesian. One of the main conclusions is that, due to the increasing complexity of electricity markets, future research would focus on game theoretic approaches that harness intelligent algorithms, such as machine learning techniques, for more efficient solutions of realistic games. Moreover, the authors suggest that Bayesian games, which may be used when information is limited, and evolutionary games, which attempt to describe the behavior of non-rational players, will increasingly be used to model transactions among private companies and citizens, since, in these situations, there are many unknown factors, and spontaneous decisions may be common.

Contrary to these comprehensive review papers, which focus on applications of game theory in energy markets [24], smart-grids [20,21], micro-grids [22,23], and power systems from a broad perspective [19], this study focuses on recent applications of game theory that are specifically related to operation and design of modern electric grids. We review four topics in this context: energy trading, energy balancing, grid planning, and system reliability, and demonstrate the advantages of using game-theoretic approaches for analyzing complex interactions among independent players. The results and conclusions of the reviewed papers provide insights regarding many aspects of design and operation, such as fair-benefit allocation among players, or efficient methodologies for expansion planning, cyber-security, and frequency stability. A central conclusion is that modeling the system from the perspective of one entity with unlimited information and control is often impractical, so correct modeling of the selfish behavior of independent players allows to develop efficient control algorithms, which may be critical for the development of future power systems. Another conclusion is that correct usage of incentives by appropriate regulation or sophisticated pricing mechanisms may improve the social welfare, and in several cases the results obtained are almost as good as can be obtained by central planning.

The paper is organized as follows. Section 2, Section 3, Section 4 and Section 5 review papers that focus on the following applications: energy trading, energy balancing, grid planning, and grid reliability. In Section 6, we discuss the advantages and disadvantages of employing different types of games for studying various power system applications, whereas, in Section 7, we discuss current trends and provide suggestions for future work. Finally, in Section 8, we conclude the paper.

2. Energy Trading

The deployment of decentralized generation and storage devices in the past two decades allows many players in the modern power grids to participate in energy trading. These players include consumers, generators and utilities, which all have the same main objective-to maximize their own profit. Since in many situations the players’ actions influence one another, more often than not conflicts arise. For this reason, game theoretic approaches are common for formulating energy trading problems.

As shown next, energy trading problems are investigated using different types of games, such as evolutionary, cooperative, and non-cooperative games, the latter being the most popular approach. In most works, the players are defined as sellers or buyers of electric energy, and the challenge is to find an equilibrium in a power market, considering various objectives and constraints. One main conclusion is that, even in competitive markets, the government should apply some profit distribution mechanism to ensure fairness. Moreover, it is concluded that reasonable bidding rules should be considered for limiting high pricing and for preventing market manipulation by generation companies. As many of the reviewed papers show, game theoretic approaches are suitable for analyzing complex trading interactions among different independent market players. However, the use of game theory for studying energy trading has its own challenges and limitations. For example, most of the studies assume that each player has complete information of the opponents’ available strategies, yet, in reality, energy companies often sign business confidentiality agreements. Another main challenge is that game-theoretic models tend to be hard to analyze and have a high computational complexity. As a result, most studies consider up to four types of players, whereas real-life energy markets are obviously more complex. The section is organized according to specific applications, and all of the reviewed works are summarized in Table 1.

2.1. General Energy Market Applications

Various works from the recent literature that employ game theory to study trading in modern energy markets are overviewed in this subsection. The works are divided according to the type of game that is modeled, as follows: non-cooperative, cooperative, and evolutionary.

Most works use a non-cooperative game to model the interactions among trading entities. For example, in Reference [25], a non-cooperative game is developed for plug-in hybrid electric vehicles that seek to sell their stored energy in an energy market. Each group can sell a desired amount of energy surplus in order to maximize a utility function that reflects a trade-off between the economic benefits from the trade and the physical costs (e.g., reduction in the batteries’ life-time). A Nash Equilibrium is reached using an iterative algorithm, and simulation results show an improvement in the average utility of the players when compared to a greedy algorithm. From the same authors, Reference [26] formulates a non-cooperative game among energy storage units that trade their stored energy in a double-auction market with multiple buyers and sellers. Each storage unit operator decides how much energy to sell in order to maximize a utility function that reflects the trade-off between the trading profits and the associated costs. A novel algorithm that guarantees convergence to equilibrium is proposed, and again the results demonstrate that the performance of each storage unit is improved when compared to a greedy approach. In the same line of thinking, Reference [27] proposes a retail electricity market model that consists of a utility company and prosumers, which locally operate and manage their own distributed resources. Here, the day-ahead electricity market price is determined based on game-theoretic algorithms, and a numerical simulation demonstrates the effectiveness of the proposed electricity market clearing schemes. In Reference [28], a Stackelberg game is suggested to model the interactions between a power station, which is considered the “leader”, and several prosumers, who are the “followers”. The rules of the game are that the station decides how much total energy to buy from the prosumers, and then each prosumer decides how much energy to sell to the station. It is shown that the game reaches a Stackelberg equilibrium, which consists of the socially optimal energy for the consumers and minimal total cost to the station. In addition, the performance of the game has been established via simulation that shows a reduction in the average total cost over a conventional feed-in-tariff scheme. Similarly, Reference [29] formulates a Stackelberg game between a power company, the leader, who defines a pricing strategy to maximize its profits, and prosumers, the followers, who choose the amount of energy to buy or sell in order to optimize their future earnings. The behavior of the prosumers is modeled according to prospect theory. The game is shown to have a unique Nash equilibrium in pure strategies, which emerges from classical game-theoretic analysis.

Studies [30,31] have used the Nash bargaining concept to study energy trading related problems. Reference [30] analyzes Nash bargaining settlement outcomes for a financial bilateral contract negotiation between a generation firm and a load-serving entity, considering risk management, strategic gaming, and multi-market interactions. It has been shown through both theoretical analysis and simulation that the negotiation outcomes vary in response to changes in risk preferences and price biases. Reference [31] proposes a cooperative-game-based bargaining scheme for an energy trading problem between demand response aggregators and a distribution company that collaboratively decide on the costs and amounts of traded energy. Based on the Nash bargaining solution, a fair and Pareto optimal outcome is achieved, which means that the players have an incentive to cooperate. Simulation results show that the suggested framework may improve the profits of the players and maximize the social welfare when compared to the non-cooperative game-based approach.

In addition, several works formulate their games using evolutionary algorithms. For example, Reference [32] proposes a non-cooperative game to analyze strategic interactions among energy suppliers in deregulated electricity markets based on competitive co-evolutionary algorithms. The suppliers are represented by their behavior strategies, and an evolutionary algorithm is used as a learning method that enables players find the best strategy according to past behavior of their opponents. The co-evolutionary algorithm is used to find a Nash equilibrium, and several case studies demonstrate the effectiveness of these algorithms to find optimal strategies in different market situations.

2.2. Micro-Grid Applications

Papers that focus on micro-grids in energy trading related problems mainly use non-cooperative games, and can be divided into two categories: (1) trading among consumers inside a single micro-grid, and (2) trading among interconnected micro-grids. Several of the latest works of the first category are [33,34,35,36]. In Reference [33], a multi-agent based reverse auction model for micro-grid market operations is proposed. The goal is to obtain the cheapest power supply for the micro-grid’s loads. The proposed model promotes competition between distributed energy resources and determines their unit commitment within an hour-ahead market framework. This approach has been tested in a micro-grid that fully complies with industrial protocols. Based on these results, the authors recommend applying such multi-agent based models in future micro-grids. Reference [34] proposes a pool strategy for a micro-grid trading with a distribution electricity market. This is done by formulating a Stackelberg game in which the micro-grid is the leader, proposing a price offering, and the competitors and consumers are the followers, choosing how to trade. It is shown that the proposed strategy creates a fair environment for all players, and that utilizing micro-grids as price makers promotes competition. Reference [36] develops a game theoretic model for real time energy trading using interactions among prosumers in a micro-grid. The interaction among the prosumers who sell energy (leaders) and prosumers who buy energy (followers) is modeled as a Stackelberg game. Furthermore, the authors formulate a non-cooperative game to model the pricing competition among sellers, and an evolutionary game to model the competition among buyers when selecting sellers. Simulation results show that the suggested model can effectively handle energy trading in a micro-grid, and that the power imported from the main grid to the micro-grid is reduced compared to conventional trading, since all the available energy within the micro-grid is efficiently allocated. The authors in Reference [35] suggest a model of economic incentives for market participants who cooperate in developing a micro-grid. The authors model a cooperative game with three types of players—a micro-grid developer, a utility company, and consumers. An analysis on how the development of a micro-grid affects prices and costs for all players under different assumptions is presented. Based on different scenarios, the study offers energy policy recommendations on how to efficiently promote micro-grids and specifies market failures that should be addressed in order to maximize the benefits of micro-grid development.

Works [37,38,39] have used game theory to study trading among interconnected micro-grids. Reference [37] suggests an energy trading mechanism among micro-grids in a competitive market where each micro-grid can be an energy provider or a consumer, according to their energy generation and local demand. The problem is formulated as a non-cooperative game among the micro-grids and the Nash equilibrium solution is given in closed-form. Based on numerical results, it is concluded that the suggested mechanism is fast enough for real-time applications. Reference [38] analyzes energy trading among micro-grids where several micro-grids have surplus energy that can be sold or stored, while other micro-grids want to buy surplus energy. A Stackelberg game is formulated where the sellers lead the competition by deciding the amount of energy for sale and the buyers follow the sellers’ actions by submitting a price bid. It is shown that the game has a unique equilibrium and that the proposed game-based trading mechanism maximizes profits for all players. Reference [39] proposes a mechanism that is based on the Nash bargaining theory to encourage proactive energy trading and fair benefit sharing among interconnected micro-grids. A Nash bargaining problem is solved as two consecutive problems: social cost minimization and trading benefit sharing. Based on several case studies, it is shown that, through the suggested mechanism, each individual micro-grid can achieve an increase in profit and the total operation cost of the interconnected micro-grids can be reduced by up to 13% compared to its performance without cooperation.

2.3. Electricity Pricing Models

In recent years, a variety of electricity pricing models that are based on electricity demand profiles have been implemented by utility companies worldwide. The pricing models mainly differ in their resolution and in the frequency in which prices change. Several known models are: Time-of-use pricing, Critical peak pricing, and Real-time pricing. Many use game theory to offer novel pricing models, where non-cooperative game theory is the leading approach. For example, Reference [40] suggests a game theoretic approach to optimize time-of-use electricity pricing strategy for utility companies and their users. The authors formulate a multi-stage non-cooperative game between utility companies, who set the pricing mechanism, and consumers, who set their demand accordingly. Through simulations, the proposed pricing mechanism is shown to increase the utilities’ profits and level the users’ demand in comparison to a scenario with flat prices. In Reference [41], a game theory based dynamic pricing model is evaluated for Singapore’s electricity market in order to achieve an efficient demand response. The game is designed to maximize benefits for both utility companies and consumers, with a focus on the residential and commercial sectors. Simulations that are based on real data show that, when a half-hourly real-time pricing strategy is used, there is maximum peak load reduction and increase in profits. Similarly, Reference [42] proposes a pricing mechanism for an electricity supply chain that consists of a single generation company, multiple consumers and competing utility companies. The interactions among the utility companies, who set retail prices, is characterized through a non-cooperative game, and an iterative algorithm is developed to obtain a Nash equilibrium. Moreover, the authors show that a revenue sharing contract among the utility companies and the generation company can be set to obtain maximal profits and social welfare. Another example is Reference [43], which formulates a real-time pricing scheme for demand response management. This is done by modeling the interactions between an energy provider (leader) and multiple energy hub operators (followers) as a Stackelberg game. The equilibrium of the game is found using a novel distributed algorithm, and the optimal strategies for each player are determined in order to balance the energy. Numerical results show that the method can improve the payoffs for all players with good convergence performance. Finally, Reference [44] analyzes an electricity market with a time-of-use electricity pricing using an evolutionary game. The game is between power generation companies, power grid companies, power supply entities, and power consumers. Based on the results, the authors describe policy implications that can assist governments in defining policies for electricity markets. A principal suggestion is that the government should promote fairness by establishing a profit distribution mechanism among all players in the market. Furthermore, the government should make reasonable bidding rules in order to limit high pricing and prevent market manipulation by the generation corporations.

2.4. Bidding Strategies

Many employ game theory to study and design bidding strategies in energy markets. An intelligent bidding strategy is crucial for generation companies that participate in a competitive energy market and aim to maximize their profits. Since a good bidding strategy has to take into account the rational and objectives of other players, game theory is a natural choice. As an example, Reference [45] presents Nash equilibrium bidding strategies in a bilateral market in which generators submit bids to consumers. Network optimization techniques are used to calculate Nash equilibrium points, and it is shown that for all solutions an efficient allocation is achieved with equal revenue for all generators. In Reference [46], conjectural variation-based bidding strategy methods are used to help generation firms maximize their profits in electricity spot markets with imperfect information. Formal analysis shows that the system equilibrium that is reached via conjectural variation-based bidding is a Nash equilibrium. Reference [47] proposes a bidding strategy for wind power producers to maximize profits. In this model, wind power producers buy energy from conventional power producers in a bilateral reserve market to minimize the risk caused by uncertainties in generation. The interaction among them is modeled through a non-cooperative game where the price is settled by finding a Nash Equilibrium. Case studies with real-world market data and different types and numbers of players are performed to show the effectiveness of the model. Reference [48] suggests a framework for implementing a retail energy market with a high distributed energy resources penetration and demand side management of prosumers. The distributed energy resources take into account uncertainties in production and try to maximize their profit by undertaking strategies through the price bidding strategy that is obtained in the Nash equilibrium. Different scenarios show that it is more profitable for the players to collaborate with one another. Reference [49] proposes an evolutionary imperfect information game for analyzing bidding strategies in electricity markets with price-elastic demand. The main observation is that the strategies of the generation companies in this imperfect information game eventually converge to the Nash equilibrium in the perfect information game.

2.5. Profit Allocations

Both the Shapley value and the Nucleolus are solution methods in cooperative game theory that describe profit allocation among cooperating entities, and many authors use these concepts to study energy trading related problems. For example, Reference [50,51] use these concepts to suggest profit allocation strategies. Reference [50] studies cooperation among independent power producers in a retail market. Based on the Shapley value and the Nucleolus solution concepts the authors propose a profit allocation among producers. Comparison between a non-cooperative game and a cooperative one shows that cooperation among power producers, with the suggested profit allocation, leads them to higher profits. Reference [51] suggests a novel stochastic programming approach to model the participation of a virtual power plant in a day-ahead market and a balancing real-time market. The virtual power plant aggregates distributed energy sources and manages them to reduce the risk caused by intermittent generation. The contribution of each resource to the virtual power plant is assessed and the Shapley value and Nucleolus solutions are used to determine the profit allocation among them. Moreover, it is shown that, when the risk-aversion level of the resources is high, their profits increase when they are coordinated by an external entity, e.g., a virtual power plant.

Table 1. Summary of works focusing on energy trading application using game-theory.

	Ref.	Game Method	Players	Application
General trading applications	[25]	Non-cooperative	Hybrid electric vehicles	Energy Trading
	[25]	Non-cooperative	Hybrid electric vehicles	Energy Trading
	[26]	Non-cooperative	Batteries	Trading in a double-auction market
	[27]	Non-cooperative	Utility company, prosumers	Retail electricity market with prosumers
	[28]	Stackelberg	Power station, consumers	Energy Trading
	[29]	Stackelberg	Generation company, prosumers	Pricing strategy for energy trading
	[30]	Cooperative	Generation companies, load-serving entity	Financial bilateral contract negotiation
	[31]	Cooperative	Demand response aggregators, distribution company	Bargaining-based cooperative energy trading
	[32]	Evolutionary	Power suppliers, consumers	Suppliers’ optimal strategies in a deregulated electricity market
Micro-grid applications	[33]	Non-cooperative	A Micro-grid’s distributed energy sources	Reverse auction model for micro-grid market operations
	[37]	Non-cooperative	Micro-grids	Trading mechanism in a competitive market
	[34]	Stackelberg	Micro-grid, consumers	Pool strategy for a micro-grid trading with a distribution electricity market
	[38]	Stackelberg	Micro-grids	Distributed mechanism for energy trading among microgrids
	[36]	Stackelberg + non-cooperative + evolutionary	Prosumers	Energy trading in a micro-grid considering demand response
	[35]	Cooperative	Utility, Micro-grid developer, customers	Model of economic incentives for market participants who cooperate to develop a micro-grid
	[39]	Cooperative	Micro-grids	Energy trading and fair benefit sharing among interconnected micro-grids
Pricing models	[40]	Non-cooperative	Utility companies, consumers	Time-of-use electricity pricing strategy
	[41]	Non-cooperative	Utility companies, consumers	Dynamic pricing model for Singapore electricity market
	[42]	Non-cooperative	Utility companies, consumers, generation company	Electricity supply chain
	[43]	Stackelberg	Energy provider, energy hub operators	Demand response management
	[44]	Evolutionary	Utility companies, consumers, generation companies, suppliers	Stability analysis of electricity markets
Bidding strategies	[45]	Non-cooperative	Generation companies, loads	Bidding strategies in a bilateral market
	[46]	Non-cooperative	Generation companies	Conjectural variation-based bidding strategy method for generation companies
	[47]	Non-cooperative	Conventional and wind power producers	Wind power producers profit maximization
	[48]	Non-cooperative	Prosumers	Retail energy market with distributed energy resources and demand side management
	[49]	Evolutionary	Generation companies	Bidding strategies in electricity markets with price-elastic demand
Profit allocation	[50]	Cooperative	Power producers	Cooperation among independent power producers in a retail market
	[51]	Cooperative	Distributed energy resources	Profit allocation among distributed energy resources

Table 1. Summary of works focusing on energy trading application using game-theory.

	Ref.	Game Method	Players	Application
General trading applications	[25]	Non-cooperative	Hybrid electric vehicles	Energy Trading
	[25]	Non-cooperative	Hybrid electric vehicles	Energy Trading
	[26]	Non-cooperative	Batteries	Trading in a double-auction market
	[27]	Non-cooperative	Utility company, prosumers	Retail electricity market with prosumers
	[28]	Stackelberg	Power station, consumers	Energy Trading
	[29]	Stackelberg	Generation company, prosumers	Pricing strategy for energy trading
	[30]	Cooperative	Generation companies, load-serving entity	Financial bilateral contract negotiation
	[31]	Cooperative	Demand response aggregators, distribution company	Bargaining-based cooperative energy trading
	[32]	Evolutionary	Power suppliers, consumers	Suppliers’ optimal strategies in a deregulated electricity market
Micro-grid applications	[33]	Non-cooperative	A Micro-grid’s distributed energy sources	Reverse auction model for micro-grid market operations
	[37]	Non-cooperative	Micro-grids	Trading mechanism in a competitive market
	[34]	Stackelberg	Micro-grid, consumers	Pool strategy for a micro-grid trading with a distribution electricity market
	[38]	Stackelberg	Micro-grids	Distributed mechanism for energy trading among microgrids
	[36]	Stackelberg + non-cooperative + evolutionary	Prosumers	Energy trading in a micro-grid considering demand response
	[35]	Cooperative	Utility, Micro-grid developer, customers	Model of economic incentives for market participants who cooperate to develop a micro-grid
	[39]	Cooperative	Micro-grids	Energy trading and fair benefit sharing among interconnected micro-grids
Pricing models	[40]	Non-cooperative	Utility companies, consumers	Time-of-use electricity pricing strategy
	[41]	Non-cooperative	Utility companies, consumers	Dynamic pricing model for Singapore electricity market
	[42]	Non-cooperative	Utility companies, consumers, generation company	Electricity supply chain
	[43]	Stackelberg	Energy provider, energy hub operators	Demand response management
	[44]	Evolutionary	Utility companies, consumers, generation companies, suppliers	Stability analysis of electricity markets
Bidding strategies	[45]	Non-cooperative	Generation companies, loads	Bidding strategies in a bilateral market
	[46]	Non-cooperative	Generation companies	Conjectural variation-based bidding strategy method for generation companies
	[47]	Non-cooperative	Conventional and wind power producers	Wind power producers profit maximization
	[48]	Non-cooperative	Prosumers	Retail energy market with distributed energy resources and demand side management
	[49]	Evolutionary	Generation companies	Bidding strategies in electricity markets with price-elastic demand
Profit allocation	[50]	Cooperative	Power producers	Cooperation among independent power producers in a retail market
	[51]	Cooperative	Distributed energy resources	Profit allocation among distributed energy resources

3. Energy Balancing

The operation of modern power systems, and specifically the balance between generation and demand, is affected by multiple decisions makers, including system operators, generators, and consumers, who may have different objectives. Thus, game theory is a popular tool for designing control algorithms that address energy balancing problems.

As shown in this section, game theory plays a pivotal role in addressing numerous energy balancing problems, which include demand side management, energy sharing in and among micro-grids, and optimal charging of electric vehicles. Different non-cooperative and cooperative games are used for modeling the selfish or cooperative behavior of utility providers, energy hubs, energy generators and electricity consumers. Several studies show that efficient regulation may result in equilibrium points in which customers distribute their load throughout the day and shift their peak consumption. This in return reduces the peak-to-average ratio of the overall load, and improves the overall efficiency of the system. In addition, different game-theoretic algorithms are proposed as solutions for consumers that have their own energy sources and storage devices. These solutions allow consumers to share energy over the grid efficiently while operating in equilibrium. A few cooperative games are proposed as tools of analysis for energy management problems in smart grids. These solutions typically provide profit-sharing schemes that ensure that all consumers are financially rewarded by minimizing the energy cost of the joint coalition. In the category of electric vehicles, non-cooperative games are used to model interactions between the aggregator and electric vehicles, and are employed to solve optimal charging problems. In many papers, exhaustive analysis illustrates that these games have a unique equilibrium that allows development of distributed control strategies for each vehicle. More specifically, it is shown that by correctly modeling the driver’s behavior, one may design incentives to influence his or her decisions, such that the social welfare is maximized.

In this section, we explore several state-of-the-art studies in which game-theoretic paradigms are deployed in order to solve energy-balancing problems. The overall section is categorized into three parts that include demand response and demand side management problems, energy management in micro-grids, and electric vehicle applications. The considered studies are also summarized in Table 2.

3.1. Demand Side Management

Demand-side management strategies are often used to balance generation and demand, by giving customers economic incentives and encouraging them to adjust their demand by leveraging the flexibility of their loads. When this technique is implemented in short time scales, it is named demand-response. Works [52,53,54] have formulated a non-cooperative game to study demand side management of residential loads. In Reference [53,54], the consumers are assumed to be selfish, and compete with each other to minimize their individual energy costs. Reference [53] proposes a distributed algorithm to solve this problem, and shows that this algorithm efficiently distributes the load and achieves lower electricity bills compared to uncontrolled energy consumption of the consumers. Similarly, in Reference [54], the selfish behavior of the consumers is modeled as a non-cooperative aggregative game, and sufficient conditions are derived for the existence and uniqueness of a Nash equilibrium. In addition, the paper develops three distributed algorithms to obtain the equilibrium of the formulated game in two scenarios: in the first scenario, consumers can access real-time information regarding the aggregated load, whereas in the second scenario they cannot. It is further revealed that these algorithms encourage the consumers to shift their peak consumption, which could be beneficial for the consumers and the grid. To control the flexible demand of consumers, a control algorithm is proposed in Reference [52] in which consumers are aiming to maximize their electricity consumption while satisfying certain constraints. To that end, the consumers propose a bid at each time interval, and based on this bid a price signal is computed by a central control authority using a proportional allocation mechanism. Following this, Reference [52] focuses on two scenarios in which the total demand is allocated among consumers: in the first, an optimization problem is solved by the central control authority, while in the second the selfish behavior of the consumers is modeled using a non-cooperative Cournot game. The authors first evaluate the Nash equilibrium of this game, and then show that the selfish behavior of the consumers may lead to reduced efficiency. The authors further explain that this loss is typically affected by the number of consumers and their functions.

Works [55,56,57] have used non-cooperative games to model demand side management in which the end users own an energy storage system. In Reference [55,57], interactions among the users are modeled as a non-cooperative game where the users aim to minimize their energy payments by controlling the charging and discharging of their storage systems. Reference [55] studies a demand side management problem in a smart grid that includes users with distributed energy sources and storage units. The authors model this problem as a non-cooperative game and prove the existence of a Nash equilibrium. Moreover, the authors suggest a distributed and iterative algorithm based on the proximal decomposition method that allows to compute the optimal strategies of the users with minimum information exchange between the central unit and the users. In Reference [57], consumers with a storage device convey their energy demand to an energy provider who determines the energy payments based on their load profiles. The authors compare two methods: a centralized and a decentralized one. In the centralized approach, the energy provider controls the user’s demand with the goal of minimizing the difference between the instantaneous and average energy demand of the system. In the decentralized approach, each user independently tries to minimize their energy payment. The second method is described as a non-cooperative game and the authors propose two distributed algorithms in which users define their demand according to a signal from the control center regarding the total load profile. In the first algorithm the users update their consumption sequentially, whereas in the second one they do so simultaneously. The authors show that both distributed algorithms achieve the global optimal solution of the centralized approach. Reference [56] formulates two games. The first one is a non-cooperative game played among residential energy consumers with storage who aim to minimize their energy consumption. The second is a Stackelberg game played between an energy provider and the consumers, where the energy provider sets the prices to maximize its profit knowing that consumers will respond by minimizing their cost. It is shown that both games have a unique equilibrium. The authors analyze the influence of the storage capacity, energy requirements and number of users on the overall cost and peak-to-average power ratio.

Reference [58] proposes demand side management algorithms for solar photovoltaic (PV) systems, where demand is scheduled in accordance with the expected power generation from the PV panels. The authors present three control schemes named centralized control, decentralized open-loop control and decentralized feedback control. The centralized control is formulated as an optimal control problem, whereas the distributed control schemes are based on differential games. Results show that the decentralized open-loop control scheme manages the power consumption efficiently when weather data is available, whereas the decentralized feedback controller performs well when such data is unavailable. Perhaps most interestingly, both decentralized control schemes outperform the centralized one whether data is available or not.

Several works study demand response problems through non-cooperative games [59,60,61,62,63]. Reference [59] proposes a game-theoretic model to manage the demand in a smart grid. The authors use a non-cooperative game to model the interaction among energy providers and consumers. In this model, the players can exchange information with one another to determine the amount of energy produced or consumed. A 0–1 mixed integer linear programming method is used to find Pareto-optimal solutions. In Reference [61], the model is extended from a single-period to a multi-period game. Moreover, the ability of the consumers to produce and store energy is added. Computation analysis shows that a Nash equilibrium can be computed in a few hundreds of seconds for even thousands of users. The authors claim that this is a reasonable time for the application of demand response.

The demand response management problem can also be formulated as a non-cooperative Stackelberg game where energy providers, as leaders, set the electricity price and consumers, as followers, adjust their energy use accordingly [60,62,63]. This is done in Reference [60] by formulating a game among utility companies and consumers. The Nash equilibrium is computed analytically and the necessary and sufficient conditions on the budget of a consumer for participating in the demand response program are given. The results show that a multi-period demand response program provides more incentive to participate in the program than a single-period one. Reference [62] addresses a real-time demand response problem and formulates a game between an energy management center and consumers. Analysis of this game reveals the existence and uniqueness of Stackelberg equilibrium and provides optimal energy demands for each customer. Reference [63] studies a demand-response problem for geographically distributed loads that represent data centers in a power grid. The problem is formulated as a two-stage Stackelberg game, where in the first stage each utility sets a price to maximize its own profit and then based on these prices, in the second stage, data centers aim to minimize its cost via work load shifting. Results show that this game has a unique equilibrium and it significantly reduces the energy demand of the data centers and improves the grid’s reliability and robustness.

3.2. Energy Management in Micro-Grids

Several recent papers use game theory to design energy management algorithms for operating and coordinating loads, distributed energy sources, and storage devices in micro-grids. These algorithms are used to optimally allocate the power output among distributed generation units and to efficiently manage stored energy to provide reliable and sustainable energy in a cost-effective way. The works are divided according to the type of game used: non-cooperative or cooperative.

Several works use non-cooperative games to study energy management of residential users in a micro-grid. For example, Reference [64] formulates a non-cooperative game among residential users for optimizing their battery capacity and scheduling their energy consumption. Results show that the game has a Nash equilibrium that is Pareto-optimal in terms of energy cost. Moreover, it assists in reducing the peak-to-average ratio of overall energy demand. Likewise, Reference [65] proposes a coupled-constraint game to solve the residential energy consumption scheduling problem. First, this game is transformed into a decoupled game by dual decomposition, and then the best response is computed employing the gradient projection method. Numerical results show that the proposed approach enhances the welfare of each user and minimizes the demand peak-to-average ratio.

Non-cooperative games are further explored to address energy management of storage devices and PV arrays in micro-grids. For instance, Reference [66] considers a hybrid energy storage system that includes an engine–generator, battery and an ultra-capacitor, and formulates the energy management problem as a non-cooperative current control game. The authors analytically derive the Nash equilibrium of the game and provide a distributed control update, which significantly improves the flexibility, scalability, and reliability of the hybrid storage systems. Similarly, given a multi-micro-grid system, Reference [67] formulates the energy management problem as a multiple leader and multiple follower Stackelberg game. In this game, micro-grids are leaders who decide the minimum generated energy, with the help of a central energy management unit, and aim to maximize their profit. Consumers with energy storage devices are the followers who choose their energy demand. The authors provide three different algorithms by which the micro-grids determine the minimum amount of energy to be generated and its price, and the customers request energy based on their real-time price. In Reference [68], a method for energy sharing among neighboring PV prosumers is provided, where the engagement of energy is done through a third party, named energy-sharing provider, that is equipped with energy storage. The key problem is formulated as a hybrid optimization problem that consists of a day-ahead energy storage scheduling and real-time internal pricing. In the day-ahead scheduling, the storage provider attempts to maximize its profit and improve the net power profile of the energy network. In addition, the real time pricing is modeled as a Stackelberg game where the storage provider is the leader who sets the energy prices considering the utilization of the storage system, and the prosumers are the followers, who respond to the energy prices by optimizing their energy consumption. A practical case study reveals that the proposed method is beneficial to improve the economic benefits of the grid and PV energy sharing. Reference [69] promotes solar PV installations in apartment buildings by studying energy sharing models between an owner of a solar PV and storage systems and the residents of the building. The first model is an optimization problem with the objective of maximizing welfare, whereas in the second model a Stackelberg game between the PV owner and the consumers is formulated. Results show that in both cases welfare may be maximized, however it is allocated differently between the PV owner and the consumers.

Several works study the utilization of components in the micro-grid through cooperation. The idea is that, through cooperation, underused resources can be utilized more efficiently, thus lowering the overall cost for the group. For example, Reference [70,71,72] study cooperation among players to purchase and manage storage systems together. Reference [70] investigates two scenarios in this context. In the first scenario, each player has a storage device and the players may cooperate and manage all devices together in order to maximize the overall profit. In the second scenario, players can purchase a common storage device and then manage it together. The authors show that, in both scenarios, sharing storage is beneficial by proving that the core of the game is non-empty, meaning that no player can benefit from forming a smaller coalition than the coalition that includes all the players. Moreover, in both scenarios, they find an allocation of the overall cost that is satisfactory for all users. Reference [71] extends the model formulated in Reference [70] by adding ramp constraints to the storage devices. While Reference [70] uses a two period time-of-use electricity tariff and takes into consideration only the peak consumption of the players, in Reference [71], the model takes into account inter-day consumption patterns and a realistic tariff. The authors show how consumption patterns and ramp constrains affect both the feasibility of forming a grand coalition and the allocation of cost among them. They prove the existence of a solution in the core for a set of cooperative games that are equivalent to a well-known class of games named ‘unitary glove market’ and demonstrate the analytic results using real data from the ‘Pecan street’ project. In their subsequent Reference [72], the authors augment the model by adding two extensions: stochasticity of the load and discreetness of the storage device capacity. They prove that, for the stochastic yet continuous case, a solution of the game always exists and they provide an efficient algorithm to find it. On the other hand, they show that the discrete case may fail to admit a solution. Accordingly, for that case, they provide an approximate solution that appears to be satisfactory for real-world deployments. In addition, they provide numerical simulations in which the cooperative scheme achieved an increase between 100% and 250% in the amount of storage hosted in residential premises compared to the setting in which consumers invest individually, when it was profitable for them to do so. In Reference [73], a cooperative energy scheduling problem is addressed for multiple neighboring energy hubs. These hubs may have different supply and load profiles, and they can exchange power to minimize their own operational costs. For ensuring cooperation among these hubs, bargaining game theory is utilized to compute a Pareto solution to the minimization problem. Further analysis illustrates that the cooperative operation of these hubs has better economic benefits than the non-cooperative operation. In Reference [74], a cooperative game is proposed to study an energy management problem in which distributed energy storage systems operate collaboratively under a centralized control strategy to minimize the joint conditional energy cost. The core of the cooperative game is shown to be non-empty, that is, the players benefit from all cooperating together. Moreover, case studies show that the cooperation among the players leads to a more leveled overall load in the local network.

3.3. Electric Vehicle Applications

Game theory is a popular tool for modeling interactions between electric vehicles and the grid, and allows to establish novel energy management and charging policies. As part of this trend, a variety of non-cooperative games are utilized to solve optimal charging problems in electric vehicles [75,76,77,78,79,80]. For example, Reference [75,76] employed Stackelberg games to research optimal charging problems in electric vehicles. In both works there is a charging service provider, who leads the game by defining the charging price, and electric vehicle owners, who follow by deciding when to charge. Reference [75], proposes a combination of non-cooperative and cooperative approaches. In the non-cooperative game, each vehicle selects its own demand strategy to optimize its benefit selfishly, while in the cooperative model, a distributed energy scheduling algorithm is proposed to optimize the overall benefit of the vehicles. Theoretical analysis proves the existence and uniqueness of an equilibrium in both approaches. Moreover, algorithms are developed in both frameworks to ensure convergence to a global optimum, and it is shown that the solutions are robust against demand uncertainty. Likewise, Reference [76] shows that the formulated game always reaches an equilibrium point where the vehicle charging requirements are satisfied and the aggregator’s profit is maximized. Moreover, the authors analyze the influence of the costumers’ weighted utility function on the results of the game, and show that they are maintained at a certain level between the optimum solution in terms of minimum generation cost and the optimal solution in terms of an equal rate-of-charge during a given period.

Apart from the Stackelberg game, several other non-cooperative games are deployed in Reference [77,78,79] to solve electric vehicle charging problems. Reference [77] proposes a stochastic mean-field theoretic framework to solve a distributed charging problem for a large-number of electric vehicles, and considers a scenario where a large number of vehicles charge simultaneously from a charging station. The proposed solution to this problem enables each vehicle to dynamically control its charging process and finish the process within an appropriate time so that the total cost of charging is minimized. A similar charging problem is addressed in Reference [78], where it is formulated as a non-cooperative game, and a Newton-type fixed-point algorithm is developed to compute the Nash equilibrium. It is shown that, in theory, this algorithm can achieve a super-linear convergence rate, and that the vehicle’s best response can be quickly implemented in a distributed way to minimize the charging cost of each vehicle.

Reference [79] suggests an electric vehicle scheduling scheme to solve the problem of variable wind generation. In the suggested model, a virtual power plant operator aims to balance the mismatch between wind generation and demand by offering electric vehicles monetary incentives to charge when needed. The key idea is that the incentives are proportional to the power needed, and the electric vehicle owners decide when to charge according to these incentives and other considerations, such as their battery’s state-of-charge and their daily schedule. The distributed approach reduces computational time since the operator only has to set the incentives, according to the overall wind generation and load, rather than to control each vehicle separately. Moreover, it provides the electric vehicle owners more freedom and prevents privacy issues. Regarding the balancing between wind generation and load, it is shown that the difference between the results of the distributed approach and the centralized one are negligible.

Reference [80] combines game theory with microeconomics to study the charging scheduling problem among a family of electric vehicle aggregators. In the proposed model, each aggregator considers the actions of their neighboring aggregators, and attempts to minimize their vehicle charging costs by determining charging start times and profiles. The interaction among these aggregators is modeled using a two-stage non-cooperative game. Next, the game is studied using two user behavioral models: expected utility theory and prospect theory. This is done in order to study the influence of irrational decisions of the aggregators on the results of the game. An exhaustive analysis reveals that the proposed charging strategy reduces electric vehicle charging cost and the peak-to-average ratio of the load in the system, and that it is resilient to irrational actions of the aggregators.

Table 2. Summary of works focusing on energy balance application using game-theory.

	Ref.	Game Method	Players	Application
Demand Response	[59]	Non-cooperative	Energy retailers, customers	Single period demand response management
	[60]	Stackelberg	Utility companies, energy consumers	Multi-period demand response management
	[61]	Non-cooperative	Residential consumers, electricity providers	Multi-periodic demand response management
	[62]	Stackelberg	Energy management center and its customers	Optimal control of customers’ load
	[63]	Stackelberg	Utility company, data centers	Demand response for geographically distributed data centers
Demand Side Management	[52]	Cournat	Energy consumers	Demand side management for balancing residential loads
	[55]	Non-cooperative	Users with storage devices	Demand side management for smart-grids with storage
	[53]	Non-cooperative	Energy consumers	Demand side management for residential loads
	[54]	Aggregative	Energy consumers	Energy consumption scheduling
	[56]	Non-cooperative + Stackelberg	Energy providers, customers	Demand side management for consumers with storage systems
	[57]	Non-cooperative	Users with storage devices	Demand side management for consumers with storage systems
	[58]	Differential	PV systems	Demand side energy management for PV systems
Energy management in micro-grids	[64]	Non-cooperative	Residential users with storage systems	Evaluation of energy consumption
	[65]	Coupled constrained	Residential users	Energy consumption scheduling for residential users
	[66]	Non-cooperative current control game	Generator, battery, ultracapacitor	Energy management in hybrid storage systems
	[67]	Stackelberg	Micro-grids, consumers	Energy management of residential consumers with storage devices
	[81]	Non-cooperative	Multiple micro-grids	Energy consumption scheduling
	[68]	Stackelberg	Storage provider, prosumers	Energy sharing for PV prosumers in the smart grid
	[69]	Stackelberg	Storage system owner, consumers	Energy allocation in a residential building
	[73]	Cooperative bargaining game	Energy hubs	Cooperative energy scheduling for neighboring energy hubs
	[70]	Coalitional Game	Consumers with storage systems	Sharing storage systems among consumers
	[71]	Coalitional Game	Consumers with storage systems that have ramp constraints	Sharing storage systems among consumers
	[72]	Coalitional Game	Consumers with storage systems that have ramp constraints	Sharing storage systems among consumers
	[74]	Cooperative	Energy storage systems	Coalitional energy management for storage systems
Electric vehicle applications	[75]	Stackelberg	Aggregator, electric vehicles	Optimal charging scheduling of multiple vehicles
	[76]	Stackelberg	Aggregator, electric vehicles	Optimal charging in the presence of demand uncertainty
	[77]	Stochastic mean field game	Aggregator, electric vehicles	Optimal charging in electric vehicles
	[78]	Non-cooperative	Aggregator, electric vehicles	Optimal charging in electric vehicles
	[79]	Incentive-based game	Wind generator, electric vehicles	Minimizing energy imbalance between a wind generator and its connected vehicles
	[80]	Non-cooperative	Multiple aggregators	Coordinated electric vehicle charging

Table 2. Summary of works focusing on energy balance application using game-theory.

	Ref.	Game Method	Players	Application
Demand Response	[59]	Non-cooperative	Energy retailers, customers	Single period demand response management
	[60]	Stackelberg	Utility companies, energy consumers	Multi-period demand response management
	[61]	Non-cooperative	Residential consumers, electricity providers	Multi-periodic demand response management
	[62]	Stackelberg	Energy management center and its customers	Optimal control of customers’ load
	[63]	Stackelberg	Utility company, data centers	Demand response for geographically distributed data centers
Demand Side Management	[52]	Cournat	Energy consumers	Demand side management for balancing residential loads
	[55]	Non-cooperative	Users with storage devices	Demand side management for smart-grids with storage
	[53]	Non-cooperative	Energy consumers	Demand side management for residential loads
	[54]	Aggregative	Energy consumers	Energy consumption scheduling
	[56]	Non-cooperative + Stackelberg	Energy providers, customers	Demand side management for consumers with storage systems
	[57]	Non-cooperative	Users with storage devices	Demand side management for consumers with storage systems
	[58]	Differential	PV systems	Demand side energy management for PV systems
Energy management in micro-grids	[64]	Non-cooperative	Residential users with storage systems	Evaluation of energy consumption
	[65]	Coupled constrained	Residential users	Energy consumption scheduling for residential users
	[66]	Non-cooperative current control game	Generator, battery, ultracapacitor	Energy management in hybrid storage systems
	[67]	Stackelberg	Micro-grids, consumers	Energy management of residential consumers with storage devices
	[81]	Non-cooperative	Multiple micro-grids	Energy consumption scheduling
	[68]	Stackelberg	Storage provider, prosumers	Energy sharing for PV prosumers in the smart grid
	[69]	Stackelberg	Storage system owner, consumers	Energy allocation in a residential building
	[73]	Cooperative bargaining game	Energy hubs	Cooperative energy scheduling for neighboring energy hubs
	[70]	Coalitional Game	Consumers with storage systems	Sharing storage systems among consumers
	[71]	Coalitional Game	Consumers with storage systems that have ramp constraints	Sharing storage systems among consumers
	[72]	Coalitional Game	Consumers with storage systems that have ramp constraints	Sharing storage systems among consumers
	[74]	Cooperative	Energy storage systems	Coalitional energy management for storage systems
Electric vehicle applications	[75]	Stackelberg	Aggregator, electric vehicles	Optimal charging scheduling of multiple vehicles
	[76]	Stackelberg	Aggregator, electric vehicles	Optimal charging in the presence of demand uncertainty
	[77]	Stochastic mean field game	Aggregator, electric vehicles	Optimal charging in electric vehicles
	[78]	Non-cooperative	Aggregator, electric vehicles	Optimal charging in electric vehicles
	[79]	Incentive-based game	Wind generator, electric vehicles	Minimizing energy imbalance between a wind generator and its connected vehicles
	[80]	Non-cooperative	Multiple aggregators	Coordinated electric vehicle charging

4. Grid Planning

Modern electric grids include new components, such as energy storage devices, renewable energy sources, and electric vehicle charging stations. Such components may be owned and managed by different entities with contradicting objectives; nevertheless, grid planning procedures today tend to ignore these contradictions and assume the existence of one entity with complete knowledge and control span. Since this fundamental assumption is gradually becoming unrealistic, several authors suggest new grid planning procedures that take these different objectives into account. Naturally, game-theory plays a major role in these works.

For instance, since modern grids include multiple generation assets that are owned by private entities, it is often a challenge to coordinate the development of these assets alongside the on-going expansion of the transmission system. As a simple example, in many countries large photovoltaic plants require development of the transmission system, which may not be optimal in view of increasing loads in other areas. To address this problem, several works formulate non-cooperative games between transmission system planners and competing generation firms, in order to study the behavior of each player at equilibrium. Furthermore, as a guideline for transmission expansion planning, theoretic concepts from cooperative game theory are used to determine the benefits obtained by different users of the transmission system and the most efficient and fair way to allocate the cost among them. A similar problem is to plan the portfolio of energy resources and storage devices, including their size, location and preferred technology. Both non-cooperative and cooperative games are used in this context. For example, to find the optimal technology for a generation asset, several papers formulate a non-cooperative game in which different technologies compete with one another. However, when planning a combination of different sources, it seems that cooperative games are more popular for finding the optimal mix. For example, many studies employ a cooperative game to find the optimal combination of renewable energy sources and storage devices in a hybrid power plant or a micro-grid. In addition, non-cooperative games are often used for optimal allocation of electric-vehicle charging stations.

In this section, we present papers that address grid planning problems in the following order: generation and transmission expansion, micro-grid design, resource sizing, and electric vehicle charging station planning. In each sub-section, we further categorize the papers according to the game theoretic approach they employ. All of the considered studies are summarized in Table 3.

4.1. Generation and Transmission Expansion Planning

Generation and transmission expansion planning requires long-term investments and coordination among multiple entities. The following papers introduce game-theory based algorithms to optimize the planning procedure and overcome inherent conflicts. Several of them suggest methods to estimate and reduce the overall costs of planned projects, while others propose methods to select an optimal deployment for different technologies. Works [82,83,84,85] have studied generation and transmission expansion planning problems using non-cooperative games. Reference [82] formulates a game among available power plant technologies to identify the ideal one in terms of profits, reliability and degree of expansion. Using the Cournot model of oligopoly behavior, it is shown that generation resources grow faster when they are distributed among competitors than when they are managed in a traditional monopolistic manner. Moreover, the formation of a coalition among the competitors, namely, a cartel, yields the greatest profits, but results in the lowest expansion rate and reliability. Reference [83] models a game between generation and transmission companies, each trying to maximize their profits while maintaining the voltage stability of the power system. Comparison of several expansion scenarios shows that simultaneous expansion is the most profitable one. Similar to this, Reference [84] formulates a game between a transmission company and a wind farm, allocating energy storage to smooth power fluctuations while meeting a governmental target of wind power curtailment rate. Based on a case study, it is shown that the companies will fail to find an equilibrium point without the government’s interference in the game, i.e., changes in regulations that may result in a sub-optimal solution. Reference [85] models a Stackelberg game between transmission (leader) and generation (follower) companies, and compares two planning models in which the incentives of the transmission operator are different: in the first, the operator maximizes profits, whereas, in the second, it maximizes welfare (thus reducing the electricity price). The results show that, if the transmission network is non-congested, there are no significant differences between the models. However, if the network is congested, then the incentive of the transmission company has major impact on the overall welfare. In such cases, the conclusion is that a regulated transmission company is more efficient for developing the grid and enhancing welfare than when it solely relies on profit incentives.

Studies [86,87,88] have employed a cooperative game for studying conflicts between generation and transmission expansion projects. Reference [86] considers expansion projects as the players of an Aumann-Shapley cooperative game, trying to minimize their costs. It is shown that the suggested method can overcome some major drawbacks of other existing planning methods. These include the inability to capture the dependency of the benefits of a projects on: (1) the interactions among different projects in the expansion plan, (2) the order of deployment of projects, (3) grouping different projects within the expansion plan, e.g., considering two new transmission lines as one project or two. Moreover, it is shown to be superior in computation time. In Reference [87], a cooperative game between a transmission company and a renewable energy (RE) generation company to maximize their profits is formulated. The suggested methodology models a negotiation between the players for the cost sharing and recovery of investment of a new transmission line permitting delivery of RE to the grid. This work also discusses the ability of RE subsidies to steer the negotiated solution towards a transmission plan that maximizes total net benefits for all market participants. Reference [88] formulated a cooperative game among flexibility providers—demand side management through flexible loads, fast-ramping gas turbines, hydro-power plants and high voltage cross-border transmission lines (inter-connectors). To this end, the Shapley value accounts for different sequences, in which technologies are deployed, from a perspective of uncertainty regarding learning and innovation, as well as lead-time. For instance, some components, e.g., transmission lines, might require a longer lead-time from day of decision to day of operation, compared to other alternatives, e.g., gas turbines. The results demonstrate the disadvantages of long lead-time of grid investments and the advantage of cost-efficient demand side management solutions.

4.2. Micro-Grids and Resource Sizing

Many studies employ game theory for planning the portfolio of resources in modern electrical grids. The planning procedure includes determining the size, location and technology of the resources, in either a micro-grid or the main grid. The following papers use game theory for predicting the behavior of the resources’ owners, for optimizing the planning procedure while taking into account the owner’s independence and private objectives, and for studying opportunities for cooperation.

Reference [89,90,91,92] used non-cooperative games for planning the portfolio of different resources in the grid. In Reference [89], a non-cooperative game among residential owners of solar photovoltaic (PV) sources and energy storage (ES) devices is formulated to optimize their capacities while minimizing the overall costs. The suggested model, which considers the varying electricity price that is a result of the individual load management of the customers, indicates that there is an optimal ratio between the user’s load and the capacity of its PV source and ES device. Reference [90] investigates the optimal sizing and siting of distributed generation sources (DGs) using a bi-level approach. The first level is used to locate the DGs while maintaining power quality parameters using multi-objective optimization. In the second level, a Stackelberg game is formulated between the DGs owners (leaders) who maximize their profits and a distribution company (follower), which selects the best pricing contract to minimize its power payments, taking into account network constraints. The results show that DGs are optimally located far from the substations, and that the profit at the equilibrium point is inversely proportional to the number and capacity of the DGs due to competition among DGs.

Several studies focus on capacity planning of components in micro-grids. For example, in Reference [92], a non-cooperative game among MGs, buying and selling electricity, is used to optimize the capacity of their renewable sources. The suggested method indicates that the needed capacity can be reduced when taking into account the participation of MGs in the electricity market. Reference [91] investigates a unique form of a MG, namely an industrial MG, that has generation surplus during weekends and a fairly neutral generation to load ratio the rest of the week. The authors use a dual non-cooperative game—an external game among MGs and an internal one among renewable energy sources that aim to maximize their profits. The results shows that PV owners who are part of an industrial MG have a shorter return of investment period and that the best compromise among players is obtained when prices for internal exchanges follow the prices of the external ones.

Several works employ cooperative games to study planning of resources and interactions among micro-grids. Works [93,94,95] have used a cooperative game to size and select the most cost effective technology for renewable energy sources and energy storage systems. In Reference [96], hybrid renewable power plants are investigated using both cooperative and non-cooperative games in which the players are optional sources-wind turbines, photovoltaic panels, and batteries, which are trying to maximize their profit. A comparison between the results of the games reveals that the equilibrium point achieved in a cooperative game, when two or more sources cooperate, increases the overall profits. In addition, this work tests the stability of the equilibrium and its sensitivity to various uncertainties and correlations. The main conclusion is that the profits and performance of hybrid plants with more than one source are more stable. Similar to this, in Reference [93], the authors model a cooperative game among renewable energy sources-wind turbines, solar panels and batteries, which try to maximize their profits while meeting the electrical load requirements. The results are that the profit is maximized when wind turbines and solar panels cooperate. Reference [94] formulates a cooperative game among battery technologies—lead-acid, lithium-ion and vanadium redox flow (VRB) to maximize the profits of an energy storage system. The results indicate that a combination of lithium-ion batteries (70%) and VRBs (30%) feature the economically optimal solution. Reference [95] uses a cooperative game between a power-to-gas (P2G) stations and the electricity network to maximize their profits. P2G technology has a two-step process: convert excess electricity generated from renewable sources (such as wind or solar energy) into methane, and then use this gas to generate electricity. Using the Nash Bargaining cooperative game between P2Gs, the electricity network and the natural gas system, it is shown that cooperation gives a good distribution of the profits among the different participants compared to other methods. In Reference [97], a cooperative game is used to solve the conflicts between micro-grids’ internal (among components) and external (with the grid and other MGs) interactions. It establishes a cooperative game among MGs, built of renewable energy resources, to minimize their overall costs. A benchmark is set using a non-cooperative game and it is shown that the cooperative game provides better results. Moreover, it is shown that additional cost reduction can be achieved by adding an incentive mechanism that encourages cooperation among interconnected MGs towards a socially optimal planning, and by distributing the total investment cost in a fair manner.

4.3. EV Charging Stations

The increasing presence of electric vehicles (EV), which have limited traveling range and long charging periods, requires careful sizing and placement of their charging stations. Moreover, there are different types of stations, which are usually divided according to their charging time: rapid, fast and slow. The optimal portfolio of charging stations in an electrical grid is affected by the behavior of the drivers and traffic patterns, both spatial and temporal, and has to be coordinated with the distribution network. Since there are many players with different objectives that influence the optimal portfolio, game theory is used in many studies to investigate this problem. We divide them by the type of game that is used: non-cooperative and cooperative.

Several papers propose a non-cooperative game for planning the placement and sizing of electric vehicle charging stations [98,99,100]. Reference [98] formulates a charging stations placement problem as a bi-level optimization problem with the goal of maximizing social welfare. The social welfare is measured as the overall travel cost of electric vehicles to charging stations and queuing cost at the stations. The behavior of the vehicle owners is modeled as a non-cooperative congestion game in which the congested elements are the roads and the charging stations. The optimal distribution of charging stations is determined by the equilibrium that yields the minimum social cost. Through experimental evaluation, the authors compare their method to three baseline methods and illustrate that this proposed method leads to a solution with less traffic congestion and queuing in charging stations. Similar to this, Reference [99] uses a Bayesian non-cooperative game in which the players are charging service providers who aim to locate their charging stations in a manner that maximizes profits, while ensuring the quality of service. The effects of the charging stations on the power grid is modeled through a penalty fee for disturbances that the grid operator gives to the charging service provider. These disturbances are represented by the load imbalance caused from charging electric vehicles. The main conclusions are that the location of charging stations is highly consistent with traffic flow, and that charging service providers prefer clustering stations, rather than separating them. Reference [100] formulates a Stackelberg game in which the leader is a distribution company setting a time-based electricity tariff, and the followers are costumers that participate in a demand response program and adjust their electricity consumption with accordance to the price. The consumption patterns are used to plan the optimal distribution of micro-turbines and electric vehicle parking lots in a distribution system, in a way that minimizes the cost for the distribution company. The results demonstrate that the reduction in consumption has a direct and positive effect on the planning cost of the distribution system.

Concepts from cooperative game theory are used both to allocate profit among stakeholders, as well as to allocate electric vehicles to charging stations. In Reference [101], the problem of planning fast charging stations in a distribution network is modeled as a mixed-integer non-linear problem. A concept from cooperative games theory, namely, the Nash bargaining solution, is used to allocate profits among the charging service providers and the electrical distribution company. In Reference [102], a pricing mechanism for charging electric vehicles and the allocation of electric vehicles among charging stations are modeled as a bi-level optimization problem. In the upper level, a coordinate descent optimization algorithm is used to define a time-based pricing mechanism of charging in different stations. In the lower level, a matching game between electric vehicles and charging stations is solved using the Gale-Shapley matching algorithm. It is shown through numerical analysis that the utility of the electric vehicle owners is improved by using the matching algorithm. Moreover, the overall utility increases when the amount of electric vehicles increases. This is true until the system capacity is nearly full, and afterwards the overall utility starts to decline since the charging service providers raise their prices.

Table 3. Summary of works focusing on grid planning application using game-theory.

	Ref.	Game Method	Players	Application
Transmission and generation expansion planning	[82]	Non-cooperative	Power plants of different technologies	Planning a new power plant with multiple technology options available
	[83]	Non-cooperative	Generation, Transmission	Planning generation and transmission expansion
	[84]	Non-cooperative static	Wind farm, Transmission	Transmission line planning between a wind farm and the grid
	[85]	Stackelberg	Generation, Transmission	Coordination between generation and transmission expansion planning
	[86]	Cooperative	Transmission expansion projects	Study of the benefits obtained by users of the transmission network (consumers, generators and transmission owners) from expansion projects
	[87]	Cooperative	Renewable energy sources, transmission	Transmission expansion planning
	[88]	Cooperative	Demand side management participants, fast-ramping generators, energy storage devices	Planning the integration of flexibility providers into the grid
Micro-grids design and resources sitting and sizing	[91]	Non-cooperative	Consumers and prosumers in a micro-grid	Investment planning of industrial micro-grids
	[92]	Non-cooperative	Micro-grids	Optimal sizing of distributed renewable energy sources in micro-grids
	[89]	Non-cooperative	PVs and ESs owners	Optimal sizing of residential PV sources and energy storage devices
	[90]	Bi-level, multiobjective optimization + Stackelberg	DG owners, distribution company	Planning the optimal location and operation of Distributed Generation
	[96]	Cooperative + non-cooperative	Wind Turbines, solar panels, batteries	Planning a hybrid power plant
	[97]	Cooperative	Micro-grids	Planning of renewable energy sources in a distribution network of micro-grids
	[93]	Cooperative	Wind turbines, Solar panels, Storage batteries	Capacity planning of generation sources and batteries for clustered micro-grids
	[94]	Cooperative	Battery technologies	Planning a battery system with optimized economic features and capacity
	[95]	Cooperative	Electricity system, natural gas system, Power to gas station	Planning an integrated electricity-gas system with power to gas (P2G) technology
EV charging stations	[98]	Non-cooperative	EV owners	Planning placement of fast EV charging stations
	[99]	Bayesian	Charging stations	Planning placement of EV charging stations
	[100]	Stackelberg	Distribution company, customers	Planning EV charging stations in parking lots
	[101]	Cooperative	Distribution company, Fast Charging Stations	Planning placement and sizing of fast EV charging stations
	[102]	Cooperative	Electric vehicles, charging stations	Allocation of electric vehicles among charging stations

Table 3. Summary of works focusing on grid planning application using game-theory.

	Ref.	Game Method	Players	Application
Transmission and generation expansion planning	[82]	Non-cooperative	Power plants of different technologies	Planning a new power plant with multiple technology options available
	[83]	Non-cooperative	Generation, Transmission	Planning generation and transmission expansion
	[84]	Non-cooperative static	Wind farm, Transmission	Transmission line planning between a wind farm and the grid
	[85]	Stackelberg	Generation, Transmission	Coordination between generation and transmission expansion planning
	[86]	Cooperative	Transmission expansion projects	Study of the benefits obtained by users of the transmission network (consumers, generators and transmission owners) from expansion projects
	[87]	Cooperative	Renewable energy sources, transmission	Transmission expansion planning
	[88]	Cooperative	Demand side management participants, fast-ramping generators, energy storage devices	Planning the integration of flexibility providers into the grid
Micro-grids design and resources sitting and sizing	[91]	Non-cooperative	Consumers and prosumers in a micro-grid	Investment planning of industrial micro-grids
	[92]	Non-cooperative	Micro-grids	Optimal sizing of distributed renewable energy sources in micro-grids
	[89]	Non-cooperative	PVs and ESs owners	Optimal sizing of residential PV sources and energy storage devices
	[90]	Bi-level, multiobjective optimization + Stackelberg	DG owners, distribution company	Planning the optimal location and operation of Distributed Generation
	[96]	Cooperative + non-cooperative	Wind Turbines, solar panels, batteries	Planning a hybrid power plant
	[97]	Cooperative	Micro-grids	Planning of renewable energy sources in a distribution network of micro-grids
	[93]	Cooperative	Wind turbines, Solar panels, Storage batteries	Capacity planning of generation sources and batteries for clustered micro-grids
	[94]	Cooperative	Battery technologies	Planning a battery system with optimized economic features and capacity
	[95]	Cooperative	Electricity system, natural gas system, Power to gas station	Planning an integrated electricity-gas system with power to gas (P2G) technology
EV charging stations	[98]	Non-cooperative	EV owners	Planning placement of fast EV charging stations
	[99]	Bayesian	Charging stations	Planning placement of EV charging stations
	[100]	Stackelberg	Distribution company, customers	Planning EV charging stations in parking lots
	[101]	Cooperative	Distribution company, Fast Charging Stations	Planning placement and sizing of fast EV charging stations
	[102]	Cooperative	Electric vehicles, charging stations	Allocation of electric vehicles among charging stations

5. Power System Reliability

Power system reliability refers to the ability of a system to deliver power to consumers under acceptable standards [103]. In recent years, the on-going deregulation of power markets alongside the continuing integration of renewable energy sources have led to increasing number of independent entities that operate within the same power system. Since power production is not always controlled directly by the system operator, maintaining an adequate level of reliability is becoming a serious challenge. This challenge may become more severe when independent entities have contradicting objectives. To address this challenge, game theory is used in various works to design incentive mechanisms that encourage players with energy sources and flexible loads to enhance the reliability of a system. It is shown that in many cases this approach is more efficient in stabilizing the system than a centralized approach, in which a single entity is solely responsible for system reliability. This is especially true in situations in which this single entity has limited resources.

In this section, we refer to real-time reliability, whereas planning for reliability in the long-term is considered in Section 4 (grid planning). We review papers that address three aspects of power system reliability: (1) frequency stability, (2) voltage stability, and (3) cyber attacks. These aspects are affected by interactions between at least two independent entities: cyber attacks involve an attacker and a defender, while frequency and voltage stability-related problems may involve all of the system’s operators, generators, and consumers. In this light, it is natural to use game theory to study such interactions. The considered studies are summarized in Table 4.

5.1. Frequency Stability

The frequency stability of a power system is strongly connected to the power balance between generation and demand and to the availability of frequency response services, such as spinning reserves and load frequency control (LFC). The power balance and the frequency response services are mostly affected by actions of active system players, such as system operators, generators, and loads. Due to the great number of such players and the interactions among them, game theory receives much attention in the literature. Reference [104,105] uses Bayesian games to study frequency reserve allocation. Reference [104] studies a game between a system operator, and frequency constrained electricity market participants. In this market, frequency reserve constraints are set in order to limit the frequency nadir of the grid, following a loss of renewable energy sources. Two types of market participant players are considered: (a) a price-taker, who cannot influence the prices, and (b) a price-maker, who is able to affect the prices. Results show that, in the game’s equilibrium point, the available frequency reserve of the system is maximized. Reference [105] proposes a game that models spinning reserve trading between neighboring power systems in order to achieve reserve requirements. These requirements increase together with wind turbine power capacity and wind uncertainty. The trading price is the mean between the price of the buyer and the seller at the Nash equilibrium. Once the trading price is set, the players set the quantities to be traded. The mechanism is decoupled from the system unit dispatch process and, as a result, it minimizes the changes to the existing optimal dispatch. Moreover, it allows each system to individually determine its reserve dispatch, instead of solving a multi-area reserve dispatch problem, and leads to higher wind power generation.

Mean-field games are useful for analyzing the coordination among a large number of agents. For example, Reference [106] proposes to coordinate a large number of thermostatically controlled loads (TCLs) in order to provide frequency response support. The coordination is based on a non-cooperative mean-field game between TCLs that receive two price signals, which are the price of electricity and the price of frequency response availability. Based on the price signals, the TCLs schedule their energy consumption and allocate frequency response provision in order to minimize their operational costs. Assuming that a single TCL is too small to significantly impact the prices, the TCLs influence the price signals through their aggregated power consumption and total frequency response availability. The equilibrium is computed numerically using an iterative algorithm. The authors compare the mean-field game based mechanism to a “business as usual” scenario, where loads do not exploit their flexibility to support the frequency, and to a centralized approach, in which the loads are controlled by a central unit. Results show that the game-based approach and the centralized one reduce system costs by 0.4% and 0.6%, respectively. Although the centralized approach achieves better results, the game-based approach promises the satisfaction of the users, since no player can achieve a better result.

Several papers use differential games to analyze power and frequency related problems that are described by differential equations, using either non-cooperative, or cooperative games. LFC is one example for a mechanism that is based on differential equations. Reference [107] analyzes an LFC mechanism through a non-cooperative differential game between suppliers and consumers, and proposes a real-time pricing mechanism. The objective is to increase the stability of LFC and to guarantee the supply-demand balance. The real-time mechanism is individually-rational in the sense that it attempts to be overall more beneficial to each player compared to their profit in a fixed-price market. In order to guarantee the supply-demand balance, the mechanism explicitly considers this dynamic balance as a constraint. The proposed mechanism converges to a Nash equilibrium that maximizes the social welfare, and, under certain conditions, its solution conforms with that of the system operator’s centralized optimization. Reference [108] considers a non-cooperative differential game to address the problem of frequency regulation in a power system with large scale wind power clusters. The power system is divided into areas, where each one has both conventional generators and wind turbine farms. The areas coordinate among themselves the active power generation in order to minimize each tie-line power fluctuations and frequency deviations. Each area regulates its own active power dispatch according to frequency and power injection measurements. This regulation is based on a distributed model predictive controller, which considers wind power fluctuations and critical frequency regulation parameters of generation units. Simulation results indicate that in cases of load step disturbances, the coordination may achieve smaller frequency deviations. Reference [109] considers a two-area LFC system and proposes several differential games to model the interaction between the areas while considering load and power fluctuations. The utility function of each area considers frequency and tie-line power errors, as well as control efforts, such as commands for active power output. The authors analyze a non-cooperative game, and two cooperative games, and conclude that the cooperative games lead to more stable system operation. Reference [110] also considers a multi-area LFC system and a differential game to model the interaction between the areas, with a similar payoff function. Differently from the previous papers, this one proposes a co-evolutionary algorithm to solve the game. The authors conclude that the proposed algorithm has a better suppression effect on the frequency and tie-line power deviations, and a shorter settling time compared to several other algorithms.

Finally, cooperative evolutionary game theory is used in Reference [111] to formulate a mechanism for controlling primary frequency control of hydro power plants. The main objective of the mechanism is to reduce the frequency nadir and the settling time following a power imbalance disturbance. The governors cooperate by sharing their measurements of frequency deviations, and are all coordinated through a central controller, which sends them a signal that is proportional to the rate of change of frequency (RoCoF). The authors conclude that the proposed RoCoF based control mechanism is effective for arresting the frequency response.

5.2. Voltage Stability

Power system voltage stability refers to the ability of the system to maintain voltage metrics at each bus within certain boundaries. We survey papers that address this problem and try to improve the voltage stability by coordinating between different entities, such as system operators, generators, and consumers. We divide these works into two types of non-cooperative games: static and sequential.

Several papers analyze non-cooperative static games, in which the players play simultaneously. Reference [112] proposes methods for power system operators to encourage PV owners to participate in voltage regulation, and coordinate among them to do so efficiently. The coordination is done through a pricing mechanism, and its objective is to minimize the operator’s cost of maintaining voltage stability, while maximizing the PV owners’ profit. The owners profit by providing two services: voltage regulation and active power input. The owners compete for the provision of these services; thus, a non-cooperative game is formulated among them. The proposed pricing mechanism is designed in a way that leads to a potential game among the owners, where the actions benefiting a single owner benefit all the others, as well.

Several papers formulate sequential games to describe voltage stability related problems that include multiple stages. Reference [113] introduces a game among buses in a HVDC grid who influence the voltage stability through the power they produce. Each bus aims at minimizing the voltage deviations, power drawn from the grid, and losses in its surroundings. Since the buses decisions influence one another, they iteratively update their decisions until they reach an equilibrium. This approach allows the general power dispatch optimization problem to be solved locally at each bus. Reference [114] proposes a dynamic game among prosumers in a distribution network. At each time step, the prosumers decide how much power to consume, generate or store. These decisions affect the power flows in the system; thus, the players affect one another. Moreover, their utility functions capture costs of power trading and penalties of violating voltage constraints, which encourage the players to comply with voltage constraints.Numerical analysis shows that the proposed mechanism can maintain the stability of the grid and can be implemented in practice. A non-cooperative differential game is proposed in Reference [115]. This paper discusses voltage magnitude and angle regulation during a transient in islanded micro-grids with parallel-connected inverters. In order to improve the voltage performance during a transient, a game between the inverters is proposed. Reference [116] introduces a Stackelberg game in order to solve a power dispatch problem between generators, which are the leaders, and micro-grids with generation capabilities, which are followers. The generators lead by determining their power generation, and the micro-grids follow by setting their power generation. The players’ cost function includes the costs of generation and penalties for voltage angle deviations. The authors show the existence and uniqueness of an equilibrium, and conclude that for players to reach their decisions, they mainly require to know the voltage angle at local buses. This may suggest that the proposed mechanism can be simply implemented without much resources.

5.3. Cyber Attacks

The digital evolution of the power grid and the increasing number of active participants increases its vulnerability to cyber attacks. Adequate analysis of such attacks can guide decision making on security measures that may increase the reliability of power systems and lower related economic losses. Cyber attacks generally involve two agents with counteracting objectives: an attacker, whose goal is to damage or destabilize the system, and a defender, whose goal is to maintain a reliable system with minimum investments. This leads to interactions among the agents that make game theory especially useful for modeling and analyzing cyber attacks. We survey papers that model cyber attacks and are mostly based on various types of non-cooperative games. While several papers consider the static versus the sequential nature of the game, other papers consider the information that the players are able to acquire before choosing their strategies.

Several papers consider a static game in which the attacker and the defender play simultaneously. For example, a static zero-sum and incomplete information game is presented in Reference [117] in order to analyze attack-defense interactions over load frequency control (LFC). The utility function of both the attacker and the defender takes into account the frequency deviations and the probability of the defender to detect the attack. The model considers two types of attackers and two defensive schemes. The attackers can be either damage oriented or detection-evasion oriented, while the defenders can follow an immediate or a cumulative false data detection scheme to identify compromised signals. A numerical analysis shows that detection-evasion oriented attackers can maintain a low probability of detection, whereas damage-oriented attackers can trigger emergency frequency controllers that cause severe damage.

Several papers utilize various types of sequential games to describe and analyze more complicated attacks. Reference [118] uses a Stackelberg game in a multi-level framework which models the relationships between defenders, attackers, and system operators, considering false data injection attacks, such as load redistribution attacks. The defenders start by allocating defensive resources, after which the attackers choose their strategy, and finally the operators react to the new state. The proposed defense mechanism is shown to be effective against false data injection attacks, and may be extended to address other types, as well. Reference [119] proposes a repeated game to analyze a framework where each synchronous generator in the system has a local energy storage that regulates the rotors’ speed. The game is between an attacker and a defending utility that reacts to their attacks. The attacker attempts to control the power injections from the storage unit towards the generators, while the defender attempts to stabilize the generators’ rotor speed by controlling their power injections. The players’ payoff function is based on the rotor speed deviation and on their control efforts. The attacker actions are stochastic, and as a result, the analysis focuses on the long term average outcome to the defender. The authors conclude that, if the defender acts at each repeated game, ignoring the control efforts penalties, then its long-term average payoff can be better compared to the game-theoretic strategy in which it attempts to optimize its actions at each repeated game. Reference [120] proposes a differential game where attackers try to control a subset of distributed energy resources (DERs) in order to destabilize the system, while a defending utility aims to stabilize it by using another subset of system resources. The cost function includes deviations from a stable state. The paper demonstrates that, if the utility is able to identify uncompromised DERs, then it can take countermeasures that will effectively reduce the impact of an attack. On the other hand, if uncompromised DERs are not identified, then a coordinated attack can lead to instabilities. Reference [121] offers a game model where both the defender and the attacker can affect the overall system damage by identifying critical substations and the chronological order in which they can be attacked. The authors conclude that subsequent attacks can incur a significantly higher damage compared to simultaneous attacks, and that the models are effective in improving the system resilience under such chronological attacks.

Several papers consider the players’ information on the system state or the other players’ strategies. A player that has no information is considered static, whereas one that has some information is considered dynamic. Reference [122] considers both a static and a dynamic attacker and proposes to analyze the worst possible outcome. During the game, the defenders allocate their budget between protection and recovery measures. Two different problems are considered: (a) how to allocate a limited budget in order to maintain low losses, and (b) how much budget is needed in order to limit the losses to an expected value. The authors conclude that the loss to the defender can be predicted and limited, and that for large systems the algorithms might take long time to compute. Reference [123] uses a non-cooperative game in order to establish a probabilistic defensive algorithm and to reduce the power system vulnerability to a cascading failure. Different types of attackers are considered that are either static or dynamic. The attacker chooses its strategy, then the possible cascading failures are identified, and finally the defender takes countermeasures based on the identification of the cascading failures and the components which are likely to be attacked. The authors suggest that highly capacity-constrained systems are especially vulnerable to cascading failures.

In addition, several papers use cooperative games to examine whether cooperation among defenders can improve their cyber protection. For example, Reference [124] proposes a game that examines cooperation among consumers to locate abnormalities in the system due to false metering and power blackouts events. It is assumed that the consumers are willing to pay in order to detect these abnormalities and reduce monetary losses. This is done by finding load changes during normal conditions, false metering events and power blackout events.

Table 4. Summary of works focusing on reliability applications using game-theory.

	Ref.	Game Method	Players	Application
Frequency stability	[104]	Non-cooperative Bayesian	System operator, generators, consumers	Frequency reserve allocation
	[105]	Non-cooperative Bayesian	Power systems	Frequency reserve allocation
	[106]	Non-cooperative mean-field	Thermostatically controlled loads	Frequency reserve allocation
	[107]	Non-cooperative differential	Suppliers and consumers	Supply-demand balancing
	[108]	Non-cooperative differential	Power system areas	Active power regulation
	[109]	Cooperative and non-cooperative differential	Power system areas	Active power regulation
	[110]	Cooperative differential	Power system areas	Primary frequency control
	[111]	Cooperative evolutionary	Hydro plants	Active power regulation
Voltage stability	[113]	Non-cooperative	High-voltage DC (HVDC) buses	Voltage regulation
	[112]	Non-cooperative static	PV owners	Voltage regulation
	[115]	Non-cooperative differential	Power inverters	Voltage regulation during transients
	[114]	Non-cooperative dynamic	Prosumers	Voltage regulation
	[116]	Non-cooperative Stackelberg	Generators and micro-grids	Voltage regulation
Cyber attacks	[117]	Non-cooperative incomplete information	Attacker and defender	False data injections
	[118]	Non-cooperative Stackelberg	Attacker, defender, system operator	Hijacking and false data injections
	[119]	Non-cooperative repeated	Attacker and defender	Hijacking
	[120]	Non-cooperative differential	Attacker and defender	Hijacking
	[121]	Non-cooperative static	Attacker and defender	Disable elements
	[122]	Non-cooperative static and dynamic	Attacker and defender	Disable elements
	[123]	Non-cooperative static and dynamic	Attacker and defender	Disable elements
	[124]	Cooperative	Attacker and defender	Locating abnormalities in the electric grid

Table 4. Summary of works focusing on reliability applications using game-theory.

	Ref.	Game Method	Players	Application
Frequency stability	[104]	Non-cooperative Bayesian	System operator, generators, consumers	Frequency reserve allocation
	[105]	Non-cooperative Bayesian	Power systems	Frequency reserve allocation
	[106]	Non-cooperative mean-field	Thermostatically controlled loads	Frequency reserve allocation
	[107]	Non-cooperative differential	Suppliers and consumers	Supply-demand balancing
	[108]	Non-cooperative differential	Power system areas	Active power regulation
	[109]	Cooperative and non-cooperative differential	Power system areas	Active power regulation
	[110]	Cooperative differential	Power system areas	Primary frequency control
	[111]	Cooperative evolutionary	Hydro plants	Active power regulation
Voltage stability	[113]	Non-cooperative	High-voltage DC (HVDC) buses	Voltage regulation
	[112]	Non-cooperative static	PV owners	Voltage regulation
	[115]	Non-cooperative differential	Power inverters	Voltage regulation during transients
	[114]	Non-cooperative dynamic	Prosumers	Voltage regulation
	[116]	Non-cooperative Stackelberg	Generators and micro-grids	Voltage regulation
Cyber attacks	[117]	Non-cooperative incomplete information	Attacker and defender	False data injections
	[118]	Non-cooperative Stackelberg	Attacker, defender, system operator	Hijacking and false data injections
	[119]	Non-cooperative repeated	Attacker and defender	Hijacking
	[120]	Non-cooperative differential	Attacker and defender	Hijacking
	[121]	Non-cooperative static	Attacker and defender	Disable elements
	[122]	Non-cooperative static and dynamic	Attacker and defender	Disable elements
	[123]	Non-cooperative static and dynamic	Attacker and defender	Disable elements
	[124]	Cooperative	Attacker and defender	Locating abnormalities in the electric grid

6. Discussion

Power system management and control problems are often studied as optimization problems, in which the underlying assumption is that there exists one entity with unlimited knowledge and control span [9,10,11,12,13]. While this approach might have been relevant for many years, it is gradually becoming unrealistic due to the decentralization and deregulation of energy markets. A major challenge is therefore to predict the development of a power system, taking into account the different objectives of many players. As many of the reviewed papers demonstrate, game theoretic approaches are especially suitable for analyzing complex interactions among different independent players. However, the type of game that is being used and the accuracy of the model are crucial for obtaining control mechanisms and policy recommendations that are both efficient and applicable.

Most studies consider non-cooperative games and address situations in which multiple players with conflicting objectives interact. In many cases formal analysis reveals that the game has a unique equilibrium, hence allowing to predict the players’ behavior and to design the system accordingly. In case of multiple equilibria the typical approach is to define appropriate regulations, in order to stir the players to the most desired outcome. In the majority of studies, this is done by designing a pricing mechanism that provides incentives for players to perform the actions needed to reach a certain goal, such as maximizing the profits of all players, balancing supply and demand, or regulating the voltage. The reviewed studies also show that taking into consideration the objectives of all players is key for designing effective mechanisms. For instance, several studies propose pricing models for demand side management, in which utility companies set prices to maximize their revenues or to improve the stability of the system, and customers respond accordingly, with an attempt to optimize their own objectives. It has been shown that in some cases such pricing mechanisms efficiently steer customers to shift their peak consumption and distribute their load throughout the day, thus reducing the peak-to-average ratio of the overall load and improving the system’s efficiency. In another example, it has been found that a competitive voltage regulation market is efficient in maintaining voltage levels within normal limits and can be designed to benefit all players who participate in voltage regulation, as well as the system operator. Based on these and similar works, several studies conclude that the results obtained using a game-theoretic model are almost as good as those obtained by central planning, even when it is assumed that full knowledge is available to the central planner. Moreover, several studies show that the results obtained from game-theory-based mechanisms for supply-demand balancing and load frequency control coincide with centralized system operator optimization.

Non-cooperative game theory is also very useful for deciding on an effective energy policy. The main challenge of policy makers is to set rules that improve the overall social welfare, without diminishing the advantages of a free market. In this context, various studies employ non-cooperative game theory to examine the influence of regulation in a competitive environment, on both the overall social welfare and on each player as an individual. For example, in energy trading problems, many studies define a non-cooperative game with competing players, typically sellers and buyers, and investigate equilibrium points of the power market under various constraints. In several cases the results may help shape suitable policies. For example, it is suggested that even in competitive markets governments should apply some profit distribution mechanisms to ensure fairness. Moreover, it has been concluded that reasonable bidding rules should be considered for limiting high prices, and for preventing market manipulations by large generation companies.

Interesting results also arise from studies that employ cooperative game theory. The reviewed studies show clearly that cooperation among different entities in power system markets often improves the benefits of all cooperating players and leads to a higher social welfare. This conclusion is true for all types of applications that have been reviewed in the present work. In energy trading problems it is shown that when players can bargain directly with each other to reach binding decisions, cooperative models can improve their profits and lead to higher social welfare. For example, it is shown that distributed energy sources or interconnected micro-grids can cooperate in competitive electricity markets to increase their profits. Moreover, in energy balancing problems, cooperation is shown to be beneficial for both generators and consumers. Cooperation is found to be especially beneficial for energy sources with variable power output and uncertainties in generation, e.g., wind turbines and solar PV sources. Through cooperation the sources gain more control over their generation, and may adjust it according to the market’s needs, to the benefit of all players. Moreover, it is shown that shared purchasing and management of energy storage devices allows to utilize them better, thus reducing the cost for all consumers.

Recent papers also shows that cooperation among players can be used to improve the stability of a power system. Several studies show that cooperation among generating companies can be designed to improve the frequency stability, thus benefiting all the participants. For example, it has been shown that RoCoF-based control, in which hydro-power plants share their frequency data to regulate the speed of their rotors, can effectively regulate frequency in the interconnected system. With regard to grid planning problems, when planning the portfolio of energy sources and storage devices, including their size, location and technology, cooperative games are useful for finding the optimal mix. For example, many studies use cooperative games to find the optimal combination of renewable energy sources and storage devices in a hybrid power plant or a micro-grid. Specifically, cooperation seems to be extremely beneficial for expansion-planning problems, as several papers show that cooperation between transmission and generation expansion projects maximizes the profits of all participants and improves the utilization of the network. Other studies use a non-cooperative approach, in which a regulated transmission company is the leader, defining the expansion of the transmission system, and the generating companies react accordingly. This approach enables to define the expansion of the transmission system based on the expected reactions of the generating companies. The role of regulation in this case is to ensure that the planning of the transmission system is done to maximize the overall social welfare, rather than solely the profits of the transmission companies.

Another important conclusion of many studies is that choosing a relevant scenario and fine-tuning the assumptions of a model are crucial for the applicability of the suggested solution. Several papers show that their assumptions regarding the regulation and market rules, such as legal limitations, governmental incentives, and the business model of the utility company, have a strong impact on the results. Moreover, in some cases the suggested solutions are not feasible in certain markets, highlighting the need to define beforehand a relevant market and regulatory framework. In addition, game theoretic models that are highly accurate often lead to complex results that are hard to implement in practice. Competitive energy markets in modern electrical grids may include a significant number of players, each having a large variety of decisions, which influence a great number of components in the system. Solving such problems from a game-theoretic perspective often leads to iterative algorithms that are too complex. In such cases, many studies take an analytical approach to find equilibrium points, thus avoiding the computational burden incurred by iterative algorithms. In addition, several studies employ evolutionary game theory due to its computational efficiency.

7. Current and Future Trends

We show here the results of a content analysis that targets applications of game theory in power systems. The analysis focuses on the four main applications reviewed in this paper and uses the Scopus and IEEE Xplore databases. To cover a large variety of relevant papers, we searched for the expression “game*” (the asterisk is used as a wild-card, allowing the search-engine to capture both game and games) and a combination of synonyms that describe the relevant applications. The search was also restricted to the fields title, abstract, and keywords, thus capturing only studies that use game theoretic concepts as a main approach. The search results and statistical analysis are summarized in

Table 5. Historical trends for typical applications of game theory in power systems.

	Proportion of Publications	Application Area	Trends	Statistical Data
	over 2013–2019		over 2013–2019	#	Slope	p-Value
Energy trading		General market		118	4.7140	0.0053
		Micro-grids		64	3.9643	0.0032
		Electricity pricing		106	0.8214	0.2578
		Bidding strategies		54	1.4643	0.0241
		Profit allocations		20	0.6429	0.2320
Energy balancing		Demand side management		126	0.8929	0.5837
		Energy management in micro-grids		94	4.4643	0.0003
		Electric vehicles		65	1.0714	0.0990
Grid planning		Generation and transmission expansion		43	1.0357	0.0369
		Micro-grids and resource sizing		90	1.9643	0.0842
		Electric vehicle charging stations		57	2.5710	0.0610
Power system reliability		Cyber attacks		47	2.1429	0.0020
		Frequency stability		40	0.3571	0.0268
		Voltage stability		35	0.9643	0.1309

. The first column denotes the category, the second and third columns depict the proportion of publications in a specific application area over a 7-year period, and the fourth column visualizes linear trends over the same period. The remaining columns summarize statistical data. We show trend lines, which were calculated via individual one-sample 2-sided t-tests, using a linear regression tool from the software “R”. Based on the results, we divide the application areas into two groups with strong ($p<0.05$) and weak statistical significance (Note that some data cannot be accurately described using linear regression; hence, nonlinear tools have to be used instead to reveal more complex trends. However, for simplicity, we report here only explicit linear trends). We further divide the strong trend group to application areas with rapidly rising interest (slope$>3$) and moderately rising interest (slope $<3$):

• Strong trends-rapidly rising interest:

  -

  energy trading & general market, or micro-grid applications;

  -

  energy balancing & energy management in micro-grids;

• Strong trends—moderately rising interest:

  -

  energy trading & bidding strategies;

  -

  grid planning & generation or transmission expansion;

  -

  power system reliability & cyber-attacks or frequency stability;

• Weak trends-fluctuating interest:

  -

  energy trading & electricity pricing or profit allocation;

  -

  energy balancing & electric vehicles or demand side management;

  -

  grid planning & resource sizing, or power system reliability, or voltage stability.

The rising trends are now in the focus of the community, probably due to global initiatives targeting the development of low-carbon and energy efficient power systems, and due to more active involvement of the end-consumers in beyond-the-meter technologies. It is the authors’ opinion that several of the weak trends are the most interesting ones. For instance, generation and transmission expansion planning or voltage stability may be under-evaluated topics, which fits nicely in line with the game-theoretical formalism.

8. Conclusions

This study reviews recent papers that employ game theoretic tools to study the operation and design of modern electric grids, with a focus on four application areas: energy trading, energy balancing, grid planning, and power systems reliability. One central conclusion is that modeling the system from the perspective of one entity with unlimited information and control span is often impractical; hence, correct modeling of the selfish behavior of independent players may be critical for the development of future power systems. In this context, game theoretic approaches seem to be particularly suitable for analyzing complex interactions among different independent players that participate in energy markets, and response to energy policies. Most studies consider non-cooperative games, in which it is consistently shown that proper incentives and regulations are crucial for optimizing the social welfare. In addition, several studies show that correct usage of incentives by appropriate regulation or sophisticated pricing mechanisms may improve the social welfare, and in several cases the results obtained are almost as good as those obtained by central planning. Interesting results also arise from studies that employ cooperative game theory, which clearly show that cooperation among different entities in power system markets often improves the benefits of all cooperating players. This conclusion holds for all types of applications that have been reviewed in the present study.

Based on an extensive content analysis, we point to several trends in the current research, and offer directions for new research. First, it is the authors’ opinion that several of the weak trends identified in Section 7 might have been overlooked and deserve more attention. More specifically, some of the less explored topics are multi-stage planning problems, profit allocation among aggregators of distributed energy sources and flexible loads, and voltage regulation in smart grids. Concerning the type of game, most works formulate a static game with a single stage, which may be used to reflect a situation in which players decide simultaneously. However, in reality, many processes are dynamic and develop over many stages; hence, there is room for additional studies that focus on multi-stage games.

Moreover, to ensure the accuracy of the results and feasibility of the suggested solutions, future studies may emphasize the operational constraints imposed by electricity and natural gas networks. In addition, a key limitation of game theory is the resulting computational complexity, which increases exponentially with the number of players. Therefore, developing algorithms that reduce the computational burden may be a promising research direction. In the context of electric vehicles, when implementing energy charging scheduling schemes that require real-time data, it may be of interest to consider the privacy of electric vehicle owners. A possible direction to tackle this problem could be employing tools from Bayesian game theory. In addition, uncertainties in the arrival and departure times of electric vehicles in charging stations can be studied in the context of dynamic or stochastic games. Regarding the topic of cyber-attacks, there seems to be a shortage of models that consider cooperation among players with common goals. Indeed, several papers study cooperation among defenders, however cooperation among attackers should be considered, as well, when evaluating cyber-security. Lastly, only a few studies use game theory, in particular cooperative game theory, in order to explore the issue of voltage stability. This may become an important topic due to the ongoing deregulation of power markets and the continuing integration of renewable energy sources.