Next Article in Journal
Historical Buildings Potential to Power Urban Electromobility: State-of-the-Art and Future Challenges for Nearly Zero Energy Buildings (nZEB) Microgrids
Previous Article in Journal
Estimation of Underground MV Network Failure Types by Applying Machine Learning Methods to Indirect Observations
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks

1
Department of Electrical Engineering, University of Azad Jammu and Kashmir, Muzaffarabad 13100, AJK, Pakistan
2
Department of Electrical Engineering, College of Engineering, Taif University, Al-Hawiyah, Taif P.O. Box 888, Saudi Arabia
*
Author to whom correspondence should be addressed.
Energies 2022, 15(17), 6297; https://doi.org/10.3390/en15176297
Submission received: 26 July 2022 / Revised: 16 August 2022 / Accepted: 26 August 2022 / Published: 29 August 2022
(This article belongs to the Section F5: Artificial Intelligence and Smart Energy)

Abstract

:
The integration of Renewable Energy Resources (RERs) into Power Distribution Networks (PDN) has great significance in addressing power deficiency, economics and environmental concerns. Photovoltaic (PV) technology is one of the most popular RERs, because it is simple to install and has a lot of potential. Moreover, the realization of net metering concepts further attracted consumers to benefit from PVs; however, due to ineffective coordination and control of multiple PV systems, power distribution networks face large voltage deviation. To highlight real-time control, decentralized and distributed control schemes are exploited. In the decentralized scheme, each zone (having multiple PVs) is considered an agent. These agents have zonal control and inter-zonal coordination among them. For the distributed scheme, each PV inverter is viewed as an agent. Each agent coordinates individually with other agents to control the reactive power of the system. Multi-agent actor-critic (MAAC) based framework is used for real-time coordination and control between agents. In the MAAC, an action is created by the actor network, and its value is evaluated by the critic network. The proposed scheme minimizes power losses while controlling the reactive power of PVs. The proposed scheme also maintains the voltage in a certain range of ±5%. MAAC framework is applied to the PV integrated IEEE-33 test bus system. Results are examined in light of seasonal variation in PV output and time-changing loads. The results clearly indicate that a controllable voltage ratio of 0.6850 and 0.6508 is achieved for the decentralized and distributed control schemes, respectively. As a result, voltage out of control ratio is reduced to 0.0275 for the decentralized scheme and 0.0523 for the distributed control scheme.

1. Introduction

1.1. General

Two main problems in the modern era are global warming and power shortages. The best power system is one that uses less energy and has less of an impact on the environment [1]. Renewable energy resources (RERs) are widely used because they have a low cost of energy and great potential. Moreover, the realization of the net metering concept encourages users to install PVs. Installing a PV system allows consumers to contribute extra energy to the grid [2,3]. Due to the two-way power flow in net metering, the power distribution network (PDN) has been impacted by sudden voltage deviations. Power losses will rise as the voltage of the PDN deviates from the target range. An artificial intelligence-based framework is needed to keep the voltage of PDN within a predetermined range. Two distinct approaches, one based on reactive power and the other on real power, can be used to regulate the voltage of PDNs [4,5]. The three primary mechanisms in the actual power scheme are the on-load tap changer transformer (OLTC-transformer), battery storage system, and power curtailment. However, real power-based control is a completely ineffective strategy, because it is a time consuming process and also reduces power contribution by RERs in case of power curtailment. On the other hand, reactive power-based schemes use techniques like capacitor banks, static var compensators, and PV inverters [6,7]. PV inverter-based reactive power control is best among all these techniques, because no extra installation of devices is required. Moreover, PV inverters have decreased response times as compared to other approaches. Agents (PVs) can be set up under a variety of control structures, including centralized, decentralized, and distributed ones [8]. This study exploits both decentralized and distributed control schemes for the coordination and control of agents [9]. Agents (PV inverterss) are grouped in a zonal layout in the decentralized scheme. Each zone contains several PV inverters, and the zone is regarded as an agent. In contrast, each PV inverter functions as an agent and collaborates with the other agents in a distributed control system [10,11]. To achieve real-time control and coordination among these PV inverters in a decentralized and distributed control scheme, a machine learning-based framework is required. Machine learning-based techniques are now widely used for real-time control and communication applications [12,13]. A reinforcement learning-based decentralized multi-agent actor-critic (MAAC) algorithm is used for the modeling of agents. The actor network of each zone performs a specific action, and the critic network produces a specific quality value generally named as Q-value for the action. The Q-value determines the performance of an action, and all actions are analyzed on the base of the Q-value. An action that has a maximal Q-value is considered the optimal action [14,15].
Different methods have been employed in the past to control the voltage of PDN. The authors of [16] suggested a swarm optimization-based control scheme for multiple wind farms installed in PDN. The swarm optimization-based control did not achieve real-time control and coordination. The authors of [17,18] control the voltage of the PDN using capacitor banks where the switching of capacitor banks regulates reactive power; however, switching capacitor banks in real-time is an issue. Grid-connected PV inverter-based techniques have been suggested by the author of the paper [19] but are insufficient to control multiple PV inverters. The author in [20,21] evaluated multi-grid system control using deep reinforcement learning (DRL). Multiple grids are modeled in the DRL algorithm. All the grids communicate with each other to meet the load demand. The system is lacking the latest deep learning algorithms due to which optimal results cannot be achieved. Q-leaning-based OLTC control is presented by the author. Different OLTC-transformers are modeled in the DRL algorithm. Each transformer performs a specific action and received a Q-value for that action [22,23]. Actions that have a maximal Q-value are considered the best actions [24]; however, switching transformers is a time-consuming process and cannot produce the required results. The authors of [25] proposed a centralized approach to power curtailment and a PV inverter-based control scheme; however, power curtailment is not an effective solution. A flexible, alternating current transmission system (FACTS) devices-based approach is proposed in [26] where the voltage is controlled by FACTS devices. This technique mentions the processing time but does not have adequate control. The authors suggested a dispersed customer resources-based approach for voltage control. Voltage is controlled at the point of connection through specific voltage regulators [27,28]. In the paper, a matrix-based nodal voltage control scheme is depicted, but it is not an efficient scheme for real-time control [29].

1.2. Research Contribution

Prior research suggests that the PV inverter-based technique is superior to alternative reactive power-based control approaches since it does not require additional installation of power devices. In addition, compared to conventional control systems, the PV inverter offers a quicker response time, better and longer life, and no power losses. However, controlling and coordinating all these PV inverters in real-time within the power distribution network (PDN) are difficult tasks. The proposed work offers a multi-agent actor-critic framework that would allow for real-time control and coordination agents (PV inverters). Agents (PV inverters) in MAAC are formulated as actor and critic networks where the actor is a policy network while the critic is a value network. Agents (PV inverters) are organized in a distributed and decentralized scheme. In the suggested scheme, time-changing loads and PVs are integrated into the IEEE-33 bus system. All of the PV inverters are modeled using the MAAC framework, and each PV system has its own individual PV inverter. In order to keep the PDN voltage within a certain range and reduce power losses, all the agents effectively communicate and work together. By keeping the voltage within a certain range of the proposed framework, the multi-agent actor-critic (MAAC) algorithm minimizes power losses. A better voltage controllable ratio is also achieved by the suggested scheme.
To sum up, the proposed framework realizes the following remarkable features:
  • Artificial Intelligence-based control and coordination;
  • Provision of improved voltage controllable ratios;
  • Realizes effective communication that leads to achieving minimum power losses;
  • System achieves a 65% voltage controllable ratio;
  • Uncontrollable voltage ratio is reduced to 0.0275.
In Section 2, decentralized and distributed control schemes are presented. The proposed framework is discussed in Section 3. The response of the framework is given in Section 4 in detail. Lastly, the paper is concluded in Section 5.

2. Decentralized and Distributed Control Scheme

All the agents (PV inverters) of the power distribution network are arranged in two different schemes. Decentralized and distributed schemes for PV inverters are discussed in this section.

2.1. Decentralized Control Scheme

Agents (PV inverters) are divided into various zones in a decentralized power distribution network. PV inverters in a zone cooperate with one another to increase the overall reward of the zone. Inter zonal coordination and zonal control are attractive features of this system. The decentralized design of power distribution networks is shown in Figure 1. Each PV inverter has an actor network that creates an action and a critic network that assesses the effectiveness of the action. The actions of every PV in a zone are interdependent and increase the overall reward of the zone. To maximize the reward function of the entire network, different zones collaborate.

2.2. Distributed Control Scheme

In a distributed control system, each PV inverter is designed in a way that allows it to control itself and work together with other agents. Each agent has an actor network and a critic network, and the actor network produces action while the critic network evaluates it. Each agent collaborates with other agents to keep the voltage of the distribution network within a specific range while controlling its reactive power in accordance with the actual power generation of PVs. The PDN distribution control strategy is depicted in Figure 2.

3. Proposed Framework

The multi-agent actor-critic-based framework is used to control and coordinate multiple agents (PV inverters) in real-time. MAAC is divided into actor and critic networks. In this section, the proposed framework is covered in detail.

3.1. Policy Function

Policy function is a technique of reinforcement learning in which the best policy is analyzed based on the objective function J ( θ ) = Ε s p π , a π θ [ R ] . It seeks to address the problem of continuous action space rather than producing value, and its value is updated using Equation (1)
θ J ( θ ) = E s p π , a π θ θ log π θ ( a s ) Q π ( s , a )
where θ is the policy gradient and J ( θ ) is the objective function of the network.

3.2. Value Function

Q ( s , a ) is the action-value function of q π ( s , a ) or q ( s , a ) . Q π ( s , a ) is the reward of an action taken in the state s following the policy π used in Q-learning algorithms. The Equation (2) gives the Q-value of a state-action pair.
Q π ( s , a ) = E s r ( s , a ) + r E a π Q π s , a
where r ( s , a ) is the expected immediate reward while Q π ( s , a ) is the Q-value of the next state.
Equation (3) represents the loss function between the targeted and predicted value.
ς ( θ ) = E s , a , r , s [ ( Q ( s , a | θ ) y ) 2 ]
In the loss function, y represent the targeted value, and its value is calculated by Equation (4).
y = r + γ max a Q ( s , a | θ )
where Q stands for targeted action value.

3.3. Multi Agent Actor Critic

The basic framework for the control schemes is MAAC. In the MAAC framework, different agents collaborate with each other to maximize the reward of the environment. Equation (5) gives a loss
Q ( φ ) = i = 1 N E ( o , a , r , o ) D [ ( Q i φ ( o , a ) y i ) 2 ]
where
y = r i + γ E a π θ ( o ) [ Q i φ ( o , a ) α log ( π θ i ( a i o i ) ) ]
The policy gradient is calculated using Equation (7).
θ i J π θ = E o D , a π θ i log π θ i a i o i α log π θ i a i o i + Q i φ ( o , a ) b o , a i
where a \ i denotes all agents except a i .

3.4. Algorithm of Multi-Agent Actor Critic

This section explain the multi agent actor critic algorithm in detail. All the agents and reply buffer is randomly initialized. Then actor and critic networks are modeled and parameters are set accordingly. Below the Algorithm 1 is explain in detail.
Algorithm 1: Multi-Agent Actor Critic.
Randomly Initialize N agents
Initialize reply buffer D
for episode one to the maximum episode do
 Reset network environment, get initial observation
 for step = 1 to tmax do
  Each agent selects the action a i π i ( . | o i )
Execute the action and get new observations o’ and the reward r from the environment
  Store transition to reply buffer ( x , a , γ , x )
 end for
for agent I = 1 to N do
 Sample mini batch from the buffer
 Update critic
 Update actor
end for
 Update target network parameters of each agent
ϕ = k ϕ + ( 1 k ) ϕ
θ = k θ + ( 1 k ) θ
end for

3.5. Update Actor and Critic Network

The actor and critics network are updated at each step. Initially an actor take and action and critics network return the Q-value. The value get stored in reply buffer and the actor network produce next action. In this way the actor and critic networks are updated. The detail steps of Algorithm 2 are explained below.
Algorithm 2: Actor and Critic Network.
Function: UPDATE CRITIC
Sample Mini batch ( x , a , r , x )
Calculate Q θ ( x , a ) and Q θ ( x , a ) by the target network
J θ ( Q θ ) = i = 1 N E ( x , a , x , a ) D [ 1 2 ( Q θ ( x , a ) y ) 2 ]
Calculate θ J θ ( Q θ ) and update critic using Adam optimizer
End Function.
Function: UPDATE CRITIC
Calculate a i π ϕ i ( . | o i ) for each agent
ϕ i J ( π ϕ i ) = ϕ i log π ϕ i ( a i | o i ) ( Q θ ( x , a ) b ( x , a i ) )
Update θ using Adam optimizer
End Function

3.6. Flow Chart

The flowchart of the proposed work is discussed in this section. The power distribution network is integrated with time changing PVs and loads. The PV systems are modeled in a decentralized and distributed control scheme. All the PV inverters are formulated in MAAC algorithm. The results are generated at the end. Figure 3 shows a flow chart of the proposed work.

4. Results and Discussion

System models and real and reactive powers of integrated PVs are briefly explained in this section. Moreover, line losses and voltage deviations of PV integrated IEEE-33 are discussed in detail. Results for the summer and winter seasons are examined, taking into account how load and PV output alters with the seasons. PVs produce less power in the winter because of lower solar radiation and a lower clearness index.
The adopted network topology is initially described before evaluating the outcomes. The generation of active and reactive electricity by PVs is then observed. Later, all the buses’ voltage deviations and line losses are noted. The comparison of the decentralized and distributed frameworks is seen at the end.

4.1. Network Topology

The model of power distribution network (considered system) is shown in Figure 4. The network has the PVs installed in various locations. The distribution network has the number of buses that can be denoted as B n = { 1 , 2 , , N } , and all nodes that are depicted are denoted as B e = { 1 , 2 , , N } . The complex, real, and reactive powers are given by the following equations: Equation (8) defines the complex power of the network.
s j = p i + j q i
Real and reactive powers are given through Equations (9) and (10).
p i P V p i L = v i 2 j B i g i j v i j B i v j ( g i j cos θ i j + b i j sin θ i j ) i B \ { 0 }
q i P V q i L = v i 2 j B i b i j v i j B i v j ( g i j cos θ i j + b i j sin θ i j ) i B \ { 0 }
p i P V and q i P V are the active and reactive powers of PV on bus i, and buses without PVs have zero value of real and reactive power. p i L and q i L are the values of load installed at bus i, and its value will be zero in case of no load. A safe range of voltage deviation exists between 0.95   p . u v i 1.05   p . u , i B \ { 0 } .
The IEEE-33 bus is modified by the integration of loads and PVs. The rated voltage of the network is 12.66 kV, PmaxL 3.5 MW, and PmaxPV is 8.75 MW. The network has 32 loads and 6 PV arrays installed at various buses. The power production of PVs in real-time is measured for the time of one year. The power production of PVs varies with the time step of 15 minutes for the whole year. In this way, different power productions can be achieved for the winter and summer seasons.

4.2. Active Power of Integrated PVs

In this section, the active power of every PV in a decentralized and distributed control architecture is discussed. The active power generation by integrated PVs is the same for both schemes. Additionally, the power output of PVs is examined for both the summer and the winter. PV-5 and PV-6, with a production of 0.0532 MW, offer the lowest output for the summer. On the other hand, PV-4 generates the most active power, at a value of 0.5191 MW. PV-5 and PV-6 produce the least amount of active power during the winter, 0.0226 MW. PV-4’s highest active power output during the winter is 0.3194 MW. Figure 5 displays the average active power generation across all PVs.
Table 1 provides a brief overview of the PVs’ active power production. In the summer, integrated PVs generate 1.6361 MW of active power collectively, while in the winter, this number is reduced to 0.9292 MW.

4.3. Reactive Power of Integrated PVs

The reactive power production for the decentralized and distributed control scheme is different. For the summer season in a decentralized control scheme, PV-2 produces the maximum amount of reactive power. Reactive power production by PV-2 is 0.7202 MVAR. PV-3 and PV-4 produce negative reactive powers that shows reactive power is absorbed by these PVs. During the winter season, PV-2 produces the maximum amount of reactive power with a value of 0.8119 MVAR. PVs generates negative reactive power, and PV-4 generates the least amount of reactive power with a value of 0.0090 MVAR. Reactive power generation for the summer is collectively 0.9193 MVAR and for the winter, the value is 1.4177 MVAR. Figure 6 and Table 2 depicted the reactive power of PVs for the summer season.
Reactive power generation in a distributed control scheme is different from the decentralized control scheme. Figure 7 and Table 3 show the reactive power of PVs for the winter season. For the summer season in a distributed control scheme, PV-2 produces the maximum amount of reactive power of 0.0438 MVAR. PV-3 and PV-4 have negative reactive power, which means that PV inverters absorb reactive power. PV-5 produced the least amount of reactive power with a value of 0.0874 MVAR. During the winter, PV-1 generated a maximum reactive power value of 0.3474 MVAR. PV-3 absorbed the reactive power, and PV-4 produced the least amount of reactive power with a value of 0.0095 MVAR. All the integrated PVs collectively produce 0.7866 MVAR and 0.6771 MVAR for summer and winter, respectively.

4.4. Voltage of IEEE-33 Bus System

The voltage of the IEEE-33 bus system in a decentralized and distributed control scheme is discussed in the section. In a decentralized control scheme, voltage fluctuation for the winter and summer is almost the same as shown in Figure 8. The voltage of the network remained within the safe range of ±5%. As shown in Figure 1, the voltage of the network remained within the safe range of 0.95–1.05 p.u.
In the distributed control scheme, voltage fluctuation is less in the winter season as compared to the summer season. During the summer season, the IEEE-33 bus system has large voltage fluctuations. However, for both summer and winter seasons, the voltage remains within the safe range of ±5%. The voltage of all 33 buses is shown in Figure 9.

4.5. Losses of IEEE-33 Bus System

Line losses of the IEEE-33 bus system for summer and winter are discussed in the following section. In the decentralized control scheme, the sum of system losses for the summer season is 0.1345 MW. For the winter season, the value of line losses is 0.1288 MW, which is less than the summer season. Figure 10 shows the line losses of the IEEE-33 bus system for the summer season.
In the distributed control scheme, the values of power losses are different than in the decentralized control scheme. For the summer season, the value of system losses is 0.1291 MW. In the distributed control scheme, the value of losses is reduced to 0.0488 MW for the winter season. Power losses for the duration of winter are depicted in Figure 11.

4.6. Comparison of Decentralized and Distributed Control Scheme

In this section, our work is compared with the most recent methods given in the literature. Since the load and PV output varies with the seasons, the results are evaluated for both the summer and winter seasons. In comparison to the existing system, the proposed framework achieves better voltage stability and controllable ratios. In comparison to the existing scheme, power losses and voltage deviation are also reduced.
Results for the summer duration are discussed in Table 4. In the decentralized control scheme, the voltage remains at 0.9980 p.u which is better compared to the distributed control scheme voltage of 0.9972 p.u. Line losses for the decentralized control scheme are 0.1345 MW which is higher than the distributed control scheme of 0.1291 MW. In the decentralized control scheme, the system faces less voltage deviation of the ratio of 0.1362, which is much less than that of the distributed control scheme of 0.01516. Better voltage control is achieved in a decentralized control scheme of 0.6850 as compared to distributed scheme of 0.6508. The value of out-of-control voltage is decreased in the decentralized control scheme (0.0275 p.u) as compared to the distributed control scheme of 0.0523.
By analyzing the following results, it can be concluded that a decentralized control scheme is better than distributed control scheme for the summer season. Decentralized control schemes have less voltage deviation and greater voltage control.
Table 5 compare the results of the decentralized and distributed control scheme for the winter season. The decentralized control scheme has a voltage of 1.003 p.u which is good compared to the distributed control scheme of 0.9949 p.u. Line losses in distributed control schemes are much less than that of a decentralized control scheme. Line losses for decentralized control schemes are 0.1288 MW and in distributed control schemes these losses are reduced to 0.0488 MW. Voltage deviation, voltage controllable ratio and voltage out-of-control ratios are the same for both the winter and summer seasons.

5. Conclusions

A multi-agent actor critic based algorithm achieved the best control and coordination of PV inverters arranged in decentralized and distributed control scheme. As a whole, decentralized control schemes achieve better results as compared to distributed control schemes. Voltage and power losses vary for the summer and winter duration. All the PV inverters in the power distribution network produce or absorb the reactive power according to the requirement to maintain the voltage of the distribution network in a certain range of ±5%.
The actor network of each PV inverter produces reactive power and critic networks to analyze the performance of actors and generate a Q-value. The actor networks of PV inverters change their actions based on these Q-values. The value of actions changes until the maximum Q-value is achieved. The proposed framework achieves a better voltage controllable ratio of 0.6850 and 0.6508 for the decentralized and distributed control schemes, respectively. The voltage out-of-control ratio is minimized up to the value of 0.275 for the decentralized scheme while maintaining a value of 0.0523 for the distributed control scheme. Moreover, the system achieves voltage control within a certain range of 0.95–1.05 p.u and also minimizes the power losses exploiting the proposed scheme.
Future work can focus on implementing new algorithms for the control and coordination of agents. There is no control scheme for the voltage of hybrid power systems like wind and PV. Future work could be done by developing artificial intelligence-based control for a hybrid system.

Author Contributions

Conceptualization, A.u.R. and M.A.; methodology, A.u.R. and S.I.; software, S.I. and A.S.; validation, N.U., M.A. and A.u.R.; formal analysis, A.S.; investigation, N.U.; resources, S.A.O.; data curation, A.u.R.; writing—original draft preparation, A.u.R.; writing—review and editing, S.I. and M.A.; visualization, S.A.O.; supervision, M.A and S.I.; project administration, S.A.O.; funding acquisition, S.A.O. and N.U. All authors have read and agreed to the published version of the manuscript.

Funding

Authors would like to thank Taif University Researchers for Supporting Project Number (TURSP-2020/228), Taif University, Taif, Saudi Arabia.

Conflicts of Interest

The authors declares no conflict of interest.

References

  1. Jamil, I.; Zhao, J.; Zhang, L.; Jamil, R.; Rafique, S.F. Evaluation of energy production and energy yield assessment based on feasibility, design, and execution of 3 × 50 MW grid-connected solar PV pilot project in Nooriabad. Int. J. Photoenergy 2017, 2017, 6429581. [Google Scholar] [CrossRef]
  2. Ceylan, O.; Paudyal, S.; Pisicay, I. Analysis of Local and Centralized Control of PV Inverters for Voltage Support in Distribution Feeders. In Proceedings of the 2021 IEEE Power & Energy Society General Meeting (PESGM), Washington, DC, USA, 25–29 July 2021; pp. 1–5. [Google Scholar] [CrossRef]
  3. Harrold, D.J.; Cao, J.; Fan, Z. Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning. Appl. Energy 2022, 318, 119151. [Google Scholar] [CrossRef]
  4. Harrold, D.J.; Cao, J.; Fan, Z. Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning. Energy 2021, 238, 121958. [Google Scholar] [CrossRef]
  5. Iqbal, S.; Jan, M.U.; Rehman, A.U.; Rehman, A.U.; Shafique, A.; Rehman, H.U.; Aurangzeb, M. Feasibility Study and Deployment of Solar Photovoltaic System to Enhance Energy Economics of King Abdullah Campus, University of Azad Jammu and Kashmir Muzaffarabad, AJK Pakistan. IEEE Access 2022, 10, 5440–5455. [Google Scholar] [CrossRef]
  6. Cao, D.; Hu, W.; Zhao, J.; Huang, Q.; Chen, Z.; Blaabjerg, F. A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters. IEEE Trans. Power Syst. 2020, 35, 4120–4123. [Google Scholar] [CrossRef]
  7. Cao, D.; Zhao, J.; Hu, W.; Ding, F.; Huang, Q.; Chen, Z.; Blaabjerg, F. Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control With High Penetration of PVs. IEEE Trans. Smart Grid 2021, 12, 4137–4150. [Google Scholar] [CrossRef]
  8. Gao, Y.; Wang, W.; Yu, N. Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks. IEEE Trans. Smart Grid 2021, 12, 3594–3604. [Google Scholar] [CrossRef]
  9. Ji, Y.; Wang, J.; Xu, J.; Fang, X.; Zhang, H. Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning. Energies 2019, 12, 2291. [Google Scholar] [CrossRef]
  10. Ali, K.H.; Sigalo, M.; Das, S.; Anderlini, E.; Tahir, A.A.; Abusara, M. Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation. Energies 2021, 14, 5688. [Google Scholar] [CrossRef]
  11. Iqbal, S.; Xin, A.; Jan, M.U.; Abdelbaky, M.A.; Rehman, H.U.; Salman, S.; Aurangzeb, M.; Rizvi, S.A.A.; Shah, N.A. Improvement of Power Converters Performance by an Efficient Use of Dead Time Compensation Technique. Appl. Sci. 2020, 10, 3121. [Google Scholar] [CrossRef]
  12. Mosa, M.A.; Ali, A. Energy management system of low voltage dc microgrid using mixed-integer nonlinear programing and a global optimization technique. Electr. Power Syst. Res. 2020, 192, 106971. [Google Scholar] [CrossRef]
  13. Muriithi, G.; Chowdhury, S. Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach. Energies 2021, 14, 2700. [Google Scholar] [CrossRef]
  14. Cosic, A.; Stadler, M.; Mansoor, M.; Zellinger, M. Mixed-integer linear programming based optimization strategies for renewable energy communities. Energy 2021, 237, 121559. [Google Scholar] [CrossRef]
  15. Iqbal, S.; Habib, S.; Khan, N.H.; Ali, M.; Aurangzeb, M.; Ahmed, E.M. Electric Vehicles Aggregation for Frequency Control of Microgrid under Various Operation Conditions Using an Optimal Coordinated Strategy. Sustainability 2022, 14, 3108. [Google Scholar] [CrossRef]
  16. Nakabi, T.A.; Toivanen, P. Deep reinforcement learning for energy management in a microgrid with flexible demand. Sustain. Energy Grids Netw. 2020, 25, 100413. [Google Scholar] [CrossRef]
  17. Kathirgamanathan, A.; Mangina, E.; Finn, D.P. Development of a Soft Actor Critic deep reinforcement learning approach for harnessing energy flexibility in a Large Office building. Energy AI 2021, 5, 100101. [Google Scholar] [CrossRef]
  18. Suanpang, P.; Jamjuntr, P.; Jermsittiparsert, K.; Kaewyong, P. Autonomous Energy Management by Applying Deep Q-Learning to Enhance Sustainability in Smart Tourism Cities. Energies 2022, 15, 1906. [Google Scholar] [CrossRef]
  19. Aurangzeb, M.; Xin, A.; Iqbal, S.; Jan, M.U. An Evaluation of Flux-Coupling Type SFCL Placement in Hybrid Grid System Based on Power Quality Risk Index. IEEE Access 2020, 8, 98800–98809. [Google Scholar] [CrossRef]
  20. Li, Z.; Yan, Y.; Qi, D.; Yan, S.; Wang, M. Distributed Voltage Optimization Control of BESS in AC Distribution Networks with High PV Penetration. Energies 2022, 15, 4120. [Google Scholar] [CrossRef]
  21. Iqbal, S.; Xin, A.; Jan, M.U.; Abdelbaky, M.A.; Rehman, H.U.; Salman, S.; Rizvi, S.A.A.; Aurangzeb, M. Aggregation of EVs for Primary Frequency Control of an Industrial Microgrid by Implementing Grid Regulation & Charger Controller. IEEE Access 2020, 8, 141977–141989. [Google Scholar] [CrossRef]
  22. Guo, X.; Zhao, Q.; Wang, S.; Shan, D.; Gong, W. A Short-Term Load Forecasting Model of LSTM Neural Network considering Demand Response. Complexity 2021, 2021, 5571539. [Google Scholar] [CrossRef]
  23. Muriithi, G.; Chowdhury, S. Deep Q-network application for optimal energy management in a grid-tied solar PV-Battery microgrid. J. Eng. 2022, 2022, 422–441. [Google Scholar] [CrossRef]
  24. Touzani, S.; Prakash, A.K.; Wang, Z.; Agarwal, S.; Pritoni, M.; Kiran, M.; Brown, R.; Granderson, J. Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency. Appl. Energy 2021, 304, 117733. [Google Scholar] [CrossRef]
  25. Domínguez-Barbero, D.; García-González, J.; Sanz-Bobi, M.A.; Sánchez-Úbeda, E.F. Optimising a Microgrid System by Deep Reinforcement Learning Techniques. Energies 2020, 13, 2830. [Google Scholar] [CrossRef]
  26. Nespoli, A.; Ogliari, E.; Pretto, S.; Gavazzeni, M.; Vigani, S.; Paccanelli, F. Electrical Load Forecast by Means of LSTM: The Impact of Data Quality. Forecasting 2021, 3, 91–102. [Google Scholar] [CrossRef]
  27. Yang, B.; Sun, S.; Li, J.; Lin, X.; Tian, Y. Traffic flow prediction using LSTM with feature enhancement. Neurocomputing 2019, 332, 320–327. [Google Scholar] [CrossRef]
  28. Gao, G.; Wen, Y.; Tao, D. Distributed energy trading and scheduling among microgrids via multiagent reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. 2022. [Google Scholar] [CrossRef]
  29. Choi, Y.-A.; Park, S.-J.; Jun, J.-A.; Pyo, C.-S.; Cho, K.-H.; Lee, H.-S.; Yu, J.-H. Deep Learning-Based Stroke Disease Prediction System Using Real-Time Bio Signals. Sensors 2021, 21, 4269. [Google Scholar] [CrossRef]
Figure 1. Decentralized Control Scheme.
Figure 1. Decentralized Control Scheme.
Energies 15 06297 g001
Figure 2. Distributed Control Scheme.
Figure 2. Distributed Control Scheme.
Energies 15 06297 g002
Figure 3. Flow chart of the proposed work.
Figure 3. Flow chart of the proposed work.
Energies 15 06297 g003
Figure 4. System Model.
Figure 4. System Model.
Energies 15 06297 g004
Figure 5. Active Power Production of Integrated PVs.
Figure 5. Active Power Production of Integrated PVs.
Energies 15 06297 g005
Figure 6. Reactive Power of Integrated PVs in Decentralized Control Scheme.
Figure 6. Reactive Power of Integrated PVs in Decentralized Control Scheme.
Energies 15 06297 g006
Figure 7. Reactive Power of Integrated PVs in Distributed Control Scheme.
Figure 7. Reactive Power of Integrated PVs in Distributed Control Scheme.
Energies 15 06297 g007
Figure 8. Voltage of IEEE-33 Bus System in Decentralized Scheme.
Figure 8. Voltage of IEEE-33 Bus System in Decentralized Scheme.
Energies 15 06297 g008
Figure 9. Voltage of IEEE-33 Bus System in Distributed Scheme.
Figure 9. Voltage of IEEE-33 Bus System in Distributed Scheme.
Energies 15 06297 g009
Figure 10. Power losses of IEEE-33 bus system in Decentralized Scheme.
Figure 10. Power losses of IEEE-33 bus system in Decentralized Scheme.
Energies 15 06297 g010
Figure 11. Power losses of IEEE-33 bus system in Distributed Scheme.
Figure 11. Power losses of IEEE-33 bus system in Distributed Scheme.
Energies 15 06297 g011
Table 1. Average active power of PVs in the decentralized and distributed control scheme.
Table 1. Average active power of PVs in the decentralized and distributed control scheme.
Active Power (MW)PV-1PV-2PV-3PV-4PV-5PV-6Total (MW)
Summer0.30680.30680.39700.51910.05320.05321.6361
Winter0.17440.17440.21580.31940.02260.02260.9292
Table 2. Average reactive power of PVs in decentralized control scheme.
Table 2. Average reactive power of PVs in decentralized control scheme.
Reactive Power (MVAR)PV-1PV-2PV-3PV-4PV-5PV-6Total
(MVAR)
Summer0.11980.7202−0.1307−0.12530.19930.13600.9193
Winter0.33900.8119−0.08140.00900.20710.13211.4177
Table 3. Average reactive power of PVs in distributed control scheme.
Table 3. Average reactive power of PVs in distributed control scheme.
Reactive Power (MVAR)PV-1PV-2PV-3PV-4PV-5PV-6Total
(MVAR)
Summer0.41250.4834−0.0133−0.32310.08740.13930.7866
Winter0.34740.2558−0.02080.00950.01940.06580.6771
Table 4. Results of Algorithm for summer season.
Table 4. Results of Algorithm for summer season.
System (Algorithm)Voltage (p.u)Line Losses (MW)Average Voltage Deviation RatioMean Test Voltage Controllable RatioMean Test Voltage Out of Control Ratio
Decentralized0.99800.13450.013620.68500.0275
Distributed0.99720.12910.015160.65080.0523
Table 5. Results of Algorithm for winter season.
Table 5. Results of Algorithm for winter season.
System
(Algorithm)
Voltage (p.u)Line Losses (MW)Average
Voltage
Deviation
Ratio
Mean Test Voltage
Controllable Ratio
Mean Test
Voltage Out of Control Ratio
Decentralized1.0030.12880.013620.68500.0275
Distributed0.99490.04880.015160.65080.0523
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rehman, A.u.; Ali, M.; Iqbal, S.; Shafiq, A.; Ullah, N.; Otaibi, S.A. Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks. Energies 2022, 15, 6297. https://doi.org/10.3390/en15176297

AMA Style

Rehman Au, Ali M, Iqbal S, Shafiq A, Ullah N, Otaibi SA. Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks. Energies. 2022; 15(17):6297. https://doi.org/10.3390/en15176297

Chicago/Turabian Style

Rehman, Anis ur, Muhammad Ali, Sheeraz Iqbal, Aqib Shafiq, Nasim Ullah, and Sattam Al Otaibi. 2022. "Artificial Intelligence-Based Control and Coordination of Multiple PV Inverters for Reactive Power/Voltage Control of Power Distribution Networks" Energies 15, no. 17: 6297. https://doi.org/10.3390/en15176297

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop