Effective Energy Management via False Data Detection Scheme for the Interconnected Smart Energy Hub–Microgrid System under Stochastic Framework

Abstract

During the last few years, attention has overwhelmingly focused on the integrated management of urban services and the demand of customers for locally-based supply. The rapid growth in developing smart measuring devices has made the underlying systems more observable and controllable. This exclusive feature has led the system designers to pursue the implementation of complex protocols to provide faster services based on data exchanges. On the other hand, the demands of consumers for locally-based supply could cause a disjunction and islanding behavior that demands to be dealt with by precise action. At first, keeping a centralization scheme was the main priority. However, the advent of distributed systems opened up new solutions. The operation of distributed systems requires the implementation of strong communication links to boost the existing infrastructure via smart control and supervision, which requires a foundation and effective investigations. Hence, necessary actions need to be taken to frustrate any disruptive penetrations into the system while simultaneously benefiting from the advantages of the proposed smart platform. This research addresses the detection of false data injection attacks (FDIA) in energy hub systems. Initially, a multi-hub system both in the presence of a microgrid (the interconnected smart energy hub-based microgrid system) and without it has been modeled for energy management in a way that allows them to cooperate toward providing energy with each other. Afterward, an FDIA is separately exerted to all three parts of the energy carrier including the thermal, water, and electric systems. In the absence of FDIA detection, the impact of FDIA is thoroughly illustrated on energy management, which considerably contributes to non-optimal operation. In the same vein, the intelligent priority selection based reinforcement learning (IPS-RL) method is proposed for FDIA detection. In order to model the uncertainty effects, the unscented transformation (UT) is applied in a stochastic framework. The results on the IEEE standard test system validate the system’s performance.

1. Introduction

The ongoing scientific progress in communications has led to an intellectual revolution in the principles of management. On the basis of the resulting technology, the connections among different parties were achieved gradually and, in turn, the management principles became multilateral. In accordance with this technological development, recently, distributed systems have been highlighted that fit their demand and the desired management schemes. In such systems, all agents are involved in an intense competition to gain benefits while they work together to realize a shared consensus [1]. The recent developments in information technology and communications, as well as the devising of a new generation of smart metering systems, have enabled the possibility of creating a multi-carrier energy system that is capable of establishing secure bilateral links between producers and consumers. In this regard, control of energy carriers’ consumption, boosting all of the parties, and distributing the benefits among them could be achieved through convertible platforms in multi-carrier energy systems. On the other side, gas and water carriers profoundly affect the management of electrical energy and vice versa [2]. Therefore, three-level descriptions of the literature are investigated: (A) energy management in distribution systems, (B) energy hub systems, and (C) false data injection (FDI) attacks.

1.1. Energy Management in Distribution Systems

The main source for generating electric energy is fossil fuels, which are a finite resource and have detrimental environmental impacts (such as increased greenhouse gases) [3]. Thus, there is a universal intention to reduce the harmful environmental impacts of electricity production [4]. Recent investigations have demonstrated that if energy consumption stays at the same rate, fossil fuel reserves will be depleted in the next 50–60 years [5]. For this reason, the necessity for a more efficient method of connection between electricity generation and the consumers is highly apparent. Energy management represents the schedule of local energy flows through the operation of local renewable energy resources [6]. In the energy management process, various resources in the power distribution network are taken into account. Photovoltaic (PV), wind turbine (WT), and tidal systems have been modelled in each part of the distribution network. The technical characteristics and parameters of microgrid and energy hub systems are available in [7]. Energy management can be executed in a microgrid (MG) or a widespread grid with multi-MGs for the achievement of goals such as minimizing the operational costs or system losses and so forth [8,9,10]. The authors in [11] implemented an energy management program among multi-microgrids in which they applied the game theory method for a demand response program in the presence of renewable energy resources and energy storage. Reference [12] introduces multi-agent energy management that specifically controls the supply side (producers) in the presence of high penetration of renewable energy sources (RES) and electric vehicles (EV). References [13,14,15,16,17,18] have also surveyed various control strategies of distributed generation to energy management. Researchers in [13] discussed a review of energy management methods. The authors in research [14] centralized the control of the microgrids’ energy management purpose. Reference [15] speaks about some scheduling strategies for multi-microgrid energy management in which the authors also consider architecture and communication issues. The authors in [16] introduced a scheduling algorithm for energy management based on artificial intelligence. However, the authors in [17,18] addressed the issue from a data security point of view. This research also moves towards dealing with the role of data security in energy management as in reference [18]. On the other hand, the interconnected smart energy hub-based MG system has been employed in energy management as a distinct type.

1.2. Energy Hub Systems

The recent developments in information technology and communications as well as devising a new generation of smart metering systems have enabled the possibility of creating a multi-carrier energy system that is capable of establishing secure bilateral links between producers and consumers. This multi-carrier energy hub is called the smart energy hub system [19]. In this regard, the control of energy carriers’ consumption, boosting all of the parties and distributing the benefits among them, could be achieved through convertible platforms in multi-carrier energy systems. On the flip side, gas and water carriers profoundly affect the management of electrical energy and vice versa.

Energy hubs with multi-carrier energy networks (such as thermal, water and electrical systems) can utilize renewable energy resources and distributed generations (DGs) reliably and securely. A hub system can operate in various situations in either an island or grid-connected mode [20]. Therefore, combined heat and power (CHP) systems can provide a substantial contribution in such systems owing to the fact that they satisfy both thermal and electrical energy demands [21]. Using smart monitoring and data communications in smart grids, transactive energy has developed a new outlook for hub operations. Supply and demand are balanced in an energy hub system by exchanging energy in various energy demands [22,23]. Evidence suggests that the use of energy hub systems has many advantages including reducing voltage oscillations by high penetration of RESs [24], decreasing the consumption of fossil fuel [25], reducing motor start-up current stresses owing to islanded operation [26], etc. Research on the subject considers several matters. Numerous studies have surveyed how cooperating among multiple energy hubs systems can reduce operation costs by using DER. Many research debates have been conducted concerning optimal energy management in decentralized and centralized control multi-hub. The literature [27] presents a multi-agent method for energy trading in grid-connected hubs.

1.3. False Data Injection Attack (FDIA)

The smart grid is increasingly associated with intelligent monitoring devices for reliability, efficiency, and control. The high integration of inter-connectivity infrastructure and pervasive use of communications has led to new types of vulnerableness, which have not been exactly covered by the civil defense entities. One of the many occurrences was a cyber-attack against distribution companies in Ukraine [28]. FDI attacks, first described in [29], import false data into the measurements system to take down the network operation [30]. In [31] a comprehensive review of FDI attacks has been carried out. An FDI attack will be effective when the system operator does not detect it. The researchers in [32] illustrated that networks could be attacked with only partial information of monitoring devices. Reference [33] introduces a type of attack called the blind FDI in which the attacker does not require any system information. In this study, an FDIA is separately exerted to all three parts of the energy carrier included the thermal, water, and electric parts. In the absence of FDIA detection, the impact of FDIA has been thoroughly illustrated in energy management, which considerably contributes to non-optimal operation. In the same vein, the intelligent priority selection based reinforcement learning (IPS-RL) method has been proposed for FDIA detection. To make out analysis more realistic, electric demand and generation uncertainties are considered using the unscented transformation (UT) method. Some researches on UT have been reported in several publications [34].

To realize the challenges expressed in this paper,

Table 1. Categorization of the different approaches.

	Reinforcement Learning	Hub Energy	Microgrid	Uncertainty	Attack Detection
[19]		✓
[29,32]					✓
[35]	✓				✓
[36]	✓		✓		✓
[37]	✓	✓			✓
Proposed Model	✓	✓	✓	✓	✓

shows the main differences between the previous investigations and the proposed frameworks.

As it can be seen, the authors in [35] have tried to provide a security platform based on machine learning for detecting cyber-attacks without considering energy carriers. However, the modified reinforcement learning method pointed out in this paper seems to be more intelligent and faster for finding false data in a system. In addition, checking the uncertainty effects on the performance of the multi-energy system equipped by the cyber-security approach against attackers can be one of the significant concerns in this paper.

Therefore, the main contributions and characteristics of this paper over prior publications are stated as follows:

• Providing an efficient and comprehensive framework based on the interconnected energy hub-based microgrid system to improve the simultaneous management of energy different carriers.
• Developing an appropriate and strong IPS-RL scheme aimed at detecting all types of attacks and guarantee the minimum detection delay.
• Validating and assessing the proposed detection method by implementing and modeling the FDI attack.
• The uncertainties of the studied system, including the electrical, thermal, and water loads, the tidal current, wind speed. and sunlight, are formulated by the UT method, which can model the correlation among uncertain parameters.

The rest of the paper is provided such that the mathematical formulations of the studied model are described in Section 2. Section 3 is aimed at expressing the proposed detection method. The solution process is represented in Section 4. Section 5 explains the UT-based uncertainty method. The effectiveness of the studied model proposed in this paper is evaluated and validated in Section 4. Our conclusions are presented in Section 7.

2. Formulation Definition of the Proposed Framework

Smart hub systems can satisfy three energy carriers (i.e., electrical, thermal, and water demands) with the use of an appropriate structure which coordinates the energy management among three load demand types. In this regard, such a structure can be incorporated in a networked microgrid by using the transaction link to supply electricity, water, and thermal loads located in the far area. To this end, this section is aimed at presenting a mathematical modeling of both the smart hub and networked microgrid systems by considering transactions between them modeled in this paper. To elaborate on such a framework, Figure 1 illustrates the studied network. The proposed microgrid comprises renewable energy units (i.e., PV, WT, and the tidal system expanded across the island). On the other side of the problem, the smart hub system requires thermal units such as a CHP and boiler, desalination unit, energy storage, and a transformer to satisfy thermal, water, and electrical demands, respectively.

It is important to first present the total cost function of the proposed networked microgrid as the objective function as follows:

$$C^{MC}=min{\displaystyle \sum _i{\displaystyle \sum _t{R}_t^{PV}{P}_{i,t}^{PV}+R_t^{WT}{P}_{i,t}^{WT}+R_t^{tidal}{P}_{i,t}^{tidal}+R^{HUB-MC}{P}_{i,t}^{HUB-MC}}}$$

(1)

$$P_{i,t}^{PV}+P_{i,t}^{WT}+P_{i,t}^{tidal}+{\displaystyle \sum _{j\in {\mathrm{\Omega }}^j}{P}_{ij,t}^{}}+P_{i,t}^{HUB-MC}=P_{i,t}^{load} ? \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T,\hspace{0.17em}\forall i\in {\mathrm{\Omega }}^i$$

(2)

Equation (1) indicates the objective function of the microgrid, which is made up of different parts including the operation cost of renewable energy units and the power transaction cost. Note that the benefit/cost arising from power transaction is dependent on the positive/negative value of exchanging power between the microgrid and hub system, as shown in the last term of the Equation (1). Such a cost function can be considered to provide an effective energy management structure within the proposed microgrid in which minimizing the operation cost of all units is necessary. An appropriate balance model between demand and generation guarantees the supplement of demands in the microgrid as can be seen by Equation (2). The left-handed parts related to Equation (2) comprise the power transaction between microgrid and hub system, the output powers of PV, WT and tidal system and power flow through lines in the networked microgrid. It is significant to say that the power transaction ($P_{i,t}^{HUB-MC}$) is bilateral which means the positive value of $P_{i,t}^{HUB-MC}$ indicates the power transaction from the hub system to the microgrid and vice versa. Besides, the right-handed part of the power balance equation is defined by load demands located at buses of the networked microgrid system. The power generation related to each unit is expressed in detail by Equations (3)–(5). The output power of PV units can be calculated by Equation (3) which complies with relevant PV capacity, the sun irradiation and the power loss consumed by cells. Looking over Equation (4), the WT unit can be able to generate its power regarding the wind speed limits in which $P{W}_{rated}$ imply to the rated power of WT for wind speeds $S_{rated}^{W-1}<S^{W-1 }$. In addition, Equation (5) represents the power generation pertaining to the tidal system, which explains the output power according to the tidal current speed ($V_{i,t})$. It should be mentioned that the tidal power would be zero, if the current speed is less than the rated speed. Starting power generation is when the current speed is high/below the cut-in/rated speeds, respectively. If so, the output power is generated according to the rated speed.

$$P_{i,t}^{PV}=\frac{D\times E_{i,t}^{PV}}{G}\times (1-P^{loss}),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T,\forall i\in {\mathrm{\Omega }}^i$$

(3)

$$P_{i,t}^{WT}=\left\{\begin{array}{l}0\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\hspace{0.17em}\hspace{0.17em}\le \hspace{0.17em}{S}^{W-1}\le S_{{}_{rated}}^{W-1}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\\ \phi (S_{i,t}{}^{W-1})\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{S}_{{}_{cutin}}^{W-1}\hspace{0.17em}\le S^{W-1}\le S_{{}_{rated}}^{W-1}\\ P_{rated}^{WT}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{S}_{{}_{rated}}^{W-1}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le S^{W-1}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\end{array}\right. t\in {\mathrm{\Omega }}^T\forall i\in {\mathrm{\Omega }}^i\sqrt{b^2-4ac}$$

(4)

$$P_{i,t}^{tidal}=\left\{\begin{array}{l}0\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}0\hspace{0.17em}\le \hspace{0.17em}{V}_{i,t}\hspace{0.17em}\le V_{rated}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\\ 0.5H_{pc}{\rho }_s{A}_{tidal}{V}_{i,t}^3\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{V}_{cutin}\hspace{0.17em}\le V_{i,t}\le V_{rated},\\ P_{raded}^{tidal}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{V}_{rated}\hspace{0.17em}\hspace{0.17em}\le V_{i,t\hspace{0.17em}}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\end{array}\right. \forall t\in {\mathrm{\Omega }}^T,\forall i\in {\mathrm{\Omega }}^i$$

(5)

As mentioned before, modeling the smart hub system is one of the significant goals in this section. Let us assume we have two hub systems with the same structure as a multi-hub system interconnected through the graph of the proposed networked microgrid. It is clear that the smart hub system can manage the energy carriers, i.e., the electrical, water, and thermal demands and provide a coordinated framework among energy carriers to get into effective energy management. To clarify such a structure, the formulations pertaining to each smart hub system are developed in the following manner. Similar to the microgrid, the energy management of the hub system comprises the objective function and constraints. Firstly, it is worth mentioning that the objective function related to the hub system includes the following terms:

$$cos{t}^{HUB}=min{\displaystyle \sum _{t\in {\mathrm{\Omega }}^t}\left(\begin{array}{l}{P}_t^C{R}_{CHP}+P_t^{boi}{C}_{boi}-P_t^{bat}{R}_{bat}+\\ W_t^G{R}_{water}-R^{HUB-MC}{P}_{i,t}^{HUB-MC}\end{array}\right)}$$

(6)

Let us consider the total cost related to the smart hub system as the objective function, which includes the operation cost of different units (i.e., CHP, boiler, energy storage and cost of grid water), all of which aim to manage three energy carriers. The last term of the objective function implies the power cost/benefit transferred between the hub system and the network microgrid. Considering the supplement of different energy carriers, the constraints related to energy management can be partitioned with regard to the different layers of the smart hub system. For more clarification, Equations (7)–(21) show the constraints pertaining to three energy carriers. Given that the electrical part is constrained by Equations (7)–(12), constraints pertaining to the thermal part are provided by (13)–(16) and finally the water part is defined based on constraints (17)–(20). By focusing on the electrical layer, the equation of the power balance between the generation units and load associated with the electrical part can be defined by (7). The electrical load, the power consumption of the desalination unit, and the power transaction need to be satisfied by the electrical generation units and thus are inserted in the left-sided part of the Equation (7). On the other hand, the right-sided part of the power balance is included the electrical power generation of CHP and charging/discharging power of the energy storage unit to get into convenient welfare from a technical point of view. Equation (8) shows that the power exchange needs to be limited within an acceptable range. Constraint (9) expresses that the power exchange is not allowed to exceed its transformer’s capacity. The energy balance for the storage unit is carried out each time by (9). Equations (10) and (11) are associated with the energy/power constraints of the storage unit.

2.1. Electrical Part

$$P_{ t }^{E-load }+P_t^{De}+{\eta }_e^T{P}_{i,t}^{HUB-MC}={\eta }_{chp}^{GtoE}{P}_{ t}^C+P_t^{bat }\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(7)

$${\underset{\_}{P}}_{ t}^{HUB-MC }\le P_{i,t}^{HUB-MC}\le {\overline{P}}_{ t }^{HUB-MC }\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(8)

$${\eta }_e^T(P_{i,t}^{HUB-MC})\le C^{Tr},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(9)

$$E_{t }^{bat}=\left( 1-E_{}^{loss}\right)E_{t-1 }^{bat}+P_{ t}^{bat }\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(10)

$${\underset{¯}{E}}_{}^{ bat}\le E_{ t }^{bat}\le {\overline{E}}_{}^{ bat},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(11)

$$\frac{1}{{\eta }_e^{}} {\underset{\_}{P}}_{ }^{ bat}\le P_{ t}^{bat }\le \frac{1}{{\eta }_e^{}}{\overline{P}}_{ }^{ bat}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(12)

As mentioned before, the second layer of the hub system is related to the thermal part whose constraints are represented by (13)–(16). The thermal generations pertaining to both the CHP and boiler units are obligated to supply the thermal load demands ($P^{heat-load}$), which is modeled by the thermal balance Equation (13). The gas grid with regards to the gas balance equation as shown by (14) serves the needed input gas of the CHP/boiler units to generate heat. Furthermore, constraints (15)–(16) show that the power output of boiler and CHP units are not permitted to exceed their transformers capacities.

2.2. Heat Part

$$P_{ t}^{heat-load}={\eta }_{chp}^{G-H}{P}_{ t}^C+{\eta }_{boi}^{G-H}{P}_{ t}^{boi}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(13)

$$P_{ t}^{Gas}=P_{ t}^C+P_{ t}^{boi}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(14)

$${\eta }_{chp}^{G-H}{P}_{ t}^C\le C^{CHP}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(15)

$${\eta }_{boi}^{G-H}{P}_{ t}^{boi}\hspace{0.17em}\le C^{Boi}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(16)

As the last part, the water load demand can be served in two ways: (1) refining the water sea by the desalination unit and (2) by usingg the water grid. Some explanations are needed to describe the water supplement process here. Firstly, a desalination tank is better considered to store the refined water, preceded by pouring into the main water tank in other to optimally manage the power consumption of the desalination unit. Then, the water output of the grid and the refined water coming out from the desalination tank are poured into the main water tank to support the water load demands. The water balance equation related to both of these tanks is modeled by constraints (17)–(18). In addition, the volume of the refined seawater can be restricted by Equation (19). The required power consumption to refine the water is determined and formulated with regards to the desalination unit coefficient and the volume of the sea water as indicated by (20). All in all, given the modeling of the smart hub system and networked microgrid explained above, the total objective function, which is a summation of the total costs of the microgrid and hub system, needs to be minimized as shown by (21). It is important to say that both of these systems are operated and controlled by a central system operator.

2.3. Water Part

$$V_t^T=V_{t-1}^T+W_t^D+W_t^{GRID}-W_t^{load},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(17)

$$V_t^T=V_{t-1}^T+W_t^D-W_t^D,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(18)

$${\underset{\_}{W}}_{}^D.I_t^D\le W_t^D\le {\overline{W}}_{}^D.I_t^D,{}_{ }^{ }\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(19)

$$P_t^{De}=W_t^D.C{F}^{Des},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall t\in {\mathrm{\Omega }}^T$$

(20)

$$C^T=C^{MC}+cos{t}^{HUB}$$

(21)

3. Proposed Cyber Attack Detection Approach

As mentioned before, the studied model as a multi-energy system is able to operate three energy carriers (i.e., the water, thermal and electrical load demands in a correlation environment). Therefore, having a disturbance in information related to each energy carrier can make injurious effects on operating the other energy carriers. In other words, manipulating the monitoring data named the FDIA can lead to inflict social/economic injuries on the proposed multi-energy carriers system. In this regard, the structure of such systems causes to augment the risk of cyber malicious attacks. For this reason, the proposed energy hub-based microgrid model needs to be developed by using an attack detection scheme not to launch any anomaly. As a result, this section is aimed to first present the FDIA modeling and introduce an effective approach in order to meet attack alarms.

3.1. FDIA Model

To make clear the different fields of system security, the attack type needs to be formulated and modeled completely. First, it better is to mention that the cyber-attack modeling is generally sorted based on different methods including the attack trees, attack graphs, and attack networks [35]. The attack model based on the graph is able to clarify somewhat all of the goals followed by hackers. Considering the network nodes, the other method is proposed to model attacks based on the acyclic directed graph. As with the last method, the attack network is introduced as a trusty model for the sake of simulating the stealthy intrusion with regards to hacker goals in the system. The above explanations indicate that the attack of FDI types can be inserted in the class of the attack networks that impose the most destructive effects on the system operation, including economic, load redistribution, and the energy deluding attacks. To focus on modeling FDI attacks, we assume that the hacker can have access to the information related to the studies system. By doing so, the problem function (Z) can be defined by (22) with the aim of changing data D within the system. Injecting the noise into the measurement process leads to be monitored the wrong data by the system, which can be modeled in Equation (23) as FDIA function (Zγ). It should be mentioned that a successful FDI attack is launched when the residue norm of the false function compared to function one is clear to zero as that is followed by (24).

$$Z=h(D_t)$$

(22)

$$Z_{\lambda }=h(D_{bad,t})$$

(23)

$$‖Z_{\lambda }-h(D_{bad,t})‖=‖Z-h(D_t)‖$$

(24)

In addition, the FDI attack performance needs to be assessed by an important criterion, which is defined in Equation (25).

$$\lambda =h(D_t+c_{t=\kappa })\hspace{0.17em}-h(D_t)$$

(25)

Here we assume that an FDI attack is launched by injecting noise (C) at time κ and λ is considered as the structured anomaly vector. All in all, to provide a targeting attack of FDI type, the bad data is generated and modeled as follows:

$$D_{bad,t}=\left\{\begin{array}{l}{D}_t+c_t\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}if\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}t\ge \kappa \\ D_t\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}otherwise\end{array}\right.$$

(26)

3.2. The Structure Definition of the Proposed IPS-RL Approach

For the last few years, machine learning methods have mainly been developed to detect anomalies in different ways such as reinforcement learning (RL), supervised learning and unsupervised learning. Some researchers have proved that the detection methods based on RL are more effective and reliable for the sake of interacting with the environment using a learnable agent than the other methods [36]. The basic structure of the RL method is indicated in Figure 2. According to these results, the RL approach includes two Sections: 2.1. environment 2.2. agent. It is axiomatic that such a method to declare attacks needs to first execute the learning phase proceeding by the detection stage. In the learning phase, the agent is learned to choose an action considering the observation of environmental conditions. In the following, the agent receives a reward arising from the environment to improve the selection of appropriate action. This learning procedure is continued when the agent gets the maximum reward. It should be mentioned that the learning phase needs to be extended in such a way that the agent would be able to settle down the best action in the unknown environment under stochastic and uncertain conditions. Technically, by focusing on the above discussion, each the RL element in the learning step t serves as follows:

The agent: (1) ibtains the observation of environment, (2) selects the action, and (3) receives the relevant scalar reward.

The environment: (1) releases the observation, (2) receives the action, and (3) emits the reward based on the action.

Proceeding with the explanations related to the proposed detection technique, it is necessary to first describe the observable Markov decision process (POMDP) idea. A POMDP problem can be defined by considering different elements such as the set of hidden states (s), set of observations of the environment (o), the transition probabilities related to each state (T), rewards (r), and set of actions selected by the agent (a). Note that the environment is hidden from point of view of the agent in a POMDP problem. In this regard, the agent observes the environment condition and chooses an actual action with regards to the current state. By doing so, the environment emits the relevant reward arising from the selected action and the present state at time t. In this situation, the next state (s_{t + 1}) of the environment is assessed and determined based on the probability related to s_{t + 1} and the reward received by the agent. Such a process of interacting between the agent and environment will be stopped if the final state is brought into the environment. It is important to say that the agent tries to make an optimal policy for mapping observation to action as follows:

$$\hspace{0.17em}RE=E\left[{\displaystyle \sum _{t=0}^{\infty }{u}_{t}r{e}_t}\right]$$

(27)

where R shows the expected value of summation of the rewards and u_t $\in$[0, 1] is defined to determine rewards, which are preferred over future rewards received by the agent. For more clarification about the proposed detection approach, it is important to say that the detection method function needs to be defined based on a POMDP problem and proposed as an effective approach in order to get into the main goals counting the attack alarm and the minimum detection delay. To this end, the proposed function for attack detection regarding the POMDP concept is defined based on Figure 3. Let us suppose that a hacker starts to launch an attack on the system with the use of the unknown strategy at time κ. Since the attack plan may be unknown, the environment states are partitioned based on the “before-attack”, “after-attack” and “final” states. In each t, the agent allows to pick up two actions of “continue” and “stop” with regards to the environment observation. Hence, when the attack is launched to the system, the agent should learn to select the “stop” action to move from the current state (before-attack or after-attack) to the “final” state and alarms the attack. If so, the agent should select the “continue” action in order to stay in the pre-state. By referring to Figure 3, the agent receives the different rewards for the sake of opting for the action in each state. In this regard, the detection problem is aimed in such a way that the agent will receive the rewards one and zero as penalty coefficients because of opting the action of “stop” and “continue” in the “before-attack” state under normal conditions, respectively.

On the other hand, the agent would take reward b due to the selection of “continue” action in the “after-attack” state. Keeping the above explanation in mind, the objective function of the agent is defined to reduce the summation of the penalty coefficients emanating from action choice for all states. To this end, the objective function can be developed as below:

$$\hspace{0.17em}min{R}^{pen}=E^{\kappa }\left[(r{e}_t|t_s<{\kappa }_t)+{\displaystyle \sum _{t=\kappa }^{\infty }b|t_s>{\kappa }_t}\right]$$

(28)

Wherein t_s indicates the stopping time and R^pen reveals the expected value of the penalty coefficient, which is received by the agent. It is also seen that the objective function comprises two key terms related to the received rewards for times t_s < κ and t_s > κ. In the first term, the agent would get the penalty coefficient because of opting for the “stop” action at time t_s < κ. Conversely, the second term shows the summation of the penalty coefficients because of the “continue” action choice in the “after-attack” state at time t_s > κ.

Following the proposed problem definition, an appropriate solution method is needed to provide with regards to the main goals (i.e., the attack alarm and the detection delay minimizing). Accordingly, this paper aims to propose an effective IPS-RL-based attack detection approach. To clarify this method, assume that the observed signal of the environment is brought by considering D_t and h (D_t). In addition, the estimate of likelihood related to each D_t, which is indicated by φ_t, is calculated as follows:

$$\hspace{0.17em}{\phi }_t={(h(D_t)-h(D_{bad,t}))}^T(h(D_t)-h(D_{bad,t}))$$

(29)

The value of φ_t may be zero or close to zero under normal conditions. On the contrary, the high value of φ_t shows the anomaly condition in the system. We assume that an action value is defined for each action-observation pair made by the agent and environment.

This method includes two fundamental steps, namely (1) the learning phase and (2) the detection phase. The first step is provided based on the proposed problem definition to absorb an action value, indicated by P (o,a), for each action-observation pair by considering many knowledge episodes. All of the learning values are stored in Y table in order to use in the second phase. Firstly, the arbitrary action and observation need to be defined based on the “before-attack” state (U) at time 1. Following collection X_t, the observation signal (o^{t + 1}) is calculated by considering the estimate of likelihood φ_t at time t + 1. With regards to o^{t + 1}, the optimal action (a^{t + 1}) is determined according to ɛ-greedy policy, which is aimed to opt for the best action based on the minimum action value (P) and probability 1 − ɛ. It is significant to say that the current action value is updated by using SARSA, which is able to carry out well over PODMP [18].

The Y table needs to be revised by using the new action value P and the action-observation pair is updated to make the new reward value and state at times t<κ and t > κ. This training process related to the first phase is stopped when the “stop” action is opted by the agent for all episodes. The second phase is aimed to detect the unknown anomaly with the use of the Y table trained by the learning phase. In fact, this phase should determine the stopping time t_s and alarms the online attack by using a “stop” action choice. Overall, according to the suggested detection method, the agent is trained to optimal action with the use of minimizing penalty coefficient. This training agent is capable of detecting the online attack as fast as possible stopping time. As it can be seen, the action value updated based on the SARSA algorithm is calculated according to the coefficient α, which is an effective coefficient to get into an optimal learning phase. In other words, the first phase can be optimized with the use of an optimal selection of the coefficient α. To this end, we would propose a proper method based on intelligent priority selection (IPS), aiming to optimize the α value.

3.3. Intelligent Priority Selection Algorithm

This section proposes a method to adjust α for optimizing the learning phase. Different methods are provided and commonly utilized by researchers to solve optimization problems according to math simplifications or artificial intelligence. Nevertheless, time-consuming and low accuracy are involved in them. This section introduces an effective method based on stochastic solutions to enhance accuracy and mitigate the CPU time. In the statistics, the number of combinations of N out of n is computed as follows:

$$\left(\begin{array}{l}N\\ n\end{array}\right)=\frac{N!}{\left(n!\right).(N-n)!}$$

(30)

This shows that the problem space can have many results. A quite reliable approach is to make use of the brute force solution to find the most fitting one. However, it takes a long time due to the very wide search space. To handle this problem, the proposed model mitigates the search space to save time as much as possible. In this regard, the subsequent process indicates the proposed method:

Step1: It is supposed that P is a set of possible solutions to the problem. The matrix K is randomly considered for the control variables initially. The remaining solutions (P-K) are stored in W. All probable sets are substituted by using the sets of K members for any point in W, which is called KT. As modeled in (34), each individual in H is calculated by the embedment of the i-th member of the W into the set K followed by computing the optimal value of the fitting function in i-th $H_{W_i}$, indicated as $F_{W_1,K_n^{\prime }}^{best}$. It should be mentioned that $K_n^{\prime }$ in (34) demonstrates the n-th element of the $K$ substituted by the elements of the $W$.

$$P=[p_1,.....,p_N]$$

(31)

$$K=[k_1,.....,k_n]$$

(32)

$$W=[w_1,.....,w_m]$$

(33)

$$\begin{gathered} KT=\left[\begin{array}{cc}\underset{}{\left.\begin{array}{c}\stackrel{\underset{\uparrow }{\cancel{k_1}=k_1^{\prime }}}{w_1}{k}_2.....k_n\\ k_1\stackrel{\underset{\uparrow }{\cancel{k_2}=k_2^{\prime }}}{w_1}.....k_n\\ k_1 k_2.....\stackrel{\underset{\uparrow }{\cancel{k_n}=k_n^{\prime }}}{w_1}& & \end{array}\right\}{H}_{w_1}}...& \underset{}{\left.\begin{array}{c}\stackrel{\underset{\uparrow }{\cancel{k_1}=k_1^{\prime }}}{w_m}{k}_2.....k_n\\ .\\ .\\ .\\ k_1 k_2.....\stackrel{\underset{\uparrow }{\cancel{k_n}=k_n^{\prime }}}{w_m}\end{array}\right\}{H}_{w_{\mathrm{m}}}}\\ \underset{\begin{array}{cc}\begin{array}{cc}F\left(H_{w_1}\right)=F_{w_{1,k_1^{″}}}^{best}& \end{array}& \end{array}}{\downarrow }& \underset{\begin{array}{cc}\begin{array}{cc}F\left(H_{w_m}\right)=F_{w_{m,k_m^{″}}}^{best}& \end{array}& \end{array}}{\downarrow }\end{array}\right], \\ {H}_{w_i}=\left[H_{w_1},H_{w_2},...,H_{w_m}\right] \forall i\in {\mathrm{\Omega }}^i \\ {k}_M^{″}=\left[k_1^{\prime },k_2^{\prime },...,k_n^{\prime }\right] \forall M\in {\mathrm{\Omega }}^M \end{gathered}$$

(34)

The element of i-th $H_{W_i}$, as illustrated in Equations (35) and (36), are sorted with regards to the objective function value. Components of W are arranged by considering the fitting function. The $W_j^{\prime }$ matrix is indicated as an array of the W matrix components (37) defined in this thread. This process is also precise for set $K_j^{″}$ (38). In this stage, the value of the objective function for $W_1^{\prime }$ is opted, eventually, as the optimal answer (39).

$$F_m^{best}=\left[\begin{array}{l}{F}_{w_{1,{k^{″}}_1}}^{best}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}.\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}.\\ F_{w_{m,{k^{″}}_m}}^{best}\end{array}\right]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall m\in {\mathrm{\Omega }}^m$$

(35)

$$F^{best\_sort}=\left[\begin{array}{l}{F}_{{w^{\prime }}_1\to {k^{″}}_1}^{best}\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}.\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}.\\ F_{{w^{\prime }}_m\to {k^{″}}_m}^{best}\end{array}\right]$$

(36)

$${w^{\prime }}_j=[{w^{\prime }}_1,...,{w^{\prime }}_m]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall m\in {\mathrm{\Omega }}^m$$

(37)

$${k^{″}}_j=[{k^{″}}_1,...,{k^{″}}_m]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall m\in {\mathrm{\Omega }}^m$$

(38)

$$F=F_{{w^{\prime }}_1\to {k^{″}}_1}^{best}$$

(39)

Step2: this stage is aimed to obtain the new $KT$ ($K{T}_r^{new})$ matrix. First, the $W_j$ the matrix needs to be updated by (40) using the $w_j^{\prime }$ matrix components. By considering the $w_1^{\prime }$ as the best option in the earlier iteration, $W_2^{\prime }$ initializes this step as stated in (19). $K{1}_j^{new}$ is defined by removing the $k_j^{″}$ and $w_j^{\prime }$ from the $K_j$ matrix described in (41). The new $K{T}_r^{new}$ component (such as Equation (33)), is created from all of these sets substituting each $W_j$ with a component of$K{1}_j^{new}$. Based on $K{T}_r^{new}$ and $w_j^{\prime }$, the combination of sets is shown ${\psi }_r$in where r is defined between 1 and $m-j$, which j is the number of iteration and m is a constant value, implying the matrix length of W in the first step as shown in (42). The objective value is calculated for each member of ${\psi }_r$ and the best the objective function ($F{1}^{Best})$ and the relevant component is stored by (43) and (44) in matrix ${\psi }_r$ (${\psi }^{Best}$), respectively. for each iteration the matrix K needs to be modified by

${\psi }^{Best}$ (45) as indicated in (46).

$$W_j^{}={w^{\prime }}_{j+1}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall j\in {\mathrm{\Omega }}^j$$

(40)

$$K{1}_j^{new}=\left\{x\left|x\in K_j^{},x\ne {k^{″}}_j,x\ne {w^{\prime }}_j\right.\right\}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall j\in {\mathrm{\Omega }}^j$$

(41)

$${\psi }_r^{}=K{T}_r^{new}\cup {w^{\prime }}_j\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall j\in {\mathrm{\Omega }}^j,\forall r\in {\mathrm{\Omega }}^r=[1,2,...,m-j]$$

(42)

$$F{\mathit{1}}_r^{}=f({\psi }_r^{})$$

(43)

$$F_j^{}=F{\mathit{1}}_{}^{Best}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall j\in {\mathrm{\Omega }}^j$$

(44)

$$K_j^{}={\psi }_{}^{Best}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\forall j\in {\mathrm{\Omega }}^j$$

(45)

Step3: In each iteration, the last component has opted as the optimal one among the other components.

$$F^{best\_total}=F_{}^{Best}$$

(46)

The flowchart of the suggested optimization algorithm is indicated in Figure 4.

4. Salp Swarm Optimization Algorithm

This section is aimed to describe an effective optimization method to solve the studied model. To this end, this paper proposes the optimization algorithm based on the salp swarm method to handle the energy management for the microgrid system interconnected to the smart hub system [38,39]. Firstly, to elaborate the algorithm procedure, some explanations need to be depicted here. For the last few years, some researchers have longingly tried to deal with the swarm nature related to the barrel-shaped bodies of salps in order to solve the optimization problems in the power system. To clarify the proposed algorithm, the mathematical formulation of the salp swarm algorithm is described and modeled by (47)–(50). According to the salp chains, the problem population needs to be partitioned into two segments: (1) followers and (2) leaders. In this regard, considering each salp as a particle of population, the first salp of the chain is assigned as the leader and the other salps make the role of followers in the salp swarm algorithm process. Let us assume a matrix U with R dimension in other to save the position of all salps, which are updated in such a way that the swarm goal (F) is satisfied. Updating process of the leader salp is considered as follows:

$$U_r=\left\{\begin{array}{l}{F}_r+c1((b_r-l{b}_r)c2+l{b}_r)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}c3\ge 0\\ F_r-c1((b_r-l{b}_r)c2+l{b}_r)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}c3<0\end{array}\right.$$

(47)

where in $F_R$ and

$U_R$ imply to the food source and the position of the leader salp, respectively. As it can be seen, the position of leader salp is limited by $B_R,L{B}_R$ in which parameters $c_1,c_2,c_3$ are randomly obtained during the updating procedure. It is worth mentioning that the updating process’s effectiveness depends on the suitable choice of parameter $c_1$, which is calculated as follows:

$$c1=2e^{-(\frac{4a}{A})}$$

(48)

In the above equation, c1is determined based on the iteration number in each iteration$a$. Also, c2 and c3 are selected in the range of [0, 1]. In addition, the position of followers is iteratively updated as follows:

$$U_r{}^i=\frac{1}{2}w{a}^2+v_{1}a$$

(49)

In process of updating the follower I_th ($U_r^i$), $v^1$ is defined as the initial speed of the salp chain and $w$is obtained based on the chain speed as below:

$$w=\frac{v}{v^1}$$

(50)

In case of having $v^1=0$, the Equation (49) is turned into the Equation (50) as follows:

$$U_r{}^i=\frac{1}{2}(U_r{}^i+U_r{}^{i-1})$$

(51)

To sum, formulation related to the salp swarm algorithm can be modeled by using Equations (47)–(50).

5. Stochastic Modeling Based on Ut Method

For the last few years, researchers have tried to model the stochastic inherent of some parameters in the power system structure. In this regard, it is important to describe a close look at their effects on energy management. By doing so, this section is dedicated to explain the stochastic effects modeled by the UT method. It is worth mentioning that the UT method is capable of modeling the correlation environment among the stochastic parameters such as the electrical, thermal and water loads, the wind speed, solar radiation, tidal current. The proposed method can be defined by using $S=\widehat{f}\left(X\right)$ with regards to 2p + 1 different points. The UT method is designed in such a way that the uncertainty parameters are brought by using the normal distribution function considering the mean and standard deviation values related to the parameter which is shown by $m\mathrm{and}\sigma$. In the following The UT method process is formulated by steps (1) to (3):

Step 1: 2p + 1 points is obtained by using (52)–(54) as follows:

$$X^0=m$$

(52)

$$X^k=m+{\left(\sqrt{\frac{p}{1-W^0}{A}_{aa}}\right)}_k\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,2,...,p$$

(53)

$$X^{k+p}=m-{\left(\sqrt{\frac{p}{1-W^0}{A}_{aa}}\right)}_k\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,2,...,p$$

(54)

where $A_{aa}$ imply to the covariance matrix and $\overline{X}=m.$

Step 2: Weight of points is computed by (55):

$$W^k=\frac{1-W^0}{2c}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}k=1,2,...,2c$$

(55)

Note that the summation of the weights related to each point should be equal to 1.

Step 3: by having the points calculated by step 1, the output values are defined by:

$$\overline{S}={\displaystyle \sum _{k=0}^{2p}{W}^k{S}^k}$$

(56)

$$P_{FF}={\displaystyle \sum _{k=1}^{2p}{W}^k}\left(S^k-\overline{S}\right){\left(S^k-\overline{S}\right)}^T$$

(57)

6. Simulation Results

This section is first dedicated to the optimal performance of the proposed energy hub-based microgrid system optimized by the salp swarm algorithm. Then, validating and assessing the proposed attack detection scheme based on the IPS-RL method in order to secure the studied model against malicious anomalies is provided as another goal of this section. To do so, firstly, it is worth saying that the proposed microgrid networked on an IEEE 24-bus test system includes renewable energy sources, such as PV, WT, and tidal units. Figure 5 shows the single line schematic of IEEE 24-bus grid. The technical characteristics of smart grid and microgrid are available in [40,41,42]. Also,

Table 2. The bus and demand parameters.

Bus Number	Vmin	Vmax	Pd (MW)	Qd (MVAR)
1	0.95	1.05	108	22
2	0.95	1.05	97	20
3	0.95	1.05	180	37
4	0.95	1.05	74	15
5	0.95	1.05	71	14
6	0.95	1.05	136	28
7	0.95	1.05	125	25
8	0.95	1.05	171	35
9	0.95	1.05	175	36
10	0.95	1.05	195	40
11	0.95	1.05	0	0
12	0.95	1.05	0	0
13	0.95	1.05	265	54
14	0.95	1.05	194	39
15	0.95	1.05	317	64
16	0.95	1.05	100	20
17	0.95	1.05	0	0
18	0.95	1.05	333	68
19	0.95	1.05	181	37
20	0.95	1.05	128	26
21	0.95	1.05	0	0
22	0.95	1.05	0	0
23	0.95	1.05	0	0
24	0.95	1.05	0	0

expresses some parameters related to each bus that is a criterion for assessing the accuracy of results. The tidal and WT units are located at buses 1,13,21,12 and 2,15,22,10, respectively. In addition, the PV unit is suggested to locate at buses 7, 16, 17 and 23. As mentioned before, this paper aims to propose an efficient structure of hub system interconnected to the proposed networked microgrid to supply electricity, water and thermal loads located in the far area.

Let us assume that the proposed hub system comprises two sub-hub systems, each of which is located at buses 5 and 11. Furthermore, the simulation is executed utilizing a computer with a 4 GHz processor, core i7 and 16 GB of RAM in GAMS software environment using CPLEX solver for MILP problems, which is linked with MATLAB software. To elaborate and evaluate the work, the analysis and assessment of the proposed model are provided based on different aspects as follows:

Case I: Energy management of the energy hub-based networked microgrid system

Case II: Assessing and validating the IPS-RL method based detection scheme against malicious attacks

Case III: Effects of uncertainties on the performance of the studied system

In the following, each one of the cases will be discussed in detail:

6.1. Energy Management of The Energy Hub-Based Networked Microgrid System

This document tries to discuss how the power exchange between the energy hub system and the networked microgrid can help to satisfy the three energy carriers (i.e., water, thermal, and electrical demands). To this end, firstly, we implement and execute the proposed model based on the explained structure in previous sections. By doing so, results related to the energy management of each system (energy hub and microgrid) for all energy carriers and the power transaction between two systems are obtained and represented in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14. As mentioned before, the supplement of water demands can be made in two ways including the grid water and the seawater, which needs to be refined with the use of the desalination unit. In this regard, the required power of these units related to the energy hub systems (hub1 and hub2) to supply the water demands is provided in Figure 6. As it can be seen, the consumed power of the desalination unit of the hub2 system is almost as much as that required for the desalination unit to supply water demands in the next hours (i.e., 150 kW at hour 1, which indicates high water volume). Similar to the hub2 system, the desalination unit of the hub1 system consumes more electrical power in hour 8 than the other hours due to the high water consumption at the same time. Note that the CHP unit is aimed to satisfy both electrical and thermal energies while the boiler unit copes to supply the thermal demands. In this regard, the output powers of CHP and boiler units for both hub1 and hub2 systems are indicated in Figure 7 and Figure 8. Based on the results elicited from energy management related to the hub1 and hub2 systems, the output power of the CHP unit overtakes that of the boiler unit most of the time for the sake of satisfying both electrical and thermal demands. On the contrary, due to the cost of power generation by the CHP and decrease of thermal demand, hub1 has decided to use the boiler unit at t = 14 − 17 and 20 − 21 to serve its thermal demand and consume power from the other agents to supply its electrical demand. The behavior of the boiler unit pertaining to Hub2 at t = 15 − 16 and t = 20 − 21 can be described for the same reason as indicated in Figure 7.

As mentioned before, the networked microgrid copes to supply the electrical loads with the use of incorporating the energy hub systems and the renewable resources including the PV, WT and tidal units. In this regard, the relevant results are indicated in Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13. As it can be seen, each generation unit has differently contributed in generated power depending on the area which is assigned them in network. Looking over Figure 9, Figure 10 and Figure 11, the results are proved that the PV4, WT4, and tidal 1 units have more generated power compared to the other units. In the view of the opposite, the least contribution of the generated power is related to the PV2, WT1, and tidal 4 units due to the load demands located near the relevant area. The previous sections mentioned that the energy hub systems are able to exchange power with the networked microgrid. Hence, the results pertaining to power transactions are shown in Figure 12 and Figure 13. The positive value of power is referred to as an exchange of the power from the microgrid to the energy hub system and vice versa. It is clear that the power injection procedure is from the microgrid to the energy hub1 system at early times while this process is altered in the hours 12 to 24 for the sake of efficiently satisfying the energy carriers during peak-load hours. Similar justification can be provided for power exchanging of energy hub2 system with the microgrid as shown in Figure 13. Besides, this paper aims to represent the effectiveness of the proposed framework, which is the interconnected smart energy hub-microgrid system, in minimizing the cost of the energy carriers. Hence, Figure 14 indicates the water, gas, investment, and total costs under conditions with/without the power transaction link. The relevant results show that such a framework leads to notably reduces the grid water cost as well as the gas cost related to the input gas of CHP and boiler units. For this reason, the total cost has approximately resulted in 4.03 × 10⁷ under the condition with the transaction link taken a marked decline in comparison with condition one.

6.2. Assessing and Validating the IPS-RL Method Based Detection Scheme against Malicious Attacks

It is important to validate the new and effective attack detection method proposed in this paper against stealthy intrusion such as attacks of the FDI type. To do so, we first implemented the FDIA with the use of injecting the false data to the measurement device during t = 80 s to 100 s and t = 125 s to 147 s as shown in Figure 15. By doing so, the monitoring system depicts the false information, which leads to disrupting the system performance and analysis under attack situation compared to the normal operation of the system. To clarify the destructive effects related to FDIA, Figure 16 compares the monitoring data fluctuation related to the attack/normal conditions. Looking over previous sections, this paper proposes an appropriate IPS-RL-based detection method to counteract the stealthy intrusions in smart systems. To ensure this, we tried to first model and assess the proposed detection scheme against FDI attack and compared it to the other detection methods. Hence, the relevant results can be seen in Figure 17 and Figure 18. Looking over the results, by launching FDIA at times 80 s and 125 s, the IPS-RL method based on the learning phase (refer to previous sections) is able to discover the false data as fast as possible time with a detection delay of 2s. Then, the monitoring system is refreshed to depict the actual data followed by the attack alarm. To validate the IPS-RL method, this paper compared this scheme to the other detection approaches (i.e., reinforcement learning (RL) and the support vector machine (VSM)). To this end, it is required that the F-score is calculated for trail numbers through the assessment of the precision versus and recall curves obtained in Equations (58)–(60).

$$Recall=\frac{True Positive}{True Positive+False negative}$$

(58)

$$Precision=\frac{True Positive}{True Positive+False pasitive}$$

(59)

$$F-score=\frac{2\left(Precision\ast Recall\right)}{Precision+Recall}$$

(60)

Hence, we obtained the F-score value regarding 10,000 and 5000 trials for three methods as shown in Figure 17. Looking over the result, the IPS-RL model is more sensitive in distinguishing the attack than the other models.

In addition, there is another consequence to further realize and validate the security scheme’s effectiveness compared to the other methods against the falsified data concerning the attack probability. To this end, we try to use the probability computing approach to reveal the probability of successful attacks. As expressed already, in this paper, we focus on an attack of FDI type in order to calculate probability and assess the functionality of the proposed method. It is significant to say that such attacks do not interrupt the network but that they will lead to some misunderstanding. Hence, the attacker tries to make a penetration in the network for creating an FDI attack. This work can be carried out everywhere in the network. When making a cyber-penetration within the network, it is necessary to compute the probability of an attack launched by hackers. Hence, the successful probability of attacks for carrying out destruction can be achieved in the following:

$$\lambda =h(X_d+c)-h(X_d)$$

(61)

$$P_a=\frac{1}{3}\left({\displaystyle \prod }_{k=1}^n{⋋}_k\right)$$

(62)

where, 0 ≤ ⋋_k ≤ 1, k = 1, 2, …, n,…, N.

In the above equation, λ is the attack vector, c is a factor that defines the alternation of data due to the attack, and Xd defines the data of the system. As h shows the function of the measurement device based on the relevant data can be different for each device. The success probability of attack ($P_a)$ can be computed by Equation (62) in which n is the number of attacks and ⋋ indicates the occurring probability of attack in iteration k for the system. In addition, the attack needs to update its data to delude the system’s devices.

According to the argument aforementioned, the sabotage probability in the network is dependent on the type of cyber-security scheme considered for preventing attacks. Firstly, we implement three cases including case1 (the IPS-RL-based security scheme), case 2 (support vector machine method), and case 3 (the autoencoder method) [43]. In the first step, we as an FDI attack to apply the incorrect data to the measurement devices based on (61) in order to monitor these data for the network operator [44]. This process continues for iteration number 50 under launching three cases based on the security scheme (second step). As the last step, we compute the successful probability of attack for three schemes considering Equation (45). In this regard, by launching an FDI attack in the network, the successful probabilities of this attack for three cases are calculated and represented in Figure 19. It can be seen in this figure, by increasing the percentage of iteration, that the successful probability of attack in the proposed method takes more of a downtrend than case 2 and case 3 for the various iterations. This means that the proposed security framework can be a hard barrier for hackers to have access to network data.

As mentioned before, the interconnected energy hub-based microgrid system can increase the risk of attack launching for the sake of having the correlation structure among the management of the energy carriers [45]. In this regard, this document is aimed to check and describe the energy management of the studied model based on different scenarios including the effects of the FDI to (1) water layer, (2) thermal layer, and (3) electrical layer within the interconnected energy hub-based microgrid system [46,47]. As mentioned before, trading the power between the energy hub systems and microgrid leads to optimally satisfying the electrical load demands. Hence, cyber hackers try to avoid getting into the optimal power transaction by injecting false data to the monitoring electrical load. As the electrical layer, we implemented the FDI attack on the electrical loads of the hub1 and hub 2 systems in hour 13 as indicated in Figure 19 and Figure 20. By doing so, the results related to the power transaction of energy hub systems (see Figure 21 and Figure 22) indicate that the FDIA leads to make a marked fluctuation in the exchanging procedure of power, which results in an upward trend in the transaction costs in the hours 13 to 24.

The second and third scenarios are defined to describe the effects of FDI attacks on the thermal/water layers of the studied system. In this regard, for instance, we carry out the FDI to the thermal load of the energy hub1 system and the water load of the energy hub2 system in the hours 10 to 18 with aim of monitoring the incorrect information as it can be seen in Figure 23 and Figure 24. It is axiomatic that the input gas consumed by the CHP and boiler units depends directly on the thermal demands of the system. Looking over Figure 25, it is clear that the input gas curve related to FDIA takes an upward trend compared to the normal operation condition for most of the attack launching times. For more clarification about the destructive effects of FDIA on the energy management of thermal and water carriers,

Table 3. Comparison of costs of energy carriers.

Operation Conditions	Input Gas Cost	Grid Water Cost	Microgrid Cost
Normal	1.3307^105	0	3.7196^108
Attack (water & thermal)	1.6795^105	2.7479^107	3.937^108

shows the comparison of costs of energy carriers and operation cost related to the microgrid. The results indicate that the FDI causes to add 0.3488 × 10⁵ to the input gas cost and makes a remarkable rise in the grid water cost. In addition, the operation cost of the microgrid pertaining to FDIA is increased nearly 6% in comparison to the normal operation condition due to the technical relationship between the electrical, water, and thermal layers of the energy hub system and the microgrid (refer to Equations (7)–(21)).

All in all, the above explanations demonstrate that launching cyber-attacks in such systems make destructive consequences from point of view of energy management. Therefore, it seems that considering the appropriate IPS-RL-based detection method in the interconnected energy hub-based microgrid system is necessary. Hence, we try to implement the proposed detection scheme on the studied model and provide the relevant results, which are depicted in Figure 26 and Figure 27. Based on the results, the IPS-RL method is capable of detecting and restoring the monitoring data related to the electrical and thermal loads, which were exposed to FDIA, as fast as possible time with a delay of 2 s. In addition,

Table 4. Comparison of the grid water cos under different conditions.

Operation Conditions	Attack without IPS-RL	Attack with IPS-RL
Grid Water Cost	2.7479^107	436

shows that the grid water cost takes a significant decline from 2.7479 × 10⁷ to 436 owing to declaring and removing the false data by the detection system. In summary, such multi-energy carriers systems equipped with the effective/strong IPS-RL-based detection scheme can reduce the risk of attack launching for a significant range. Concerning the results related to Section 1 and Section 2, it can be deduced that some load demands in sundry forms of electrical, thermal, and water are located far from the main grid that needs to be satisfied effectively and optimally. Therefore, the proposed interconnected energy hub-based microgrid system seems to be capable of well supporting all energy carriers (such as water, electrical, and thermal carriers). This means that such framework can remarkably decline the need for the main grid in the far area. On the other hand, considering the simple security structure related to the machining learning based software compared to the detection methods such as blockchain, this model can be suitable for the wide use in detecting the false data in the far area.

6.3. Effects of Uncertainties on the Performance of the Studied System

One of the significant goals of this paper is to check the uncertain effects arising from renewable energy resources on the energy management of the proposed framework. Hence, this document investigates the performance of the interconnected smart energy hub-microgrid system under uncertainty condition and highlights the effects of that on energy management compared to the normal condition. To this end, we implement the UT model on the studied system and see the consequence related to the operating costs and the total operation cost for both the normal and uncertainty conditions as shown in Figure 28. It is clear that the stochastic effect leads to increasing the operating costs compared to condition one. Looking over Figure 29, the operation cost of microgrid has a marked increase because of the uncertainty related to the power output of the renewable energy resources and load demands. Similar to the microgrid, this trend is expanded for the operation cost of the energy hub system. It is significant to say that the uncertain condition leads to change the total cost approximately increased from 4.12 × 10⁸ to 4.56 × 10⁸ in comparison with the deterministic operation.

7. Conclusions

In the present study, the interconnected smart energy hub–based microgrid system was modeled. Initially, the formulation of each unit was extracted. Afterward, the proposed method for cyber-attack detection based on the IPS-RL approach was introduced. According to this network, the optimal algorithm was executed for energy management in which multi-energy hub systems and microgrids cooperate for supplying loads. As well as, to close the reality, the uncertainty based on the UT method was explained for simulation. The most important results have been obtained as follows: the most important result of the first scenario is the fact that the total cost when the power exchange among all units has been taken into account. It is worth mentioning that the total cost of the water network is equal to zero when the power exchange is carried out, owing to its power electric is not supplied by the electrical grid. The second scenario was addressed the impact of attack detection. From the results obtained, it can be seen that a timely detection attack is too important for system stability. The most striking result to emerge from the second scenario is that in the best situation, the operation cost severely goes up in the absence of a detection attack. The second issue that must be considered in the future is the uncertainty regarding energy transactions that might temp the authorized members to fool the system, do more transactions or inject false data through data loggers that may lead the system to mix these data up with the actual uncertainty associated with the renewable energy sources of the system and causes serious problems specifically during critical situations. Hence, recognition of the proposed abuse must also investigate in the future to be added to the security platform to make this technology more implementable and compatible with the features of energy management. Finally, in the third scenario, the performance accuracy of the proposed method was evaluated in presence of uncertainty.