A Survey on Hybrid SCADA/WAMS State Estimation Methodologies in Electric Power Transmission Systems

Abstract

State estimation (SE) is an essential tool of energy management systems (EMS), providing power system operators with an overall grasp of the actual power system operating conditions and aiding them in sustaining reliable and secure operation of the grid. In modern transmission sectors, two main measurement systems are deployed, namely the supervisory control and data acquisition (SCADA) and the wide area monitoring systems (WAMS). The multiple advantages of augmenting conventional SCADA-based SE algorithms with synchrophasor measurements from WAMS are already well-established; thus, an abundance of different methodologies has been reported in the field of hybrid SE (HSE). Under this premise, this paper provides a thorough literature review of novel HSE methods in transmission systems and proposes a classification based on the scope and mathematical modeling of each method. Following a brief introduction to the concept of SE based on WAMS and SCADA measurements, an insight into the main challenges emerging in HSE implementations is provided. Various HSE methods which overcome these challenges are reviewed, for both static and dynamic SE implementations. In conclusion, the research trends in the area of HSE are summarized, and the main findings of this literature review are discussed.

1. Introduction

Power system State Estimation (SE) is arguably one of the most important functions of the Energy Control Center (ECC), being essential for near real-time monitoring of the power system, as well as for supporting various applications of modern Energy Management Systems (EMS). The first formulation of SE dates to the 1970s, when [1] described the SE procedure as the result of the combination of load flow and statistical estimation theory. The objective of the state estimator is to determine the state vector (positive sequence bus voltage phasors) with the highest probability of appearing, considering a system-wide measurement set and the current network topology [1].

In the EMS, SE plays a vital role, enabling the operation of the monitoring and control functions and contingency analysis. In modern ECCs, the state estimator identifies and corrects anomalies in field data, suppresses the effect of gross measurement errors, and refines the measurements, thus providing a reliable set of system states to be used by the operator and as inputs to other computational functions of the EMS. Accurate knowledge of the system operating conditions is directly related to the accuracy, availability, and reliability of the measurements provided to SE [2,3]. State estimators generally comprise the following functions [2,3]:

• Topology processor: the status of switches and circuit breakers are processed to determine the current network topology.
• Observability analysis: the observability of the system for executing SE is verified by analyzing accrued field measurements. In the case of insufficient measurement redundancy, full observability of the network is not achieved; thus, observable islands must be detected for the execution of SE, or observability is reinstated using pseudo-measurements.
• SE algorithm: an optimization process that utilizes the aggregated real-time measurements in a certain time frame and provides the estimated state of the network. The random noise (due to instrument transformers, communication errors, limited meter accuracy etc.) intrinsically existing in field data is filtered out; then, these data are used to calculate the most probable operating state of the power system.
• Bad Data (BD) detection and identification: an algorithm that enables detection, identification, and elimination of gross measurements in the dataset, based on the statistical properties of measurements. Depending on the employed SE algorithm, this step may be integrated directly into the estimation process, or it can be a post-processing step; in the latter case, if BD are detected and eliminated, the SE process is repeated.
• Topology error identification and system parameter estimation: similar to the process of handling BD, the SE results are analyzed to diagnose errors in the assumed network topology due to erroneous reporting of switching component states. Finally, parameter estimation is executed to extract the updated (most probable) values of network parameters based on the SE solution.

In Figure 1, the data flow diagram of a typical SE implementation in the ECC is presented. Modern measurement networks usually employ IEC 60870-5, DNP3, IEC 61850, or Modbus messaging protocols for transmitting unsynchronized measurements. Synchronized data are transmitted according to the IEEE C37.118.2-2011 standard for synchrophasor data transfer, using either TCP or UDP as the transport protocol [3,4].

The fundamental problem of SE, solved using the SE algorithm, is essentially an overdetermined system of nonlinear equations. The relation between the state variables and the measurements comprises the so-called measurement model, which, for a power system of N buses, is generally given by [2,5]:

$$z=h(x)+e$$

(1)

where $h(\cdot )\in {ℝ}^n\to {ℝ}^m$ is the vector of nonlinear functions relating the measurement vector $z\in {ℝ}^m$ and the true (unknown) state vector $x\in {ℝ}^n$, $n=2\mathrm{N}<m$, $e\in {ℝ}^m$ is the measurement error vector that is assumed to have zero mean, and a diagonal covariance matrix $R\in {ℝ}^{m\times m}$, with $R_{ii}={\sigma }_i^2$ being the variance of the ith measurement.

Under the assumption of Gaussian measurement noise, the mathematical formulation of the SE problem is usually obtained by maximizing the log-probability function of observation z, resulting in the following optimization problem with objective function J(x) [2,5]:

$$\underset{x}{\min }J(x)={[z-h(x)]}^T{R}^{-1}[z-h(x)]=e^T{R}^{-1}e$$

(2)

According to (2), the SE is solved by minimizing the Weighted Least-Squares (WLS) criterion, which is accomplished via iterative numerical methods. Besides the WLS algorithm, other SE methods, such as decoupled WLS and Least Absolute Value (LAV) SE, have also been proposed. However, WLS still prevails in practical implementations [2]. Irrespective of the SE solution algorithm, most methods in the literature refer to the measurement model (1), which is used as a basis for subsequently defining the objective function employed in the SE algorithm.

Equation (1) is used to formulate the Static SE (SSE) problem, disregarding any information pertaining to state transition, i.e., a memory of the state estimates of previous SE executions. Information about the state vector at previous time instant(s) can be exploited in a prediction scheme to aid subsequent SE executions, resulting in Dynamic SE (DSE) formulations. With the increased uncertainties introduced by the rapid integration of Distributed Energy Resources (DERs) on the generation side, and volatile loads/modern demand-response technologies on the demand side, DSE offers the added benefit of accurately capturing system dynamics, thus becoming crucial to power system monitoring, control, and protection [6].

According to [6], the majority of proposed DSE methods are variants of the Kalman Filter (KF) technique, which is derived from the Bayesian framework of leveraging prior knowledge of a system’s states obtained over time, along with the available measurement data. KF methods utilize the measurements and conjoin them with the state transition model of the system to compute an optimal state estimate. This process typically consists of two stages, namely, the prediction (time update) and the correction (measurement update). Depending on how the state statistics are propagated, i.e., the presumed state transition model, different KF methods have been developed [7,8]: Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), Cubature Kalman Filter (CKF), and Ensemble Kalman Filter (EnKF).

The most common measurement data sources in modern power systems are the Supervisory Control and Data Acquisition (SCADA) system and the Wide Area Monitoring System (WAMS). Classic SE algorithms utilize voltage magnitude and active/reactive power injection/flow measurements gathered by the SCADA system via Remote Terminal Units (RTUs) deployed across the grid. Existing SCADA systems generally have long data update periods by today’s standards, at approximately 2–8 s, and offer moderate measurement accuracy. Nevertheless, SCADA technology is now quite mature and well-established and has thus become a powerful tool for dispatching operators [3].

WAMS generally comprise diversely placed Phasor Measurement Units (PMUs) and Phasor Data Concentrators (PDCs). PMUs have been under development since the early 1980s and have since become a widely accepted measurement system of modern WAMS. PMUs are devices capable of providing high-fidelity GPS-synchronized timestamped snapshots of bus voltage and branch current phasors, as well as frequency and Rate of Change of Frequency (ROCOF) measurements, at high reporting rates (10–120 Hz, depending on nominal system frequency and PMU manufacturer) [9,10]. Synchrophasor data can be best utilized in modern SE algorithms, effectively enhancing their accuracy and performance, due to the following reasons [10,11]:

• The quality of the state estimates is significantly improved, owing to the high measurement accuracy of PMUs (0.1% for magnitudes, 0.001 rad for phase angles [4]). This aspect provides operators with greater confidence in the system conditions and reliable data for downstream control functions in the EMS.
• PMUs directly measure the system states, i.e., bus voltage phasors, hence simplifying the mathematical formulation of the SE problem into a linear one.
• Measurement synchronization is crucial for obtaining accurate snapshots of the estimated system operating conditions in different areas. Leveraging the absolute GPS-dictated time reference is decisive for achieving an overall and detailed picture of the operating state of the network.
• PMUs offer superfast measurement reporting rates (~100 times faster than RTUs), which is important for tracking the trajectory of the system states, above all in scenarios characterized by high dynamics.

In this context, the utilization of PMU data provides important benefits for all stages of the SE process, and, to this end, synchrophasor networks are being rapidly deployed worldwide. However, as the deployment of PMUs is still somewhat limited due to technical and economic constraints, in the vast majority of transmission systems, the synchrophasor measurements are not sufficient to ensure full network observability for SE execution. Therefore, it is reasonable to anticipate that WAMS and SCADA will still coexist and complement each other in the foreseeable future, and conventional RTU measurements will be used, along with a limited number of PMU measurements, for implementing viable SE algorithms [11,12]. Under this premise, the integration of SCADA/WAMS measurements in SE has drawn considerable attention in both transmission and distribution levels, resulting in the accumulation of a large amount of corresponding literature.

This paper aims to give insight into recent research progress and achievements considering the development and implementation of Hybrid SE (HSE) algorithms for transmission systems, utilizing both RTU and PMU measurements. Based on the examined research trends, future work on such SE algorithms is proposed. The rest of the paper has the following structure. In Section 2, the major challenges of integrating synchrophasor measurements in conventional SE methods are described. In Section 3 and Section 4, several SE approaches proposed for overcoming these challenges are categorized and analyzed, according to recent literature, for both SSE and DSE, respectively. Finally, in Section 5, the prospects of this research area, based on the outcomes of the examined work, are identified, and the paper is concluded.

2. State Estimation under Sensor Diversity

Incorporating measurements from different types of sensors (RTUs and PMUs) improves SE performance in terms of precision and BD processing, as these are closely related to measurement redundancy. However, the integration of data from multiple sources in SE is not straightforward; the most widely recognized challenges can be summarized in two aspects, according to research into HSE formulation:

• Different sensor reporting rates and time-inconsistent data. The first aspect is that PMU measurements are updated at a much higher rate than RTU measurements. Furthermore, there is no coordinated timing of measurement arrivals between different data sources, referred to as the asynchronization or time-skew problem, which indicates that the field measurements will likely not form a dataset captured at an exact time instance. Apart from the uncoordinated timestamping of RTU data, time inconsistencies also occur due to different communication delays among sensors [12,13].
• Different types and accuracy levels of measured quantities. The measured quantities provided by RTUs and PMUs are different, which leads to challenges in practical implementations, as existing SE software needs to be modified. Numerical problems may also arise at flat start involving current phasor measurements, when these are expressed in polar coordinates [14]. Different accuracy levels between sensor types have an impact on the choice of appropriate measurement weights, which in turn negatively affects SE convergence in case of excessively diverse values [15].

In the context of these challenges, various methods proposed for effectively overcoming the above issues in HSE are elaborated on and discussed in the following. Figure 2 depicts the hierarchical structure of the diverse categories of HSE methods that are proposed in this paper. It is worth mentioning that SSE methods are categorized according to their scope and algorithmic process, while considering the challenges discussed in this Section. DSE methods are principally classified with respect to their employed state space model, and then subcategories are formed considering the contribution and mathematical background of each method.

3. Hybrid Static State Estimation

Utilizing both RTU and PMU measurements, the Hybrid Static State Estimation (HSSE) is the most widely deployed form of SE in ECCs to date. RTUs are the conventional data source, while the deployment of PMUs enriches and improves the measurement profile in transmission systems. Various proposed HSSE methods are organized into different categories with respect to the main problem tackled, as they pertain to the arising challenges discussed in Section 2.

3.1. Fusion of Asynchronous and Multi-Rate Data

With higher reporting rates of PMUs, numerous PMU scans are available during the time interval between two SCADA scans, as shown in Figure 3. However, the system will likely be unobservable at instants when only PMU measurements are received, on account of the limited number of PMU measurements in most transmission systems. Under this premise, several methods have been suggested to address the significantly different reporting rates of PMUs and RTUs, by restoring system observability between consecutive RTU measurement sets.

3.1.1. State/Measurement Reconstruction

One approach relies on the utilization of a linear PMU-only SE method for reconstructing and tracking the network state between instances of the arrival of RTU measurements, while a nonlinear SE is executed when data from PMUs and RTUs are retrieved simultaneously [16,17,18,19,20]. In [16], the linear SE relies on the refreshed synchrophasor measurements together with power and (generator) voltage pseudo-measurements, derived either from the last execution of the HSSE algorithm with both PMU and RTU data (fixed values) or the last reconstructed values pertaining to the latest PMU-only SE execution (recursively calculated values). For the linear SE problem, the authors in [17,18] propose that PMU measurements are jointly processed with voltage and current phasor pseudo-measurements from the PMU-unobservable subnetwork, calculated using the most recent state estimate. Work [19] introduces an HSSE method that shifts between using a WLS estimator at instances of simultaneous PMU and RTU measurement update and a robust weighted LAV (WLAV) estimator when only the PMU measurements are refreshed; in the latter case, PMU data, along with a minimum number of reconstructed RTU measurements, are leveraged to attain system observability, similarly to [17,18]. For improving SE performance, in publication [20], the author proposes a SE decentralization method by exploiting islands of phasor measurements with common GPS-dictated reference angle and RTU-observable sub-islands, formed using only critical RTU measurements. This enables the deployment of computationally demanding HSSE methods, such as the one proposed in [19], against bad data and cyber-attacks, by exploiting their inherent robustness to the appearance of outliers in the measurement dataset.

Alternative methods for overcoming the issue of limited PMU data between SCADA scans are proposed in [21,22,23,24]. Work [21] relies on distributed Compressive Sensing (CS). At time instants when only PMU measurements are available, RTU measurements are reconstructed using distributed CS according to the spatial and temporal correlation among the most recent estimated states to achieve complete system observability. A classic WLS-based method incorporating PMU-derived power flows, PMU-measured voltage magnitudes, and RTU measurements, is then used to solve the SE problem. Paper [22] proposes a robust HSSE method in which processed PMU data are used as a priori information for a modified WLS-based SE. In the time gap between two SCADA scans, the states of PMU-unobservable buses are interpolated from limited PMU data using an interpolation matrix, which is updated upon the arrival of both RTU and PMU measurements. Robustness is achieved through dynamic readjustment of measurement weights based on the occurrence of events that can affect measurement reliability. Article [23] proposes a real-time recursion correction linear HSSE method utilizing stream processing. RTU and PMU measurements are processed asynchronously, and PMU measurements, along with the most recent SE results, are utilized in between two consecutive RTU measurement sets to run a recursion correction. The efficiency of this process is optimized for large-scale power systems by implementing multithreaded stream processing. Authors in [24] propose an HSSE method employing a Sequential Quadratic Programming (SQP) algorithm. When only PMU measurements are refreshed, line current pseudo-measurements reconstructed from the most recent SE results are leveraged to satisfy observability. Non-linearities in the optimization problem can be efficiently handled by SQP, resulting in a decent performance in the case of equality-constrained SE problems and large power systems.

3.1.2. Measurement Buffering

Buffering of PMU measurements is also a possible method for tackling the issue of multi-rate data by considering the statistical properties of a retained set of consecutive PMU measurements [25,26,27,28]. This approach attempts to sanitize PMU data received in a certain time window (buffer) by mitigating the effects of measurement noise and the deviation due to variation of system states. The derived information is then used to perform SE upon receiving new RTU measurements. Hence, buffering methods are applicable, assuming that the HSSE is executed periodically at intervals larger than the reporting period of RTUs. Various methods for optimization of the PMU data buffering process have been proposed in recent literature: in [25], the optimal buffer length is determined using hypothesis testing, and in [26], three methods are tested to determine the buffer length by evaluating the mean and variance shift of a set of consecutive PMU measurements. In [27,28], the time skew present in unsynchronized measurements is tackled by utilizing buffered PMU measurements. In [27], a procedure for considering temporal and space correlations in PMU measurement datasets for HSSE is proposed. A time series of PMU data can be modeled by stationary Vector Autoregressive (VAR) models to filter measurement and communication noise stemming from different sensors. Authors in [28] introduce a robust HSSE method in which the correlations of diverse measurement types are considered to improve SE accuracy. The Unscented Transformation (UT) is applied to calculate the self- and the cross-correlations among RTU measurements, while correlations among the PMU measurements are modeled as in [27].

3.1.3. Summary

This subsection discussed the issue of incomplete observability that arises under the usually limited number of high-rate PMU measurements. Most methods resort to WLS linear SE implementations or non-WLS approaches that significantly improve execution times in order to achieve a high frequency of SE execution while leveraging measurements or results of previous SE runs. This way, SE results become available in the EMS, even at time instants of partial (PMU) observability of the system. If the HSSE is solved upon the arrival of both RTU and PMU data, then buffering methods are also applicable for enhancing the accuracy and reliability of SE.

3.2. Fusion of Different Measured Quantities

Another important aspect of HSSE formulations lies in combining phasor measurements with conventional measurements in a unified estimator. Due to the different properties of measured quantities, direct inclusion of phasor measurements in existing state estimators requires significant modifications to the traditional EMS software. Methods proposed in literature for incorporating measurements from different sensor types can be divided into three categories, as reported in [12], which are also illustrated in Figure 4: One-Stage HSSE methods (OSHSSE), Two-Stage HSSE methods (TSHSSE), and Fusion HSSE methods (FHSSE).

3.2.1. One-Stage HSSE Methods

One-Stage HSSE (OSHSSE) methods are used to directly combine RTU and PMU measurements into a single mathematical formulation (Figure 4a). Apart from the necessary modification of existing SE algorithms in the EMS to incorporate phasor measurements, several other challenges appear when the above methods are employed:

Numerical stability: The inclusion of current measurements and large variations between RTU and PMU measurement weights may lead to matrix ill-conditioning at flat start and poor algorithm convergence, respectively [15]. Methods for circumventing the emerging numerical issues are reported in [17,29,30,31,32]. Work in [29] proposes a unified HSE, in which RTU measurements are jointly processed with PMU data, and branch current phasors are simply converted from polar to rectangular coordinates, thus avoiding ill-conditioning issues. The employed state space model and calculation of current measurement covariances using error propagation theory are described in detail. Similarly, a nonlinear WLS formulation of the HSSE problem is presented in [30], which avoids numerical problems encountered at flat start or for lightly loaded lines when current measurements are expressed in polar form by employing rectangular coordinates for current measurements when necessary. Work in [17] expands upon [29,30] by applying the matrix inversion lemma (Sherman–Morrison–Woodbury formula) so that the structure of conventional SCADA-based state estimators remains intact, while current phasors are again expressed in rectangular coordinates. In [31], the authors propose an approach of including only voltage phasor measurements by formulating current measurement equations as a function of bus voltage phasors adjacent to PMU buses. The state vector comprises bus voltage phasors, along with the polar form of the branch currents measured by PMUs, and equality constraints are used to relate PMU buses and their respective adjacent buses. A regularized HSSE method is proposed in [32] to tackle the problem of numerical instability of WLS based on least squares optimization and the minimization of the solution norm. The L-curve method is applied for selecting the regularization parameter due to its robustness and its ability to handle measurement correlation. Zero injections are modeled as equality constraints and are included at the post-estimation stage.

SE algorithm performance: Various formulations have been proposed for enhancing OSHSSE performance, leveraging a linear measurement model, a distributed multi-area approach, or by solving SE in the complex domain [33,34,35,36,37,38,39,40,41,42,43,44,45,46]:

• Linear models: Work [33] proposes converting all power measurements into equivalent current phasors, resulting in the formulation of a linear iterative WLS-based HSSE. The Jacobian and gain matrices remain constant during the iterative process, consequently improving SE execution times. In a similar fashion, [34] proposes an estimator based on the non-iterative linear equality-constrained WLS approach by transforming RTU-measured quantities into voltages and currents expressed in rectangular form, while equality constraints are used to model zero injections. A study in [35] proposes a fully linear robust LAV-based HSSE, which is solved non-iteratively using linear programming. Reference [36] introduces a linear robust HSSE, employing a Schweppe-type M-estimator with Huber loss function. The method of Iteratively Reweighted Least Squares (IRLS) is used to maximize the likelihood function in the M-estimator. In [37], the authors propose two LAV-based robust HSSE methods, both leveraging linear measurement models. The first method is based on the linear LAV approach and is formulated as a single linear programming problem, while the second builds upon an alternative LAV-based estimator that can be solved by gradient-based methods. In [38], a linear Equivalent Circuit Formulation (ECF) of the power system is derived by relating power flows/injections to bus voltage and branch current phasors. Both RTU and PMU measurement models are expressed using linear circuits; thus, the estimated state is obtained by solving a linear set of optimality conditions. In [39], the authors further enhance ECF for practical implementations by including circuit models for all possible combinations of RTU measurements, null injections, and the possibility of having no measurements at a bus.
• Multi-area: Work [40] proposes a decentralized multi-area HSSE, by first implementing a PMU-only SE concerning area boundary buses and then introducing the boundary bus state estimates as equality constraints imposed upon the local SE of each area. Similarly, [41] introduces a decentralized method, which enables exploiting the computation capability of each subarea to solve for a global state vector in parallel via the Gossip-based Gauss-Newton algorithm. The robustness of the proposed scheme is achieved by dynamically adjusting the measurement weights based on the measurement quality to suppress the influence of BD on the SE solution. In [42], the authors propose a multi-area HSSE based on a fully distributed Gauss-Newton method, in which each area carries out the SE locally and independently, relying on local measurements and limited communication with neighboring areas. Alternatively, [43] proposes an iterative multi-area HSSE approach, in which all subareas run their SE sequentially in each iteration, and the problem of area slack bus angle referencing in case of insufficient PMUs is tackled using pseudo-measurements derived from the SE solution of boundary buses.
• Complex variables: Various papers have also addressed the solution of the SE problem in the complex domain, which is also proven to be computationally advantageous. Publication [44] first presents an implementation of the WLS-based HSSE problem in complex variables by employing the complex Taylor series expansion, which is based on Wirtinger calculus. It is worth noting that current measurements do not require any special handling, unlike HSSE implementations over the real domain. In [45], the complex normal equations of [44] are expanded to incorporate equality constraints. Finally, [46] proposes a constant gain matrix method utilizing the perturbed Gauss-Newton method for nonlinear least-squares formulated in the complex domain as in [44].

3.2.2. Two-Stage HSSE Methods

The concept of Two-Stage HSSE (TSHSSE) algorithms is commonly employed to decouple RTU and PMU measurements in the SE problem. Generally, they consist of a conventional (RTU-only) SE followed by a linear (PMU-only) SE, or vice versa, so that two types of measurements appear in separate formulations (Figure 4c).

Publications [47,48,49,50,51,52] discuss various methods of incorporating phasor measurements and the results of the traditional state estimator in a post-processing step. This approach formulates a linear (non-iterative) estimation step that requires no modification of the traditional EMS software. Refs. [47,48,49,50] propose the inclusion of the estimated states from the conventional WLS SE in the measurement vector of a subsequent linear SE, along with PMU measurements in rectangular coordinates. Based on this concept, publication [51] formulates the HSSE problem using various approaches for mixing polar and rectangular coordinates of measurements and states. In [52], a similar cascaded architecture is employed, in which the output of the first stage is then taken as a priori state information by the second PMU-based estimation module. This method also avoids the assumption of measurement uncorrelatedness while defining measurement weighting factors assigned to real and imaginary complex measurement components.

Alternatively, in [53], the authors propose a TSHSSE scheme, in which a linear state estimator is formulated first, using only synchrophasors in rectangular coordinates, to estimate the states for the PMU-only observable subnetwork. Subsequently, the SE solution from the first stage and the RTU measurements are simultaneously processed by a nonlinear WLS SE in the polar form to determine the system-wide state vector. The estimated states of the first stage are introduced into the second stage either as high-accuracy measurements or as equality constraints. By adopting this TSHSSE method, in [54], the states of PMU-observable buses are introduced directly in the final state vector, along with the results of the second estimation stage. Expanding upon this approach, [15] proposes a two-stage process by executing a LAV-based SE with PMU measurements only and then a WLS-based post-processing step, as in [53]. Thus, numerical problems stemming from incorporating PMU current measurements in WLS-based algorithms are alleviated.

Publication [55] proposes a decentralization method for the TSHSSE approach. The network is partitioned into several ‘linear’ and ‘nonlinear’ areas, according to the presence of PMUs or RTUs at substations, respectively. For each ‘linear’ area, a linear WLS SE is utilized to estimate subsystem states, while for each ‘nonlinear’ area, the conventional nonlinear SE is used. As linear SE is solved at a higher rate, estimated states of ‘linear’ area boundary buses are treated as highly weighted pseudo-measurements for the nonlinear SE.

3.2.3. Fusion HSSE Methods

The Fusion HSSE (FHSSE) and TSHSSE algorithms are similar in that they both comprise separate SE modules for each measurement type. However, in FHSSE, these two estimators work in parallel, and their estimates are combined in a post-estimation fusion scheme to produce the final estimate (Figure 4b). The two produced state estimates are commonly fused using the following formula [56]:

$$\widehat{x}=W_R{\widehat{x}}_R+W_P{\widehat{x}}_P$$

(3)

where ${\widehat{x}}_R$ and ${\widehat{x}}_P$ are the estimated state vectors from the RTU- and PMU-based modules, respectively, $W_R$ and $W_P$ are the weighting matrices derived from the covariance matrices of RTU and PMU measurements, respectively, and $\widehat{x}$ is the fusion state vector. The main advantage of FHSSE formulations is the ability to execute the two modules in parallel, leading to reduced computational burden. However, it is noteworthy that the application of (3) requires both complete RTU- and PMU-observability of the system.

In [57], the authors introduce a multi-stage parallel SE architecture based on (3) to optimally combine results independently obtained from RTU- and PMU-based estimation modules. In order to satisfy the complete PMU-observability of the system, the inclusion of a priori information in the SE is proposed. Work [58] enhances this approach and presents a fast algorithm for implementing FHSSE, improving SE execution times by utilizing parallel processing of RTU- and PMU-based modules and expediting the bad data handling process.

References [59,60] propose robust FHSSE methods. In [59], a data fusion architecture is proposed in which RTU and PMU measurements are separately processed by BD-resilient maximum correntropy-based estimators. In [60], authors have devised a robust FHSSE framework to deal with the unknown (non-Gaussian) statistical properties of measurement noise and the issue of time skew. The method employs robust Mahalanobis distances combined with a statistical test for determining the appropriate buffering length and weights of PMU measurements. A Schweppe-type Huber generalized maximum-likelihood estimator is then used to filter out non-Gaussian noise and suppress the effects of measurement outliers.

In [61], a strategy for distributed FHSSE is suggested using a multi-stage approach, where the RTU- and PMU-based SE modules compute the local state vector separately and in parallel. In order to overcome PMU observability issues, a local state vector extension is used, enabling the RTU- and PMU-based local estimators to obtain the SE results for the same set of buses in each sub-area. The consistency requirement of local SE results obtained for overlapping regions between sub-areas is satisfied via the exchange of SE results between neighboring estimators. Finally, the fusion of local state estimates is attained via (3).

3.2.4. Summary

In this Subsection, the HSSE methods have been categorized with respect to their mathematical modeling and employed method of handling different types of measurements. Although OSHSSE methods are prone to numerical instability under certain conditions and require modification of existing EMS software, extensive research on the subject has led to the formulation of easily implementable, numerically stable, and computationally efficient algorithms. Both TSHSSE and FHSSE methods have been proven to be effective ways of handling PMU measurements separately from the existing RTU-based SE algorithms using pre- or post-processing software modules. These formulations are also suitable for parallel or decentralized SE implementations. For TSHSSE methods, communication of data between the two estimation stages or between locations/ECCs (in the case of decentralized approaches) is still an issue, hindering SE efficiency. For FHSSE methods, the assumption of complete PMU-observability of the system is rarely satisfied in practice, leading to the utilization of a priori (historical) data to achieve observability instead of highly accurate real-time PMU measurements. Similar to TSHSSE, implementation of FHSSE using parallel processing still poses a challenge and should concern future research.

4. Hybrid Dynamic State Estimation

Most of the state estimators deployed in modern ECCs are based on steady-state power system models. Therefore, they do not consider system dynamics, that is, the power system’s varying operating conditions over time. This primarily stems from the unsynchronized and low-density information provided by classic SCADA-based measurements [6]. With the increasing number of deployed PMUs, the development of Hybrid Dynamic State Estimation (HDSE) methods for power system monitoring and control becomes realizable, significantly enhancing the capabilities of existing SE processes.

The HSE methods examined in Chapter 3 are static in the sense that each measurement snapshot relates to a single instance of the state vector and does not capture the system’s dynamics. When changes in the power system are mainly driven by slow load fluctuation, analysis is performed under a quasi-steady regime, and in this scenario, HSSE is adequate for providing a reliable state estimate [62]. In the face of large-scale penetration of stochastic and intermittent renewable energy generation, responsive loads, and microgrids, the power system is subject to many different types of dynamics. To perform SE for quasi-steady operating conditions, the use of conventional measurements from RTUs is indeed sufficient. Conversely, when the system is operating under transient conditions, synchrophasors could be the only dependable measurements to reliably carry out SE. In this case, low-rate RTU measurement scans could be used to enhance measurement redundancy and SE robustness [63].

Various challenges emerge in implementing HDSE methods, similar to the ones arising for HSSE: fusion of asynchronous measurements with different reporting rates, robustness against corrupted, missing or delayed data from diverse sensors, the inclusion of heterogeneous measurement datasets, and correlation among measured quantities. In recent literature [6], HDSE methods have been divided with respect to the mathematical formulation of power system dynamics in their adopted state space model into three major categories: Dynamic State Estimation (DSE), Forecasting-aided State Estimation (FASE), and Tracking State Estimation (TSE). In the following, methods appertaining to all three categories are reviewed.

4.1. DSE Methods

Generally, in DSE methods, the state vector is augmented with the internal states of various system components, such as synchronous machines or dynamic loads, and a nonlinear discrete-time state space model is adopted, involving system inputs, model parameters and (in)equality constraints. The continuous-time general state space model associated with DSE is the one customarily adopted for transient stability analysis, given by [6,63]:

$$\begin{array}{l}\dot{x}(t)=f(x(t),u(t),p)\\ 0=g(x(t),u(t),p)\end{array}$$

(4)

where $f(.)$ and $g(.)$ are nonlinear functions, $x\in {ℝ}^n$ is the unknown state vector, comprising voltage phasors and dynamic states, $u$ is the system input vector, and $p$ represents the model’s parameters. Equation (4) may also be subject to equality and inequality constraints pertaining to various electrical quantities. For SE implementation, the continuous time-domain model in (4) is discretized, yielding:

$$\begin{array}{l}{x}_k=f(x_{k-1},u_k,p)+w_k\\ z_k=h(x_k,u_k,p)+e_k\end{array}$$

(5)

where subscript $k$ denotes the time sample, and $w_k$ is assumed to be the normally distributed state transition noise with zero mean and covariance matrix $Q_k$. The remaining quantities have already been defined in (1) and (4).

The dynamic state vector (or model parameters) in (5) are estimated using various proposed methods within the KF framework. UKF-based approaches [64,65,66,67] propose DSE frameworks under the presence of multi-rate data from RTUs and PMUs for tracking the dynamic system state during the transient operating condition. In [64], the dynamic model (4) is discretized at appropriate sampling periods depending on each sensor’s reporting rate, and the RTU/PMU equations are then decoupled so that a different estimator is applied to each discrete model. The final estimated state is calculated using (3). In [65], a UKF-based covariance intersection method for multi-rate data fusion is considered. Publication [66] proposes a discrete-time model for state transition derived from an Artificial Neural Network (ANN) trained for short-term load forecasting. The DSE problem is solved using a dual-UKF approach, considering the interactions between the state vector and the dynamic power system model. Different reporting rates of RTUs and PMUs are addressed using a parameterized process model and a state reconstruction technique. In [67], the application of the method in [66] is extended to include dynamic state variables of synchronous machines, and the SE problem is distributed over a multi-agent system.

In [68], a multi-scale SE framework is proposed, which effectively enables the integration of SSE and DSE in EMS. The system is monitored in real-time using the Singular Spectrum Analysis (SSA)-based change point detection approach. A robust HSSE algorithm is executed, and if a disturbance is detected by the SSA, the HSSE results are used for the initialization of a PMU-only DSE algorithm for real-time monitoring of transient conditions.

4.2. FASE Methods

Various HDSE methods have been developed under the assumption of a quasi-steady regime, where the state transition depends only on slow load variations and corresponding generation adjustments. Under this premise, FASE methods neglect the dynamics of $x(t)$ in (4), i.e., $\dot{x}(t)=0$, and exploit a linear transition model, resulting in the following discretized state space representation [69]:

$$\begin{array}{l}{x}_k=F_k{x}_{k-1}+g_{k-1}+w_k\\ z_k=h(x_k,p)+e_k\end{array}$$

(6)

where state vector $x_k$ now only contains bus voltage magnitudes and angles, matrix $F_k$ represents the linear state transition, and vector $g_{k-1}$ captures the trend of state trajectory. Conventionally, $F_k$ and $g_{k-1}$ are model parameters to be estimated, and essentially express the system memory derived from historical information.

Pioneering work on hybrid FASE methods is presented in [70], by combining the concepts of unscented filtering and SE. The proposed derivative-free HSE approach updates the state transition and trajectory parameters via Holt’s linear, exponential smoothing technique and applies the UT to achieve enhanced estimation accuracy with a relatively simple implementation.

In [71,72,73,74], authors propose FASE methods addressing the issue of missing data. In [71], an EKF-based FASE framework for HDSE, which is robust against randomly missing RTU data, is devised. Based on the probability-maximization method, the SE problem is formulated as a constrained optimization one, with PMU measurements considered as inequality constraints, which is solved using the Particle Swarm Optimization (PSO) algorithm. Work [72] introduces a multi-area FASE method for large-scale power grids by implementing a modified distributed KF capable of independently estimating local states using local measurements, taking the appearance of multiple missing measurements into consideration. The internodal transformation theory is employed to deal with the communication problem between the distributed subsystems, while the SE formulation and solution are identical to that in [71]. Work [73] presents a CKF-based FASE, which utilizes a state forecasting technique for state prediction during periods of missing PMU data under an unreliable communication network. State forecasting is accomplished via Holt’s smoothing technique. In [74], the authors leverage the spherical cubature and the Gaussian quadrature rules to estimate the properties of the prior and the posterior probability densities of the state space and the measurement space. The estimated mean and the associated covariance are then utilized by the CKF to estimate the final states of the power system. State forecasting during the interval between SCADA scans is accomplished as in [73].

Papers [75,76,77,78,79] focus on the time alignment of different data sources for FASE. Ref. [75] proposes a robust UKF algorithm based on constrained quadratic estimation. The synchronization of PMU and RTU data is achieved by combining the maximum correlation and interpolation synchronization methods, and robustness is attained by introducing the strong tracking UT algorithm to modify the prediction covariance matrix in the presence of gross measurement errors. Authors in [76] propose a FASE method considering irregular sampling and random delay of measurements, in which an EKF-based algorithm is used for the time alignment of measurements and SE results. Work [77] suggests the utilization of the Unscented Rauch–Tung–Streibel (URTS) smoothing algorithm, which handles the time-skew problem by optimally predicting the values of RTU measurements during the absence of SCADA information, thus reducing the corresponding estimation errors. Paper [78] introduces a two-stage FASE considering the different sensor reporting rates. The first stage employs traditional RTU-only WLS SE and computes an a priori evaluation for the state variables; the second stage leverages PMU data to update the posterior distribution obtained according to Bayesian inference concepts. Work [79] addresses random time delays between SCADA updates in FASE, using Bernoulli distributed random variables. Results obtained from two separate unsynchronized SEs are fused, based on the covariance intersection scheme.

Parallelization techniques for mitigating the increased computational load of FASE methods are reported in [80,81,82]. In [80], the application of parallel processing for FASE is proposed, aiming to accelerate the DSE solution for large-scale systems using PMU and RTU measurements. A massively parallel HDSE is developed on a Graphics Processing Unit (GPU), employing a lateral two-level FASE solution algorithm based on the EKF method. In [81], an ANN-assisted dual-UKF FASE algorithm is developed using a multi-agent-based model. A dynamic ANN is applied for developing a discrete-time state transition model, which is used for short-term load forecasting. The dual UKF simultaneously estimates the state vector and determines the parameters of the dynamic ANN. In the same spirit, [82] proposes a distributed CKF algorithm for FASE implementation in large-scale power systems consisting of non-overlapping sub-areas. The FASE algorithm is executed in parallel among the subsystems to forecast and estimate each local state, mitigating the computational and communication load of SE, lacking a central coordinator.

Various two-stage FASE methods have also been proposed [83,84]. Work [83] is based on a KF-aided two-stage FASE, in which a linear PMU-only state estimator serves as a preprocessing first stage, and the second stage is an iterative WLS formulation that combines RTU measurements with pseudo-measurements obtained from the first stage. The method introduces a recursive KF, which uses consecutive scans of PMU measurements to provide more accurate pseudo-measurements for the second stage. Likewise, [84] proposes a two-stage FASE, in which a limited number of PMU measurements, along with an UKF-based estimation of the RTU measurements, are utilized to obtain the SE solution at intervals between two successive SCADA scans. A conventional HSE method is executed at each arrival of both measurement sets.

4.3. TSE Methods

By introducing the assumption that $F_k=I$ and dropping the term $g_{k-1}$ in (6), the simplified state-space model of TSE is obtained as follows:

$$\begin{array}{l}{x}_k=x_{k-1}+w_k\\ z_k=h(x_k,p)+e_k\end{array}$$

(7)

Work [85] addresses the effects of time-skew between simultaneously processed RTU and PMU data for TSE implementation. The proposed method includes a prediction step, an innovation analysis and event detection step, and a correction step. Predicted RTU measurements are used to ensure observability, a combined analysis of synchronized measurement variations and innovation vectors is utilized to distinguish between abrupt system state changes and gross errors, and a constrained least-squares optimization problem is solved in the correction step.

The parallelization and performance enhancement of TSE is reported in [86], which introduces a decentralized UKF-based method employing a consensus algorithm for multi-area TSE. The UKF is used to execute TSE locally in each of the non-overlapping power system subareas, and the consensus algorithm carries out the exchange of local state information between neighboring areas.

A TSE approach considering the temporal aspects of the estimation process within a maximum correntropy-based EKF is proposed in [87]. By representing the behavior of the state variables with a nonparametric probabilistic model within the kernel density estimation, this approach considers sudden state transitions as part of the non-Gaussian process noise. To suppress the effects of suspect BD, a novel strategy to update the size of Parzen windows in the kernel estimation is introduced.

Correlated prediction and measurement errors are addressed in [88], which proposes a KF-based TSE method for joint state and parameter estimation, capable of incorporating RTU and PMU measurements and abrupt state change detection by implementing an adaptive filter based on optimal tracking. The estimation problem is formulated as two loosely coupled linear subproblems of state and parameter tracking.

4.4. Summary

Based on this review of HDSE methods, it becomes clear that the inclusion of system dynamics in the SE model can greatly benefit power system monitoring and control. The computationally demanding DSE methods generally adopt highly detailed state transition models and mainly exploit high-rate PMU data. On the other hand, SCADA measurements are often utilized only as supplementary information. TSE methods offer easily implementable enhancements to existing SSE algorithms and are proven useful for sustaining SE reliability under the appearance of BD. However, they are not suitable for tracking system dynamics. FASE methods form a middle ground between DSE and TSE. These approaches assume a linear transition model. While they are not ideal for tracking dynamics, they are more computationally efficient and suitable for augmenting SSE in the EMS compared to DSE.

5. Discussion

After extensively discussing the various HSSE and HDSE methods developed, a summary of the identified challenges to be addressed, along with possible research directions in the area of SE, are presented.

• Numerical stability and convergence: For OSHSSE formulations, attention has been mainly given to issues such as estimation optimality and computational performance. However, diverse accuracy classes of sensors, and widely differing measurement values, may result in poor SE convergence under direct RTU and PMU measurement fusion. Recent research has proven that decoupling the SE formulations of SCADA and WAMS in TSHSSE and FHSSE architectures offers the benefits of improved numerical stability and good convergence properties. Optimality of the estimated states, communication load, parallelization and observability are issues that arise with these methods. Future works on HSE should also cover the effect of PMU measurements on systems with insufficient redundancy levels in traditional measurements and systems that are observable only under both SCADA and WAMS measurements.
• Diverse reporting rates: The issue of multi-rate RTU and PMU measurement fusion is taken into consideration in numerous proposed approaches. However, the effect of measurement time-skew and observability with respect to PMU measurements on the quality of the SE results should be investigated further. Uncoordinated SCADA scans and individual asynchronous measurements (e.g., from FACTS controllers, DERs, etc.) may still not arrive periodically, as is usually assumed in the literature. Therefore, the case of SE under unsynchronized multi-rate/multi-sensor measurement infrastructure should be investigated further, in order to develop implementable SE algorithms. Possible solutions would be to implement HSE methods that consider estimation fusion without any assumptions on sensor reporting rates or utilizing buffering techniques and treating asynchronous and low-rate data as a statistical trend that enhances real-time SE. Groundwork on this research has been recently laid in [89,90] under the concepts of extreme learning machines and Bayesian SE, respectively.
• Robustness: With respect to BD and cyberattack resilience, further investigation in BD detection and identification methods is needed, especially for practical implementations of TSHSSE and FHSSE. Future works should consider more realistic modeling of the communication networks and HSE algorithms that are robust against missing, corrupted, or even deliberately altered measurements [91]. By examining recent literature, one can infer that the utilization of machine learning-based or forecasting-aided SE methods could provide acceptable SE solutions by mitigating the effect of such anomalies in the measurement network. Most inherently robust non-WLS SE approaches (e.g., [19,32]) involve a trade-off between computational efficiency and accuracy/convergence/BD resilience. Practical implementations of such methods need to be addressed in future work, and ways for them to possibly complement existing WLS algorithms should be investigated further [92,93,94].
• Performance: SE performance also becomes a crucial factor for fully exploiting high-rate PMU data between SCADA scans. In the face of the increasing complexity of modern power systems, the need to optimize SE algorithms for large-scale systems arises, especially when considering HDSE techniques could be vital to fast control actions. This has led to the implementation of distributed HSE methods, which provide accuracy comparable to centralized HSE methods while significantly enhancing SE performance by solving subarea SE problems in parallel. Such performance enhancements are of utmost importance for the real-time implementation of HSE in the ECC and should thus concern future publications [95,96]. Interarea communication and information exchange between boundary buses is a topic that should be expanded upon in future research, particularly in the case of missing measurements or BD and the unobservability of subareas. It would also be interesting to investigate employing distributed algorithms for handling BD in such SE methods.
• Measurement model: Improvements in the HSE measurement model could also concern future publications. The measurement noise statistics are usually unknown or time-varying, an issue that is accentuated in the presence of multiple data sources. The assumptions pertaining to the statistical properties of measurements (noise Gaussianity and uncorrelatedness) should be alleviated in future work, expanding upon the research in [97,98,99]. Furthermore, network modeling also emerges as an issue, with wide integration of FACTS, HVDC, and DERs that need to be included accordingly in the measurement model and considered in the parameter estimation step. Mathematical formulations for different combinations of system components and measurement data need to be derived, to broaden the current SE modeling framework.
• State transition models: Future work on DSE should focus on more accurate and detailed state transition models. In order to obtain reliable state estimates for dynamic systems, state prediction and filtering should be robust against the various uncertainties inherently present in the power system, possibly using pattern recognition which could capture the effects of stochastic components, such as DERs. Multi-area, numerically robust and efficient data-driven DSE methods also comprise fruitful approaches that deserve further investigation [100,101]. Considering that the accuracy of PMUs is reduced during transient conditions, it is also imperative to test and validate DSE methods with real field data and assess their real-time accuracy and performance under such circumstances. It is worth noting that already existing static state estimators could benefit significantly in terms of accuracy and convergence by incorporating a simple state space model, such as the ones adopted in TSE or FASE methods. In order to improve upon such methods, future research could consider simultaneous topology and parameter estimation, the correlation between different PMU channels and measurement scans, as well as more advanced techniques for state forecasting and formulating the transition function.

6. Conclusions

To conclude, this paper conducted a comprehensive literature review of various HSE methods by categorizing them depending on their scope and applicability for facing the most prominent challenges of multi-sensor SE in modern transmission systems. Overall, HSSE algorithms are widespread owing to their simple implementation, good convergence properties, and estimation quality. HDSE methods prove to be most suitable for leveraging high-refresh-rate PMU data in real-time monitoring of the system while utilizing RTU data as a secondary source of information. The key topics that should be investigated in future HSE implementations are found in computational performance, accuracy and BD resilience, adaptability and tracking of operating conditions, as well as optimal exploitation of multi-rate and historical data.