# A Review on Distribution System State Estimation Algorithms

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. DSSE Special Attributes

## 3. DSSE Fundamentals and Main Algorithms

#### 3.1. State Vectors

NV-DSSE | BC-DSSE | |
---|---|---|

Configurations | All (radial, meshed, etc.) [23] | Mostly limited to radial |

Coordinates | Polar or rectangular [17,23,24,25,26] | Mostly rectangular [27,28,29,30] |

Implementation | More complex | Relatively easy [9] |

Sensitivity (weight variations, etc.) [11] | Higher (might affect convergence) | Lower |

Computational time [9,11] | Higher | Lower |

Compatibility with transmission system state estimation | Yes [22] | No |

#### 3.2. DSSE Algorithms

#### 3.2.1. Conventional, Model-Based Algorithms

#### 3.2.2. Forecasting-Aided Algorithms

#### 3.2.3. Data-Driven Algorithms

#### 3.2.4. Summary and Comparison between DSSE Algorithms

## 4. Auxiliary Algorithms

#### 4.1. Observability

Method | Description | Advantages and Disadvantages |
---|---|---|

GMM and EM [70,71,72] | Probabilistic and statistical approaches using Gaussian distributions, correlations, etc. | Mature algorithms, easy implementation, but with increased sensitivity and difficulties when it comes to large systems. |

Correlation coefficients [72] | ||

DNN [74] | Solutions with learning-based approaches, using mostly neural networks. | Modern, advanced algorithms, but with database requirements. |

PDP [75] | ||

NARX [76] |

#### 4.2. Bad Data Detection

Method | Description | Advantages and Disadvantages |
---|---|---|

Model-based [78,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98] | ${\mathrm{L}}_{2}$-norm [84,85,86,87,88,89], LNR [90,91,92,93,94], Chi-square [78,95,96,97], Combination [98] | Robust and simple methods. Not as accurate. |

Data-driven [82,96,99,100,101,102,103,104,105,106,107,108,109] | LR [99], SVM [96,100,101,102,103], ANN [100,104], RNN [82,103], CNN [105], MSA [102], k-NN [106], ENN [106], KMC [107], FC [108,109] | Very accurate methods. Need large amounts of data. |

#### 4.3. Meter Placement

## 5. Technical Requirements and Applications

#### 5.1. Technical Requirements

#### 5.2. Applications

Application | Approach |
---|---|

PV [51,134,135] | Model based, such as WLS [134], WLAV [135] but also FASE, such as EKF variations [51]. |

PV and WT [136] | Data-driven DSSE, with DNN [136]. |

PV, WT and diesel generator [137] | Model-based WLS DSSE coupled with DNN [137]. |

EV charging [50] | FASE DSSE, with a variation of EKF [50]. |

## 6. Future Trends and Challenges

## 7. Conclusions

- WLS is the most commonly used algorithm, not only in theoretical development but also in actual applications.
- Data-driven algorithms challenge the dominance of model-based counterparts.
- DSSE can play an important role in the ongoing energy transition, but in order to do so in a large scale, standardized solutions should be established.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Mah, D.N.-Y.; Wu, Y.-Y.; Ip, J.C.-M.; Hills, P.R. The role of the state in sustainable energy transitions: A case study of large smart grid demonstration projects in Japan. Energy Policy
**2013**, 63, 726–737. [Google Scholar] [CrossRef] [Green Version] - Mahmud, N.; Zahedi, A. Review of control strategies for voltage regulation of the smart distribution network with high penetration of renewable distributed generation. Renew. Sustain. Energy Rev.
**2016**, 64, 582–595. [Google Scholar] [CrossRef] - Das, C.K.; Bass, O.; Kothapalli, G.; Mahmoud, T.S.; Habibi, D. Overview of energy storage systems in distribution networks: Placement, sizing, operation, and power quality. Renew. Sustain. Energy Rev.
**2018**, 91, 1205–1230. [Google Scholar] [CrossRef] - Anjana, K.; Shaji, R. A review on the features and technologies for energy efficiency of smart grid. Int. J. Energy Res.
**2018**, 42, 936–952. [Google Scholar] [CrossRef] - Gust, G.; Biener, W.; Brandt, T.; Dallmer-Zerbe, K.; Neumann, D.; Wille-Haussmann, B. Decision Support for Distribution Grid Planning. In Proceedings of the 2016 IEEE International Energy Conference (ENERGYCON), Leuven, Belgium, 4–8 April 2016; pp. 1–7. [Google Scholar]
- Alassery, F. Advanced metering infrastructure smart metering based on cloud architecture for low voltage distribution networks in application of smart grid monitoring. Sustain. Comput. Inform. Syst.
**2022**, 35, 100747. [Google Scholar] [CrossRef] - Wang, G.; Giannakis, G.; Chen, J.; Sun, J. Distribution system state estimation: An overview of recent developments. Front. Inf. Technol. Electron. Eng.
**2019**, 20, 4–17. [Google Scholar] [CrossRef] - Watitwa, J.; Awodele, K. A Review on Active Distribution System State Estimation. In Proceedings of the 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa (SAUPEC/RobMech/PRASA), Bloemfontein, South Africa, 28–30 January 2019; pp. 726–731. [Google Scholar]
- Primadianto, A.; Lu, C.-N. A Review on Distribution System State Estimation. IEEE Trans. Power Syst.
**2017**, 32, 3875–3883. [Google Scholar] [CrossRef] - Sarada Devi, M.S.N.G.; Yesuratnam, G. Comparison of State Estimation Process on Transmission and Distribution Systems. In Advances in Decision Sciences, Image Processing, Security and Computer Vision; Satapathy, S.C., Raju, K.S., Shyamala, K., Krishna, D.R., Favorskaya, M.N., Eds.; Learning and Analytics in Intelligent Systems; Springer International Publishing: Cham, Switzerland, 2020; Volume 4, pp. 414–423. ISBN 978-3-030-24317-3. [Google Scholar]
- Ahmad, F.; Rasool, A.; Ozsoy, E.; Sekar, R.; Sabanovic, A.; Elitaş, M. Distribution system state estimation-A step towards smart grid. Renew. Sustain. Energy Rev.
**2018**, 81, 2659–2671. [Google Scholar] [CrossRef] [Green Version] - Radhoush, S.; Bahramipanah, M.; Nehrir, H.; Shahooei, Z. A Review on State Estimation Techniques in Active Distribution Networks: Existing Practices and Their Challenges. Sustainability
**2022**, 14, 2520. [Google Scholar] [CrossRef] - Dehghanpour, K.; Wang, Z.; Wang, J.; Yuan, Y.; Bu, F. A Survey on State Estimation Techniques and Challenges in Smart Distribution Systems. IEEE Trans. Smart Grid
**2019**, 10, 2312–2322. [Google Scholar] [CrossRef] - Bhattar, P.L.; Pindoriya, N.M.; Sharma, A. A combined survey on distribution system state estimation and false data injection in cyber-physical power distribution networks. IET Cyber-Phys. Syst. Theory Appl.
**2021**, 6, 41–62. [Google Scholar] [CrossRef] - Táczi, I.; Sinkovics, B.; Vokony, I.; Hartmann, B. The Challenges of Low Voltage Distribution System State Estimation—An Application Oriented Review. Energies
**2021**, 14, 5363. [Google Scholar] [CrossRef] - Gómez-Expósito, A.; Conejo, A.J.; Cañizares, C.A. Electric Energy Systems: Analysis and Operation; CRC Press: Boca Raton, FL, USA, 2016; ISBN 978-1-4200-0727-5. [Google Scholar]
- Majumdar, A.; Pal, B.C. A three-phase state estimation in unbalanced distribution networks with switch modelling. In Proceedings of the 2016 IEEE First International Conference on Control, Measurement and Instrumentation (CMI), Kolkata, India, 8–10 January 2016; pp. 474–478. [Google Scholar] [CrossRef] [Green Version]
- Hansen, C.; Debs, A. Power system state estimation using three-phase models. IEEE Trans. Power Syst.
**1995**, 10, 818–824. [Google Scholar] [CrossRef] - Xygkis, T.C.; Korres, G.N. Optimized Measurement Allocation for Power Distribution Systems Using Mixed Integer SDP. IEEE Trans. Instrum. Meas.
**2017**, 66, 2967–2976. [Google Scholar] [CrossRef] - Clements, K.A. The Impact of Pseudo-Measurements on State Estimator Accuracy. In Proceedings of the 2011 IEEE Power and Energy Society General Meeting, Detroit, MI, USA, 24–28 July 2011; pp. 1–4. [Google Scholar]
- Baran, M.E.; Zhu, J.; Kelley, A.W. Meter Placement for Real-Time Monitoring of Distribution Feeders. In Proceedings of the Power Industry Computer Applications Conference, Salt Lake City, UT, USA, 7–12 May 1995; pp. 228–233. [Google Scholar]
- Kotha, S.K.; Rajpathak, B. Power System State Estimation using Non-Iterative Weighted Least Square method based on Wide Area Measurements with maximum redundancy. Electr. Power Syst. Res.
**2022**, 206, 107794. [Google Scholar] [CrossRef] - Gholami, M.; Tehrani-Fard, A.; Lehtonen, M.; Moeini-Aghtaie, M.; Fotuhi-Firuzabad, M. A Novel Multi-Area Distribution State Estimation Approach for Active Networks. Energies
**2021**, 14, 1772. [Google Scholar] [CrossRef] - Kotsonias, A.; Asprou, M.; Hadjidemetriou, L.; Kyriakides, E. State Estimation for Distribution Grids with a Single-Point Grounded Neutral Conductor. IEEE Trans. Instrum. Meas.
**2020**, 69, 8167–8177. [Google Scholar] [CrossRef] - Yao, Y.; Liu, X.; Li, Z. Robust Measurement Placement for Distribution System State Estimation. IEEE Trans. Sustain. Energy
**2019**, 10, 364–374. [Google Scholar] [CrossRef] - Lu, C.; Teng, J.; Liu, W.-H. Distribution system state estimation. IEEE Trans. Power Syst.
**1995**, 10, 229–240. [Google Scholar] [CrossRef] - Melo, I.D.; Pereira, J.L.; Ribeiro, P.F.; Variz, A.M.; Oliveira, B.C. Harmonic state estimation for distribution systems based on optimization models considering daily load profiles. Electr. Power Syst. Res.
**2019**, 170, 303–316. [Google Scholar] [CrossRef] - Wang, H.; Schulz, N. A Revised Branch Current-Based Distribution System State Estimation Algorithm and Meter Placement Impact. IEEE Trans. Power Syst.
**2004**, 19, 207–213. [Google Scholar] [CrossRef] - Pau, M.; Pegoraro, P.A.; Sulis, S. Efficient Branch-Current-Based Distribution System State Estimation Including Synchronized Measurements. IEEE Trans. Instrum. Meas.
**2013**, 62, 2419–2429. [Google Scholar] [CrossRef] - Pau, M.; Ponci, F.; Monti, A.; Sulis, S.; Muscas, C.; Pegoraro, P.A. An Efficient and Accurate Solution for Distribution System State Estimation with Multiarea Architecture. IEEE Trans. Instrum. Meas.
**2017**, 66, 910–919. [Google Scholar] [CrossRef] [Green Version] - Meriem, M.; Bouchra, C.; Abdelaziz, B.; Jamal, S.O.B.; Faissal, E.M.; Nazha, C. Study of State Estimation Using Weighted-Least-Squares Method (WLS). In Proceedings of the 2016 International Conference on Electrical Sciences and Technologies in Maghreb (CISTEM), Marrakech, Morocco, 26–28 October 2016; pp. 1–5. [Google Scholar]
- Pau, M.; Pegoraro, P.A.; Sulis, S. Performance of Three-Phase WLS Distribution System State Estimation Approaches. In Proceedings of the 2015 IEEE International Workshop on Applied Measurements for Power Systems (AMPS), Aachen, Germany, 23–25 September 2015; pp. 1–6. [Google Scholar]
- Shafiei, M.; Nourbakhsh, G.; Arefi, A.; Ledwich, G.; Pezeshki, H. Single Iteration Conditional Based DSSE Considering Spatial and Temporal Correlation. Int. J. Electr. Power Energy Syst.
**2019**, 107, 644–655. [Google Scholar] [CrossRef] - Angioni, A.; Shang, J.; Ponci, F.; Monti, A. Real-Time Monitoring of Distribution System Based on State Estimation. IEEE Trans. Instrum. Meas.
**2016**, 65, 2234–2243. [Google Scholar] [CrossRef] - Kumar, C.S.; Rajawat, K.; Chakrabarti, S.; Pal, B.C. Robust distribution system state estimation with hybrid measurements. IET Gener. Transm. Distrib.
**2020**, 14, 3250–3259. [Google Scholar] [CrossRef] - Jin, T.; Shen, X. A Mixed WLS Power System State Estimation Method Integrating a Wide-Area Measurement System and SCADA Technology. Energies
**2018**, 11, 408. [Google Scholar] [CrossRef] [Green Version] - Gol, M.; Abur, A. LAV Based Robust State Estimation for Systems Measured by PMUs. IEEE Trans. Smart Grid
**2014**, 5, 1808–1814. [Google Scholar] [CrossRef] - Dobakhshari, A.S.; Azizi, S.; Abdolmaleki, M.; Terzija, V. Linear LAV-based state estimation integrating hybrid SCADA/PMU measurements. IET Gener. Transm. Distrib.
**2020**, 14, 1583–1590. [Google Scholar] [CrossRef] - Celik, M.; Abur, A. A robust WLAV state estimator using transformations. IEEE Trans. Power Syst.
**1992**, 7, 106–113. [Google Scholar] [CrossRef] - Weng, Y.; Negi, R.; Liu, Q.; Ilić, M.D. Robust state-estimation procedure using a Least Trimmed Squares pre-processor. In Proceedings of the ISGT 2011, Anaheim, CA, USA, 17–19 January 2011; pp. 1–6. [Google Scholar] [CrossRef]
- Mili, L.; Phaniraj, V.; Rousseeuw, P. Least median of squares estimation in power systems. IEEE Trans. Power Syst.
**1991**, 6, 511–523. [Google Scholar] [CrossRef] - Zhao, J.; Huang, C.; Mili, L.; Zhang, Y.; Min, L. Robust Medium-Voltage Distribution System State Estimation Using Multi-Source Data. In Proceedings of the 2020 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA, 17–20 February 2020; pp. 1–5. [Google Scholar]
- Zhao, J.; Zhang, G.; La Scala, M.; Wang, Z. Enhanced Robustness of State Estimator to Bad Data Processing Through Multi-innovation Analysis. IEEE Trans. Ind. Inform.
**2017**, 13, 1610–1619. [Google Scholar] [CrossRef] - Huang, Y.-F.; Werner, S.; Huang, J.; Kashyap, N.; Gupta, V. State Estimation in Electric Power Grids: Meeting New Challenges Presented by the Requirements of the Future Grid. IEEE Signal Process. Mag.
**2012**, 29, 33–43. [Google Scholar] [CrossRef] [Green Version] - Liu, H.; Hu, F.; Su, J.; Wei, X.; Qin, R. Comparisons on Kalman-Filter-Based Dynamic State Estimation Algorithms of Power Systems. IEEE Access
**2020**, 8, 51035–51043. [Google Scholar] [CrossRef] - Shafiei, M.; Ledwich, G.; Nourbakhsh, G.; Arefi, A.; Pezeshki, H. Layered Based Augmented Complex Kalman Filter for Fast Forecasting-Aided State Estimation of Distribution Networks. arXiv
**2018**, arXiv:1804.08298. [Google Scholar] - Durgaprasad, G.; Thakur, S.S. Robust dynamic state estimation of power systems based on M-Estimation and realistic modeling of system dynamics. IEEE Trans. Power Syst.
**1998**, 13, 1331–1336. [Google Scholar] [CrossRef] - da Silva, A.M.L.; Do Coutto Filho, M.; de Queiroz, J.F. State forecasting in electric power systems. IEE Proc. C Gener. Transm. Distrib.
**1983**, 130, 237–244. [Google Scholar] [CrossRef] - Zhang, Z.; Qiu, J.; Ma, W. Adaptive Extended Kalman Filter with Correntropy Loss for Robust Power System State Estimation. Entropy
**2019**, 21, 293. [Google Scholar] [CrossRef] [Green Version] - Antončič, M.; Papič, I.; Blažič, B. Robust and Fast State Estimation for Poorly-Observable Low Voltage Distribution Networks Based on the Kalman Filter Algorithm. Energies
**2019**, 12, 4457. [Google Scholar] [CrossRef] - Barchi, G.; Macii, D. A photovoltaics-aided interlaced extended Kalman filter for distribution systems state estimation. Sustain. Energy Grids Netw.
**2021**, 26, 100438. [Google Scholar] [CrossRef] - Ćetenović, D.; Ranković, A. Optimal parameterization of Kalman filter based three-phase dynamic state estimator for active distribution networks. Int. J. Electr. Power Energy Syst.
**2018**, 101, 472–481. [Google Scholar] [CrossRef] - Wan, E.A.; Van Der Merwe, R. The Unscented Kalman Filter for Nonlinear Estimation. In Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373), Lake Louise, AB, Canada, 4 October 2000; pp. 153–158. [Google Scholar]
- Simpson, R.H. Power System Data Base Management. In Proceedings of the Conference Record of 2000 Annual Pulp and Paper Industry Technical Conference (Cat. No.00CH37111), Atlanta, GA, USA, 19–23 June 2000; pp. 79–83. [Google Scholar]
- Wang, Q.; Li, F.; Tang, Y.; Xu, Y. Integrating Model-Driven and Data-Driven Methods for Power System Frequency Stability Assessment and Control. IEEE Trans. Power Syst.
**2019**, 34, 4557–4568. [Google Scholar] [CrossRef] - Matijašević, T.; Antić, T.; Capuder, T. A systematic review of machine learning applications in the operation of smart distribution systems. Energy Rep.
**2022**, 8, 12379–12407. [Google Scholar] [CrossRef] - Sah, S. Machine Learning: A Review of Learning Types. Math. Comput. Sci.
**2020**. [Google Scholar] [CrossRef] - Sharma, P.; Said, Z.; Kumar, A.; Nižetić, S.; Pandey, A.; Hoang, A.T.; Huang, Z.; Afzal, A.; Li, C.; Le, A.T.; et al. Recent Advances in Machine Learning Research for Nanofluid-Based Heat Transfer in Renewable Energy System. Energy Fuels
**2022**, 36, 6626–6658. [Google Scholar] [CrossRef] - Rajamoorthy, R.; Arunachalam, G.; Kasinathan, P.; Devendiran, R.; Ahmadi, P.; Pandiyan, S.; Muthusamy, S.; Panchal, H.; Kazem, H.A.; Sharma, P. A novel intelligent transport system charging scheduling for electric vehicles using Grey Wolf Optimizer and Sail Fish Optimization algorithms. Energy Sources Part A Recover. Util. Environ. Eff.
**2022**, 44, 3555–3575. [Google Scholar] [CrossRef] - Dridi, S. Supervised Learning–A Systematic Literature Review. 2021. Available online: https://www.researchgate.net/publication/354996999_Supervised_Learning_-_A_Systematic_Literature_Review (accessed on 26 September 2022).
- Bernieri, A.; Betta, G.; Liguori, C.; Losi, A. Neural networks and pseudo-measurements for real-time monitoring of distribution systems. IEEE Trans. Instrum. Meas.
**1996**, 45, 645–650. [Google Scholar] [CrossRef] - Ferdowsi, M.; Benigni, A.; Lowen, A.; Zargar, B.; Monti, A.; Ponci, F. A Scalable Data-Driven Monitoring Approach for Distribution Systems. IEEE Trans. Instrum. Meas.
**2015**, 64, 1292–1305. [Google Scholar] [CrossRef] - Zhang, L.; Wang, G.; Giannakis, G.B. Distribution System State Estimation Via Data-Driven and Physics-Aware Deep Neural Networks. In Proceedings of the 2019 IEEE Data Science Workshop (DSW), Minneapolis, MN, USA, 2–5 June 2019; pp. 258–262. [Google Scholar] [CrossRef]
- Zamzam, A.S.; Sidiropoulos, N.D. Physics-Aware Neural Networks for Distribution System State Estimation. IEEE Trans. Power Syst.
**2020**, 35, 4347–4356. [Google Scholar] [CrossRef] [Green Version] - Tran, M.-Q.; Zamzam, A.S.; Nguyen, P.H. Enhancement of Distribution System State Estimation Using Pruned Physics-Aware Neural Networks. In Proceedings of the 2021 IEEE Madrid PowerTech, Madrid, Spain, 28 June–2 July 2021. [Google Scholar] [CrossRef]
- Tran, M.-Q.; Zamzam, A.; Nguyen, P.; Pemen, G. Multi-Area Distribution System State Estimation Using Decentralized Physics-Aware Neural Networks. Energies
**2021**, 14, 3025. [Google Scholar] [CrossRef] - Huang, M.; Wei, Z.; Sun, G.; Zang, H. Hybrid State Estimation for Distribution Systems With AMI and SCADA Measurements. IEEE Access
**2019**, 7, 120350–120359. [Google Scholar] [CrossRef] - Zamzam, A.S.; Fu, X.; Sidiropoulos, N.D. Data-Driven Learning-Based Optimization for Distribution System State Estimation. IEEE Trans. Power Syst.
**2019**, 34, 4796–4805. [Google Scholar] [CrossRef] [Green Version] - Fantin, C.A.; Castillo, M.R.C.; de Carvalho, B.E.B.; London, J.B.A. Using Pseudo and Virtual Measurements in Distribution System State Estimation. In Proceedings of the 2014 IEEE PES Transmission & Distribution Conference and Exposition–Latin America (PES T&D-LA), Medellin, Colombia, 10–13 September 2014; pp. 1–6. [Google Scholar]
- Singh, R.; Pal, B.; Jabr, R. Distribution system state estimation through Gaussian mixture model of the load as pseudo-measurement. IET Gener. Transm. Distrib.
**2010**, 4, 50–59. [Google Scholar] [CrossRef] - Singh, R.; Pal, B.C.; Jabr, R.A. Statistical Representation of Distribution System Loads Using Gaussian Mixture Model. IEEE Trans. Power Syst.
**2010**, 25, 29–37. [Google Scholar] [CrossRef] [Green Version] - Manitsas, E.; Singh, R.; Pal, B.; Strbac, G. Modelling of Pseudo-Measurements for Distribution System State Estimation. In Proceedings of the CIRED Seminar 2008: SmartGrids for Distribution, Frankfurt, Germany, 23–24 June 2008; p. 42. [Google Scholar]
- Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. Ser. B (Methodol.)
**1977**, 39, 1–22. [Google Scholar] - Manitsas, E.; Singh, R.; Pal, B.C.; Strbac, G. Distribution System State Estimation Using an Artificial Neural Network Approach for Pseudo Measurement Modeling. IEEE Trans. Power Syst.
**2012**, 27, 1888–1896. [Google Scholar] [CrossRef] - Wu, J.; He, Y.; Jenkins, N. A robust state estimator for medium voltage distribution networks. IEEE Trans. Power Syst.
**2013**, 28, 1008–1016. [Google Scholar] [CrossRef] - Hayes, B.P.; Gruber, J.K.; Prodanovic, M. A Closed-Loop State Estimation Tool for MV Network Monitoring and Operation. IEEE Trans. Smart Grid
**2015**, 6, 2116–2125. [Google Scholar] [CrossRef] - Monticelli, A.; Garcia, A. Reliable Bad Data Processing for Real-Time State Estimation. IEEE Trans. Power Appar. Syst.
**1983**, PAS-102, 1126–1139. [Google Scholar] [CrossRef] - Manandhar, K.; Cao, X.; Hu, F.; Liu, Y. Detection of Faults and Attacks Including False Data Injection Attack in Smart Grid Using Kalman Filter. IEEE Trans. Control Netw. Syst.
**2014**, 1, 370–379. [Google Scholar] [CrossRef] - Liu, Y.; Ning, P.; Reiter, M.K. False data injection attacks against state estimation in electric power grids. ACM Trans. Inf. Syst. Secur.
**2011**, 14, 1–33. [Google Scholar] [CrossRef] - Rahman, M.A.; Mohsenian-Rad, H. False Data Injection Attacks against Nonlinear State Estimation in Smart Power Grids. In Proceedings of the 2013 IEEE Power & Energy Society General Meeting, Vancouver, BC, Canada, 21–25 July 2013; pp. 1–5. [Google Scholar]
- Liang, G.; Zhao, J.; Luo, F.; Weller, S.R.; Dong, Z.Y. A Review of False Data Injection Attacks Against Modern Power Systems. IEEE Trans. Smart Grid
**2017**, 8, 1630–1638. [Google Scholar] [CrossRef] - Ayad, A.; Farag, H.E.Z.; Youssef, A.; El-Saadany, E.F. Detection of False Data Injection Attacks in Smart Grids Using Recurrent Neural Networks. In Proceedings of the 2018 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA, 19–22 February 2018; pp. 1–5. [Google Scholar]
- Musleh, A.S.; Chen, G.; Dong, Z.Y. A Survey on the Detection Algorithms for False Data Injection Attacks in Smart Grids. IEEE Trans. Smart Grid
**2020**, 11, 2218–2234. [Google Scholar] [CrossRef] - Kallitsis, M.G.; Bhattacharya, S.; Stoev, S.; Michailidis, G. Adaptive Statistical Detection of False Data Injection Attacks in Smart Grids. In Proceedings of the 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Washington, DC, USA, 7–9 December 2016; pp. 826–830. [Google Scholar]
- Khalid, H.M.; Peng, J.C.-H. Immunity Toward Data-Injection Attacks Using Multisensor Track Fusion-Based Model Prediction. IEEE Trans. Smart Grid
**2017**, 8, 697–707. [Google Scholar] [CrossRef] - Xu, R.; Wang, R.; Guan, Z.; Wu, L.; Wu, J.; Du, X. Achieving Efficient Detection Against False Data Injection Attacks in Smart Grid. IEEE Access
**2017**, 5, 13787–13798. [Google Scholar] [CrossRef] - Sreenath, J.G.; Meghwani, A.; Chakrabarti, S.; Rajawat, K.; Srivastava, S.C. A recursive state estimation approach to mitigate false data injection attacks in power systems. In Proceedings of the 2017 IEEE Power & Energy Society General Meeting, Chicago, IL, USA, 16–20 July 2017; pp. 1–5. [Google Scholar] [CrossRef]
- Duan, J.; Zeng, W.; Chow, M.-Y. Resilient Distributed DC Optimal Power Flow Against Data Integrity Attack. IEEE Trans. Smart Grid
**2018**, 9, 3543–3552. [Google Scholar] [CrossRef] - Musleh, A.S.; Khalid, H.M.; Muyeen, S.M.; Al-Durra, A. A Prediction Algorithm to Enhance Grid Resilience Toward Cyber Attacks in WAMCS Applications. IEEE Syst. J.
**2019**, 13, 710–719. [Google Scholar] [CrossRef] - Chung, H.-M.; Li, W.-T.; Yuen, C.; Chung, W.-H.; Wen, C.-K. Local Cyber-Physical Attack with Leveraging Detection in Smart Grid. In Proceedings of the 2017 IEEE International Conference on Smart Grid Communications (SmartGridComm), Dresden, Germany, 23–27 October 2017; pp. 461–466. [Google Scholar]
- Khalaf, M.; Youssef, A.; El-Saadany, E. Detection of False Data Injection in Automatic Generation Control Systems Using Kalman Filter. In Proceedings of the 2017 IEEE Electrical Power and Energy Conference (EPEC), Saskatoon, SK, Canada, 22–25 October 2017; pp. 1–6. [Google Scholar]
- Karimipour, H.; Dinavahi, V. On False Data Injection Attack against Dynamic State Estimation on Smart Power Grids. In Proceedings of the 2017 IEEE International Conference on Smart Energy Grid Engineering (SEGE), Oshawa, ON, Canada, 14–17 August 2017; pp. 388–393. [Google Scholar]
- Karimipour, H.; Dinavahi, V. Robust Massively Parallel Dynamic State Estimation of Power Systems Against Cyber-Attack. IEEE Access
**2018**, 6, 2984–2995. [Google Scholar] [CrossRef] - Shi, W.; Wang, Y.; Jin, Q.; Ma, J. PDL: An Efficient Prediction-Based False Data Injection Attack Detection and Location in Smart Grid. In Proceedings of the 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Tokyo, Japan, 23–27 July 2018; pp. 676–681. [Google Scholar]
- Liu, T.; Sun, Y.; Liu, Y.; Gui, Y.; Zhao, Y.; Wang, D.; Shen, C. Abnormal traffic-indexed state estimation: A cyber–physical fusion approach for Smart Grid attack detection. Futur. Gener. Comput. Syst.
**2015**, 49, 94–103. [Google Scholar] [CrossRef] - Chen, P.-Y.; Yang, S.; McCann, J.A.; Lin, J.; Yang, X. Detection of false data injection attacks in smart-grid systems. IEEE Commun. Mag.
**2015**, 53, 206–213. [Google Scholar] [CrossRef] [Green Version] - Jiang, Y.; Hui, Q. Kalman Filter with Diffusion Strategies for Detecting Power Grid False Data Injection Attacks. In Proceedings of the 2017 IEEE International Conference on Electro Information Technology (EIT), Lincoln, NE, USA, 14–17 May 2017; pp. 254–259. [Google Scholar]
- Zhao, J.; Zhang, G.; La Scala, M.; Dong, Z.Y.; Chen, C.; Wang, J. Short-Term State Forecasting-Aided Method for Detection of Smart Grid General False Data Injection Attacks. IEEE Trans. Smart Grid
**2017**, 8, 1580–1590. [Google Scholar] [CrossRef] - Yip, S.-C.; Wong, K.; Hew, W.-P.; Gan, M.-T.; Phan, R.C.-W.; Tan, S.-W. Detection of energy theft and defective smart meters in smart grids using linear regression. Int. J. Electr. Power Energy Syst.
**2017**, 91, 230–240. [Google Scholar] [CrossRef] - Ferragut, E.M.; Laska, J.; Olama, M.M.; Ozmen, O. Real-Time Cyber-Physical False Data Attack Detection in Smart Grids Using Neural Networks. In Proceedings of the 2017 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 14–16 December 2017; pp. 1–6. [Google Scholar]
- Esmalifalak, M.; Liu, L.; Nguyen, N.; Zheng, R.; Han, Z. Detecting Stealthy False Data Injection Using Machine Learning in Smart Grid. IEEE Syst. J.
**2017**, 11, 1644–1652. [Google Scholar] [CrossRef] - Wang, Y.; Amin, M.M.; Fu, J.; Moussa, H.B. A Novel Data Analytical Approach for False Data Injection Cyber-Physical Attack Mitigation in Smart Grids. IEEE Access
**2017**, 5, 26022–26033. [Google Scholar] [CrossRef] - Binna, S.; Kuppannagari, S.R.; Engel, D.; Prasanna, V.K. Subset Level Detection of False Data Injection Attacks in Smart Grids. In Proceedings of the 2018 IEEE Conference on Technologies for Sustainability (SusTech), Long Beach, CA, USA, 11–13 November 2018; pp. 1–7. [Google Scholar]
- El Hariri, M.; Youssef, T.A.; Habib, H.F.; Mohammed, O. Online false data detection and lost packet forecasting system using time series neural networks for IEC 61850 sampled measured values. In Proceedings of the 2017 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA, 23–26 April 2017; pp. 1–5. [Google Scholar] [CrossRef]
- Wang, D.; Wang, X.; Zhang, Y.; Jin, L. Detection of power grid disturbances and cyber-attacks based on machine learning. J. Inf. Secur. Appl.
**2019**, 46, 42–52. [Google Scholar] [CrossRef] - Yan, J.; Tang, B.; He, H. Detection of False Data Attacks in Smart Grid with Supervised Learning. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 1395–1402. [Google Scholar]
- McLaughlin, S.; Holbert, B.; Fawaz, A.; Berthier, R.; Zonouz, S. A Multi-Sensor Energy Theft Detection Framework for Advanced Metering Infrastructures. IEEE J. Sel. Areas Commun.
**2013**, 31, 1319–1330. [Google Scholar] [CrossRef] - Viegas, J.L.; Vieira, S.M. Clustering-Based Novelty Detection to Uncover Electricity Theft. In Proceedings of the 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Naples, Italy, 9–12 July 2017; pp. 1–6. [Google Scholar]
- Zanetti, M.; Jamhour, E.; Pellenz, M.; Penna, M.; Zambenedetti, V.; Chueiri, I. A Tunable Fraud Detection System for Advanced Metering Infrastructure Using Short-Lived Patterns. IEEE Trans. Smart Grid
**2019**, 10, 830–840. [Google Scholar] [CrossRef] - Camacho, E.F.; Bordons, C. Model Predictive Control: Advanced Textbooks in Control and Signal Processing, 2nd ed; Springer: London, UK; New York, NY, USA, 2004; ISBN 978-0-85729-398-5. [Google Scholar]
- Abdel-Basset, M.; Abdel-Fatah, L.; Sangaiah, A.K. Metaheuristic Algorithms: A Comprehensive Review. In Computational Intelligence for Multimedia Big Data on the Cloud with Engineering Applications; Elsevier: Amsterdam, The Netherlands, 2018; pp. 185–231. ISBN 978-0-12-813314-9. [Google Scholar]
- Pereira, J.L.J.; Oliver, G.A.; Francisco, M.B.; Cunha, S.S.; Gomes, G.F. A Review of Multi-objective Optimization: Methods and Algorithms in Mechanical Engineering Problems. Arch. Comput. Methods Eng.
**2022**, 29, 2285–2308. [Google Scholar] [CrossRef] - Klanšek, U. A comparison between MILP and MINLP approaches to optimal solution of nonlinear discrete transportation problem. Transport
**2014**, 30, 135–144. [Google Scholar] [CrossRef] [Green Version] - Liu, J.; Tang, J.; Ponci, F.; Monti, A.; Muscas, C.; Pegoraro, P.A. Trade-Offs in PMU Deployment for State Estimation in Active Distribution Grids. IEEE Trans. Smart Grid
**2012**, 3, 915–924. [Google Scholar] [CrossRef] - Raposo, A.A.; Rodrigues, A.B.; da Silva, M.D.G. Optimal meter placement algorithm for state estimation in power distribution networks. Electr. Power Syst. Res.
**2017**, 147, 22–30. [Google Scholar] [CrossRef] - Ramachandran, B.; Bellarmine, G.T. Improving observability using optimal placement of phasor measurement units. Int. J. Electr. Power Energy Syst.
**2014**, 56, 55–63. [Google Scholar] [CrossRef] - Peng, J.; Sun, Y.; Wang, H. Optimal PMU placement for full network observability using Tabu search algorithm. Int. J. Electr. Power Energy Syst.
**2006**, 28, 223–231. [Google Scholar] [CrossRef] - Xygkis, T.C.; Korres, G.N.; Manousakis, N.M. Fisher Information-Based Meter Placement in Distribution Grids via the D-Optimal Experimental Design. IEEE Trans. Smart Grid
**2018**, 9, 1452–1461. [Google Scholar] [CrossRef] - El-Hawary, M.E. The Smart Grid—State-of-the-art and Future Trends. Electr. Power Compon. Syst.
**2014**, 42, 239–250. [Google Scholar] [CrossRef] - Croteau, D.; Carre, O. Distribution state estimation: Outcomes from a field implementation aimed at tackling MV mastering in the presence of distributed energy resources (DER). CIRED–Open Access Proc. J.
**2017**, 2017, 1715–1717. [Google Scholar] [CrossRef] - Al-Wakeel, A.; Wu, J.; Jenkins, N. State estimation of medium voltage distribution networks using smart meter measurements. Appl. Energy
**2016**, 184, 207–218. [Google Scholar] [CrossRef] [Green Version] - Pegoraro, P.A.; Tang, J.; Liu, J.; Ponci, F.; Monti, A.; Muscas, C. PMU and Smart Metering Deployment for State Estimation in Active Distribution Grids. In Proceedings of the 2012 IEEE International Energy Conference and Exhibition (ENERGYCON), Florence, Italy, 9–12 September 2012; pp. 873–878. [Google Scholar]
- Heydt, G.T. The Next Generation of Power Distribution Systems. IEEE Trans. Smart Grid
**2010**, 1, 225–235. [Google Scholar] [CrossRef] - Albu, M.; Heydt, G.T.; Cosmescu, S.-C. Versatile platforms for wide area synchronous measurements in power distribution systems. In Proceedings of the North American Power Symposium 2010, Arlington, TX, USA, 26–28 September 2010. [Google Scholar] [CrossRef]
- Deka, D.; Backhaus, S.; Chertkov, M. Structure Learning in Power Distribution Networks. IEEE Trans. Control. Netw. Syst.
**2015**, 5, 1061–1074. [Google Scholar] [CrossRef] - Peppanen, J.; Reno, M.J.; Broderick, R.J.; Grijalva, S. Distribution System Model Calibration With Big Data From AMI and PV Inverters. IEEE Trans. Smart Grid
**2016**, 7, 2497–2506. [Google Scholar] [CrossRef] - Weng, Y.; Liao, Y.; Rajagopal, R. Distributed Energy Resources Topology Identification via Graphical Modeling. IEEE Trans. Power Syst.
**2017**, 32, 2682–2694. [Google Scholar] [CrossRef] - Carta, D.; Benigni, A.; Muscas, C. Model Order Reduction for PMU-Based State Estimation in Distribution Grids. IEEE Syst. J.
**2018**, 12, 2711–2720. [Google Scholar] [CrossRef] - SCADA Communication and Protocols. Available online: https://instrumentationtools.com/scada-communication-and-protocols/ (accessed on 26 September 2022).
- Tomsovic, K.; Bakken, D.; Venkatasubramanian, V.; Bose, A. Designing the Next Generation of Real-Time Control, Communication, and Computations for Large Power Systems. Proc. IEEE
**2005**, 93, 965–979. [Google Scholar] [CrossRef] [Green Version] - Hippert, H.S.; Pedreira, C.E.; Souza, R.C. Neural networks for short-term load forecasting: A review and evaluation. IEEE Trans. Power Syst.
**2001**, 16, 44–55. [Google Scholar] [CrossRef] - Fotopoulou, M.; Rakopoulos, D.; Blanas, O. Day Ahead Optimal Dispatch Schedule in a Smart Grid Containing Distributed Energy Resources and Electric Vehicles. Sensors
**2021**, 21, 7295. [Google Scholar] [CrossRef] - Petridis, S.; Blanas, O.; Rakopoulos, D.; Stergiopoulos, F.; Nikolopoulos, N.; Voutetakis, S. An Efficient Backward/Forward Sweep Algorithm for Power Flow Analysis through a Novel Tree-Like Structure for Unbalanced Distribution Networks. Energies
**2021**, 14, 897. [Google Scholar] [CrossRef] - Bandara, W.G.C.; Almeida, D.; Godaliyadda, R.I.; Ekanayake, M.P.; Ekanayake, J. A complete state estimation algorithm for a three-phase four-wire low voltage distribution system with high penetration of solar PV. Int. J. Electr. Power Energy Syst.
**2021**, 124, 106332. [Google Scholar] [CrossRef] - Song, S.; Xiong, H.; Lin, Y.; Huang, M.; Wei, Z.; Fang, Z. Robust three-phase state estimation for PV-Integrated unbalanced distribution systems. Appl. Energy
**2022**, 322, 119427. [Google Scholar] [CrossRef] - Menke, J.-H.; Bornhorst, N.; Braun, M. Distribution system monitoring for smart power grids with distributed generation using artificial neural networks. Int. J. Electr. Power Energy Syst.
**2019**, 113, 472–480. [Google Scholar] [CrossRef] [Green Version] - Huang, M.; Zhao, J.; Wei, Z.; Pau, M.; Sun, G. Decentralized robust state estimation for hybrid AC/DC distribution systems with smart meters. Int. J. Electr. Power Energy Syst.
**2022**, 136, 107656. [Google Scholar] [CrossRef] - Teixeira, H.; Pereira Barbeiro, P.N.; Pereira, J.; Bessa, R.; Matos, P.G.; Lemos, D.; Morais, A.C.; Caujolle, M.; Sebastian-Viana, M. A State Estimator for LV Networks: Results from the EvolvDSO Project. In Proceedings of the CIRED Workshop 2016, Helsinki, Finland, 14–15 June 2016; p. 136. [Google Scholar]
- Dragovic, J.; Pudjianto, D.; Djiapic, P.; Bilton, M.; Strbac, G. Application of Distribution System State Estimation on Engineering Instrumentation Zones of Low Carbon London. In Proceedings of the 23rd International Conference on Electricity Distribution (CIRED 2015), Lyon, France, 15–18 June 2015. [Google Scholar]
- Eggenschwiler, L.; Favre-Perrod, P.; Nauts, O.; Rampazzo, J.; Mousavi, O. Performance evaluation of distribution system state estimator using different measurement devices. In Proceedings of the CIRED 2019, Madrid, Spain, 3–6 June 2019. [Google Scholar]
- Sustainable Project. Advanced Local Distribution Grid Monitoring/State-Estimation; Sustainable Project: Porto, Portugal; Available online: https://repositorio-aberto.up.pt/bitstream/10216/83284/2/44855.pdf (accessed on 20 September 2022).
- Pokhrel, B.R.; Bak-Jensen, B.; Pillai, J.R. Integrated Approach for Network Observability and State Estimation in Active Distribution Grid. Energies
**2019**, 12, 2230. [Google Scholar] [CrossRef] [Green Version] - Waeresch, D.; Brandalik, R.; Wellssow, W.H.; Jordan, J.; Bischler, R.; Schneider, N. Field test of a linear three-phase low-voltage state estimation system based on smart meter data. CIRED–Open Access Proc. J.
**2017**, 2017, 1773–1776. [Google Scholar] [CrossRef] [Green Version] - Barbeiro, P.; Pereira, J.; Teixeira, H.; Seca, L.; Silva, P.; Silva, N.; Melo, F. LV SCADA Project: In-Field Validation of a Distribution State Estimation Tool for LV Networks. In Proceedings of the CIRED Workshop 2016, Helsinki, Finland, 14–15 June 2016; p. 134. [Google Scholar]
- Petridis, S.; Blanas, O.; Rakopoulos, D.; Stergiopoulos, F.; Nikolopoulos, N.; Voutetakis, S.; Andritsakis, E.; Tagkoulis, N. LV Network Topology and Parameter Estimation with High Data Availability; Centre for Research and Technology Hellas (CERTH): Thessaloniki, Greece, 2020. [Google Scholar]
- Petridis, S.; Bintoudi, A.D.; Veliskaki, A.; Karapatsias, V.; Fotopoulou, M.; Rakopoulos, D.; Ioannidis, D.; Tzovaras, D. State Estimation in Unbalanced Smart Grids. In Proceedings of the 2022 International Conference on Smart Energy Systems and Technologies (SEST), Eindhoven, The Netherlands, 5–7 September 2022; pp. 1–6. [Google Scholar]
- Manousakis, N.M.; Korres, G.N. Application of State Estimation in Distribution Systems with Embedded Microgrids. Energies
**2021**, 14, 7933. [Google Scholar] [CrossRef] - Rui, W.; Qiuye, S.; Pinjia, Z.; Yonghao, G.; Dehao, Q.; Peng, W. Reduced-Order Transfer Function Model of the Droop-Controlled Inverter via Jordan Continued-Fraction Expansion. IEEE Trans. Energy Convers.
**2020**, 35, 1585–1595. [Google Scholar] [CrossRef] - Wang, R.; Sun, Q.; Qin, D.; Li, Y.; Li, X.; Wang, P. Steady-state Stability Assessment of AC-busbar Plug-in Electric Vehicle Charging Station with Photovoltaic. J. Mod. Power Syst. Clean Energy
**2020**, 8, 884–894. [Google Scholar] [CrossRef]

Method | Description | Advantages and Disadvantages |
---|---|---|

WLS [31,32,33,34,35,36] | Minimization of the weighted residuals between the estimated and measured values. Objective function (11). | Common, simple, fast model-based approach, but sensitive to data. |

LAV [37,38] | Minimization of the sum of absolute values of measurement residuals. Objective function (12). | Model-based approaches. Mostly variations of WLS, trying to overcome the issue of sensitivity to bad data and model parameters. Enhanced robustness but also limited implementation in literature and higher resources requirements such as computational cost, memory usage, etc. Information regarding the architecture and parameters of the distribution system is a necessity. |

WLAV [39] | Variation of LAV including weights. Objective function (13). | |

LTS [40] | Utilization of selected number of residuals in the objective function (14). | |

LMS [41] | Use of the median in the objective function (15). | |

GM [42,43] | Generalized maximum-likelihood approach using the Huber cost function in (16). | |

Kalman filter [46,47,48] | FASE, providing recursive updates of the estimated states with the use of Kalman filter and dynamic models such as (17). | Changes in normal operation can be tracked. Further, since FASE is based on forecasts by nature, missing measurements can be addressed with the use of the forecasted states. However, this sort of algorithms is more complex and not as widely used as WLS. |

EKF [49,50,51,52] | FASE, extended variation of Kalman filter with the use of Taylor series for linearization of nonlinear systems. | Widely used variation of Kalman filter for linearization of nonlinear systems. Yet, not as widely used as WLS. |

UKF [52,53] | FASE, variation of Kalman filter with the use of UT for linearization of nonlinear systems. | Variation of Kalman filter for linearization of nonlinear systems where no explicit calculation of Jacobian or Hessian matrices is necessary. Limited use of this approach in literature. |

ANN [61,62] | Purely data-driven DSSE approach including three layers, trained with data provided by the DSO. | Modern, data-driven approach, where the sensitivity/accuracy regarding model parameters, initial conditions, etc. does not affect the result. However, an adequate database is required. |

DNN [63] | Data-driven approach, sub-category of ANN, with more hidden layers. | |

Physics-aware neural network [64,65,66] | Data-driven approach where the neural network takes into account the structure of the distribution network. | Advanced, data-driven approach where the physics of the network are not overlooked, thus prohibiting the overfitting behavior of the algorithm. Yet, in this case too, an adequate database is required. |

Hybrid [67,68] | Various possibilities, e.g., [67]: DNN-based DSSE, supported by random forest algorithm for topology identification. WLAV is used to ensure the robustness of the estimation. | Combination of the advantages of various sorts of algorithms at the cost of having a less simple methodology. |

Method | Description | Advantages and Disadvantages |
---|---|---|

Rule-based [21] | Set of rules for meter placement with low cost. | Easy and fast algorithms, but not optimal. |

Metaheuristic [114,115,116,117] | GA [114], PSO [115], fruit fly [116], TS [117], etc. for: (i) maximization of accuracy or, (ii) minimization of units while sustaining observability, etc. | Efficient bio-inspired algorithms which do not guarantee global optimal solutions. |

Optimization [19,25,118] | MISDP, etc., usually for the maximization of accuracy. | Optimal, advanced algorithms, more demanding than the others. |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Fotopoulou, M.; Petridis, S.; Karachalios, I.; Rakopoulos, D.
A Review on Distribution System State Estimation Algorithms. *Appl. Sci.* **2022**, *12*, 11073.
https://doi.org/10.3390/app122111073

**AMA Style**

Fotopoulou M, Petridis S, Karachalios I, Rakopoulos D.
A Review on Distribution System State Estimation Algorithms. *Applied Sciences*. 2022; 12(21):11073.
https://doi.org/10.3390/app122111073

**Chicago/Turabian Style**

Fotopoulou, Maria, Stefanos Petridis, Ioannis Karachalios, and Dimitrios Rakopoulos.
2022. "A Review on Distribution System State Estimation Algorithms" *Applied Sciences* 12, no. 21: 11073.
https://doi.org/10.3390/app122111073