# Comparison of the Stability and Accuracy of Deterministic Project Cost Prediction Methods in Earned Value Management

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Literature Review

#### 2.1. Limitations of Previous Research on the Stability of EAC Methods

#### 2.2. Limitations of Previous Research on the Accuracy of EAC Methods

## 3. Research Methods

#### 3.1. Database

#### 3.2. Measurement of the Stability of EAC Methods

_{mk}is the difference between the order classification values of EAC

_{AT}and RC

_{mk}for project m, and the K = 100 simulation runs in a given tracking period AT, that is ${d}_{mk}\equiv rank\left(\right(\mathrm{EAC}{)}_{AT}^{k})-rank\left({\mathrm{RC}}_{mk}\right)$ for k = 1,2... K.

^{2}. In this study, K

^{2}= 10,000, which already makes the calculations extremely time consuming for the 4100 projects, 30 EAC methods, and 10 tracking periods. For this reason, the number of simulations for each of the 4100 projects was limited to K = 100. The Kendall coefficient formula τ in each AT tracking period is:

_{m}represents the number of concordant pairs between the EAC estimates and the RC values for each project m, that is:

#### 3.3. Measuring the Accuracy of the EAC Methods

_{mk}). Specifically, the estimated cost at completion at each progress interval AT of the project is called the EAC

_{AT}and is calculated for each of the 30 prediction methods compared. RC

_{mk}is equivalent to the real cost of the artificially generated project as the sum of the costs of the project activities for project m (m = 1, 2… M projects) and simulation k (k = 1, 2… K iterations for each of the m projects). As indicated above, in our research, we assumed M = 4100 projects, K = 100 iterations, and AT = 10%, 20%...100% of the real duration (RD

_{mk}) of the project. The formulation of the MSE is as follows:

## 4. Results

#### 4.1. Stability of the EAC Methods

#### 4.2. Accuracy of the EAC Methods

#### 4.3. Summary of Results

#### 4.4. Consideration of the Project Network Topology

## 5. Discussion

- The accuracy is high when measured from the middle of the project (AT ≥ 50% RD) if the deviations are measured with squared errors (MSE);
- The accuracy is high when measured from early stages (AT ≥ 30% RD) if the deviations are measured with percentage errors (MPE); or
- The accuracy is high when measured late into the project (AT ≥ 80% RD) if the deviations are measured with absolute percentage errors (MAPE).

## 6. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Pérez, P.B.; González-Cruz, M.; Pastor-Ferrando, J.P. Analysis of Construction Projects by Means of Value Curves. Int. J. Proj. Manag.
**2010**, 28, 719–731. [Google Scholar] [CrossRef] - Martens, A.; Vanhoucke, M. Integrating Corrective Actions in Project Time Forecasting Using Exponential Smoothing. J. Manag. Eng.
**2020**, 36, 04020044. [Google Scholar] [CrossRef] - Wauters, M.; Vanhoucke, M. Study of the Stability of Earned Value Management Forecasting. J. Constr. Eng. Manag.
**2015**, 141, 04014086. [Google Scholar] [CrossRef] - Anysz, H.; Buczkowski, B. The Association Analysis for Risk Evaluation of Significant Delay Occurrence in the Completion Date of Construction Project. Int. J. Environ. Sci. Technol.
**2019**, 16, 5369–5374. [Google Scholar] [CrossRef] - Jie, D.; Wei, J. Estimating Construction Project Duration and Costs upon Completion Using Monte Carlo Simulations and Improved Earned Value Management. Buildings
**2022**, 12, 2173. [Google Scholar] [CrossRef] - Buildings and Constructed Assets—Service Life Planning—Part 5: Life-Cycle Costing. ISO 15686-5:2017; ISO: Geneva, Switzerland, 2017.
- Cerezo-Narváez, A.; Pastor-Fernández, A.; Otero-Mateo, M.; Ballesteros-Pérez, P. Integration of Cost and Work Breakdown Structures in the Management of Construction Projects. Appl. Sci.
**2020**, 10, 1386. [Google Scholar] [CrossRef] - Zhang, J.; Lu, G.; Skitmore, M.; Ballesteros-Pérez, P. A Critical Review of the Current Research Mainstreams and the Influencing Factors of Green Total Factor Productivity. Environ. Sci. Pollut. Res.
**2021**, 28, 35392–35405. [Google Scholar] [CrossRef] [PubMed] - Ballesteros-Pérez, P.; González-Cruz, M.C.; Cañavate-Grimal, A. Mathematical Relationships between Scoring Parameters in Capped Tendering. Int. J. Proj. Manag.
**2012**, 30, 850–862. [Google Scholar] [CrossRef] - Browning, T.R. A Quantitative Framework for Managing Project Value, Risk, and Opportunity. IEEE Trans. Eng. Manag.
**2014**, 61, 583–598. [Google Scholar] [CrossRef] - Konior, J. Determining Cost and Time Performance Indexes for Diversified Investment Tasks. Buildings
**2022**, 12, 1198. [Google Scholar] [CrossRef] - Kim, B.-C. Probabilistic Evaluation of Cost Performance Stability in Earned Value Management. J. Manag. Eng.
**2016**, 32, 04015025. [Google Scholar] [CrossRef] - Mortaji, S.T.H.; Noorossana, R.; Bagherpour, M. Project Completion Time and Cost Prediction Using Change Point Analysis. J. Manag. Eng.
**2015**, 31, 04014086. [Google Scholar] [CrossRef] - Barrientos-Orellana, A.; Ballesteros-Pérez, P.; Mora-Melià, D.; González-Cruz, M.C.; Vanhoucke, M. Stability and Accuracy of Deterministic Project Duration Forecasting Methods in Earned Value Management. Eng. Constr. Archit. Manag.
**2022**, 29, 1449–1469. [Google Scholar] [CrossRef] - Farris, J.A.; Groesbeck, R.L.; Van Aken, E.M.; Letens, G. Evaluating the Relative Performance of Engineering Design Projects: A Case Study Using Data Envelopment Analysis. IEEE Trans. Eng. Manag.
**2006**, 53, 471–482. [Google Scholar] [CrossRef] - Babar, S.; Thaheem, M.J.; Ayub, B. Estimated Cost at Completion: Integrating Risk into Earned Value Management. J. Constr. Eng. Manag.
**2017**, 143, 04016104. [Google Scholar] [CrossRef] - Mahmoudi, A.; Bagherpour, M.; Javed, S.A. Grey Earned Value Management: Theory and Applications. IEEE Trans. Eng. Manag.
**2021**, 68, 1703–1721. [Google Scholar] [CrossRef] - Ayman, H.M.; Mahfouz, S.Y.; Alhady, A. Integrated EDM and 4D BIM-Based Decision Support System for Construction Projects Control. Buildings
**2022**, 12, 315. [Google Scholar] [CrossRef] - Wauters, M.; Vanhoucke, M. A Nearest Neighbour Extension to Project Duration Forecasting with Artificial Intelligence. Eur. J. Oper. Res.
**2017**, 259, 1097–1111. [Google Scholar] [CrossRef] - Proaño-Narváez, M.; Flores-Vázquez, C.; Vásquez Quiroz, P.; Avila-Calle, M. Earned Value Method (EVM) for Construction Projects: Current Application and Future Projections. Buildings
**2022**, 12, 301. [Google Scholar] [CrossRef] - de Andrade, P.A.; Martens, A.; Vanhoucke, M. Using Real Project Schedule Data to Compare Earned Schedule and Earned Duration Management Project Time Forecasting Capabilities. Autom. Constr.
**2019**, 99, 68–78. [Google Scholar] [CrossRef] - Chang, C.-J.; Yu, S.-W. Three-Variance Approach for Updating Earned Value Management. J. Constr. Eng. Manag.
**2018**, 144, 04018045. [Google Scholar] [CrossRef] - Kim, B.C. Integrating Risk Assessment and Actual Performance for Probabilistic Project Cost Forecasting: A Second Moment Bayesian Model. IEEE Trans. Eng. Manag.
**2015**, 62, 158–170. [Google Scholar] [CrossRef] - Christensen, D.S.; Antolini, R.C.; McKinney, J.W. A Review of Estimate at Completion Research. In Cost Estimating and Analysis; Springer: New York, NY, USA, 1992; pp. 207–224. [Google Scholar]
- Anbari, F. Earned Value Project Management Method and Extensions. IEEE Eng. Manag. Rev.
**2003**, 32, 97. [Google Scholar] [CrossRef] - Lipke, W. Schedule Is Different. Meas. News
**2003**, 2, 31–34. [Google Scholar] - Jacob, D.S.; Kane, M. Forecasting Schedule Completion Using Earned Value Metrics. Meas. News
**2004**, 2004, 11–17. [Google Scholar] - Lipke, W. Schedule Adherence and Rework. CrossTalk
**2012**, 25, 4–8. [Google Scholar] - Batselier, J.; Vanhoucke, M. Improving Project Forecast Accuracy by Integrating Earned Value Management with Exponential Smoothing and Reference Class Forecasting. Int. J. Proj. Manag.
**2017**, 35, 28–43. [Google Scholar] [CrossRef] - Ballesteros-Pérez, P.; Sanz-Ablanedo, E.; Mora-Melià, D.; González-Cruz, M.C.; Fuentes-Bargues, J.L.; Pellicer, E. Earned Schedule Min-Max: Two New EVM Metrics for Monitoring and Controlling Projects. Autom. Constr.
**2019**, 103, 279–290. [Google Scholar] [CrossRef] - Kim, B.-C.; Pinto, J.K. What CPI = 0.85 Really Means: A Probabilistic Extension of the Estimate at Completion. J. Manag. Eng.
**2019**, 35, 04018059. [Google Scholar] [CrossRef] - Henderson, K.; Zwikael, O. Does Project Performance Stability Exist? A Re-Examination of CPI and Evaluation of SPI(t) Stability. CrossTalk
**2008**, 21, 7–13. [Google Scholar] - Petter, J.L.; Ritschel, J.D.; White, E.D. Stability Properties in Department of Defense Contracts: Answering the Controversy. J. Public Procure.
**2015**, 15, 341–364. [Google Scholar] [CrossRef] - Vanhoucke, M.; Coelho, J.; Debels, D.; Maenhout, B.; Tavares, L.V. An Evaluation of the Adequacy of Project Network Generators with Systematically Sampled Networks. Eur. J. Oper. Res.
**2008**, 187, 511–524. [Google Scholar] [CrossRef] - Soltan, S.; Ashrafi, M. Predicting Project Duration and Cost, and Selecting the Best Action Plan Using Statistical Methods for Earned Value Management. J. Proj. Manag.
**2020**, 5, 157–166. [Google Scholar] [CrossRef] - Vanhoucke, M. Project Management with Dynamic Scheduling: Baseline Scheduling, Risk Analysis and Project Control; Springer: Berlin/Heidelberg, Germany, 2012; ISBN 9783642404382. [Google Scholar]
- Christensen, D.S. Using the Earned Value Cost Management Report to Evaluate the Contractor’s Estimate at Completion. Acquis. Rev. Q.
**1999**, 6, 283–296. [Google Scholar] - Khamooshi, H.; Abdi, A. Project Duration Forecasting Using Earned Duration Management with Exponential Smoothing Techniques. J. Manag. Eng.
**2017**, 33, 04016032. [Google Scholar] [CrossRef] - Payne, K. An Investigation of the Stability of the Cost Performance Index; Air Force Institute of Technology: Montgomery, OH, USA, 1990. [Google Scholar]
- Christensen, D.S.; Heise, S.R. Cost Performance Index Stability. Natl. Contract Manag. J.
**1992**, 25, 7–15. [Google Scholar] - Zwikael, O.; Globerson, S.; Raz, T. Evaluation of Models for Forecasting the Final Cost of a Project. Proj. Manag. J.
**2000**, 31, 53–57. [Google Scholar] [CrossRef] - Christensen, D.; Templin, C. EAC Evaluation Methods: Do They Still Work? Acquis. Rev. Q.
**2002**, 9, 105–117. [Google Scholar] - Batselier, J.; Vanhoucke, M. Empirical Evaluation of Earned Value Management Forecasting Accuracy for Time and Cost. J. Constr. Eng. Manag.
**2015**, 141, 05015010. [Google Scholar] [CrossRef] - De Koning, P.; Vanhoucke, M. Stability of Earned Value Management Do Project Characteristics Influence the Stability Moment of the Cost and Schedule Performance Index. J. Mod. Proj. Manag.
**2016**, 4, 9–25. [Google Scholar] - Khafri, A.A.; Dawson-Edwards, J.; Simpson, R.; Abourizk, S. Empirical Study of Relationship between Cost Performance Index Stability and Project Cost Forecast Accuracy in Industrial Construction Projects. In Part of the Canadian Society for Civil Engineering Annual Conference 2018, Proceedings of the CSCE General Conference 2018, 13–16 June 2018, Fredericton, Canada; Canadian Society for Civil Engineering: Ontario, ON, Canada, 2018; pp. 269–276. [Google Scholar]
- Kim, D.B.; White, E.D.; Ritschel, J.D.; Millette, C.A. Revisiting Reliability of Estimates at Completion for Department of Defense Contracts. J. Public Procure.
**2019**, 19, 186–200. [Google Scholar] [CrossRef] - Demeulemeester, E.; Vanhoucke, M.; Herroelen, W. RanGen: A Random Network Generator for Activity-on-the-Node Networks. J. Sched.
**2003**, 6, 17–38. [Google Scholar] [CrossRef] - Vandevoorde, S.; Vanhoucke, M. A Comparison of Different Project Duration Forecasting Methods Using Earned Value Metrics. Int. J. Proj. Manag.
**2006**, 24, 289–302. [Google Scholar] [CrossRef] - Colin, J.; Vanhoucke, M. Setting Tolerance Limits for Statistical Project Control Using Earned Value Management. Omega
**2014**, 49, 107–122. [Google Scholar] [CrossRef] - Trietsch, D.; Mazmanyan, L.; Gevorgyan, L.; Baker, K.R. Modeling Activity Times by the Parkinson Distribution with a Lognormal Core: Theory and Validation. Eur. J. Oper. Res.
**2012**, 216, 386–396. [Google Scholar] [CrossRef] - Ballesteros-Pérez, P.; Sanz-Ablanedo, E.; Soetanto, R.; González-Cruz, M.C.; Larsen, G.D.; Cerezo-Narváez, A. Duration and Cost Variability of Construction Activities: An Empirical Study. J. Constr. Eng. Manag.
**2020**, 146, 04019093. [Google Scholar] [CrossRef] - Vanhoucke, M.; Vandevoorde, S. A Simulation and Evaluation of Earned Value Metrics to Forecast the Project Duration. J. Oper. Res. Soc.
**2007**, 58, 1361–1374. [Google Scholar] [CrossRef] - Elshaer, R. Impact of Sensitivity Information on the Prediction of Project’s Duration Using Earned Schedule Method. Int. J. Proj. Manag.
**2013**, 31, 579–588. [Google Scholar] [CrossRef]

**Figure 2.**Comparison of the accuracy of the 30 EAC methods as a function of the project progress (% RD).

**Figure 3.**Summary of the results. Some EAC methods are highlighted to compare stability and accuracy. More asterisks (*) indicate better performance. Specifically, ***: 1st quartile (top 1st–8th methods), **: 2nd quartile (methods ranked 9th–15th), *: 3rd quartile (methods ranked 16th–22nd), " ": 4th quartile (methods ranked 23rd–30th).

**Figure 4.**Stability and accuracy results of the top-performing EAC methods as a function of the projects’ series–parallel indicator.

ID | Method | EAC | Mathematical Expression | Description |
---|---|---|---|---|

1 | EVM | PV1 | EAC_{PV1} = AC + (BAC − EV) | Methods 1–5 [37] assume that PF = 1, CPI, SPI, SCI and a weighted average of time and cost. This study has assumed that α = 0.8 and β = 0.2, as for cost prediction it is necessary that the cost performance factor (CPI) significantly outweighs the time performance factor (SPI). |

2 | EVM | PV2 | EAC_{PV2} = AC + (BAC − EV)/CPI | |

3 | EVM | PV3 | EAC_{PV3} = AC + (BAC − EV)/SPI | |

4 | EVM | PV4 | EAC_{PV4} = AC + (BAC − EV)/SCI | |

5 | EVM | PV5 | EAC_{PV5} = AC + (BAC − EV)/(α CPI+β SPI) | |

6 | EVM | ES1 | EAC_{ES1} = AC + (BAC − EV)/SPI(t) | Methods 6–8 [26] accept that PF = SPI(t), SCI(t) and a weighted average of time and cost. Method 8 also assumes that α = 0.8 and β = 0.2 |

7 | EVM | ES2 | EAC_{ES2} = AC + (BAC − EV)/SCI(t) | |

8 | EVM | ES3 | EAC_{ES3} = AC + (BAC − EV)/α CPI+β SPI(t) | |

9 | EVM | SP1 | EAC_{SP1} = BAC/SCI | Method 9 [25] considers the actual performance of the project in terms of progress (duration) and costs (this, as SCI = SPI·CPI) |

10 | ESM | ESM1 | EAC_{ESM1} = AC + (BAC − EV(e)) | Methods 10–19 [28] are the counterparts of Methods 1–8. Methods 20 and 21 assume that PF = T _{t,SPI(t)(e)} and T_{t,EV(e)/}T_{t,AC}, respectively. Both methods use the exponential smoothing technique to determine the PF value. Method 20 [38] considers that γ = 0.25. Additionally, Method 21 [29] assumes that δ = 0.05. |

11 | ESM | ESM2 | EAC_{ESM2} = AC + (BAC − EV(e))/CPI(e) | |

12 | ESM | ESM3 | EAC_{ESM3} = AC + (BAC − EV(e))/SPI(e) | |

13 | ESM | ESM4 | EAC_{ESM4} = AC + (BAC − EV(e))/SCI(e) | |

14 | ESM | ESM5 | EAC_{ESM5} = AC + (BAC − EV(e))/SPI(t)(e) | |

15 | ESM | ESM6 | EAC_{ESM6} = AC + (BAC − EV(e))/SCI(t)(e) | |

16 | ESM | ESM7 | EAC_{ESM7} = AC + (BAC − EV(e))/αCPI+βSPI(e) | |

17 | ESM | ESM8 | EAC_{ESM8} = AC + (BAC − EV(e))/α CPI+β SPI(t)(e) | |

18 | ESM | ESM9 | EAC_{ESM9} = AC + (BAC − EV(e))/αCPI(e)+βSPI(e) | |

19 | ESM | ESM10 | EAC_{ESM10} = AC + (BAC − EV(e))/αCPI(e)+βSPI(t)(e) | |

20 | ESM | ESM11 | EAC_{ESM11} = AC + (BAC − EV(e))/T_{t,SPI(t)(e)} | |

21 | ESM | ESM12 | EAC_{ESM12} = AC + (BAC − EV(e))/T_{t,EV(e)/}T_{t,AC} | |

22 | XSM | XSM1 | EAC_{XSM1} = AC + (BAC − EV)/T_{t,SPI(t)} | Methods 22 and 23 [38] assume that PF = T_{t,SPI(t)} and T_{t,EDI}, respectively. Method 24 [29] considers PF = T_{t,EV}/T_{t,AC}. |

23 | XSM | XSM2 | EAC_{XSM2} = AC + (BAC − EV)/T_{t,EDI} | |

24 | XSM | XSM3 | EAC_{XSM3} = AC + (BAC − EV)/T_{t,EV}/T_{t,AC} | |

25 | ES min | ES1 | EAC_{ESmin ES1} = AC + (BAC − EV)/SPI(t)_{ESmin} | Methods 25–30 [30] are the counterparts of Methods 6–8. Methods 27 and 30 also consider α = 0.8 and β = 0.2. |

26 | ES min | ES2 | EAC_{ESmin ES2} = AC + (BAC − EV)/SCI(t)_{ESmin} | |

27 | ES min | ES3 | EAC_{ESmin ES3} = AC + (BAC − EV)/α CPI+β SPI(t)_{ESmin} | |

28 | ES max | ES1 | EAC_{ESmax ES1} = AC + (BAC − EV)/SPI(t)_{ESmax} | |

29 | ES max | ES2 | EAC_{ESmax ES2} = AC + (BAC − EV)/SCI(t)_{ESmax} | |

30 | ES max | ES3 | EAC_{ESmax ES3} = AC + (BAC − EV)/α CPI+β SPI(t)_{ESmax} |

Study | Accuracy | Stability ^{2} | NO. Projects | EAC methods | ||||
---|---|---|---|---|---|---|---|---|

MAPE | MPE | MSE | Artificial | Real | Compared | Top Performers | ||

Payne [39] | - | - | - | ✓ | - | 26 | 2 | - |

Christensen and Heise [40] | - | - | - | ✓ | - | 400 | 2 | - |

Zwikael et al. [41] | ✓ | - | ✓ | ✓ | - | 12 | 1, 4 and 9 | 9 |

Christensen and Templin [42] | - | - | - | ✓ | - | 240 | 2 | - |

Henderson and Zwikael [32] | - | - | - | ✓ | - | 45 | 2 and 6 | 2 |

Petter et al. [33] | - | - | - | ✓ | - | 209 | 2 and 6 | - |

Wauters and Vanhoucke [3] | ✓ | - | - | Ad hoc ^{1} | 90 | 2 | 1–8 | 1 and 2 |

Batselier and Vanhoucke [43] | ✓ | - | - | - | - | 51 | 1–8 | 1 |

De Koning and Vanhoucke [44] | - | - | - | ✓ | - | 9 | 2 and 6 | - |

Batselier and Vanhoucke [29] | ✓ | - | - | - | - | 23 | 1 and 2 | 2 |

Khafri et al. [45] | - | - | - | ✓ | - | 35 | 2 | - |

Kim et al. [46] | - | - | - | ✓ | - | 451 | 2 and 4 | - |

This study | ✓ | ✓ | ✓ | R,ρ,τ | 4100 | - | 1–30 | 1, 8, 10 and 30 |

^{1}A mean lags indicator was used that compares the prediction of the method between two consecutive AT periods.

^{2}The three stability parameters used in this study (R, ρ, τ) are described later in Section 3.2.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Barrientos-Orellana, A.; Ballesteros-Pérez, P.; Mora-Melià, D.; Cerezo-Narváez, A.; Gutiérrez-Bahamondes, J.H.
Comparison of the Stability and Accuracy of Deterministic Project Cost Prediction Methods in Earned Value Management. *Buildings* **2023**, *13*, 1206.
https://doi.org/10.3390/buildings13051206

**AMA Style**

Barrientos-Orellana A, Ballesteros-Pérez P, Mora-Melià D, Cerezo-Narváez A, Gutiérrez-Bahamondes JH.
Comparison of the Stability and Accuracy of Deterministic Project Cost Prediction Methods in Earned Value Management. *Buildings*. 2023; 13(5):1206.
https://doi.org/10.3390/buildings13051206

**Chicago/Turabian Style**

Barrientos-Orellana, Alexis, Pablo Ballesteros-Pérez, Daniel Mora-Melià, Alberto Cerezo-Narváez, and Jimmy H. Gutiérrez-Bahamondes.
2023. "Comparison of the Stability and Accuracy of Deterministic Project Cost Prediction Methods in Earned Value Management" *Buildings* 13, no. 5: 1206.
https://doi.org/10.3390/buildings13051206