Reducing Cost Overrun in Public Housing Projects: A Simplified Reference Class Forecast for Small Island Developing States

Abstract

Inaccuracies in cost estimation on construction projects is a contested topic in praxis. Among the leading explanations for cost overrun (CO), factors accounting for large variances in actual cost are shown to have psychological or political roots. The context of public sector social housing projects (PSSHPs) in Small Island Developing States (SIDS) is positioned with similar CO challenges. This study is the fifth phase of a series of research projects on the vulnerability of PSSHPs to COs, and the need to de-risk cost estimates. The aim of this study is to present a simple and practical application of Reference Class Forecasting (RCF), a promising solution utilizing an “outside view” approach, as an effective control to reduce the variance of forecasted cost inaccuracies. Using a sample set of 82 housing projects, a reference class of 23 projects was selected based on properties such as design-build procurement type and local contractor involvement. A probability distribution was then established for this reference class, and required cost uplifts to be applied were based on the level of risk a housing agency is willing to accept for PSSHPs. Finally, the accuracy of the reference class was tested using a recently completed project. The results showed that the RCF method, based on a 50th percentile risk acceptance of CO, provides a closer estimate to the actual costs of the project as compared to the contracted costs. This empirical study is the first to undertake and implement RCF in the 52 SIDS and presents the first instance of practical RCF in public housing projects worldwide, thus providing a platform for improvement in future PSSHPs’ budget forecasting. The research can be applied to lessen societal and economic welfare losses as well as significant financial risks for governments. The implementation of practical safeguards, such as RCF, together with contemporary standard project controls, provides immediate advantages for enhancing accuracy in present forecasting approaches against financial risks. It allows for improved value derived from social infrastructure projects, improved supply of public housing, and consequently progress for these nations towards achieving their sustainable development goals.

1. Introduction

On an annual basis, democratic governments provide accounts on the progress of economic initiatives undertaken to improve and protect the livelihoods of their citizenry. To achieve these objectives, as well as to drive economic growth and create job opportunities, infrastructure development is often targeted as a critical solution. A rationale for capital investments into these public infrastructure programmes is the derived benefit of improved standard of living, productivity, and business opportunities [1]. This view is supported by Kaliba et al. [2], who indicated that the wealth of a nation can be measured by its infrastructure outlay. Comparatively, reducing investments into infrastructure provision hinders economic recovery and growth [3]. Infrastructure development is also extremely capital intensive, and typically commissioned through major programmes. Major programmes are multi-billion-dollar transformational undertakings that last for several years with the goal of delivering a positive outcome [4]. Flyvbjerg et al. [3] recognized that a 10% increase in infrastructure provisions can result in an improved long-term economic growth output of 1% to 5%, demonstrating the importance of public infrastructure investment in fostering economic growth. Public sector infrastructure programmes (PSIPs) are thus positioned as key strategic change initiatives to allocate capital investment into infrastructure development. However, a major problematic with PSIPs is their historical inability to deliver intended benefits within forecasted projections of time and cost [5]. Thus, funding loans and financial arrangements for public infrastructure are primarily serviced through taxpayer dollars, and the projected capital demands envisioned to finance these projects through to completion rarely consider the effects based on the complexity and volatility of the industry. This ultimately leads to unfavourable outcomes [6].

Public sector social housing programmes (PSSHPs), a class of PSIPs, is a government’s strategic intervention to promote socio-economic growth and raised standard of living. These social change programmes are aimed to assist lower income citizenry to acquire the basic need of shelter through lowering costs to improve its affordability. Public housing is typically subsidized [7] through measures ranging from low interest loan provisions, to nominal fees on land and infrastructure development, to tax exemptions for first time homeowners. In the SIDS the demand for public housing provisions far outpaces the actual supply of affordable housing units. This is true in Trinidad and Tobago, where over 175,000 applicants are on a ministry’s waiting list; this sum equates to about 15% of the 1.3 million citizens. To ensure the supply of affordable housing is achieved, the Government of the Republic of Trinidad and Tobago (GORTT) enacted The Housing Act of 2005. This allowed for the incorporation of a special purpose state housing agency (SHA), the Housing Development Corporation, and the dissolution of the existing Ministry’s National Housing Authority, with a stated policy objective to deliver 10,000 low-income residents’ homes annually. However, the expected benefit realization for this policy objective never materialized. Chadee et al. [6] recognized that the housing delivery fell far short of the intended goal, with a maximum delivery of roughly 2000 units in 2013 and an annual average of approximately 1200 units since SHA’s founding. Explanations for the lack of benefits to society and the slow delivery of housing sites were mainly due to extreme cost overruns and delays on all housing projects, which inevitably choked off the supply of homes necessary to satiate the rising demand. “Over budget, over time, over and over again” [8] seems to be the common theme, highlighting the mounting demand on public authorities to properly account for infrastructure development projects’ expenditures.

Cost estimation errors in construction projects are a contentious issue in practice. The cost overrun scenario, however, is not a recent occurrence; rather, it is the outcome of a long-standing systemic culture that has led to the formation of symptoms that defy theoretical explanations, further obscuring the origins and explanations of cost overruns. Factors explaining a significant difference between actual costs and the agreed projection and cost overruns have been proven to have technical, psychological, or political-economic underpinnings [5]. Similar cost overrun issues are present in the context of public sector social housing projects (PSSHPs) in Small Island Developing States (SIDS), including the necessity to protect against financial risks to address the public housing supply gap. In response to the knowledge deficiency identified, this study is the fifth phase of ongoing research into the CO phenomenon in Caribbean SIDS and advocates the necessity to de-risk cost estimates and the vulnerability of social housing projects to cost overruns. The aim is to formulate a RCF to empirically assess (1) the probability distribution of CO for a reference class of design-build housing projects, (2) deriving the uplifts to contract sums based off the probability distribution, and (3) provide an empirical and visual representation to managers of the direction of COs based off the probability of cost exceedance and the associated uplift. RCF is an innovative approach that takes an “outside view” as a potential control to lessen the variety of anticipated cost inaccuracies [9]. Although RCF has had little global applicability over the past 20 years, after being endorsed by the American Planning Association in 2005 and being required in the UK to reduce risks of human biases in decision-making on significant projects, this study is the first to undertake and apply RCF in the 52 SIDS. It introduces the first instance of real-world RCF in public housing projects, and provides a more comprehensive understanding of cost overruns in contract sums. This report is significant because it clarifies how project managers can put cost control mechanisms in place to prevent the recurrence of these issues.

2. Theoretical Background

2.1. Causes vs. Root Causes of Cost Overruns

For decades project planners have consistently used inefficient cost, demand, and other forecasting methods which continue to produce inaccurate results [8]. This is evident by the negligible gains in forecasting accuracy despite the growing promises of enhanced modern forecasting models. For example, the overrun costs for transportation infrastructure were roughly 44.7% for rail, 33.8% for bridges and tunnels, and 20.4% for roadways [8]. For public housing in SIDS, the overrun cost averages 75% [6]. As a justification strategy, cost-benefit analyses are frequently employed to support these projects’ acceptability, where the derived benefit of the project outweighs the costs incurred. However, these cost-benefit ratios are frequently inaccurate as true costs are not reported and benefits are often inflated [1]. Since estimates and socioeconomic and environmental assessments are frequently inaccurate, managing social infrastructure projects is extremely difficult. Investment choices are frequently based on inaccurate, and/or incomplete, and potentially biased information that leads to poor decision making and inferior project selection.

Cost overruns are found to reside around four main root causes: technological, psychological, political, and economic [7,9,10]. Technical variables, such as inaccurate forecasting, inadequate project data, project specification variations, and technical experience errors, are the focus of technological factors. Psychological explanations include mistakes made as a result of various biases of the cognitive propensity to predict outcomes optimistically even though past experiences show a different response [11,12]. The psychological explanations provide accounts for the cognitive tendency of individuals to perceive future events more favourably than perceptions based on real experience [12]. The economic elements also consider macroeconomic issues including currency fluctuations, the GDP, investments based on global outlooks, and public-private interests based on self-interests.

The critical root cause of CO, however, is political influence [6,13]. Political influence has the power to challenge and alter informed technical models to support and approve a particular project. Due to political and institutional factors, such as partisanship, resource availability, and financial gains, project professionals are compelled to offer false information about the costs and benefits of their initiatives to boost the likelihood of their chances of receiving funding [14]. This results in exceedingly risky ventures [11] using false data throughout the decision-making stage of a new project [13,15]. This type of deception creates inaccurate cost and demand projections with a consequential negative impact on social sustainability through the production of inaccurate estimates, socioeconomic assessments, and environmental assessments.

2.2. Delusive vs. Deceptive Behaviours

Flyvbjerg et al. [14] concluded that the cost estimates used in the choice to construct were “systematically and significantly false”. This statement was subsequently confirmed for PSSHPs in SIDS [1,6]. A psychological explanation given for cognitive biases was optimism bias, whereas a political explanation given was strategic deception. It was concluded that both optimism bias (delusion) and strategic misrepresentation (deception) involve elements of dishonesty, but the underlying causes are fundamentally different. To understand and lessen the occurrence of incorrect forecasting estimates, both explanations are required [9].

The traditional perspective of risk occurrence was to identify its cause as external elements to the program. However, experts frequently make judgement errors that are compelling even when one is fully aware of their nature. The main cause of risk, according to Flyvbjerg [16], is managers’ persistent misperception of risk, which leads to risk being perceived as internal to the program. As a result, a complete shift in viewpoint is necessary. This shift in viewpoint is grounded on the planning fallacy. This led to the Nobel Prize winners Kahneman and Tversky’s [17,18] research on decision-making under uncertainty. There is a recognized systematic tendency for an individual to be excessively enthusiastic about the results of planned acts, and this tendency is based on self-deception on the side of project forecasters and decision makers [16]. It refers to the “inside view” that project players adopt in which the results of comparable historical events or acts are not considered. Bias, rather than perplexity, generally manifests itself as mistakes of judgement because they are systematic rather than random. Thus, project actors exaggerate the probability of pleasant events while underestimating the probability of negative ones. Comparatively, strategic deception is widespread when organizational and political pressures are high. Strategic misrepresentation is deliberate deception or lying [19] in the estimating process to gain a favourable outcome. To profit, estimates are systematic distorted, which directly leads to a misrepresentation of reality [16]. As a mitigation strategy to combat against delusive and deceptive behaviours, incentive alignment and structures are proposed to reward ethical conduct and penalize unethical behaviours [13].

The Planning Fallacy theory, as related to the actor-observer behaviour study by Buehler, Griffin, and Ross [20], looked at people’s propensity to forget past failures. However, Pezzo, Pezzo, and Stone [21] critiqued these findings in defence of the planning fallacy’s self-preservation explanation, whereby they investigated the paradox of the planning fallacy, wherein accountability rises rather than falls, leading to forecasting bias [20,21]. According to this idea, people who display optimism bias and strategic misrepresentation should be terminated from important programmes and decisions should be debiased by “making salient the repercussions of a failed prediction” [21]. Since forecasters concentrated on the symptoms of these errors, which were mostly technical (scope changes, delays, geotechnical, design changes, etc.), rather than the root causes, which included cognitive bias (optimism bias) and strategic misrepresentation, systematic errors happened during the planning phase. Therefore, despite the principal-agent problem and the idea of information asymmetry being addressed thoroughly by incentives and incentive alignment, incentives by themselves are insufficient to encourage decision makers to produce reliable data [1]. Some decision makers lack the self-discipline and inner drive to gather facts due to various “preoccupations” in their everyday lives. For instance, a serious personal circumstance may have negative psychological effects, which could raise the likelihood of an individual making suboptimal decisions. Utilizing the same baseline quality techniques, these “preoccupations” lead to collation of unreliable and inaccurate data across programmes rather than an effort-driven approach to build a database of reliable distributional information across projects and programmes.

2.3. Reducing Optimism Bias and Strategic Misrepresentation—The Outside View and RCF

Traditionally, most individuals and organizations choose to create their projects from an “inside view” [22] or bottom-up approach. This intuitive approach decomposes a project into individual tasks, looks at its complexities, and estimates what is known. The inside view pays particular attention to the project’s unique characteristics in order to foresee how future events may affect it [5]. Bottom-up costing strategies such as analogous and parametric estimating are commonly used in SIDS [6]. These traditional strategies attempt to reconcile estimates of individual components and cumulatively sum the various costs into a total estimate, with a plus/minus percentage deviation attached to the forecast. These convectional costing strategies are common to “roll-up” cost estimates from lower levels towards the overarching highest level of total costs. Other methods such as Building Information Modelling (BIM) have also attracted better success in cost estimating internationally [23,24]. The challenge, however, is the uptake of BIM in SIDS [6]. BIM technology has been around for decades, but its popularity is extremely sparce, and rarely practiced in SIDS.

A key rationale for the over reliance in the traditional costing strategies and the ignorance of BIM leads to engineers and planners being highly optimistic of their estimates. In a study conducted in the Caribbean SIDS, planners and engineers were shown to be highly optimistic, and obtained higher bias scores, on issues of delays and CO on projects [11]. These same participants were also shown to be highly pessimistic on issues such as lack of funding and poor planning habits. Most engineers and cost estimators do not often see themselves going out and gathering basic facts on related endeavours influencing cost variables from actual distributional data. Unknown risks, such as pandemics that adversely influences costs [25], are treated in isolation to compensate for reduced supplies of building materials and associated escalated costs. Other issues such as the actual implications of irregular cash flows for a project [26] are rarely given any merit in cost consideration, if ever in SIDS, to the overall contract sum.

Project managers have a strong tendency to ignore distributional information, which may be the direct source of errors in forecasting [3]. This results in prejudice from errors in judgement that are typically purposeful and predictable, rather than random, and any remedial prescription must take this into account [9]. To resolve this issue, Flyvbjerg et al. [5] argued that distributional information is the cure for the planning fallacy. As there were no assurances that similar reference projects were considered to obtain distributional information on actual performances, Kahneman and Lovallo [27] proposed taking an “outside view” of planned actions to be compared with the performance that has already occurred. Reference Class Forecasting (RCF) [8,9] was created to account for the specific sort of cognitive bias discovered in decision-making under uncertainty. By adopting an “outside view” approach and using distributional information from other ventures of a similar nature, a better and accurate forecast can be generated.

RCF is a top-down method of predicting future costs by evaluating comparable project data and results and developing graphically various uplift factors to be applied to initial forecasts based on the decision maker’s appetite for risk [3,28]. Thus, based on the project’s risk profile and the decision makers’ tolerable risk of budget overrun, a corresponding uplift factor is then applied to the base estimate to offer a genuine cost of the project. The uplift factor removes subjectivity based on biases in the estimating process and can be reported as a standard gauge of the base estimate of a reference class of similar historical projects. Flyvbjerg [3,5,9,10,14,29] pioneered the use of RCF on major projects and created a due diligence process based on the outside perspective. Prior to developing a cost forecast, the scope of the project must be identified and thoroughly defined. Any changes will result in cost variance from the initial agreed scope. The establishment of a representative outside perspective benchmark for performance comparison is another requirement. The performance of the initial estimate in question is then evaluated against this benchmark. Any extra risks related to costs and benefits must be taken into consideration when estimating the expected outcome. The final step before calculating how far the forecast deviates from the specified benchmark is the forecaster’s feedback.

Budgets uplifts and contingency sums are sometimes conflated, creating ambiguity in definition and use [30,31]. One commonality is that both sums are presented to support a project and raise the likelihood of its success. A contingency sum is an additional allocated sum based on a percentage of the awarded contract sum. The contingency sum, derived from the contingency management strategy of the project [32] is typically drawn down for any unforeseeable event or to cover additional variation works. Best practices and expert judgement suggest that the contingency amount varies from between 10% to 20%. Project contingencies frequently adhere to the modified Parkinson’s principle, which stipulates that “money allotted equals money spent” [33,34]. This also suggests that when project managers access big contingencies, they are likely to manage these sums sub-optimally, as is the typical pattern in public sector initiatives [6,12]. Budget uplifts however, are not based on best practice or expert judgements, and are subjected to overconfidence and a lack of distributional knowledge which can skew most human judgements [35]. It provides a more precise estimation based on the client’s risk appetite. Instead of projecting the specific unknown events that may affect the project, the budget uplifts in RCF attempt to position the project within a statistical distribution derived from the class of reference projects.

By showing that reference class forecasting failed to produce an accurate cost estimate in at least one megaproject, the effectiveness of RCF as a best-practice technique for project forecasting is questioned [36]. As the study indicates, reference class forecasting did not stop the project participants from using their own “biased” judgement when predicting the project’s costs. It only changed how biases affected the result and the dynamics of cost estimates. To understand this seeming failure, Themsen [36] contends that cost estimation is a reflexive behaviour that goes beyond subordinate-superior relationships, individual actors’ cognitive abilities, and strategic objectives. By accepting biases as an inevitable component of the process and reducing the requirement for exact cost estimates, this method focuses on the project’s linked network of mutually constitutive stakeholders across time. The author also emphasizes that Flyvbjerg ignored technical justifications for erroneous cost estimates, such as “imperfect methodologies, limited data, honest blunders, and lack of expertise,” and instead pointed to socio-political justifications. They stated that as just one megaproject was looked at in their study and the results are only suggestive, further empirical case studies on the topic would need to be carried out.

Given the pervasive nature of CO in practice, and the continued debates in scholarship on its origins and treatments [37], caution is exercised in generalizability and replication of the theory. Context is important in CO studies [38], and this phase of the research aligns with the contingency theory perspective that a phenomenon is influenced by context [7,39]. As a result, given the sustainability sensitivities of SIDS, based on their economic, social, and environmental dependencies [40,41], it is assumed that the effectiveness of treatments to critical risks associated with CO varies, based on time, location, and project environments. However, the scholarship is very sparce on risk and decision-making optimization techniques consistently used in SIDS. Chadee et al. [1,6] began the difficult discussion in scholarship to move away from the industry’s rhetoric of adverse realities of political influences on PSSHPs and public projects, and to document these realities with the aims of minimizing wastages of finite resources and building a culture of sustainability in the SIDS’s construction industry. A measurable starting point is needed to benchmark overall economic performances in projects [42]. Based on the successes and limitations of RCF in the literature, RCF can be easily implemented on a variety of projects as an intervention mechanism to improve cost forecasting. With RCF, managers and policy makers are forced to undertake a due diligence check to improve the accuracy of contractual estimates and unrealistic schedules [5]. These decision makers are now equipped with a range of possible risk levels and cost uplifts to draw a consensus for a more informed choice for forecasting and budgetary control [3]. In turn, informed decision-making leads to better management of taxpayer funds, contains public outcry over corrupt activities, and improves accountability for public sector housing projects [1]. The CO’s causal chain can thus be gradually reduced by the continued improvement of existing systems, governance structures, and construction management practices to support technocrats and academia to identify and remove optimism bias and strategic misrepresentation that are widely prevalent in PSSHPs [6].

3. Methodology

This study is the fifth phase of a sequential mixed methods study into PSSHPs in SIDS [43,44]. By adopting a pragmatist perspective, this study expands the existing body of knowledge into CO in public housing by adopting a sequential exploratory design to collect, analyse, and interpret both qualitative and quantitative CO data in public housing programmes. To recap, the first phase of the overall study into CO in PSSHPs showed CO’s theoretical limitations through two concepts of ideological distancing of philosophical positioning, and through encapsulation of perspectives and alternatives within their respective schools of thought [37]. This phase of the research advocated for both the need of, and congruency in, a structured cross-fertilization of ideas, concepts, and theories, to depart from the current practice of fragmented and ad hoc conflation of concepts/theories, and to link context to process to outcome in CO studies. The second phase, qualitative in nature, selected a critical case study to investigate actual outcomes in PSSHPs [1,6]. A method of process tracing was adopted to demonstrate that the current reliance on decision making and judgements from project management practitioners led to biases, strategic mismanagement, and further uncertainty, which ultimately led to further societal implications [1]. A call was made to actively remove subjective judgements and vagueness in decision making that creates additional uncertainty in the PSSHP, and for the urgent need to improve transparency and accountability through “informed and empirically” driven decision support methods to address CO risks.

To address the “informed” aspect of CO risks, the third phase of the overall study determined the critical risks factors (CRFs) and root risk groups in PSSHPs [7]. One hundred and twenty-three (123) CRFs were extracted through a detailed literature review, and through the use of the Delphi technique, a panel of experts was created to determine the relevance, suitability, additional occurrences, and applicability of these CRFs within the Caribbean SIDS context. In all, 41 critical factors were found valid, grouped into four root causes, and ranked in this phase [7]. These three phases all concluded that the origins of CO primarily reside in the political domain, where its agency negatively influences technical decision making on PSSHPs. The fourth phase recognised the lack of a formal risk decision support system to guide academics, professionals, and policy makers in addressing the true root causes of COs. To address the “empirical” aspect of CO risks, the latter phase proposed a fuzzy synthetic evaluation model [45] as a soft computing model to empirically determine and rank the multi-criteria levels of risk, i.e., the individual CRFs, the root causes, and the overall risk level index.

This phase of the research continues along with sequential mixed methodology [46]. The qualitative component followed a critical case study [47,48] to investigate the causal chain of cost performances on social housing programmes in Trinidad and Tobago to gather data for the quantitative component of the research. A critical case study methodology can be beneficial in obtaining details of a particular phenomenon [49] through the use of multiple instruments to provide analytical inferences based on hindsight and tacit knowledge [43,48]. The quantitative component assessed the project performances within two programmes based on the “outside view” method and the development of the RCF model. Data on a recently completed project was provided to back test the simplified RCF model to determine whether the newly proposed RCF model provided a better estimate of the actual costs as compared to the traditional bottom-up estimating approach.

3.1. Qualitative Component Case Study: The State Housing Agency

For the qualitative component, literature surveys and semi-formal interviews were the primary and secondary data collection methods used in order to gain deeper insights into decision-making rationales associated with public housing construction delivery. All ethical protocols were observed during the administration of the semi-structured interviews. Participants were informed of the objectives of the study and were notified that the interview was non-mandatory and could be stopped for any reason [50]. Three major contractor’s representatives were interviewed utilizing scripted open-ended questions. The general feedback was positive and there was a general interest in delivering sustainable affordable housing. The critical case study focused on the public housing agency of Trinidad and Tobago. This state housing agency (SHA) was established in 2005 by statute law passed by the Parliament and functions as a state agency under the Ministry of Housing and Urban Development. The SHA was created with the aim to fulfil the government’s 2020 vision, i.e., every citizen would have access to social affordable housing that is both acceptable and inexpensive. The SHA’s mandate was to produce 8000 dwelling units annually in order to realize this aim [51]. To promote affordability, the cost of the infrastructure is wholly subsidized by the government. An additional 30% subsidy of the cost of building houses was given at the discretion of the Cabinet, depending on the development’s location. Reduced mortgage terms, rent-to-own programs, and other subsidy programs represent several interventions by the government to address affordability issues [52]. Land value was also subsidized, from about USD $8 market value to USD $0.8. Any resident making less than $15,500 is eligible with no down payment for a mortgage with a 2% interest rate instead of the market rate of 7%. For instance, in 2009–2010, the cost to the government for a 1200 square foot, three-bedroom, two-bathroom single family home was almost USD 110,000. A prospective homeowner instead pays about USD 60,000, or a subsidy of 45.5% total for property ownership [1].

The Trinidad and Tobago government anticipated delivering about 8000 units and starting about 2000 housing projects in 2010. Data for 82 projects were collected and analysed to ensure all relevant costs were captured [6]. Projects were grouped using three procurement methods: The traditional Design-Bid-Build (BOQ), Bespoke (non-traditional format), and Design-Build (DB) [53]. For several projects, information for the traditional BOQ procurement and the bespoke housing contracts that were tailored for the organisation were found to be inconsistent and incomplete. For Design-build projects, however, data was found to be consistent and mostly complete. This is because a requirement at the end of the contract is for the contractor to submit a complete bundle of documentation for the entire project, inclusive of all communications, as-built drawings, approvals, payment drawdown schedules, and applications for the release of retention. Therefore, costs can be easily verified and validated for associated milestones such as infrastructure, utilities, and housing [6]. Another property associated with these 24 DB projects (nineteen projects initiated in the 2005–2010 political cycle and seven DB projects commenced in the 2010–2015 political cycle) is the involvement of local medium to large contractors who were experienced in various types of civil engineering projects. These DB projects were selected to form the reference class of Design-Build Housing projects.

Though several definitions of CO exist in the literature [54], the baseline cost used to estimate the deviation of the final cost was the accepted contract sum on the award of the project to the DB contractor. The contract award sum is shown to be much more conservative than the decision to build estimate [55]. However, in terms of verification and validity, the contract award sum provides a verifiable baseline benchmark to measure cost deviations, and is the only measure widely used in the SIDS construction sector. Thus, CO is calculated as follows:

$$\mathrm{CO}=\raisebox{1ex}{$\left(FS-OS\right)$}\!\left/ \!\raisebox{-1ex}{$OS$}\right.$$

where $FS$ = final contract sums and $OS$ = original contract award amount. The $FS$ did not consider transactional costs as this information was not captured in the project accounts, and consistent and verifiable data was unavailable.

Table 1. Analysis of project performance within 2005–2018 (constant prices) [6].

#	Description-Infrastructure and Housing	Contract Award/Completion Date	Original Contract Amount	Variation	Cost Overrun
1	202 units	July 2005/July 2012	$163,394,697.12	$116,186,634.22	71%
2	Infrastructure		$57,484,766.09	$87,722,785.23	153%
3	408 units	July 2005/October 2011	$133,129,000.00	$54,707,606.84	41%
4	78 units	October 2007/October 2011	$88,660,471.91	$13,380,664.79	15%
5	544 units	February 2005/February 2013	$290,506,855.14	$322,953,166.80	111%
6	124 units	December 2007/December 2012	$65,110,289.00	$24,886,100.80	38%
7	41 units	October 2007/October 2012	$27,433,576.45	$26,072,749.75	95%
8	422 units	April 2006–Present	$85,448,540.82	$174,569,708.09	204%
9	288 units	December 2005 to June 2015	$298,944,490.50	$379,577,411.70	127%
10	807 units	July 2004/October 2013	$366,085,547.83	$153,781,415.29	42%
11	96 units	6 January to November 2015	$51,669,306.54	$75,026,601.64	145%
12	277 units	September 2005/October 2013	$76,558,282.30	$7,768,970.88	10%
13	152 units	December 2004 to December 2011	$66,434,777.80	$44,274,141.41	67%
14	701 units	October 2003/Ongoing	$313,448,071.21	$18,832,761.46	6%
15	368 units	7 March/June 2015	$277,697,724.00	$339,735,129.56	122%
16	50 units	January 2007/2009	$23,902,129.00	$17,764,374.21	74%
17	1128 units	January 2007/May 2013	$382,636,740.07	$125,959,613.22	33%
18	171 units	November 2005/January 2012	$96,585,870.98	$84,367,604.19	87%
19	256 units	March 2005 to December 2013	$71,061,239.00	$115,569,014.53	163%
20	1190 units	April 2012/March 2015	$1,236,704,755.27	$130,598,232.88	11%
21	572 units	August 11/ February 2015	$545,337,160.88	$-104,848,117.79	−19%
22	34 units	14 March/March 2017	$31,992,302.00	$1,438,584.00	4%
23	743 units	April 2012/July 2015	$713,153,487.09	$81,717,815.09	11%
24	70 units	May 2012–December 2014	$65,252,980.39	$-1,471,963.54	−2%

provides a summary of contract award date and sum, as well as the final $\mathrm{CO}$ percentages on completion.

According to the information in Table 1, the best-performing project was project 21, which had cost savings of 19%; therefore this project is to be used as a success case study. Project 8 had a 204% cost overrun and was the worst performing project recorded. The reference class of project sizes ranges from 41 housing units to 1190 housing units. The following sections provide a three-step process of conducting a RCF [28].

3.2. Quantitative Approach: Simplified RCF Method

The principle of RCF is based on the “outside view” whereby examining past projects and their outcomes is used to predict cost overruns for the present project [3,56]. The use of realized outcomes rather than manipulated estimates can provide a more reliable estimate of the true costs and benefits while bypassing the cognitive biases associated with optimism bias and strategic misrepresentation [10]. This is determined using distributional analysis and produces more realistic results on cost overruns and benefits. The process of conducting RCF is shown in the flowchart below (refer to Figure 1) [3,9].

The first step is the identification of the relevant reference class of past projects. Table 1 shows the reference class of DB projects that were selected to undergo statistical analyses. From this reference class, if the project managers and estimators were concerned that the contract sums are in fact correct, then the distribution of outcomes on these DB projects would have been established, as shown in Figure 2. To test this hypothesis, an informal survey was undertaken on all ten (10) of the inhouse project managers (PMs) who are directly responsible for providing oversight on similar ongoing DB projects. The question asked the PMs what they perceived is the average CO on their DB projects. The average CO experienced by these PMs are approximately 20%. Moreover, all PMs agreed that the contingence within the contract of 10% is not a realistic figure.

Of the 24 projects selected, 23 projects were found to have complete and accurate data that could be used to form the reference class. Project 2 was eliminated from the reference class as new works have been added; therefore, the cost overrun figure of 153% is an underestimated percentage. A normal distribution curve representing the cost overrun percentages with its associated probability densities is shown in Figure 3.

The curve is positively skewed, with long positive tails, indicative of high CO on PSSHPs. The mean cost overrun is 69%, with a standard deviation of 63%. The skewness was found to be 0.42, indicating that the data is fairly symmetrical (−0.5 ≤ x ≤ 0.5). The kurtosis of the distribution was negative (−1), indicating that more of the CO values are located nearer the centre of the distribution as compared to the tails. With these basic statistics, a one tail t-test was administered to test the hypothesis that the mean CO on DB public housing projects is 20%. Based on the hypothesis test, there is strong evidence to reject the null hypothesis at 1% level of significance (t = 4.29, d.o.f = 22, p < 0.0001). Therefore, a 20% cost increase on DB public housing projects is a highly optimistic and inaccurate account for cost overruns.

Table 2. Breakdown of cost inaccuracies in public sector DB housing projects.

Category of Project	Housing
Types of Projects	Single family units, Duplex, Apartments, Town Homes
No. of projects in reference class	23
Average Inaccuracy (%)	69 (sd = 63)
Level of Significance, p	<0.0001
Percentage of Projects with CO larger than ±20%	74%
Percentage of Projects with CO larger than ±40%	70%
Percentage of Projects with CO larger than ±50%	57%
Percentage of Projects with CO larger than ±80%	43%

provides a breakdown of the actual percentages of projects with various percentages of projects experiencing CO exceeding 20%, 40%, 50%, and 80%, respectively.

The second step is the establishment of a statistical distribution. For this step, a reference class of completed, comparable housing projects was selected to produce a probability distribution curve for cost overruns for projects in the reference class (refer to Figure 4).

Step 3 develops the necessary uplift after applying RCF to the reference class mentioned above. From the distribution curve of cost overruns, a probable distribution curve of acceptable Chance of Cost Overrun vs. Required Uplift was plotted as shown in Figure 5 [9]. The curve provides a systematic way of forecasting cost overruns for any given uplift at constant prices. The percentage value of the uplift is set by the client.

Before uplifts to the original contract sum can be applied, the organisation first needs to understand its risk appetite for cost overruns. If the organisation is undertaking a major project, and cost certainty is the leading performance metric, then uplifts at the 80% range and above (20% or less chance of cost overruns) is recommended [3,57]. However, if the organisation has a portfolio or programme of various projects, where CO in one project is offset by cost savings in another, then the 50% percentile uplift can be used as a contingency measure (a 50% chance that CO can occur).

Table 3. Applicable capital expenditure uplifts for the 50% and 80% percentile.

Category	Types	Applicable CO Uplift
		50% Percentile	80% Percentile
Housing	Single family units, Duplex, Apartments, Town Homes	60 %	132 %

summarizes the capital expenditure uplifts for DB housing projects with the recommendation that the state agency adopts the 50% percentile, i.e., apply a 60% uplift to the contract estimate to achieve better cost certainty.

3.3. Testing the Simplified RCF Method

To determine whether the first instant of the RCF model in public housing is indeed robust, two recently completed projects were examined. The data of the two cases are presented in

Table 4. Comparing the predictive robustness of the RCF Model.

#	Size of Project	Original Contract Amount	Actual CO	P50 Uplift	% CO/Savings from P50
25	102 units	$55,852,890	$37,200,000 (67%)	$33,511,734	7%
26	57 units	$47,760,000	$9,119,000 (19%)	$28,656,000	−11%

. Using the 50% percentile, or a 50% chance of CO, the required uplift to apply to the contract award sums is 60%, as shown in Table 3. For project 25, the actual cost performance deviated by 7% as compared to the simplified RCF prediction. This immediately shows that the predictive capability of RCF to bypass the daily decision-making nuances and cut to actual outcomes is legitimate, as a 7% CO will be welcomed as the industry benchmark for any class of construction projects. For project 26, the RCF model was conservative by 11%. Project 26 performed better that the average reference class of DB housing projects, with a 19% cost overrun. In terms of uplifts, the acceptable chance of cost overruns on the project is approximately 70% or P30. The P50 uplift provided sufficient contingency cover for this project. It is also noted that project 26 performed as per the PM’s perception, i.e., the CO on DB housing projects is, on average, 20%. Given the two test cases, the RDF predictions shows that the contingency sums can now be allocated with a ±10% accuracy. Overall, back testing the RCF model shows that RCF provides a better representation of overall project costs and can be applied to PSSHPs to improve the probabilities of projects staying on budget.

4. Results and Discussion

4.1. Using Reference Class Forecasting as a Tool for Risk Management

In the Caribbean SIDS, RCF can be readily integrated into public housing to safeguard taxpayers’ dollars and allows for improved economic sustainability of PSSHPs. As such, RCF might be viewed as adding to the social advantages of public housing as it results in long-term cost savings and establishes a benchmark for staying within budget. Cost savings from better management can be allocated to other projects and infrastructure services, such as the installation of playgrounds, streetlights, etc., and thus add to the safety and security of low-income public housing developments. By objectively examining risk exposures, RCF can provide an indication as to whether technical or political influences are present, and ethically driven professionals can assess an acceptable degree of risk and project certainty to be achieved via a risk profile range. When deciding on contingencies and uplift requirements, RCF makes informed predictions simpler for decision makers, and presents a range of possible risk outcomes with associated additional capital outlay as a graphical representation. Additionally, RCF enables the governing organizations to foresee the entire project cost, which may then be set aside at the start of the project in a management reserve contingency. Project managers, quantity surveyors, and analysts will also find RCF useful as a tool when making decisions about cost certainty. RCF is also a learning model, as performances of new projects, such as the two test cases, can be fed into the existing reference class; the overall distribution will regress to a new lower mean, and the variance will tighten.

A major challenge from the RCF, however, is the justification to achieve a P80 percentile or a 20% chance of CO. Based on the uplift curve and Table 3, to achieve P80, a capital uplift of 132% is to be applied to the contract sum. For any public sector professionals and decision makers, this may be an unrealistic estimate to apply, and further, extremely difficult to justify. Public housing construction is not highly technical or innovative, and proven methods of construction used for over three decades are adopted. Then the question for any rational decision maker is why the need for a 132% increase in contract sums to achieve a 20% or less risk of CO? The real issue to address in public housing cost increases is not one of technical limitations, but the reality of the relationships between public housing and a country’s politics.

Chadee et al. [6] demonstrated in their study on systematic impacts that political influences follow an exponential trajectory, with more political influence translating to more loss of control of the project, and unsustainable high cost overruns. A framework was provided as a heuristic to gauge what are influencing cost overruns on PSSHPs. Technical and psychological influences contribute to at most 20% of CO on projects, and this CO can be reversed and removed by due diligence checks. CO associated with socio-economic influences accounts for project COs ranging from 20% to 35%. However, political influences result in cost overruns of at least 40%. Political influences are very dangerous for project success since cost overruns follow an exponential and non-reversible profile with the more political factors present on the project.

To test these cost profiles a review of the cost overrun contributors for projects 25 and 26 were undertaken. The critical risk factors from Chadee et al. [7] study were used to qualitatively assess the presence of these factors under four root causes of CO as perceived by the project managers on each project.

Table 5. Test for levels of CO percentages [7]: Perceived presence of critical risk factors.

Critical Risk Group and Factors	Project 25	Project 26
Psychological Critical Risk Group: C.O < 10%
Optimism bias	√	√
Technical Critical Risk Group: 10% ≤ C.O < 20%
Design change	√	√
Underestimation		√
Rework/Errors
Technical uncertainty, i.e., poorly defined project objectives, scope	√
Socio-Economic Critical Risk Group: 20% ≤ C.O < 40%
Lengthy bureaucratic processes
Economic business cycles
Acquiring regulatory approvals
Labour strikes
Financial shortfalls
Political Critical Risk Group: 40% ≤ C.O < 200%
Selection of politically aligned contractors	√
Contract poorly designed (intentionally)
Selection of politically aligned project management team (i.e., consultant, team lead, directors, etc.)	√
Strategic Misrepresentation, i.e., lying, e.g., underestimating costs	√
Project actors deliberately overestimating the benefits of projects
Pre-election commitments
Direct political influences (i.e., ministerial influences, location and type)
Political election cycles	√
Escalating commitment
Governance shortfall in the organisation
OVERALL COST OVERRUN	67%	19%

summarizes the findings on each project.

Project 25 was completed in 2020, an election year, as compared to project 26 which were completed in 2018. For SIDS, this is the number one indicator for large CO on PSSHPs. Other factors such as partisan contractors and project management team, and strategic misrepresentation, have a cumulative effect on the overall cost outcomes on projects being delivered on election years. Further investigations into the reference class of projects showed that all projects with over 100% CO were affected by the election year in political cycles. As the delivery of housing is seen as a measure of a government’s performance, and if existing organisational cultures are not improved [6,26], it could be a good idea to establish a risk/uplift factor of greater than 60% (P50) and 132% (P80) during election years as policy for this organisation where political pressures are present.

RCF was created as a safeguard to lessen any cognitive bias forecasters may have while making choices in the face of uncertainty. RCF eliminates subjectivity, which lowers the amount of bias present in human judgement [9,28]. To improve forecasting accuracy, the RCF model will require a consistent input of credible data [36,56]. However, data is typically not accessible in current active projects to aid in decision-making and accuracy in cost estimates [3]. Once credible data are made available that allows causalities and relationships among political and technical influences to be traced, and the underlying gaps can be identified and closed to give better results, cost estimators can gain from models that generate more consistent conclusions. Future budget calculations will be based on informed and valid distributional data for the purpose of allocating project resources. The decision maker’s willingness to tolerate risk is then taken into consideration while evaluating the uplift factor. Consequently, based on the simplicity of the outside view formulation, and with very few barriers to using this method, managers and forecasters should welcome the use of the outside view merely for the reason that no judgement can be made against a method that seeks to improve forecasting results.

4.2. Obstacles to Forecasting Reference Class

In PSSHPs, project decisions are made by professionals based on their own predictions and without consultation towards other previous endeavours to improve the project [6]. This is because optimism bias tends to lean towards the intuition of the individual forecaster, which results in inaccurate decision making or cost estimation. Few studies are undertaken prior to deciding and putting into action decisions based on projected results. Where optimism bias is high, managers will want to better their forecasts using RCF [5].

As previously mentioned, a challenge in RCF is obtaining valid and reliable data that leads to dependable forecasts as well as using a credible reference class to allow comparison with the forecaster’s estimate. A second challenge is the high prevalence of strategic misrepresentation in PSSHPs, where morally challenged professionals intentionally want the forecast to be erroneous and will create barriers to adopt RCF [58]. To appease political affiliation demands and political allies, project professionals are under pressure to make judgments that are incorrect, have a high risk of failure, and result in significant cost overruns [59,60]. For instance, a housing project would need to be completed on schedule before the following general election only to produce a host of issues due to poor judgement and inaccurate estimations. If a barrier is placed between the politics and the technical issues of the project, ethical professionals in charge of these public sector projects can carry out the duties that are required of them thanks to the elimination of strategic misrepresentation [61,62]. When this occurs, more realistic expectations and lower costs are the outcomes. All parties involved benefit from increased confidence, engagement, and passion as well as the effective utilization of available resources and public money [63].

RCF has issues in both obtaining accurate and trustworthy data that results in trustworthy forecasts and in using a trustworthy reference class to allow comparison with the forecaster’s estimate. A second problem is strategic deception, in which management aims to make the prediction inaccurate and puts obstacles in the way of RCF implementation. The forecasting process should incorporate accountability and transparency, with managers having “skin in the game” and being held responsible for their forecast with the prospect of being fired or disciplined legally if they falsify data [5,10,64].

5. Conclusions

RCF has been shown to improve forecasting accuracy on mega to micro projects through an unbiased viewpoint and makes use of distributional information from properties belonging to the same reference class. To recap, this study’s aim was to formulate the first RCF to address the high CO phenomenon for PSSHPs in SIDS. This aim was achieved through three objectives, namely: (1) to establish the probability distribution of CO for a reference class of design-build housing projects, (2) to derive the uplifts to contract sums based off the probability distribution, and (3) to provide an empirical and visual representation to managers of the direction of COs based of the probability of cost exceedance and the associated uplift. This study developed the first instance of a reference class of design-build (DB) public housing projects to address the CO challenges in PSSHPs. From a sample of 82 projects, 23 projects met the requirements of properties to be selected for the reference class. The reference class followed a positively skewed normal distribution with a mean CO of 69%, standard deviation of 63%, and a skewness of 0.42. A one tail t-test was administered to test the current project managers following the organisation hypothesis, whereby the mean CO on DB public housing projects is 20%. Based on the hypothesis test, there is strong evidence to reject the null hypothesis at 1% level of significance (t = 4.29, d.o.f = 22, p < 0.0001). Thereafter, a probability distribution curve for CO for projects in the reference class was produced of completed and comparable DB housing projects. From this distribution curve of CO, a probable distribution curve of acceptable chance of CO vs. required uplift was plotted to provide a range of uplift values based on the organisation risk appetite for CO. The P50 percentile (50% chance of CO) uplift factor and the P80 percentile (20% chance of CO) uplift factor were found to be 60% and 132%, respectively.

The RCF was validated against two recently completed projects using the P50 percentile uplift. For project 25, the actual cost performance deviated by 7% as compared to the simplified RCF prediction. For project 26, the RCF model was conservative by 11%. Project 26 performed better than the average reference class of DB housing projects, with a 19% cost overrun. Given the two test cases, the RCF predictions show that the contingency sums can now be allocated with an approximate accuracy of ±10% contingency. Overall, back testing the RCF model shows that RCF provides a better representation of final project costs and can be applied to PSSHPs to improve the probabilities of projects staying on budget. A major challenge for RCF, however, is the justification to achieve a P80 percentile or a 20% chance of CO. Based on the uplift curve and Table 3, to achieve P80, a capital uplift of 132% is to be applied to the initial contract sum. Further investigations into the reference class of projects showed that all projects with over 100% CO were affected by the election year in political cycles, and partisan project management teams. As the delivery of housing is seen as a measure of a government’s performance, and if existing organisational cultures are not improved [6], it is a good idea to establish a risk/uplift factor of greater than 60% (P50) and 132% (P80) during election years as policy for this organisation where political pressures are present.

Another challenge in RCF is obtaining data that enables trustworthy forecasts and using valid reference classes for comparison with the forecaster’s estimate. The data indicates that adopting an inside perspective concentrates on the intended action rather than the results of related previous efforts. Another problem identified is the pervasive strategic mismanagement during political election years, in which management aims for the prediction to be deliberately inaccurate in order to put obstacles in the way of RCF implementation. Forecasters should be held accountable and transparent, with the potential for termination or legal repercussions if they falsify data.

As with all forecasting methods, RCF has limitations, some identified above. However, the benefit of the predictive capability of RCF to shift the percentage overrun of CO into manageable and acceptable thresholds far outweighs its limitations. The results of this study will significantly benefit construction practitioners and academia by raising the bar in contemporary forecasting in SIDS. This study shows the need to unearth the hidden dangers of the negative outcomes of political influences on the noble and equitable deed of providing subsidized homes to the lower income earning population. Consequently, this results in tens of millions of dollars saved, and these monies can be better spent on socially sustainable projects.