# Validation of a Computer Code for the Energy Consumption of a Building, with Application to Optimal Electric Bill Pricing

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{*}

## Abstract

**:**

## 1. Introduction

_{2}emissions by four by 2050. This challenge implies, among other things, to be able to predict the energy performance of existing or future buildings, a task which can be tackled using thermal building models, implemented in numerical simulation computer codes, which can be thought of as deterministic functions with an input, some fixed parameters that can be uncertain, and an output.

## 2. Overview of the Case Study

#### 2.1. Dymola Computer Code

#### 2.2. Experimental Data

#### 2.3. Observation Model

#### 2.4. Sensitivity Analysis

- First of all, the sensitivity indices were defined as the ratios of output over input deviations:$$\begin{array}{ccc}\hfill {\mathit{S}}_{i}& =& \frac{\mathit{Y}({\mathit{\theta}}^{*}+{h}_{i}{\mathit{e}}_{i})-\mathit{Y}({\mathit{\theta}}^{*}-{h}_{i}{\mathit{e}}_{i})}{2{h}_{i}}\approx \frac{\partial \mathit{Y}\left(\mathit{\theta}\right)}{\partial {\theta}_{i}}{|}_{\mathit{\theta}={\mathit{\theta}}^{*}},\hfill \end{array}$$
- Next, the mean ${S}_{i,m}$ and the standard deviation ${S}_{i,std}$ were computed over time, resulting in a hybrid index ${S}_{i,d}=\sqrt{{S}_{i,m}^{2}+{S}_{i,std}^{2}}$;
- Finally, only the variables with ${S}_{i,d}$ larger than a certain threshold underwent uncertainty quantification. By thresholding this indicator, the number of parameters was eventually downsized from $p=193$ to $p=3$.

- ${\theta}_{1}\in [0,1]$ which is the albedo factor;
- ${\theta}_{2}>0$ which encodes the effect of the thermal bridges;
- ${\theta}_{3}>0$ which is the convective factor of the HVAC system.

## 3. Calibration

- A statistical model that links the available field measurements $\mathbf{Z}$ with the code outputs. This equation provides a likelihood function $\mathcal{L}\left(\mathbf{Z}\right|\mathit{\theta},\mathit{\psi})$, where $\mathit{\psi}$ is a vector of nuisance parameters attached to the model, specifying for instance the error structure between code outputs and field measurements;
- A prior density $\pi \left(\mathit{\theta}\right)$ encoding the uncertainty as a prior belief in favor of some values of $\mathit{\theta}$, which are more probable than others, based on expert opinion. If no such prior information is available, a uniform prior distribution can be adopted. Similarly to $\mathit{\theta}$, $\mathit{\psi}$ is endowed with a prior density $\pi \left(\mathit{\psi}\right)$, which we can choose independently from $\pi \left(\mathit{\theta}\right)$, meaning that we form a priori independent opinions about the plausible values of both parameters.

- Statistical Model

#### 3.1. Prior Densities

#### 3.2. Posterior Distribution

#### 3.3. Results

## 4. Validation

- Generate a sample $({\mathit{\theta}}_{1},\cdots ,{\mathit{\theta}}_{M})$ from the posterior distribution $\pi \left(\mathit{\theta}\right|\mathbf{Z})$, as described in Section 3;
- Run the code over the M samples $({\mathit{\theta}}_{1},\dots ,{\mathit{\theta}}_{M})$2. This leads to a sample $(\mathit{Y}\left({\mathit{\theta}}_{1}\right),\dots ,$$\mathit{Y}\left({\mathit{\theta}}_{M}\right))$ from the posterior distribution $\pi \left(\mathit{Y}\right(\mathit{\theta}\left)\right|\mathbf{Z})$ of the electric power over the time-period. A point estimate can then be derived, such as the posterior mean:$$\mathbb{E}[\mathit{Y}(\mathit{\theta})|\mathbf{Z}]=\int \mathit{Y}(\mathit{\theta})\pi (\mathit{\theta}|\mathbf{Z})\mathrm{d}\mathit{\theta},$$
- Estimate the posterior distribution of $\pi \left(\overline{\mathit{P}}\right|\mathbf{Z})$ by $\pi \left(\overline{\mathit{Y}}\left(\mathit{\theta}\right)\right|\mathbf{Z})$, following (4), a sample of which is given by $(\overline{\mathit{Y}}\left({\mathit{\theta}}_{1}\right),\dots ,\overline{\mathit{Y}}\left({\mathit{\theta}}_{M}\right))$, where the upper bar denotes the mean over the time period, as defined by (8).

#### Statistical Testing

- Assumption (9) leads to a statistical model in which only $\mathit{Y}\left(\mathit{\theta}\right)+\mathit{b}$ can be estimated; this results in a confounding between $\mathit{\theta}$ and $\mathit{b}$ which is solved in the literature by adopting a Gaussian process (GP) prior distribution on $\mathit{b}$. This means that the estimated value of $\mathit{\theta}$ depends entirely on the prior chosen for $\mathit{b}$, irrespective of the number of available data;
- The definition of the ‘real value’ ${\mathit{\theta}}_{0}$ for the parameter becomes problematic as soon as the code itself is no longer considered an exact depiction of reality;
- Adding the GP term makes model estimation as well as prediction more complex from a purely technical point of view.

## 5. Optimal Power Consumption Forecasts

- m is the marginal electricity price,
- c characterizes the probability $1-{(c(d-\overline{\mathit{P}})+1)}^{-1}$ that the customer breaks the contract, given that he/she pays more than he/she has consumed. For small values of c, this probability is approximately equal to $c(d-\overline{\mathit{P}})$, hence c can be interpreted as a customer defection rate, assuming that the number of defecting customers is proportional to the amount of unused energy they have paid for.

## 6. Discussion

- Statistical testing of the code’s goodness of fit;
- Parametric uncertainty propagation through the code;
- And posterior utility maximization;

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Notes

1 | Complete conjugacy is only attained for linear codes, for which the full calibration posterior distribution is explicit. |

2 | In fact, these runs are necessarily performed during the calibration step. A good practice is therefore to store all the computer code evaluations done during calibration, to avoid having to do them all over again for the validation. |

3 | This generality comes at a cost, since the Monte-Carlo variance associated with this second estimator is systematically higher than that of the first one. |

## References

- AIAA. 1998. Guide for the Verification and Validation of Computational Fluid Dynamics Simulations. Reston: American Institute of Aeronautics and Astronautics. [Google Scholar]
- Baudin, Michaël, Anne Dutfoy, Bertrand Iooss, and Anne-Laure Popelin. 2017. Open TURNS: An industrial software for uncertainty quantification in simulation. In Springer Handbook on Uncertainty Quantification. Edited by Roger Ghanem, David Higdon and Houman Owhadi. Cham: Springer, pp. 2001–38. [Google Scholar]
- Bayarri, Maria Jesus, James O. Berger, Rui Paulo, Jerry Sacks, John A. Cafeo, James Cavendish, Chin-Hsu Lin, and Jian Tu. 2007. A framework for validation of computer models. Technometrics 49: 138–54. [Google Scholar] [CrossRef]
- Berger, James O. 1985. Statistical Decision Theory and Bayesian Analysis, 2nd ed. New York: Springer. [Google Scholar]
- Bernardo, Jose Maria, and Adrian Frederick Melhuish Smith. 1994. Bayesian Theory. London: Wiley. [Google Scholar]
- Blin, David, Fabrice Casciani, Pierre Imbert, Benjamin Mousseau, Alberto Pasanisi, Pascal Terrien, and Pablo Viejo. 2015. A software platform to help Singapore to build a more smart and sustainable city. Paper presented at Energy Science Technology Conference, Karlsruhe, Germany, May 20–22. [Google Scholar]
- Bontemps, Stéphanie. 2015. Empirical Validation of Models: Application to Low-Energy Buildings. Ph.D. thesis, HESAM University, Paris, France. (In French). [Google Scholar]
- Bontemps, Stéphanie, Aurélie Kaemmerlen, Rémi Le Berre, and Laurent Mora. 2013. La fiabilité d’outils de simulation thermique dynamique dans le contexte des bâtiments basse consommation. Paper presented at Congrés Français de Thermique 2013, Gerardmer, France, May 28–31. [Google Scholar]
- Campbell, Katherine. 2006. Statistical calibrations of computer simulations. Reliability Engineering & System Safety 91: 1358–63. [Google Scholar]
- Chang, Yen-Chang, and Wen-Liang Hung. 2007. LINEX Loss Functions with Applications to Determining the Optimum Process Parameters. Quality & Quantity 41: 291–301. [Google Scholar]
- Cox, Dennis D., Jeong Soo Park, and Clifford E. Singer. 2001. A statistical method for tuning a computer code to a data base. Computational Statistics and Data Analysis 37: 77–92. [Google Scholar] [CrossRef]
- Damblin, Guillaume, Merlin Keller, Pierre Barbillon, Alberto Pasanisi, and Eric Parent. 2016. Bayesian Model Selection for the Validation of Computer Codes. Quality and Reliability Engineering International 32: 2043–54. [Google Scholar] [CrossRef]
- Damblin, Guillaume, Pierre Barbillon, Merlin Keller, Alberto Pasanisi, and Eric Parent. 2018. Adaptive Numerical Designs for the Calibration of Computer Codes. SIAM/ASA Journal on Uncertainty Quantification 6: 151–79. [Google Scholar] [CrossRef][Green Version]
- Eastman, Chuck, Paul Tiecholz, Rafael Sacks, and Kathleen Liston. 2011. CBIM Handbook: A Guide to Building Information Modeling for Owners, Managers, Designers, Engineers and Contractors, 2nd ed. Hoboken: Wiley. [Google Scholar]
- Efron, Bradley. 1981. Nonparametric Estimates of Standard Error: The Jackknife, the Bootstrap and Other Methods. Biometrika 68: 589–99. [Google Scholar] [CrossRef]
- Elmqvist, Hilding. 1978. A Structured Model Language for Large Continuous Systems. Ph.D. thesis, Lund University, Lund, Sweden. [Google Scholar]
- Fonseca, Jimeno A., Ido Nevat, and Gareth W. Peters. 2020. Quantifying the uncertain effects of climate change on building energy consumption across the United States. Applied Energy 277: 115556. [Google Scholar] [CrossRef]
- French, Simon, and David Rios Insua. 2000. Statistical Decision Theory. London: Wiley. [Google Scholar]
- Gelman, Andrew, and Donald B. Rubin. 1992. Inference from Iterative Simulation Using Multiple Sequences. Statistical Science 7: 457–72. [Google Scholar] [CrossRef]
- Gelman, Andrew, Xiao-Li Meng, and Hal Stern. 1996. Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica 6: 733–807. [Google Scholar]
- Heo, Yeonsook, Ruchi Choudhary, and Godfried A. Augenbroe. 2012. Calibration of Building Energy Models for Retrofit Analysis under Uncertainty. Energy and Buildings 47: 550–60. [Google Scholar] [CrossRef]
- Kennedy, Marc C., and Anthony O’Hagan. 2001. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63: 425–64. [Google Scholar] [CrossRef]
- Liu, Fei, and Mike West. 2009. A dynamic modelling strategy for Bayesian computer model emulation. Bayesian Analysis 4: 393–412. [Google Scholar] [CrossRef]
- Mirakyan, Atom, Alexandru Nichersu, Alberto Pasanisi, Muhammad Saed, Nico Schweiger, Maria Sipowicz, and Jochen Wendel. 2015. Applied Statistics in Support of Cities Simulation: Some Examples and Perspectives. Paper presented at ENBIS-2015 Conference, Prague, Czech Republic, September 6–10. [Google Scholar]
- Pasanisi, Alberto, and Anne Dutfoy. 2012. An Industrial Viewpoint on Uncertainty Quantification in Simulation: Stakes, Methods, Tools, Examples. In Uncertainty Quantification in Scientific Computing. Edited by Andrew M. Dienstfrey and Ronald F. Boisvert. Berlin: Springer, pp. 27–45. [Google Scholar]
- Pasanisi, Alberto, and Joseph Ojalvo. 2008. A multi-criteria decision tool to improve the energy efficiency of residential buildings. Foundations of Computing and Decision Sciences 33: 71–82. [Google Scholar]
- Pasanisi, Alberto, Merlin Keller, and Eric Parent. 2012. Estimation of a quantity of interest in uncertainty analysis: Some help from Bayesian Decision Theory. Reliability Engineering & System Safety 100: 93–101. [Google Scholar]
- Plessis, Gilles, Aurélie Kaemmerlen, and Amy Lindsay. 2014. BuildSysPro: A Modelica library for modelling buildings and energy systems. Paper presented at 10th International Modelica Conference, Lund, Sweden, March 10–12. [Google Scholar]
- Rivalin, Lisa. 2016. Vers une démarche de garantie des consommations énergétiques dans les bâtiments neufs: Méthodes d’évaluation des incertitudes associées à la simulation thermique dynamique dans le processus de conception et de réalisation. Ph.D. thesis, HESAM University, Paris, France. [Google Scholar]
- Roache, Patrick J. 1998. Verification of codes and calculations. AIAA Journal 36: 696–702. [Google Scholar] [CrossRef]
- Robert, Christian P., and George Casella. 2004. Monte Carlo Statistical Methods, 2nd ed. Berlin: Springer. [Google Scholar]
- Roy, Christofer J., and William L. Oberkampf. 2011. A comprehensive framework for verification, validation and uncertainty quantification in scientific computing. Computer Methods in Applied Mechanics and Engineering 20: 2131–44. [Google Scholar] [CrossRef]
- Rysanek, Adam Martin, and Ruchi Choudhary. 2012. A decoupled whole-building simulation engine for rapid exhaustive search of low-carbon and low-energy building refurbishment options. Building and Environment 50: 21–33. [Google Scholar] [CrossRef]
- Sacks, Jerome, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. 1989. Design and analysis of computer experiments. Technometrics 31: 41–47. [Google Scholar] [CrossRef]
- Saltelli, Andrea, Karen Chan, and Evelyn Marian Scott. 2000. Sensitivity Analysis. New York: Wiley. [Google Scholar]
- Shamsi, Mohammad Haris, Usman Ali, Eleni Mangina, and James O’Donnell. 2020. A framework for uncertainty quantification in building heat demand simulations using reduced-order grey-box energy model. Applied Energy 275: 115141. [Google Scholar] [CrossRef]
- Spitz, Clara. 2012. Analyse de la fiabilité des outils de simulation et des incertitudes de métrologie appliquée à l’efficacité énergétique des bâtiments. Ph.D. thesis, Université de Grenoble, Grenoble, France. [Google Scholar]
- Tian, Wei, and Ruchi Choudhary. 2011. Energy use of buildings at urban scale: A case study of London school buildings. Paper presented at Building Simulation 2011: 12th Conference of International Building Performance Simulation Association, Sydney, Australia, November 14–16; pp. 1702–9. [Google Scholar]
- van der Vaart, Aad. 2000. Asymptotic Statistics. Cambridge: Cambridge University Press. [Google Scholar]
- Wate, Parag, Marco Iglesias, Volker Coors, and Darren Robinson. 2020. Framework for emulation and uncertainty quantification of a stochastic building performance simulator. Applied Energy 258: 11375. [Google Scholar] [CrossRef]

**Figure 1.**General layout of the EDF BESTLAB experimental platform. We will focus on the thermal behavior of one of the lower level cells.

**Figure 2.**Experimental cell under study, equipped with captors monitoring temperature, power consumption, and other variables of interest, and whose thermal behavior is modeled by the DYMOLA code.

**Figure 3.**Four month sequence acquired in the experimental cell, comprising six periods: 1. temperature maintained at 20 °C (7 days), 2. 150 °C (15 days), 3. 30 °C (29 days), 4. no air conditioning (42 days), 5. temperature maintained at 20 °C (7 days), 6. heating with a constant power of 160 W (20 days). Red: inside temperature, blue: outside temperature.

**Figure 4.**19 time-averaged power measurements from the first period of the experimental data. The time step is approximately 5 h and 30 min.

**Figure 5.**Raw outputs of the MCMC algorithm: a Markov chain approximating the joint posterior distribution. Each color corresponds to a different MCMC run.

**Figure 6.**Componentwise autocorrelation of MCMC algorithm output. Each color corresponds to a different MCMC run.

**Figure 7.**Cumulative ergodic means computed from the MCMC output. Each color corresponds to a different MCMC run.

**Figure 8.**From left to right: marginal posterior densities of $\phantom{\rule{0.166667em}{0ex}}{\theta}_{1},\phantom{\rule{0.166667em}{0ex}}{\theta}_{2}$ and ${\theta}_{3}$.

**Figure 9.**Measurements versus calibrated code predictions of the electric power delivered inside the cell at each time step of the period.

**Figure 10.**Probability distribution of the averaged electric power (kWh) delivered inside the cell over the time period.

**Figure 11.**Left: Histogram of the posterior sample of ${p}_{val}(\mathit{\theta},{\lambda}^{2},\mathbf{Z})$: the posterior p-value is estimated by the average over this sample. Right: Scatterplot of predictive vs. realized ${\mathcal{X}}^{2}$ discrepancies: the posterior p-value is estimated by the proportion of points above the $y=x$ line.

**Figure 12.**(

**Left**): posterior median (solid line) and 95% credible intervals of the normal QQ-plot. (

**Right**): Posterior distribution of code outputs versus observed values.

**Figure 13.**Expected utilities as functions of the fixed contract fee d, for different defection rates c.

**Table 1.**Acceptance rates for the component-wise random-walk Metropolis–Hastings algorithm. Each line corresponds to a different MCMC run.

Chain | Parameter | ${\mathit{\theta}}_{1}$ | ${\mathit{\theta}}_{2}$ | ${\mathit{\theta}}_{3}$ |
---|---|---|---|---|

1 | Acceptance rate | $21\%$ | $20\%$ | $52\%$ |

2 | Acceptance rate | $26\%$ | $71\%$ | $58\%$ |

3 | Acceptance rate | $23\%$ | $70\%$ | $55\%$ |

Parameter | ${\mathit{\theta}}_{1}$ | ${\mathit{\theta}}_{2}$ | ${\mathit{\theta}}_{3}$ |
---|---|---|---|

Gelman-Rubin test | 1.014 | 1.017 | 1.018 |

**Table 3.**Marginal posterior laws characteristics: central values, dispersion metric, and $95\%$ bilateral credible interval (lower/upper credible levels).

Parameter | Mean | Median | LCL | UCL |
---|---|---|---|---|

${\theta}_{1}$ | 0.203 | 0.204 | 0.181 | 0.222 |

${\theta}_{2}$ | 3.200 | 3.205 | 2.997 | 3.377 |

${\theta}_{3}$ | 39.960 | 36.446 | 24.884 | 75.378 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Keller, M.; Damblin, G.; Pasanisi, A.; Schumann, M.; Barbillon, P.; Ruggeri, F.; Parent, E.
Validation of a Computer Code for the Energy Consumption of a Building, with Application to Optimal Electric Bill Pricing. *Econometrics* **2022**, *10*, 34.
https://doi.org/10.3390/econometrics10040034

**AMA Style**

Keller M, Damblin G, Pasanisi A, Schumann M, Barbillon P, Ruggeri F, Parent E.
Validation of a Computer Code for the Energy Consumption of a Building, with Application to Optimal Electric Bill Pricing. *Econometrics*. 2022; 10(4):34.
https://doi.org/10.3390/econometrics10040034

**Chicago/Turabian Style**

Keller, Merlin, Guillaume Damblin, Alberto Pasanisi, Mathieu Schumann, Pierre Barbillon, Fabrizio Ruggeri, and Eric Parent.
2022. "Validation of a Computer Code for the Energy Consumption of a Building, with Application to Optimal Electric Bill Pricing" *Econometrics* 10, no. 4: 34.
https://doi.org/10.3390/econometrics10040034