Acceleration of Nuclear Reactor Simulation and Uncertainty Quantification Using Low-Precision Arithmetic

Abstract

In recent years, interest in approximate computing has been increasing significantly in many disciplines in the context of saving energy and computation cost by trading off on the quality of numerical simulation. The hardware acceleration based on the low-precision floating-point arithmetic is anticipated by the upcoming generation of microprocessors and code compilers and has already proven to be beneficial for weather and climate modelling and neural network training. The present work illustrates the application of low-precision arithmetic for the nuclear reactor core uncertainty analysis. We studied the performance of an elementary transient reactor core model for the arbitrary precision of the floating-point multiplication in a direct linear system solver. Using this model, we calculated the reactor core transients initiated by the control rod ejection taking into account the uncertainty of the model input parameters. Then, we evaluated the round-off errors of the model outputs for different precision levels. The comparison of the round-off errors and the model uncertainty showed the model could be run using a 15-bit floating-point number precision with an acceptable degradation of the result’s accuracy. This precision corresponds to a gain of about 6× in the bit complexity of the linear system solution algorithm, which can be actualized in terms of reduced energy costs on low-precision hardware.

1. Introduction

Progress in nuclear power technology heavily depends on the expansion of computing performance. The cutting-edge multiphysics [1,2] and multiscale frameworks [3,4,5], designed to provide more realistic predictions of a reactor core’s safety parameters and to unveil the fundamental mechanisms of nuclear fuel behaviour beyond heuristic correlations, require higher-than-ever computing resources. Over decades, the advance in computing performance has been achieved through transistor miniaturization, which translates into the improvement of the processor design, clock rate speed-up and the high-density multicore architecture. However, nowadays the processor architecture scale is close to its physical limit and it becomes more difficult to shrink [6]. New computing architectures and models will thus be required to continue gaining improvements in chip performance. Optical computing [7], DNA computing [8], coupled oscillating networks [9], approximate computing [10], quantum computing [11] and analogue–digital in-memory computing [12,13] have become important areas of exploration. Another big advance in computing performance is envisioned with the use of traditional computing hardware accelerated for specific engineering tasks. The hardware acceleration of nuclear engineering applications can rely on machine learning methods and approximate computing techniques aimed to trade off the computational quality with speed/energy efficiency.

One of the most promising strategies of approximate computing is the use of a smaller numerical precision in floating-point operations. This idea is motivated by the fact that a high precision of computing hardware is often excessive considering a much lower accuracy is required in practice for engineering applications. Moreover, the solution accuracy is usually limited by physical approximations, numerical errors and the uncertainty of the input data. Therefore, the computing requirement can be decreased in order to redirect the resources into higher spatial, energy or angular resolution or simply to improve the statistics in simulations. This idea has proven to be beneficial for several applications of ocean, weather and climate modelling [14,15] as well as for neural networks training [16]. For a full survey of approximate computing techniques, including their challenges and obstacles, one can read [10].

The usage of a lower precision provides three advantages that can make computation faster. Firstly, a floating-point unit processor requires less work per operation. As an example, switching from double to half precision reduces complexity almost 9 times assuming the best-known Harvey–Hoeven binary multiplication algorithm [17] of complexity $O(nlogn)$ and 14 times for the Karatsuba algorithm [18] of complexity $O\left(n^{1.585}\right)$. Secondly, more data can fit in a chip’s cache allowing to speed up memory-bound computations, whose performance depends primarily on the time cost to access memory storing working data. The larger the cache size, the greater number of accesses to the cache that actually find data, hence, the average memory access time becomes less. Empirical studies [19] have shown that the cache miss rate decreases as a power law of the cache size, where the power lies between −0.3 and −0.7, which is equivalent to decreasing by a factor of two if the precision switches from double to half. Thirdly, the lower precision facilitates the communication between parallel processors, whose impairment can seriously slow down computations [20]. Anticipating all these advantages, the low- and multiprecision architectures are now considered as prospective techniques for the acceleration of large-scale applications on the forthcoming high-performance computer systems.

In this paper, we investigated the advantage of low-precision arithmetic for nuclear engineering applications. In nuclear safety analysis, the traditional conservative methods and hypotheses have gradually been replaced with the best estimate plus uncertainty (BEPU) approach [21] meaning that one should produce a series of high-fidelity simulations to quantify the best estimate vector and its uncertainty, for example, by means of the random sampling method. Uncertainty quantification (UQ) is a computationally very expensive part of the BEPU approach as it implies the multiple execution of a high-fidelity code to propagate the uncertainty from the model’s input parameters and the nuclear data to the responses of interest. To address this issue, researchers have developed many acceleration techniques, such as reduced-order modelling [22,23,24], neural networks [25] and automatic differentiation [26] to setup a fast-working heuristic model and bootstrap the best-estimate solution and its variance on the fly. Although reduced order modelling acceleration have achieved a high level of maturity, the simulation of representative samples to train the heuristic model remains quite a time-consuming operation. One can note, however, that there is no much sense in producing high-fidelity simulations with a high accuracy since the uncertainty of engineering parameters likely would exceed the numerical errors by several orders of magnitude. Therefore, the simulations could be performed with a much lower accuracy as long as it did not violate the simulation’s stability and the probability distribution of the responses. This research aims to verify this hypothesis and evaluate the potential speed-up of the UQ for the reactivity-initiated accident in a pressurized water reactor (PWR) core at least for the elementary point-kinetic reactor core model with temperature feedback. Note that this study is focused on the multiprecision analysis of a reactor core simulation rather than on the programming of code on mixed-precision hardware.

The paper is structured as follows. Section 2 discusses the computing hardware that implements the arbitrary-precision arithmetic and the programming software that performs the multiprecision analysis of a computer code. Section 3 shows the results of the multi-precision analysis applied to the uncertainty quantification of a reactor core model. Section 4 contains the conclusions.

2. Multiprecision Hardware

2.1. SIMD

Switching from double to single precision on modern consumer processors does not automatically mean that the computation will be two times faster as the floating-point operation units are designed to perform optimally for 64-bit calculations. However, one can achieve a boost by a vectorization of the algorithm and utilizing SIMD (single instruction, multiple data) or SWAR (SIMD with a register) instructions supported by many modern CPUs and compilers. Notwithstanding SIMD has great savings for cryptography and the processing of 3D graphics, the implementation of a specific algorithm with SIMD is tricky and not automatized.

2.2. Half-Precision

In recent years, lower-precision multiplications accompanied by a higher-precision accumulation have proven useful in training deep neural networks [27]. Traditionally, a neural network has been trained using a single 32-bit precision data type for matrix multiplication and the convolution of vectors. However, it was recognised that half-precision hardware delivers a significant boost of speed while providing similar training quality. That motivated major hardware vendors to settle on mixed-precision hardware units to further enhance their performance [28]. Furthermore, a new floating-point format, BF16, has drawn attention as it is better suited to support deep learning tasks than the standardized FP16 variant due to the right balance between exponent and significant parts of a floating-point number [16]. The comparison of floating-point data types are given in

Table A1. Floating-point number size, bits.

	BF16	FP16	FP32	FP64
Width	16	16	32	64
Significand	8	10	23	52
Exponent	7	5	8	11
Max	3.38 $\times {10}^{+38}$	6.55 $\times {10}^{+4}$	3.40 $\times {10}^{+38}$	1.79 $\times {10}^{+308}$
Min	1.17 $\times {10}^{-38}$	6.10 $\times {10}^{-5}$	1.17 $\times {10}^{-38}$	2.22 $\times {10}^{-308}$
eps	3.06 $\times {10}^{-03}$	9.76 $\times {10}^{-4}$	1.19 $\times {10}^{-7}$	2.22 $\times {10}^{-16}$

of Appendix A.

2.3. FPGA

Notwithstanding the usefulness for neural network applications, the IEEE-754 and BF16 data formats are not necessary the best choices for any other scientific problem. In order to achieve a better optimization of computing resources for arbitrary tasks, a field-programmable gate array (FPGA) has been invented. An FPGA is an integrated circuit which can be configured for a specific functionality and/or data type as it contains an array of transistor logic elements and switchers that can be interwired in various configurations. The circuit is reconfigured using a configuration file developed on a hardware description language such as Verilog, VHDL or OpenCL. The recent Intel C++ and Fortran compilers support programming FPGA devices. Nowadays, FPGA research is an active area of computer science research [14,29] and many hardware vendors invest their efforts into the development of high-efficiency programmable chips.

2.4. Novel Data Types

A software solution is not meant to be efficient for real time execution, as it has to emulate nonstandard numbers and arithmetic on the IEEE-754 floating-point arithmetic units. To address this issue, novel floating-point data types were developed to replace the common floating-point arithmetic on a hardware level. Significance arithmetic relies on the concept of significant digits that carry meaning, contributing to a number, and insignificant digits used for the approximation of numerical errors. Significance arithmetic was extended by [30] to unums and posits which are based on a variable-width storage format for the significand and exponent potentially help to minimize the number of bits used. However, there are several obstacles for the computation in the posit space, including the change of bit length of the number structure and concerns about the numerical stability of algorithms. Another approach is the floating-point adaptive noise reduction (FP-ANR) format suggested by [31]. FP-ANR is compatible with the existing floating-point representation format as it replaces the uninformative bits of the mantissa by a self-detectable pattern. Scanning the mantissa from right to left, one can deduce the number of cancelled bits without using extra memory fields. To conclude, there is a wide range of software solutions emulating the new formats and a short list of implementations on FPGA [32].

2.5. Multiprecision Emulators

The straightforward execution of a piece of code on low-precision hardware likely will produce unacceptable results due to the quality loss or iteration divergence. Therefore, it is useful to experiment with different precision levels to understand the performance of a numerical model prior the porting it onto specialized hardware. To address this goal, a variety of programming tools have been developed to be able to propagate the round-off errors, tune the floating-point numbers’ precision and produce a mixed-precision version of a specific piece of code. One can find a comprehensive review on multiprecision analysis in [33].

A reduced-precision emulator (RPE) [34] provides the capability to emulate calculations in arbitrary reduced floating-point precision. The software program is implemented as a Fortran module that can be included into existing projects. The module defines a new derived type “rpe-var” that emulates a reduced-precision variable and can replace Fortran’s built-in real type in the user’s code. The module provides an overloaded assignment, arithmetic and logic operators that accept “rpe-var” instances as inputs, along with overloaded versions of many of Fortran’s intrinsic functions. The reduction of the precision is applied after each operator or intrinsic function call, while the operations are performed using single-precision arithmetic.

2.6. Multiprecision Analysis Tools

The CADNA (Control of Accuracy and Debugging for Numerical Applications) library [35] estimates the propagation of rounding errors using discrete stochastic arithmetic (DSA). DSA executes each arithmetic operation within user code several times with a randomly chosen rounding error. Then, the statistical analysis of the obtained samples is used to evaluate the number of exact significant digits of the computed result. In addition, the statistical analysis allows one to detect a few types of numerical instabilities during the code execution: nonsignificant operands of multiplication, division or a mathematical function; indeterminism in a branching test; and the sudden loss of accuracy on an addition or a subtraction. The stochastic operations are implemented by overloading the arithmetic operators for stochastic types. The library is designed for C/C++ and Fortran applications

3. Numerical Results

For the numerical illustration, we considered an elementary reactor core model based on the point kinetics equations and the lumped-parameter system of fuel and coolant temperatures; the mathematical formulation of the model is presented in Appendix B. Using this model, we simulated several hot zero power transients initiated by the ejection of control rods worth 1, 1.2, 1.5 and 2 dollars (1$ is equal to one effective delayed neutron fraction). The excessive reactivity was injected gradually over 10 ms and then remained unchanged.

The numerical scheme employed a second-order multistep backward differentiation formula (BDF) method with a variable step size to achieve better performance. The step size selection algorithm benefited Nordsik’s formulation and minimized the local truncation error with respect to the given absolute and relative tolerances. The fixed-point iteration method solved the nonlinear system of equations at every time step. The linear system of equations was resolved by a direct LU decomposition algorithm [36]. The description of the numerical scheme is presented in Appendix C.

The presented reactor core model was analysed as follows

• Code profiling;
• Uncertainty propagation;
• Multiprecision analysis;
• Multiprecision emulation;
• Speed-up evaluation.

The first step evaluated the computational time utilized by each part of the code and determined the bottleneck routine; in our example, the most demanding routine was the matrix–vector multiplication performed for the solution of the in-step iteration. The second step propagated the model uncertainty by a random sampling of the input data. The third step propagated the rounding error based on the given uncertainty of the code inputs and evaluated the minimum required precision of every arithmetic operation. The fourth step verified the simulation stability and accuracy; in this step, we evaluated the actual rounding error that must be small compared to the model’s uncertainty. Finally, the fifth step was to compare the performance of the two code versions: (a) the full-precision version that was run on a CPU and (b) the low-precision version programmed on an FPGA. In this work, however, we did not program the low-precision hardware component; instead, we evaluated the potential speed-up in terms of the bit complexity.

3.1. Code Profiling

The model was coded in Fortran and compiled by GCC 8.2.0 using a double precision for all variables. The time spent by each routine was obtained at runtime by the performance analysis tool “GProf” [37]. As was expected, the solution of the in-step linear system $(I-hA)x=b$, where h is the time step, by the LU decomposition was the most demanding routine of the code accounting for 95% of the runtime; of that, 85% was devoted to the floating-point operations ×, ÷ and ±, 5% on the cycle loops and 5% on the variable assignment.

The listing of the LU routine is presented in

Table 1. Arithmetic complexity of the LU algorithm applied to the linear system of equations $(I-hA)x=b$ of dimension $n=9$.

		Arithmetic Complexity
#	Code Listing	%	FLOP
01:	do j = 1, n
02:	do i = 1, j
03:	do k = 1, i − 1
04:	A (i, j) = A (i, j) − A (i, k) ∗ A (k, j)	38	$n(n^2-1)/6$
05:	do i = j + 1, n
06:	S = A (i, j)
07:	do k = 1, j − 1
08:	S = S − A (i, k) ∗ A (k, j)	26	$n(n-1)(n-2)/6$
09:	A (i, j) = S / (A (j, j) − 1/h)	10	$n(n-1)/2$
10:	do i = 1, n
11:	do k = 1, i − 1
12:	b(i) = b(i) − A (i, k) ∗ b(k)	10	$n(n-1)/2$
13:	do i = n, 1, −1
14:	S = −b (i) / h	3	n
15:	do k = i + 1, n
16:	S = S − A (i, k) ∗ b (k)	10	$n(n-1)/2$
17:	b (i) = S / (A (i,i) − 1/h)	3	n

; the arithmetic complexity was evaluated in percent for a linear system of dimension $n=9$. The algorithm operated over $n^2+n+2$ floating-point variables contained in input matrix A, input vector b, input parameter h and additional parameter S used for the summation. The entries of the matrix and the vector were changed during the algorithm run. At the end of the execution, vector b contained the final result and matrix A consisted of the lower and upper triangular matrices of the input matrix; the diagonal elements of A referred to matrix U, assuming the diagonal entries of L were filled with one.

3.2. Uncertainty Propagation

The model uncertainty given in

Table A3. Parameter values and uncertainties.

Par	Units	Value	$3\mathit{\sigma },\%$	Description
$P_0$	W	0.1	${10}^{-2}$	Initial power
$l_0$	$\mu$s	10	${10}^{-2}$	Mean generation time
$M_f$	t	95	${10}^{-2}$	Fuel mass
$M_c$	t	15	${10}^{-2}$	Coolant mass
$C_{pf}$	J/kg K	260	6	Fuel heat capacity
$C_{pc}$	J/kg K	5400	6	Coolant heat capacity
$a_f$	pcm/K	−5	9	Doppler coefficient
$a_c$	pcm/K	1	9	Coolant total temp/dens coefficient
$a_h$	MW/K	5.4	9	Heat transfer × surface
$T_{in}$	K	566	${10}^{-2}$	Inlet temperature
v	m/s	5	${10}^{-2}$	Coolant velocity

of Appendix B was propagated by a run of the 10,000 transients with randomly perturbed inputs. The random samples were generated by the Latin hypercube method using the experimental design package for Python “pyDOE” [38]; this method ensured that the set of randomly generated inputs was a good representative of the real variability of the model input parameters. The variation of the reactor core reactivity $\rho$, power P, fuel temperature $T_f$ and coolant temperature $T_c$ for all reactivity perturbations is presented in Figure 1 as a function of time.

3.3. Multiprecision Analysis

This section presents the multiprecision analysis of the LU algorithm shown in Table 1. The multiprecision analysis aimed to evaluate the minimum precision of A and b entries that was required to propagate the code’s input data without information loss; in general, the larger the uncertainty of the inputs, the smaller the required precision. In order to estimate the exact number of bits and/or digits of the variables of merit, we replaced all the floating-point variables in our code with the stochastic data type provided by CADNA software. The required precision was evaluated for all the variables of the LU algorithm at each time-step iteration.

Figure 2 shows the results of the CADNA analysis applied to matrix A and vector b. At the beginning, matrix $I-hA$ defined the system of neutron kinetics and lumped temperature equations coupled by means of the reactor core power; therefore, most entries of the input matrix were equal to zero. The LU algorithm gradually changed the entries of matrix A; however, 21 entries were still kept equal to zero for all iterations due to the input sparse structure. In order to simplify the multiprecision analysis, these entries were excluded from consideration.

The division operation on the lines 9 and 17 of the LU algorithm in Table 1 required special treatment. If the time step h is small, then denominator $1-hx$ has to preserve a much higher precision than term $hx$. In order to reduce the complexity of this operation, one should hardcode the function $f\left(x\right)=1/(1-x)$. Note, this function can be resolved by the Newton–Raphson iterative method with the on-the-fly precision correction algorithm proposed by [39]. In our implementation, we computed the division function using double-precision arithmetic; however, the argument of this function was given in the reduced precision.

Figure 2 shows the precision of A and b varied from two to six decimal digits, which is equivalent to 7 and 20 bits, respectively. The digits referred to the decimal representation of a floating-point number as the binary representation was not available in the CADNA version used. The number of digits varied as a function of time and reactivity as can be seen in Figure 3. One can notice the correlation between the required precision and the time-step history presented in Figure A1 of Appendix B; the smaller the time step used, the higher the precision required. The step size was a function of the local truncation error evaluated by the BDF as the discrepancy between corrector and predictor vectors. The correlation between the required precision and the local truncation error might be useful for a further optimization of the algorithm.

3.4. Multiprecision Emulation

In this section we analyse the stability and accuracy of the low-precision version of the code using RPE [34]. We assumed that the emulator assigned the same number of significant bits for all variables of merit; otherwise, one would have to develop a floating-point arithmetic unit for varied precision, which can be unstable and too complex. The multiprecision analysis provided by CADNA showed that a 17-bit precision arithmetic was required to guarantee that the accuracy of the LU algorithm was enough to propagate the input uncertainty. Firstly, we tested the code for this guaranteed precision; however, the actual precision requirements might have been even lower due to the inaccuracy of the DSA estimation method, and also because the precision estimate varied in time and depended on the index of the matrix entry. Therefore, it was useful to run the code several times while making the precision lower and lower until either the rounding error became too large, or simulation became unstable. In the next paragraph, we show how these criteria were met for the reactor core’s transient model.

The RPE emulation aimed to confirm that the rounding error of the low-precision simulations did not exceed the uncertainty originated from the model’s input. Hence, we calculated and compared two data sets referring to (a) the full-precision simulations produced by the random sampling and (b) the full-to-reduced precision simulation discrepancy produced for the same random samples. Figure 4 shows the full-to-reduced discrepancy in the peaking power $P_{max}$, the total energy release $P_{tot}$ and the fuel and coolant temperatures at specific time moments. The rounding error computed for the 15-bit precision was comparable to the variation of the parameters of merit due to the uncertainty of the input data. A further drop in precision led to a much higher scope of the rounding errors, and at some point, it resulted in an unstable simulation resulting in a split of the cloud of points.

3.5. Speed-Up Evaluation

In this section, we estimate the potential speed-up of the reduced-precision version of the code in terms of the bit complexity. Figure 4 shows that simulations could be done with an acceptable degradation of the accuracy using 15-bit instead of 52-bit precision for the LU algorithm Table 1. The algorithm performed $\frac{n^3}{3}+\frac{n^2}{2}-\frac{5n}{6}$ multiplications and $\frac{n(n+1)}{2}$ divisions, where $n=9$ was the rank of matrix A. In our implementation, the reduced-precision code computed the divisions using double-precision arithmetic. Hence, we evaluated the speed-up accounting for the multiplications only, which was about 86% of the floating-point operations.

Table A2. Bit complexity gain of multiplication of two floating-point numbers with n-bit significance compared to the multiplication of two floating-point numbers with 52-bit significance (FP64).

	Long Multiplication	Karatsuba [18]	Harvey–Hoeven [17]
Precision n	$\mathit{O}\left({\mathit{n}}^2\right)$	$\mathit{O}\left({\mathit{n}}^{1.585}\right)$	$\mathit{O}\left(\mathit{n}\mathbf{log}\right(\mathit{n}\left)\right)$
11	22	12	7
13	16	9	6
15	12	7	4
17	9	6	3
19	7	5	3
21	6	4	2

shows the gain in the bit complexity of the multiplication function that uses the operands of the reduced precision compared to the double precision. As can be seen, one could reach a 4× to 12× gain in the bit complexity depending on the multiplication algorithm. For example, for the Karatsuba method, the potential speed-up of our algorithm was about six times. Moreover, the use of the reduced precision required four times less inner iterations than that for 52 bits. As a result, the reduced-precision version of the code could potentially be accelerated by about 24 times.

4. Conclusions

The numerical precision of many engineering applications is often excessive in view of the model approximation and input uncertainty. The mixed-precision technique tries to resolve this maladjustment and optimizes the exploitation of the computational infrastructure to make computations faster or more energy efficient. This work presented a brief overview of the existing multiprecision methods, hardware and software; it discussed the status and prospects of half-precision computing, FPGA chips and the novel floating-point number formats. The multiprecision analysis was illustrated for an elementary reactor core model and the uncertainty quantification of the transient initiated by the insertion of reactivity. The analysis performed using RPE and CADNA codes allows us to conclude that there is a large room for the acceleration of nuclear engineering applications on mixed-precision hardware. Future work will be focused on the multiprecision analysis of spatial reactor core models.