In this section, we extend the maximum entropy design to the case of multiple layers and propose the corresponding construction algorithms.
3.1. Nested Maximum Entropy Designs
Nested designs with multiple layers are usually used for multi-fidelity computer experiments [
11,
12,
22,
24,
25]. Assume that we have a computer experiment with
K levels, and the accuracy declines gradually from level 1 to level
K. For each
, the Kriging model for computer experiments at the
kth level of accuracy is
where
and
are straightforward extensions to those in (
1), and
.
is the
matrix whose
th entry is
with the known vector
.
Let a nested design
with
K layers
, where
denotes the
kth layer of the nested design for each
and
. The vector
represents the structure of
. Please note that
with smaller
k is used for computer experiments with higher accuracy, which are more important. Similar to some definitions of optimal nested designs [
11,
22], we call
a nested maximum entropy design if the following conditions hold: the first layer
is a maximum entropy design that maximizes
; for each
,
maximizes
with fixed optimized
.
By the above definition, a nested maximum entropy design can be constructed by a sequential algorithm; see Algorithm 1.
Algorithm 1 Construction of a nested maximum entropy design with K layers |
Initialization: , randomly construct the first layer . Optimize using the maximum criteria to obtain . Recursive step: for do Enlarge to , where is the kth layer of the design with the structure . Maximize the entropy corresponding to by optimizing . Output . end for
|
3.2. Multi-Layer DETMAX Algorithm
This subsection presents optimization algorithms for constructing each layer of a nested maximum entropy design.
The maximum entropy design can be obtained by a DETMAX-based algorithm [
19]. It is optimized through a series of “excursions” to improve the det(
) corresponding to the current design by adding or removing appropriate points from the current design until the det(
) for the resulting
n-point design cannot be increased. Except for the initial and final designs which have exactly
n points, the number of chosen points of any designs constructed on this excursion can be greater or less than
n. We extend this algorithm to the multi-layer case.
The flow chart in
Figure 1 describes the procedure of the multi-layer DETMAX algorithm. The layer-by-layer optimization strategy [
11,
22] is adopted here. First, the initial design of the first layer, denoted as
, is randomly generated and then optimized. If
obtained through a series of excursions meets the stopping condition, then the first layer has been completed. Subsequently, optimize the second layer with the first layer fixed. Each layer is optimized in turn until the last layer is optimized, and then output
.
We give the details for optimizing each layer in the above algorithm. Suppose we now optimize the kth layer of the design . One excursion starts with the -point design and ends when the number of points in reaches exactly again. The procedure for making excursions is described as follows. Let denote a prespecified threshold.
Step 1. Add a point at which the variance function is largest, or subtract a point corresponding to the maximum element of the diagonal of .
Step 2. The current design has points.
If , remove a point if is not in and add a point otherwise.
If , add a point if is not in and remove a point otherwise.
The new design updated by this step has points and correlation matrix .
Step 3. If , back to Step 2. Otherwise, if , stop the procedure and output ; if , do the following:
If det() < det(), let , place all the designs generated on this excursion into . Go to Step 1.
If , let = , , and . Go to Step 1.
In Step 1, the selection of whether to add or subtract a point is made randomly. The best point
is obtained by maximizing
, which is given by
where
. To determine the best site
to add to the current design, we adopt a grid search procedure [
26] for
, and the simulated annealing algorithm [
23] for
(see Algorithm 2). The point we subtract in Step 1 is always in
.
Algorithm 2 Simulated annealing in excursions |
Step 0: Input the starting temperature , the ending temperature , the length of Markov chain L, search step size , Boltzmann’s constant , reduction factor () and an initial solution , which is randomly generated in . while do for , do Step 1: , where is generated by sampling random values from . Step 2: if , then , . else Randomly generate a real number r in . if then , . else Go back to Step 1. end if end if end for Step 3: T = T. end while Step 4: Output the best solution .
|
Since the design points are bounded, the determinant of the corresponding covariance matrix is bounded. According to the monotone convergence theorem, our algorithm can converge to a limit after considerable iteration times. However, because of the nonconvex feature of the problem like other design construction problems [
27], the limit may not be the global solution. To better approximate the global solution, the above algorithm can be conducted repeatedly with several random initial designs, and the final output is the best one among the corresponding results. Several two-dimensional nested maximum entropy designs constructed by the proposed algorithm can be seen in
Figure 2.