A Deficiency of the Weighted Sample Average Approximation (wSAA) Framework: Unveiling the Gap between Data-Driven Policies and Oracles

Abstract

This paper critically examines the weighted sample average approximation (wSAA) framework, a widely used approach in prescriptive analytics for managing uncertain optimization problems featuring non-linear objectives. Our research pinpoints a key deficiency of the wSAA framework: when data samples are limited, the minimum relative regret—the discrepancy between the expected optimal profit realized by an oracle aware of the genuine distribution, and the maximum expected out-of-sample profit garnered by the data-driven policy, normalized by the former profit—can approach towards one. To validate this assertion, we scrutinize two distinct contextual stochastic optimization problems—the production decision-making problem and the ship maintenance optimization problem—within the wSAA framework. Our study exposes a potential deficiency of the wSAA framework: its decision performance markedly deviates from the full-information optimal solution under limited data samples. This finding offers valuable insights to both researchers and practitioners employing the wSAA framework.

1. Introduction

In practical optimization problems, uncertainty is a pervasive feature—encountered in diverse scenarios such as uncertain travel durations in routing problems and unpredictable future demand in revenue management [1]. Two distinct methodologies exist for tackling optimization under uncertainty, primarily differentiated by their reliance on data [2].

The first category, which includes stochastic optimization [3] and robust optimization [4,5], does not treat data as a primary component. These methods typically predefine distributions for uncertain parameters without using empirical data, which can lead to inaccuracies in distribution specifications and subsequently, suboptimal decisions [6]. On the other hand, the rapid advancement of internet technologies, facilitating the collection and storage of vast quantities of data, has given birth to a second category—using data to characterize uncertainty.

This second category further branches out based on the nature of the data utilized: historical data of uncertain parameters or auxiliary data (contextual covariates) that assist in predicting these uncertain parameters. One subset of this category models scenarios or distributions of uncertain parameters using historical data, without taking into account any useful auxiliary data. Techniques such as the sample average approximation (SAA) method [7] and the data-driven distributionally robust optimization framework exemplify this subcategory [8,9].

The other subset capitalizes on machine learning (ML) techniques to predict uncertain parameters, integrating both historical data and relavant auxiliary data. This integration of ML techniques with mathematical optimization models is termed prescriptive analytics. The most commonly used method in this domain is the two-stage predict-then-optimize (PTO) framework. Here, ML models are used to obtain point predictions using available auxiliary data, effectively transforming uncertain problems into simpler and deterministic problems. However, the PTO framework may not be apt for optimization problems in which the objective functions are not linear in their uncertain parameters, as stochastic problems may involve complex integrals with respect to the conditional distribution model [1,6,10,11]. In such instances, the estimate-then-optimize (ETO) methods often yield superior decisions compared to PTO methods. ETO methods first estimate the distributions of uncertain parameters and then solve the resulting contextual stochastic optimization (CSO) problems. A prevalent non-parametric ETO method is the weighted SAA (wSAA) method, which estimates the conditional distribution by assigning weights to historical data, with the weights determined by the covariates [6,10].

Within the context of the wSAA framework, this paper scrutinizes its decision-making performance based on the minimum relative regret metric, defined as the difference between the expected optimal profit (benefits) of an oracle possessing full information (the actual distribution) and the maximum expected out-of-sample profit realized by the data-driven policy, normalized by the former profit.

Related Literature and Contributions

Our work contributes to two key areas of literature. Firstly, it relates to prescriptive analytics frameworks, in which decision-makers leverage contextual data about uncertainty to make informed decisions [1,2,11,12,13,14]. Several advanced frameworks have been developed, including the smart predict-then-optimize framework [13,15], the wSAA framework [10,14], the empirical risk minimization framework [12,14], and the kernel optimization framework [2,12,14,16].

Among these frameworks, our focus lies on the wSAA framework, which predicts the distributions of uncertain parameters by assigning weights to empirical data [10,12]. One of its strengths lies in its ability to reveal all necessary information for solving the stochastic problem when a perfect distributional prediction of the underlying conditional distribution can be achieved [6]. Bertsimas and Kallus [10] construct weights based on various ML models, such as kernel density estimation, k-nearest-neighbors (kNN) regression, and tree-based methods (i.e., classification and regression trees and random forest). Bertsimas and Kallus [10] demonstrate that the resulting wSAA problem can be solved in polynomial time given certain conditions (Theorem 2 in Bertsimas and Kallus [10]). They also demonstrate the asymptotic optimality of this framework—whether the data-driven decision converges to the full-information optimal solution as the data size grows to infinity. Ban and Rudin [12] propose a kernel optimization method to solve the feature-based newsvendor problem, in which they assign weights to each historical sample by the Nadaraya–Watson estimator [17,18]. They also show the generalization bounds—how well the performance of a prediction model, fitted on finite training data, generalizes out of sample [6,19]. Kallus and Mao [11] further learn effective forest-based CSO policies that integrate prediction and optimization, in which they also reweight each historical sample and solve the weighted sample analogue of the original CSO problem. For detailed reviews of corresponding prescriptive analytics frameworks for CSO problems, we refer to Tian et al. [1] and Qi and Shen [6].

Secondly, our work contributes to the literature that explores the influence of data size on data-driven decisions. Much of this literature focuses solely on previous samples of uncertainty, not considering contextual covariates. Our main idea is closely aligned with a recent study by Besbes and Mouchtaki [20], which presents a surprising finding: the worst-case relative regret of the SAA method is not monotonically related to the number of available samples, implying an influx of information could potentially harm decision quality. In contrast, our work examines the decision performance of the wSAA framework across a limited data size.

To achieve this aim, we show two CSO problems with non-linear maximization objectives. We examine the minimum relative regret—the difference between the expected optimal profit of the oracle and the maximum expected out-of-sample profit incurred by the data-driven policy, normalized by the former profit—of the wSAA framework with limited data samples. Our analysis reveals the following intriguing finding:

• The minimum relative regret of the wSAA framework can approach one with limited data samples.

The value of one in this context indicates a total divergence between the expected profit from the oracle and the maximum expected profit obtained by the data-driven policy under the wSAA framework. That is to say, when the relative regret equals one, the data-driven policy under the wSAA framework performs as poorly as possible compared to the oracle. This interpretation underlines a considerable disparity between the decision-making efficacy of the wSAA framework and that of an oracle. Consequently, the primary contribution of our work lies in highlighting that the wSAA framework may not always effectively manage CSO problems when data samples are limited. We illustrate this observation through two straightforward examples, thus paving the way for future research to potentially attain optimal decision performance of the wSAA framework, contingent upon data size.

The remainder of this paper is organized as follows. In Section 2, we introduce the wSAA framework mathematically. In Section 3, we present our findings through two CSO problems. Section 4 concludes and outlines future research directions.

2. Methodology

Consider a stochastic optimization problem with a given profit function $c(\mathit{y},\mathit{z})$, where $\mathit{y}\in \mathcal{Y}\subset {\mathbb{R}}^{d_y}$ are the random variables that affect the value of the profit function, and $\mathit{z}\in \mathcal{Z}\subset {\mathbb{R}}^{d_z}$ are the decision variables that are subject to a set of constraints which define the feasible region within which solutions can exist. Let $\mathit{x}\in \mathcal{X}\subset {\mathbb{R}}^{d_x}$ denote the contextual features that are related to the distribution of $\mathit{y}$. Suppose we have a new observation of the contextual data, denoted as ${\mathit{x}}_0\in \mathcal{X}\subset {\mathbb{R}}^{d_x}$. The CSO problem can be mathematically formulated as:

$${\mathit{z}}^{*}\left({\mathit{x}}_0\right)\in arg\underset{\mathit{z}\in \mathcal{Z}}{max}{\mathbb{E}}_{\mathit{y}}\left[c(\mathit{y},\mathit{z})|\mathit{x}={\mathit{x}}_0\right],$$

(1)

where ${\mathit{z}}^{*}\left({\mathit{x}}_0\right)$ denotes the expected optimal decision (oracle) under the given context ${\mathit{x}}_0$.

A historical dataset ${\left\{({\mathit{x}}_i,{\mathit{y}}_i)\right\}}_{i=1}^N$ is available to solve the optimization problem (1), where ${\mathit{x}}_i$ and ${\mathit{y}}_i$ represent the historical auxiliary data of $\mathit{x}$ and the historical realization of $\mathit{y}$ for the ith sample, respectively. To solve (1), we can adopt various ML techniques, that is, we leverage the data on $\mathit{x}$ to predict the value of $\mathit{y}$ given $\mathit{x}={\mathit{x}}_0$. Under mild conditions, and with infinite data, we can obtain the point prediction $\widehat{\mathit{y}}=\mathbb{E}\left[\mathit{y}\right|\mathit{x}={\mathit{x}}_0]$. If $c(\mathit{y},\mathit{z})$ is linear in $\mathit{y}$, we can see from

$$\mathbb{E}\left[c(\mathit{y},\mathit{z})|\mathit{x}={\mathit{x}}_0\right]=c\left(\mathbb{E}\left[\mathit{y}\right|\mathit{x}={\mathit{x}}_0],\mathit{z}\right),$$

(2)

that the point prediction $\widehat{\mathit{y}}$ is appropriate to be incorporated into the linear objective function to derive decisions. However, (2) does not hold when $c(\mathit{y},\mathit{z})$ is not linear in $\mathit{y}$. In this case, we thus need to estimate the conditional distribution of $\mathit{y}$ given $\mathit{x}={\mathit{x}}_0$. Several ways can be used to estimate this conditional distribution using ${\left\{({\mathit{x}}_i,{\mathit{y}}_i)\right\}}_{i=1}^N$ and approximate (1). This paper focuses on one possible approach termed the wSAA framework, motivated by a string of non-parametric ML models [10,14], such as kNN and decision trees [10]. The wSAA framework approximate (1) as follows:

$${\widehat{\mathit{z}}}_N^{\mathrm{wSAA}}\left({\mathit{x}}_0\right)\in arg\underset{\mathit{z}\in \mathcal{Z}}{max}\sum _{i=1}^N{w}_i\left({\mathit{x}}_0\right)c({\mathit{y}}_i,\mathit{z}),$$

(3)

where ${\widehat{\mathit{z}}}_N^{\mathrm{wSAA}}\left({\mathit{x}}_0\right)$ denotes the predictive prescriptions and $w_i\left({\mathit{x}}_0\right)$ are known as weight functions [10] that measure the distances between the new observation ${\mathit{x}}_0$ and historical covariates ${\mathit{x}}_i(i\in \{1,\dots ,N\})$; there are various ways to determine $w_i\left({\mathit{x}}_0\right)$. As long as the weights are nonnegative and sum to one, they can be seen as the estimated conditional distribution of $\mathit{y}$ given $\mathit{x}={\mathit{x}}_0$ [10]. Notably, point prediction can be seen as a specific case of the distributional prediction, where the predicted distribution has all its mass at a single point.

For instance, considering the kNN method, $w_i\left({\mathit{x}}_0\right)$ can be defined as:

$$w_i^{k\mathrm{NN}}\left({\mathit{x}}_0\right)=\frac{1}{k}\mathbb{I}\left[{\mathit{x}}_i\mathrm{is}\mathrm{a}k\mathrm{NN}\mathrm{of}{\mathit{x}}_0\right].$$

(4)

Accordingly, (1) can be reformulated as:

$${\widehat{\mathit{z}}}_N^{k\mathrm{NN}}\left({\mathit{x}}_0\right)\in arg\underset{\mathit{z}\in \mathcal{Z}}{max}\sum _{i\in {\mathcal{N}}_k\left({\mathit{x}}_0\right)}\frac{1}{k}c({\mathit{y}}_i,\mathit{z}),$$

(5)

where ${\mathcal{N}}_k\left({\mathit{x}}_0\right)=\{i=1,\dots ,N:{\sum }_{j=1}^N\mathbb{I}\left[\parallel {\mathit{x}}_0-{\mathit{x}}^i\parallel \ge \parallel {\mathit{x}}_0-{\mathit{x}}^j\parallel \right]\le k\}$ is the set of k historical data samples that are the closest to ${\mathit{x}}_0$. Note that k is a hyperparameter in kNN regression, and there are multiple methods for measuring the distance between data points in kNN, such as Euclidean distance, Mantattan distance, and Minkowski distance.

Our interest lies in the performance of the data-driven decision under the wSAA framework. To this end, we assess the performance of a data-driven decision for an out-of-sample ${\mathit{x}}_0$ through the minimum relative regret, defined as

$${\mathcal{R}}_N^{\mathrm{wSAA}}\left({\mathit{x}}_0\right):=\frac{\mathbb{E}\left[c\left(\mathit{y},{\mathit{z}}^{*}\left({\mathit{x}}_0\right)\right)\right]-\mathbb{E}\left[c\left(\mathit{y},{\widehat{\mathit{z}}}_N^{\mathrm{wSAA}}\left({\mathit{x}}_0\right)\right)\right]}{\mathbb{E}\left[c\left(\mathit{y},{\mathit{z}}^{*}\left({\mathit{x}}_0\right)\right)\right]},$$

(6)

where $\mathit{y}$ remains a random variable given ${\mathit{x}}_0$, and the expectation is taken over $\mathit{y}$ given ${\mathit{x}}_0$. Here, $\mathbb{E}\left[c\left(\mathit{y},{\mathit{z}}^{*}\left({\mathit{x}}_0\right)\right)\right]$ denotes the expected profit from the oracle, and $\mathbb{E}\left[c\left(\mathit{y},{\widehat{\mathit{z}}}_N^{\mathrm{wSAA}}\left({\mathit{x}}_0\right)\right)\right]$ denotes the maximum expected profit obtained by the wSAA decision. Observing the above equation, if the minimum relative regret approaches 1, it indicates a total divergence between the expected profit from the oracle and the profit obtained by the data-driven decision under the wSAA framework.

Notably, the wSAA framework tends to discard data that is not close to the new observation, thereby requiring a substantial amount of data, particularly data that is close to the current observation, to accurately estimate the distributions of the uncertain variables and subsequently maximize the given profit function [2]. This suggests that the wSAA framework may struggle when extrapolating to new observations that lie outside the range of known data samples. To comprehensively characterize this minimum relative regret, we design two distinct CSO problems. One problem features a new observation that is outside the range of known data samples, and the other features a new observation that is within the range of known data samples.

3. The Minimum Relative Regret of the wSAA Framework

In this section, we consider two CSO problems: the production decision-making problem (PDP), where the new observation is outside the range of known data samples, and the ship maintenance problem (SMP), where the new observation is within the range of known data samples. We use these two examples to demonstrate that the minimum relative regret of the wSAA framework can approach one when data samples are limited, no matter whether the new observation falls outside the range of known data samples or within the range of known data samples.

3.1. The Analysis of the PDP

In this section, we will analyze the PDP. Consider the demand function D, where $D:{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$, $D\left(x\right)=10x$. In this context, x represents a feature related to the demand, and $D\left(x\right)$ corresponds to the actual demand in terms of quantity. Historical data is given as two points in this demand space, $(x_1=1, y_1=10)$ and $(x_2=2, y_2=20)$, where $y_1$ and $y_2$ denote the historical realizations of demand. We consider a unit production cost of 1, a selling price per unit of 2, and the to-be-determined production quantity as z. Now, we encounter a new $x_0$, which equals a very large positive value M and thus lies outside the range of known data samples. Assuming we only have access to the two historical data points and do not know the underlying relationship (distribution) of demand given the feature, we must make the production decision. Therefore, we formulate the CSO problem with the aim of maximizing the profit (price minus cost) as follows:

$$\underset{z}{max}\mathbb{E}\left[c(\tilde{y},z)|x=x_0\right]=\underset{z\ge 0}{max}\mathbb{E}[2min\{\tilde{y},z\}-z|x=x_0],$$

(7)

where $\tilde{y}$ denotes the uncertain demand given $x_0$.

To solve this problem, we apply the wSAA framework, which assigns varying weights to samples 1 and 2. These weights are determined based on the similarity between the contextual data of the new observation and the contextual data of each historical sample. Additionally, the weights are normalized to ensure that they sum to one. We denote the weight of sample 1 as $\pi$ and the weight of sample 2 as $1-\pi$. Therefore, we need to solve the following wSAA of (7):

$$\underset{z\ge 0}{max}\pi [\underset{\mathrm{decision}\mathrm{problem}\mathrm{of}{y}_1}{\underbrace{2min\{y_1,z\}-z}}]+(1-\pi )[\underset{\mathrm{decision}\mathrm{problem}\mathrm{of}{y}_2}{\underbrace{2min\{y_2,z\}-z}}],$$

(8)

where the data-driven optimal decision is ${\widehat{z}}_{2,\mathrm{PDP}}^{\mathrm{wSAA}}\left(x_0\right)=\pi y_1+(1-\pi )y_2=20-10\pi$. Because $\pi$ can span any any value on $[0,1]$ due to the use of different weighting schemes, the problem has an infinite number of data-driven optimal solutions. Given the underlying relationship $\tilde{y}=D\left(x\right)=10x$, an upper bound of the expected profits for all these data-driven optimal solutions can be obtained as follows:

$$\begin{array}{cc}\hfill & \underset{\genfrac{}{}{0pt}{}{{\widehat{z}}_{2,\mathrm{PDP}}^{\mathrm{wSAA}}\left(x_0\right)=20-10\pi }{\pi \in [0,1]}}{max}{\mathbb{E}}_{\tilde{y}=10x}\left[c\left(\tilde{y};{\widehat{z}}_{2,\mathrm{PDP}}^{\mathrm{wSAA}}\left(x_0\right)\right)|x=x_0\right]\hfill \\ \hfill =& \underset{\pi \in [0,1]}{max}{\mathbb{E}}_{\tilde{y}=10x}\left[2min\{\tilde{y},20-10\pi \}-(20-10\pi )|x=x_0\right]\hfill \\ \hfill =& 20,\hfill \end{array}$$

(9)

where ${\pi }^{*}=0$, ${\widehat{z}}_{2,\mathrm{PDP}}^{\mathrm{wSAA},*}\left(x_0\right)=20$, and no weighting scheme can achieve an expected profit higher than 20. This result signifies that the optimal weighting scheme that can achieve the highest expected profit is to assign sample 1 a weight of 0 and sample 2 a weight of 1.

However, if we knew the underlying relationship $\tilde{y}=10x$ and given the CSO problem (7), we should solve

$$\underset{z\ge 0}{max}{\mathbb{E}}_{\tilde{y}=10x}[2min\{\tilde{y},z\}-z|x=x_0],$$

(10)

where the oracle with knowledge of the demand function should be $z^{*}\left(x_0\right)=M$, and the corresponding expected profit would be M. Therefore, the minimum relative regret of the PDP would be

$${\mathcal{R}}_{2,\mathrm{PDP}}^{\mathrm{wSAA}}\left(x_0\right):=\frac{M-20}{M}⟶1,\mathrm{when}M⟶\infty .$$

(11)

This analysis clearly shows that as M increases significantly, the minimum relative regret ${\mathcal{R}}_{2-\mathrm{PDP}}^{\mathrm{wSAA}}\left(x_0\right)$ tends towards one, highlighting the inherent limitations of using the wSAA framework when data samples are limited. We further use another problem to show that the minimum relative regret of the wSAA framework tends towards one even when the new observation lies within the range of known data samples.

3.2. The Analysis of the SMP

This section analyzes the SMP. A shipping company considers purchasing preventive maintenance services for a ship at a cost denoted by c (set to $c=100$ for this discussion). For a ship, the generated income relies heavily on its operational condition and regular maintenance, as a well-kept vessel allows for uninterrupted service leading to consistent revenue, whereas a neglected ship might face service disruptions, reducing its earning potential. The ship’s state is represented by its deficiency number, denoted by $\tilde{y}$, which ranges from 0 to Q. Specifically, the income, falling into the range of 0 to K, is assumed to be Lipschitz continuous with respect to the number of deficiencies, regardless of the maintenance decision, as depicted in Figure 1 (with $Q=20$ and $K=1000$).

Figure 1 indicates that a ship’s state significantly impacts its income. A pristine ship, with zero deficiencies, can generate a maximum income of $K=1000$, irrespective of maintenance. As deficiencies rise to $0.5Q=10$, a maintenance-less ship becomes inoperable and profitless. However, maintenance at this stage rectifies all issues, restoring maximum income generation. When deficiencies reach the peak $Q=20$, the ship becomes irreparably inoperable, making maintenance futile and income unattainable. Importantly, the cost of preventive maintenance is less than the maximum potential income. Therefore, when deficiencies reach $0.5Q=10$, maintenance becomes a financially prudent decision.

We assume that a ship’s deficiency number, $\tilde{y}$, is a function of its age, denoted by x, with the relationship defined as $\tilde{y}=x$. As such, an ageing ship accumulates more deficiencies. A brand-new ship ($x=0$) is free of deficiencies and does not need maintenance, while a ship at the end of its lifespan ($x=Q$) is highly deficient, and failures may occur regardless of maintenance attempts.

Assuming a ship has aged to $x_0=0.5Q$, uncertainty arises regarding the relationship between age and deficiency number, that is, we do not know their underlying relationship. However, employing the wSAA framework to two known data points, $(x_1=0, y_1=0)$ and $(x_2=Q, y_2=Q)$, can guide the decision of maintenance purchasing. In this case, the new observation falls within the range of known data samples. Similarly, the wSAA framework assigns weight $\pi$ to sample 1 and $1-\pi$ to sample 2 ($\pi \in [0,1]$). We denote z as the decision variable, with 1 signifying the purchase of maintenance service and 0 the opposite. By reweighting $y_1\left(\pi \right)$ and $y_2(1-\pi )$ and fixing $\pi$, we solve the following wSAA problem with the objective of maximizing profit (income minus maintenance cost):

$$\underset{z\in \{0,1\}}{max}\pi (\underset{\mathrm{decision}\mathrm{problem}\mathrm{when}{y}_1=0}{\underbrace{(1-z)K+z(K-c)}})+(1-\pi )(\underset{\mathrm{decision}\mathrm{problem}\mathrm{when}{y}_2=Q}{\underbrace{(1-z)0+z(0-c)}}),$$

(12)

where the data-driven optimal decision is ${\widehat{z}}_{2,\mathrm{SMP}}^{\mathrm{wSAA}}\left(x_0\right)=0$, which holds regardless of the value of $\pi$. Given the underlying relationship $\tilde{y}=x$, the expected number of deficiencies $\mathbb{E}\left[\tilde{y}\right|x_0=0.5Q]=0.5Q$. Therefore, without maintenance (${\widehat{z}}_{2,\mathrm{SMP}}^{\mathrm{wSAA}}\left(x_0\right)=0$) and referring to Figure 1, the expected profit from this data-driven decision is zero when the expected deficiency number is 0.5Q.

However, with knowledge of the underlying relationship $\tilde{y}=x$, the expected deficiency number $\mathbb{E}\left[\tilde{y}\right|x_0=0.5Q]=0.5Q$. Examining Figure 1, in this case, the fully informed oracle would recommend $z^{*}\left(x_0\right)=1$, yielding an expected profit of $K-c$. Consequently, the minimum relative regret of the SMP becomes:

$${\mathcal{R}}_{2,\mathrm{SMP}}^{\mathrm{wSAA}}\left(x_0\right):=\frac{K-c-0}{K-c}=1.$$

(13)

This analysis again shows that the minimum relative regret ${\mathcal{R}}_{2,\mathrm{SMP}}^{\mathrm{wSAA}}\left(x_0\right)$ equals one even when the new observation lies within the range of known data samples, further highlighting the inherent limitations of using the wSAA framework when data samples are limited.

4. Conclusions and Future Research Directions

Our study provides a focused analysis on specific examples of the wSAA framework, which is commonly used in prescriptive analytics for dealing with uncertain optimisation problems featuring non-linear objectives. We highlight a potential deficiency of the wSAA framework: when data samples are limited, the minimum relative regret—an indicator of the deviation between a data-driven policy and a full-information oracle—can approach one. This is illustrated through the analysis of two CSO problems: the PDP and the SMP. These analyses consider the wSAA framework’s different capacities for extrapolation and interpolation. The primary implication of our finding is that the wSAA framework may not perform well when data is limited. This insight is of value to both researchers and practitioners who utilize the wSAA framework in their work, providing them with an important perspective on its limitations and potential areas for improvement.

The findings of this study open up several avenues for future research. The first and most immediate would be to examine ways to mitigate the high relative regret identified in this study, especially when working with limited data samples. This could involve developing new frameworks or improving upon the existing wSAA model. Another direction could be to analyze whether the same deficiency apply to other prescriptive analytics approaches. This research could contribute to a broader understanding of how these models perform under the condition of limited data and inform the development of more robust strategies. Lastly, future research could apply our findings to more specific, practical scenarios, examining how the limitations of the wSAA framework impact decision-making in various industries. This could lead to industry-specific recommendations for enhancing the utility of prescriptive analytics approaches.