A Semiparametric Tilt Optimality Model

Abstract

Practitioners often face the situation of comparing any set of k distributions, which may follow neither normality nor equality of variances. We propose a semiparametric model to compare those distributions using an exponential tilt method. This extends the classical analysis of variance models when all distributions are unknown by relaxing its assumptions. The proposed model is optimal when one of the distributions is known. Large-sample estimates of the model parameters are derived, and the hypotheses for the equality of the distributions are tested for one-at-a-time and simultaneous comparison cases. Real data examples from NASA meteorology experiments and social credit card limits are analyzed to illustrate our approach. The proposed approach is shown to be preferable in a simulated power comparison with existing parametric and nonparametric methods.

1. Introduction

Classical applied statistical techniques often depend heavily on the assumption that observations are normally distributed. The benefit of this assumption is that it helps to produce exact inferences in many popular methods, such as, t-test, F-test, chi-squared tests, analysis of variance (ANOVA) models and multivariate analysis. In reality, however, observations often show departures from normality (or near-normality). Statistical texts address this issue and discuss remedies, such as the Box-Cox transformations of data to normality. Developing techniques that use fewer assumptions (normality or others) is an important area of statistical research. When comparing different populations, this paper proposes an alternative to the one-way ANOVA by relaxing some of the assumptions.

Previous study described experimental radar reflectivity data obtained from independent radars deployed during NASA’s Tropical Rainfall Measuring Mission Kwajalein Experiment in the republic of the Marshall Islands on 15 July to 12 September 1999 [1]. The data are skewed, and we investigate whether the data from two radars are from identical populations. A credit limit data set [2] with three different education levels; graduate school, university and high school was obtained from the University of California Irvine repository. The data are skewed with high variability, and we studied whether the three credit limit populations are identical. Such data have unknown statistical distributions, and known statistical procedures may fail to work properly.

If $g_i\left(x\right)$ is the probability density function (pdf) of N $({\mu }_i,{\sigma }^2)$ for $i=1,\dots ,k$ [3], then one can write

$$g_i\left(x\right)=g_k\left(x\right)exp({\alpha }_i+{\beta }_{i}x),i=1,\dots ,k-1,$$

(1)

where

$${\alpha }_i=\frac{{\mu }_k^2-{\mu }_i^2}{2{\sigma }^2},{\beta }_i=\frac{{\mu }_i-{\mu }_k}{{\sigma }^2},i=1,\dots ,k-1.$$

(2)

From (1), denoting $g_k\left(x\right)$ as a reference distribution, one can think of $g_i\left(x\right)$ as an exponential distortion or tilt of the reference. Furthermore, using (2), the test of equality of ${\mu }_i$’s (as needed in the one-way ANOVA) reduces to the test of equality of the ${\beta }_i$’s to zero, or

$$H_0:{\mu }_1=\cdots ={\mu }_k⟺H_0:{\beta }_1=\cdots ={\beta }_{k-1}=0.$$

(3)

Motivated by (3), the same authors proposed a generalization of (1) by replacing $g_k\left(x\right)$ with any pdf $g\left(x\right)$, x in the exponent of (1) with any known function $h\left(x\right)$ and considered all pdfs $g_i\left(x\right)$ that can be expressed as exponential tilt pdfs of $g\left(x\right)$. In this way, (1)–(2) are updated as

$$g_i\left(x\right)=g\left(x\right)exp({\alpha }_i+{\beta }_{i}h\left(x\right)),$$

(4)

where

$${\alpha }_i=-ln\left({\int }_{-\infty }^{\infty }{e}^{{\beta }_{i}h\left(x\right)}g\left(x\right)dx\right),i=1,\dots ,k-1,$$

(5)

for some ${\mathbf{\beta }}_i\in \mathbb{R}$. Then, $H_0:{\beta }_1=\cdots ={\beta }_{k-1}=0$ can also be used to test for the equality of any k pdfs $g\left(x\right),g_i\left(x\right),1\le i\le k-1$ satisfying (4), which are not necessarily normal.

In classical ANOVA, the parameters of the normal distributions are estimated using the maximum likelihood method. To estimate the ${\beta }_i$ parameters in (4), Consider the restricted class of distributions obtained by multiplicative exponential distortions with $g_k=g$ as a reference and, based on k independent samples, used the profile maximum likelihood method to estimate $g,g_i,\forall i$ in that class [3]. This paper, instead, considers the class $\mathbb{C}$ of all distributions (which are restricted by a given mean for $h\left(X\right)$) and estimates the ${\beta }_i$ by minimizing the Kullback–Leibler divergence between $g\left(x\right)$ and the class $\mathbb{C}$.

Often the criterion of comparison between distributions may be clear from the context of the data, which helps to formulate the constraint $\mathbb{C}$. In the radar data, we are interested to know if the mean rain rate is equal in two different radars. In the credit limit data, we are interested to know if the mean credit limit is the same in three education groups. The constraints can be multiple criteria as well. Once the constraints are fixed in $\mathbb{C}$, only those aspects are considered in the comparison between distributions.

This approach matches with the maximum entropy (ME) principle, which may be stated as follows: when selecting a model for a given situation, it is often appropriate to express the prior information in terms of the constraints. However, one must be careful so that no information other than these specified constraints are used in model selection. That is, other than the constraints that we have, the uncertainty associated with the probability distribution to be selected should be kept at its maximum [4]. In this paper, we extended the ME principle to general information projection using the Kullback–Leibler divergence.

We show in Section 2 that the solution (4) is optimum under specified constraints using h when g is known and, in this way, the proposed approach yields an optimality interpretation for the exponential tilt models. The proposed approach extends the comparisons of means in ANOVA to comparisons of means and variances for the normal and other known distributions using duality. In Section 3, we develop a semi-parametric approach when $g\left(x\right)$ is unknown and derive asymptotic test statistics for testing the equality of populations for the cases when sample sizes are equal or different.

In Section 4, we present simulation studies that evaluate the performance of the ${\beta }_i$-parameters with respect to the classical ANOVA methods. We also compare the test statistics developed in Section 4 with existing parametric and nonparametric procedures. Section 5 shows the details of the proposed methods for the applications with radar data and credit limit data sets. Section 6 contains discussions on the choice of the function(s) h for particular cases of reference distributions $g\left(x\right)$. The Appendix contains additional results and proofs.

2. Tilt Optimality Models

Kullback–Liebler (KL) discriminant information or divergence is a measure of ‘distance’ between two probability distributions. For pdf’s $f\left(x\right),g\left(x\right)$, the KL discriminant information for $f\left(x\right)$ against $g\left(x\right)$ is given by ${\int }_{-\infty }^{\infty }f\left(x\right)ln\left(\frac{f\left(x\right)}{g\left(x\right)}\right)dx,$ which is always nonnegative, and $=0$ if and only if $f\equiv g$.

Let $\mathbb{C}$ be a convex set of pdf’s with $g\notin \mathbb{C}$. If $f^{*}\left(x\right)$ is the solution to

$$\underset{f\left(x\right)\in \mathbb{C}}{inf}{\int }_{-\infty }^{\infty }f\left(x\right)ln\left(\frac{f\left(x\right)}{g\left(x\right)}\right)dx$$

(6)

then $f^{*}\left(x\right)$ (information projection) is the closest to $g\left(x\right)$ among all pdf’s in $\mathbb{C}$ with respect to the KL distance. Let $h_i\left(x\right)$ be arbitrary but known functions of x. Define $\mathbb{C}$ to be the class of all pdf’s f where $E_f\left(h_i\left(X\right)\right)=0,\forall i$—that is,

$$\mathbb{C}=\left\{f\left(x\right):{\int }_{-\infty }^{\infty }{h}_i\left(x\right)f\left(x\right)dx=0,\forall i\right\}.$$

(7)

In order to solve (6), Fenchel’s duality theorem can be applied as shown by [5]. The corresponding dual cone is ${\mathbb{C}}^{*}=\{{\sum }_{i=1}^k{\beta }_i{h}_i\left(x\right):{\beta }_i\in \mathbb{R}\}.$

The dual problem can be shown to be equivalent to

$$\underset{{\beta }_i\in \mathbb{R},\forall i}{inf}{\int }_{-\infty }^{\infty }exp\left(\sum _{i=1}^k{\beta }_i{h}_i\left(x\right)\right)g\left(x\right)dx.$$

(8)

As the dual problem (8) is a function of scalars only, it could be substantially easier to solve than the primal problem (6), depending on the form of $h_i\left(x\right)$. In particular, setting the derivative of the integral quantity in (8) with respect to ${\beta }_i$ equal to zero, we obtain

$${\int }_{-\infty }^{\infty }{h}_i\left(x\right)exp\left(\sum _{i=1}^k{\beta }_i{h}_i\left(x\right)\right)g\left(x\right)dx=0,1\le i\le k,$$

(9)

which can be solved easily for ${\beta }_i$’s by the Newton–Raphson method. If the solution of (9) is ${\hat{\beta }}_i$, then, from [5], the solution of the primal problem (6), say $f^{*}\left(x\right)$, is of the form

$$f^{*}\left(x\right)=\frac{g\left(x\right)exp\left({\sum }_{i=1}^k{\hat{\beta }}_i{h}_i\left(x\right)\right)}{{\int }_{-\infty }^{\infty }g\left(x\right)exp\left({\sum }_{i=1}^k{\hat{\beta }}_i{h}_i\left(x\right)\right)dx}.$$

(10)

When $k=1$, setting $\hat{\beta }={\beta }_i,f^{*}\left(x\right)=g_i\left(x\right)$, (10) can be simplified as (4). The above derivation explains the exponential structure of $g_i\left(x\right)$, and (9) verifies that the exponential tilt model $g_i\left(x\right)$ is in $\mathbb{C}$ (see (7)). These developments are summarized in the following theorem.

Theorem 1.

When $g\left(x\right)$ is known, the exponential tilt model (4) is the optimum model in the sense that it is the closest to $g\left(x\right)$ among all probability distributions in $\mathbb{C}$ from (7) in the KL distance.

The model (10) will be referred to in the sequel as the tilt optimality (TO) model. Note that, when $\mathbb{C}$ in (7) specifies $h\left(x\right)=x-{\mu }_i$ and $g\left(x\right)=g_k\left(x\right)\sim N({\mu }_k,{\sigma }^2)$, then the solution $g_i\left(x\right)$ is found to be $N({\mu }_i,{\sigma }^2)$ as was seen in (1)–(2). When $g\left(x\right)$ in (6) is uniform (or Lebesgue measure), then the minimizing $f^{*}$ is known as the maximum entropy model (distribution) in $\mathbb{C}$ [4].

Theorem A1 in the Appendix A shows that closed form solutions for the dual problem are obtained for the normal distributions with constraints on both the mean and variance. However, this may not always be the case. The final solution depends on the form of $g\left(x\right)$ and the restrictions in $\mathbb{C}$.

While (1) compares each $g_i\left(x\right)$ with $g_k\left(x\right)$ one-at-a-time, another approach would be to compare all k distributions simultaneously. For $x=(x_1,\dots ,x_k)$, let $\mathit{X}=(X_1,\dots ,X_k)\sim g\left(x\right)=N_k(\mathbf{\mu },\mathbf{\Sigma })$ with known $\mathbf{\mu }=({\mu }_1,\dots ,{\mu }_k)$ and covariance matrix $\mathbf{\Sigma }=\left({\sigma }_{ij}\right)$ with ${\sigma }_{ii}={\sigma }_i^2$ and ${\sigma }_{ij}=0$, elsewhere. Considering equality of the k means (unspecified) with possibly unequal variances, we define

$$\mathbb{C}=\{f\left(x\right):E\left(X_i\right)=E\left(X_{i+1}\right),1\le i\le k-1\}.$$

(11)

Following (10), the solution is given by

$$f^{*}\left(x\right)=\frac{g\left(x\right)e^{{\sum }_{i=1}^k{\hat{\beta }}_i(x_i-x_{i+1})}}{{\int }_{-\infty }^{\infty }g\left(x\right)e^{{\sum }_{i=1}^k{\hat{\beta }}_i(x_i-x_{i+1})}dx}.$$

(12)

However, a closed form expression is only available when $k=3,4$.

Theorem 2 (proof in Appendix A) considers equality of means as in classical ANOVA (deals with unknown means and unknown but equal variances) but with unrestricted variances.

Theorem 2.

For $k=3$ with known reference pdf $g\left(x\right)=N_3(\mathbf{\mu },\mathbf{\Sigma })$ with the above Σ, we consider the minimization problem

$$\underset{f\left(x\right)\in \mathbb{C}}{inf}{\int }_{x}f\left(x\right)ln\left(\frac{f\left(x\right)}{g\left(x\right)}\right)dx$$

(13)

with $\mathbb{C}$ in (11). The solution to (13) is given by

$$f^{*}\left(x\right)=g\left(x\right)e^{{\hat{\alpha }}_1+{\hat{\beta }}_1(x_1-x_2)+{\hat{\beta }}_2(x_2-x_3)}=N_3({\mu }^{*}\mathbf{1},\mathbf{\Sigma })$$

(14)

where ${\hat{\alpha }}_1=-ln\left[\int g\left(x\right)e^{{\hat{\beta }}_1(x_1-x_2)+{\hat{\beta }}_2(x_2-x_3)}dx\right]$,

$${\mu }^{*}=\frac{{\mu }_1{\sigma }_2^2{\sigma }_3^2+{\mu }_2{\sigma }_1^2{\sigma }_3^2+{\mu }_3{\sigma }_1^2{\sigma }_2^2}{{\sigma }_1^2{\sigma }_2^2+{\sigma }_1^2{\sigma }_3^2+{\sigma }_2^2{\sigma }_3^2},\mathbf{1}={(1,1,1)}^{\prime },$$

(15)

$${\hat{\beta }}_1=-\left(\frac{{\mu }_1{\sigma }_2^2+{\mu }_1{\sigma }_3^2-{\mu }_2{\sigma }_3^2-{\mu }_3{\sigma }_2^2}{{\sigma }_1^2{\sigma }_2^2+{\sigma }_1^2{\sigma }_3^2+{\sigma }_2^2{\sigma }_3^2}\right),{\hat{\beta }}_2=-\left(\frac{{\mu }_1{\sigma }_2^2+{\mu }_2{\sigma }_1^2-{\mu }_3{\sigma }_1^2-{\mu }_3{\sigma }_2^2}{{\sigma }_1^2{\sigma }_2^2+{\sigma }_1^2{\sigma }_3^2+{\sigma }_2^2{\sigma }_3^2}\right).$$

(16)

Note that the solution $f^{*}\left(x\right)$ has the same covariance $\mathbf{\Sigma }$ as the reference $g\left(x\right)$; nonetheless, $\mathbf{\Sigma }$ influences the mean ${\mu }^{*}$ in solution (15) as a weighted average of its elements. When ${\sigma }_1^2={\sigma }_2^2={\sigma }_3^2$ as in one-way ANOVA, then ${\mu }^{*}=\frac{{\mu }_1+{\mu }_2+{\mu }_3}{3}$,

$${\hat{\beta }}_1=-\left(\frac{2{\mu }_1-{\mu }_2-{\mu }_3}{3{\sigma }^2}\right),{\hat{\beta }}_2=-\left(\frac{{\mu }_1+{\mu }_2-2{\mu }_3}{3{\sigma }^2}\right).$$

(17)

Theorem 2 can be extended to $k=4$ with closed form solutions for $\mathbf{\beta },{\mu }^{*}$, which is tedious [6]. Unique solutions exist for higher k but obtaining their closed forms seems intractable. This is also the case of extension to a general $\mathbf{\Sigma }$ with ${\sigma }_{ij}\ne 0,i\ne j$.

Beyond the one-way ANOVA, the above approach allows us to simultaneously compare both the means and variances of k independent normal distributions (Theorem A2).

3. Semiparametric Approach

For an unknown data generating process, however, the true form of $g\left(x\right)$ in (13) (with $\mathbb{C}$ in (11)) may not be known. Then, $g_i\left(x\right)$ in the solution (4) becomes not well-defined (replacing x with x). Note that the model (10) now becomes a ‘semiparametric tilt optimality restricted model’ because, along with the parametric component $\mathbf{\beta }$, there is also the nonparametric component $g\left(x\right)$, about which no distributional assumption is made. Using the sample, we define a discrete version of $\mathbb{C}$ expressed as moment constraints. Assuming that these sample (moment) constraints represent the corresponding population (moment) constraints efficiently and consistently, the resulting model is expected to perform well.

The dual problem corresponding to (13) is

$$\underset{{\beta }_i\in \mathbb{R},\forall i}{inf}{\int }_x{e}^{{\sum }_{i=1}^{k-1}{\beta }_i(x_i-x_{i+1})}g\left(x\right)dx.$$

(18)

The relevant score equations are

$${\displaystyle {\int }_x(x_i-x_{i+1})e^{{\sum }_{i=1}^{k-1}{\beta }_i(x_i-x_{i+1})}g\left(x\right)dx=0,1\le i\le k-1.}$$

(19)

To study the asymptotic properties of the model (similar in spirit to [7]), we consider the cases when the sample sizes are equal and when they are different.

3.1. Equal Sample Sizes

Suppose k independent random samples $\{x_{ij}1\le j\le n\},1\le i\le k$, each of size n, are available from independent populations with unknown means ${\mu }_i$ and unknown variances ${\sigma }_i^2,1\le i\le k$, respectively. If we rearrange all the $nk$ sample values as $({\mathit{x}}_1,\dots ,{\mathit{x}}_n)$, where ${\mathit{x}}_j={(x_{1j},\dots ,x_{kj})}^{\prime },1\le j\le n$, then ${\mathit{x}}_j$’s form a random sample from a multivariate distribution (say, pdf $g\left(x\right)$), with mean $\mathbf{\mu }={({\mu }_1,\dots ,{\mu }_k)}^{\prime }$ and covariance diag $({\sigma }_1^2,\dots ,{\sigma }_k^2)$.

Let $\widehat{\mathit{p}}={({\widehat{p}}_1,\dots ,{\widehat{p}}_n)}^{\prime }$ be the empirical distribution that has mass ${\widehat{p}}_j=\frac{1}{n}$ at each ${\mathit{x}}_j,1\le j\le n$. The constraint of equality of k means, $\mathbb{C}$ in (11), is discretized below as $\mathbb{K}$, appropriately, using the probability mass function (pmf) $\mathit{q}=(q_1,\dots ,q_n)$ and sample values $x_{ij}$’s as

$$\mathbb{K}=\left\{\mathit{q}:\sum _{j=1}^n(x_{ij}-x_{i+1,j})q_j=0,1\le i\le k-1\right\}.$$

Here, the primal problem (6) becomes (replacing pdfs with pmfs) ${inf}_{\mathit{q}\in \mathbb{K}}{\sum }_{j=1}^n{q}_{j}ln\left(\frac{q_j}{{\widehat{p}}_j}\right),$ and the dual problem is ${inf}_{{\beta }_i\in \mathbb{R},\forall i}{\sum }_{j=1}^n{\widehat{p}}_{j}exp\left[{\sum }_{i=1}^{k-1}{\beta }_i(x_{ij}-x_{i+1,j})\right].$ The score equations for the dual problem are

$$\sum _{j=1}^n{\widehat{p}}_j(x_{ij}-x_{i+1,j})exp\left[\sum _{i=1}^{k-1}{\beta }_i(x_{ij}-x_{i+1,j})\right]=0,1\le i\le k-1.$$

(20)

Suppose ${\mathbf{\beta }}_n=({\beta }_{ni},1\le i\le k-1)$ solve the score Equation (20). Then, the primal solution ${\widehat{q}}_j^{*}$ is given by

$${\widehat{q}}_j^{*}=\frac{{\widehat{p}}_{j}exp\left[{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})\right]}{{\sum }_{j=1}^n{\widehat{p}}_{j}exp\left[{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})\right]},j=1,\dots ,n.$$

For arbitrary $\mathbf{\delta }={({\delta }_1,\dots ,{\delta }_{k-1})}^{\prime }$, define

$$h_j\left(\mathbf{\delta }\right)=\sum _{i=1}^{k-1}{\delta }_i(x_{ij}-x_{i+1,j}),1\le j\le n,$$

${\mathit{m}}_n\left(\mathbf{\delta }\right)=(m_{nr}\left(\mathbf{\delta }\right):1\le r\le k-1)$, ${\mathsf{\Sigma }}_n\left(\mathbf{\delta }\right)=({\sigma }_{nrs}\left(\mathbf{\delta }\right);1\le r,s\le k-1)$,

$$\begin{array}{c}\hfill \begin{array}{ccc}\hfill & m_{nr}\left(\mathbf{\delta }\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n(x_{rj}-x_{r+1,j})e^{h_j\left(\mathbf{\delta }\right)}}{\frac{1}{n}{\sum }_{j=1}^n{e}^{h_j\left(\mathbf{\delta }\right)}},\mathrm{and}\hfill & \\ & \hfill & \\ \hfill & {\sigma }_{nrs}\left(\mathbf{\delta }\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n\left[(x_{rj}-x_{r+1,j})-m_{nr}\left(\mathbf{\delta }\right)\right]\left[(x_{sj}-x_{s+1,j})-m_{ns}\left(\mathbf{\delta }\right)\right]e^{h_j\left(\mathbf{\delta }\right)}}{\frac{1}{n}{\sum }_{j=1}^n{e}^{h_j\left(\mathbf{\delta }\right)}}.\hfill \end{array}\end{array}$$

(21)

By (20) and (21), ${\mathit{m}}_n\left({\mathbf{\beta }}_n\right)=\mathbf{0}$, a vector of zeroes of length $k-1$. Furthermore, by Taylor’s expansion of ${\mathit{m}}_n\left({\mathbf{\beta }}_n\right)$ around $\mathbf{\beta }$ ($\mathbf{\beta }=({\beta }_1,\dots ,{\beta }_{k-1})$),

$$\mathbf{0}={\mathit{m}}_n\left({\mathbf{\beta }}_{\mathit{n}}\right)={\mathit{m}}_n\left(\mathbf{\beta }\right)+{\mathsf{\Sigma }}_n\left({\mathit{w}}_{\mathit{n}}\right)({\mathit{\beta }}_{\mathit{n}}-\mathbf{\beta }),$$

(22)

where ${\mathit{w}}_n$ satisfies max{ $|{\mathit{w}}_n-{\mathbf{\beta }}_n|$, $|{\mathit{w}}_n-\mathbf{\beta }|$ } ≤$|{\mathbf{\beta }}_n-\mathbf{\beta }|$. As $n\to \infty$, by the strong law of large numbers, ${\mathbf{\beta }}_n\to \mathbf{\beta },{\mathit{w}}_n\to \mathbf{\beta }$, and

$$\begin{array}{cc}\hfill \frac{1}{n}\sum _{j=1}^n{(x_{rj}-x_{r+1,j})}^a{(x_{sj}-x_{s+1,j})}^b{e}^{h_j\left({\mathit{w}}_n\right)}& ⟶\int {(x_r-x_{r+1})}^a{(x_{sj}-x_{s+1,j})}^b{e}^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx,\hfill \end{array}$$

(23)

$a,b=0,1,1\le r,s\le k-1$, with probability 1 where $h\left(\mathbf{\beta }\right)={\sum }_{i=1}^{k-1}{\beta }_i(x_i-x_{i+1})$.

By (21) and (23), it follows that, when ${\mu }_1=\dots ={\mu }_k$,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill m_{nr}\left({\mathit{w}}_{\mathit{n}}\right)& ⟶\frac{\int (x_r-x_{r+1})e^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx}{\int e^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx}=0,\hfill \\ & \\ \hfill {\sigma }_{nrs}\left({\mathit{w}}_n\right)& ⟶{\sigma }_{rs}=\frac{\int (x_r-x_{r+1})(x_s-x_{s+1})e^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx}{\int e^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx},\hfill \end{array}\end{array}$$

(24)

$1\le r,s\le k-1$, with probability 1. Let $\mathsf{\Sigma }=\left({\sigma }_{rs}\right)$. As $n\to \infty$, by the central limit theorem

$$\sqrt{n}({\mathit{m}}_n\left(\mathbf{\beta }\right)-\mathbf{0})⟶N(\mathbf{0},{\mathsf{\Sigma }}^{*}),$$

(25)

where ${\mathsf{\Sigma }}^{*}=\left({\sigma }_{rs}^{*}\right)$,

$${\sigma }_{rs}^{*}=\frac{\int (x_r-x_{r+1})(x_s-x_{s+1})e^{2h\left(\mathbf{\beta }\right)}g\left(x\right)dx}{{\left(\int e^{h\left(\mathbf{\beta }\right)}g\left(x\right)dx\right)}^2}.$$

(26)

By (22)–(26),

$$\sqrt{n}\left({\mathbf{\beta }}_n-\mathbf{\beta }\right)={\left[{\mathsf{\Sigma }}_n\left({\mathit{w}}_{\mathit{n}}\right)\right]}^{-1}\sqrt{n}\left({\mathit{m}}_n\left({\mathbf{\beta }}_n\right)-{\mathit{m}}_n\left(\mathbf{\beta }\right)\right)\stackrel{D}{\to }N\left(\mathbf{0},{\mathsf{\Sigma }}^{-1}{\mathsf{\Sigma }}^{*}{\mathsf{\Sigma }}^{-1}\right),$$

as $n\to \infty$. The above developments are summarized in the following theorem. This theorem establishes the asymptotic normality of the parameters of the proposed model when sample sizes are equal.

Theorem 3.

For a general reference pdf g, assume that the solution of (13), $f^{*}\left(x\right)$, exists and $\int e^{h\left(\mathbf{\beta }\right)}{f}^{*}\left(x\right)dx<\infty$ where $h\left(\mathbf{\beta }\right)={\sum }_{i=1}^{k-1}{\beta }_i(x_i-x_{i+1})$ for β in an open neighborhood of 0. When $\int (x_i-x_{i+1})(x_j-x_{j+1})e^{2h\left(\mathbf{\beta }\right)}g\left(x\right)dx<\infty ,\forall i,j$, and all sample sizes are equal to n,

$$\sqrt{n}({\mathbf{\beta }}_{\mathit{n}}-\mathbf{\beta })\underset{}{\overset{D}{\to }}N(\mathbf{0},{\mathsf{\Sigma }}^{-1}{\mathsf{\Sigma }}^{*}{\mathsf{\Sigma }}^{-1})$$

as $n\to \infty$ where $\mathsf{\Sigma },{\mathsf{\Sigma }}^{*}$ are defined in (24) and (26), respectively.

When ${\mu }_1=\dots ={\mu }_k$, or equivalently, $\mathbf{\beta }=\mathbf{0}$, the quantity $n{\mathbf{\beta }}_n^{{}^{\prime }}{\left({\mathsf{\Sigma }}^{-1}{\mathsf{\Sigma }}^{*}{\mathsf{\Sigma }}^{-1}\right)}^{-1}{\mathbf{\beta }}_n$ has an asymptotic chi-square distribution with $k-1$ degrees of freedom as $n\to \infty$.

Thus, the test statistic

$${\chi }_1=n{\mathbf{\beta }}_n^{{}^{\prime }}{\left({\hat{\mathsf{\Sigma }}}^{-1}{\hat{\mathsf{\Sigma }}}^{*}{\hat{\mathsf{\Sigma }}}^{-1}\right)}^{-1}{\mathbf{\beta }}_n$$

(27)

can be used for testing the hypothesis $H_0:\mathbf{\beta }=\mathbf{0}$ where ${\sigma }_{rs},{\sigma }_{rs}^{*}$ are estimated by ${\sigma }_{nrs}\left({\mathbf{\beta }}_n\right)$, ${\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right)$, respectively, (since $\mathsf{\Sigma }={\mathsf{\Sigma }}^{*}$, we replaced ${\sigma }_{nrs}\left({\mathbf{\beta }}_n\right),{\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right)$ by $({\sigma }_{nrs}\left({\mathbf{\beta }}_n\right)+{\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right))/2$).

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {\sigma }_{nrs}\left({\mathbf{\beta }}_n\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n\left[(x_{rj}-x_{r+1,j})\right]\left[(x_{sj}-x_{s+1,j})\right]e^{{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})}}{\frac{1}{n}{\sum }_{j=1}^n{e}^{{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})}},\hfill \\ \hfill & {\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n\left[(x_{rj}-x_{r+1,j})\right]\left[(x_{sj}-x_{s+1,j})\right]e^{2{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})}}{{\left(\frac{1}{n}{\sum }_{j=1}^n{e}^{{\sum }_{i=1}^{k-1}{\beta }_{ni}(x_{ij}-x_{i+1,j})}\right)}^2}.\hfill \end{array}\end{array}$$

(28)

Clearly, the above developments can be extended for simultaneous mean and variance comparisons for k populations by modifying the $\mathbb{C}$ in (11).

3.2. Different Sample Sizes

The simultaneous approach developed above for equal sample sizes does not allow the sample sizes to be different. To that end, we consider $k-1$ independent one-at-a-time population optimization problems by adopting the development in Section 2 reversing the roles of $g\left(x\right)=g_k\left(x\right)$ and $g_i\left(x\right)$ in (4), along with setting $h\left(x\right)=x$, and, finally, we combine the $k-1$ results. Both procedures work for equal sample sizes.

For $1\le i\le k-1$, consider the ith problem as finding the pdf $f\left(x\right)$ in

$$\mathbb{C}=\{f\left(x\right):E\left[X\right]={\mu }_k\},$$

(29)

which is the closest to $g_i\left(x\right)$, assuming that ${\mu }_k$ is known. Following similar steps as in Section 2, the pdf in (29), which is the closest to $g_i\left(x\right)$, is given by

$$f_i^{*}\left(x\right)=g_i\left(x\right)exp({\gamma }_i+{\eta }_{i}x),{\gamma }_i=-ln\left({\int }_{-\infty }^{\infty }{e}^{{\eta }_{i}x}{g}_i\left(x\right)dx\right),$$

(30)

$i=1,\dots ,k-1$, where ${\eta }_i$ solves

$${\int }_{-\infty }^{\infty }(x-{\mu }_k)exp\left({\eta }_{i}x\right)g_i\left(x\right)dx=0.$$

(31)

If ${\eta }_i=0$, then $f_i^{*}\left(x\right)=g_i\left(x\right),\forall x,\forall i$.

To develop the corresponding $k-1$ sample optimization problems, suppose independent random samples $\{x_{ij}1\le j\le n_i\},1\le i\le k$ are available from k populations with the means ${\mu }_i,1\le i\le k$, respectively. Although we assume that ${\mu }_k$ is known, in reality, it maybe unknown. Thus, we suggest choosing the kth sample as the one that is the largest in size. Let ${\overline{x}}_k=\frac{{\sum }_{j=1}^{n_k}{x}_{ij}}{n_k}$ be the mean of the kth sample. Then, take ${\mu }_k={\overline{x}}_k$ (see Section 6).

For the ith ($1\le i\le k-1$) sample optimization problem, let ${\widehat{\mathit{p}}}_i=({\widehat{p}}_{i1},\dots ,{\widehat{p}}_{i,n_i})$ be an empirical distribution that has mass ${\widehat{p}}_{ij}=\frac{1}{n_i}$ at each $x_{ij},1\le j\le n_i$. Let the ith sample version of $\mathbb{C}$ in (29), say ${\mathbb{K}}_i$, containing ${\mathit{q}}_i=(q_{i1},\dots ,q_{i,n_i})$, be defined as

$${\mathbb{K}}_i=\left\{{\mathit{q}}_i:\sum _{j=1}^{n_i}(x_{ij}-{\overline{x}}_k)q_{ij}=0\right\},1\le i\le k-1.$$

The ith ($1\le i\le k-1$) sample version of (6) and its dual problem becomes

$$\underset{{\mathit{q}}_i\in {\mathbb{K}}_i}{inf}\sum _{j=1}^{n_i}{q}_{ij}ln\left(\frac{q_{ij}}{{\widehat{p}}_{ij}}\right),\mathrm{and},\underset{{\eta }_i\in \mathbb{R}}{inf}\sum _{j=1}^{n_i}{\widehat{p}}_{ij}exp\left[{\eta }_i(x_{ij}-{\overline{x}}_k)\right],$$

respectively. The ith score equation is

$$\sum _{j=1}^{n_i}{\widehat{p}}_{ij}(x_{ij}-{\overline{x}}_k)exp\left[{\eta }_i(x_{ij}-{\overline{x}}_k)\right]=0.$$

(32)

Suppose ${\eta }_{n_i}$ solves (32). Then, the primal solution ${\widehat{q}}_{ij}^{*}$ is given by

$${\widehat{q}}_{ij}^{*}=\frac{{\widehat{p}}_{ij}exp\left[{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)\right]}{{\sum }_{j=1}^n{\widehat{p}}_{ij}exp\left[{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)\right]},j=1,\dots ,n_i.$$

For arbitrary ${\psi }_i\in \mathbb{R},1\le i\le k-1$, define

$$\begin{array}{ccc}\hfill h_{ij}\left({\psi }_i\right)={\psi }_i(x_{ij}-{\overline{x}}_k),& & {\displaystyle m_{n_i}\left({\psi }_i\right)=\frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}(x_{ij}-{\overline{x}}_k)e^{h_{ij}\left({\psi }_i\right)}}{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{h_{ij}\left({\psi }_i\right)}},\mathrm{and}}\hfill \\ & & \\ \hfill {\sigma }_{n_i}^2\left({\psi }_i\right)& =& {\displaystyle \frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{\left[(x_{ij}-{\overline{x}}_k)-m_{n_i}\left({\psi }_i\right)\right]}^2{e}^{h_{ij}\left({\psi }_i\right)}}{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{h_{ij}\left({\psi }_i\right)}}.}\hfill \end{array}$$

(33)

By (32), $m_{n_i}\left({\eta }_{n_i}\right)=0,\forall i$. Using Taylor’s expansion of $m_{n_i}\left({\eta }_{n_i}\right)$ around ${\eta }_i$,

$$0=m_{n_i}\left({\eta }_{n_i}\right)=m_{n_i}\left({\eta }_i\right)+{\sigma }_{n_i}^2\left(w_{n_i}\right)({\eta }_{n_i}-{\eta }_i),$$

(34)

where $w_{n_i}$ is such that max{ $|w_{n_i}-{\eta }_{n_i}|$, $|w_{n_i}-{\eta }_i|$ } ≤$|{\eta }_{n_i}-{\eta }_i|$. With $h_i={\eta }_i(x-{\overline{x}}_k)$, by the strong law of large numbers, as $n_i\to \infty$, ${\eta }_{n_i}\to {\eta }_i,w_{n_i}\to {\eta }_i\forall i$ and

$$\frac{1}{n_i}\sum _{j=1}^{n_i}{(x_{ij}-{\overline{x}}_k)}^a{e}^{h_{ij}\left(w_{n_i}\right)}⟶\int {(x-{\overline{x}}_k)}^a{e}^{h_i}{g}_i\left(x\right)dx,$$

(35)

$a=0,1,2,1\le i\le k-1$, with probability 1.

From (31), $\int (x-{\overline{x}}_k)e^{h_i}{g}_i\left(x\right)dx=0$. When ${\mu }_i={\mu }_k$, by (33) and (35), it follows that, as $n_i\to \infty$,

$$m_{n_i}\left(w_{n_i}\right)⟶\frac{\int (x-{\overline{x}}_k)e^{h_i}{g}_i\left(x\right)dx}{\int e^{h_i}{g}_i\left(x\right)dx}=0,{\sigma }_{n_i}^2\left(w_{n_i}\right)⟶{\sigma }_i^2=\frac{\int {(x-{\overline{x}}_k)}^2{e}^{h_i}{g}_i\left(x\right)dx}{\int e^{h_i}{g}_i\left(x\right)dx},$$

(36)

$1\le i\le k-1$, with probability 1. As $n_i\to \infty$, by the central limit theorem

$$\sqrt{n_i}(m_{n_i}\left({\eta }_i\right)-0)⟶N(0,{\sigma }_i^{2*}),$$

(37)

where

$${\sigma }_i^{*2}=\frac{\int {(x-{\overline{x}}_k)}^2{e}^{2h_i}{g}_i\left(x\right)dx}{{\left[\int e^{h_i}{g}_i\left(x\right)dx\right]}^2}.$$

(38)

By (34), (35) and (25),

$$\sqrt{n_i}\left({\eta }_{n_i}-{\eta }_i\right)=\frac{\sqrt{n}\left(m_{n_i}\left({\eta }_{n_i}\right)-m_{n_i}\left({\eta }_i\right)\right)}{{\sigma }_{n_i}^2\left(w_{n_i}\right)}\stackrel{D}{\to }N\left(0,\frac{{\sigma }_i^{*2}}{{\sigma }_i^4}\right),$$

(39)

as $n_i\to \infty ,1\le i\le k-1$.

Next, we combine the results from the $k-1$ sample-optimization problems.

Theorem 4.

For a general reference pdf g, assume that the solution of (13) subject to (29), $f_i^{*}\left(x\right)$, exists and $\int e^{{\eta }_{i}x}{f}_i^{*}\left(x\right)dx<\infty$ for some ${\eta }_i$ in an open neighborhood of 0, $1\le i\le k-1$. Assume $\int x^2{e}^{{\eta }_{i}x}{f}_i^{*}\left(x\right)dx<\infty ,\forall i.$ Since all k samples are independent, $\frac{\sqrt{n_i}({\eta }_{n_i}-{\eta }_i)}{{\sigma }_i^{*}/{\sigma }_i^2}\stackrel{D}{\to }N(0,1),$ as $n_i\to \infty$.

Then, under $H_0:{\eta }_i=0,1\le i\le k-1$, ${\sum }_{i=1}^{k-1}\frac{n_i{\eta }_{n_i}^2}{{\sigma }_i^{*2}/{\sigma }_i^4}$ has an asymptotic chi-square distribution with $k-1$ degrees of freedom as $n_i\to \infty ,\forall i$. Thus, the test statistic

$${\displaystyle \sum _{i=1}^{k-1}\frac{n_i{\eta }_{n_i}^2}{{\sigma }_{n_i}^{*2}\left({\eta }_{n_i}\right)/{\sigma }_{n_i}^4\left({\eta }_{n_i}\right)}}$$

(40)

can be used for testing the hypothesis $H_0:{\eta }_i=0,1\le i\le k-1$, where we replaced ${\sigma }_i^{*2},{\sigma }_i^2$ by ${\sigma }_{n_i}^{*2}\left({\eta }_{n_i}\right),{\sigma }_{n_i}^2\left({\eta }_{n_i}\right)$, respectively, with

$${\sigma }_{n_i}^{*2}\left({\eta }_{n_i}\right)=\frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{(x_{ij}-{\overline{x}}_k)}^2{e}^{2{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)}}{{\left[\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)}\right]}^2},{\sigma }_{n_i}^2\left({\eta }_{n_i}\right)=\frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{(x_{ij}-{\overline{x}}_k)}^2{e}^{{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)}}{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{{\eta }_{n_i}(x_{ij}-{\overline{x}}_k)}}.$$

(41)

Since ${\sigma }_i^{*2}={\sigma }_i^2$ under $H_0$, we replace each of ${\sigma }_{n_i}^{*2}\left({\eta }_{n_i}\right),{\sigma }_{n_i}^2\left({\eta }_{n_i}\right)$ by a common estimate ${\sigma }_{n_i}^{**2}\left({\eta }_{n_i}\right)=({\sigma }_{n_i}^{*2}\left({\eta }_{n_i}\right)+{\sigma }_{n_i}^2\left({\eta }_{n_i}\right))/2$, and then calculate a pooled estimate of variance ${\hat{\sigma }}^2=({\sum }_{i=1}^{k-1}$$n_i{\sigma }_{n_i}^{**2}\left({\eta }_{n_i}\right)/$${\sum }_{i=1}^{k-1}{n}_i$ over $k-1$ populations. With this substitution, the test statistic from (40) is simplified as

$${\displaystyle {\chi }_2={\hat{\sigma }}^2\sum _{i=1}^{k-1}{n}_i{\eta }_{n_i}^2.}$$

(42)

The above developments can be extended for simultaneous mean and variance comparisons for k populations by modifying the $\mathbb{C}$ in (29).

For one h:

$$\frac{1}{n}\sum _{j=1}^n{e}^{{\beta }_{n}h\left(x_j\right)}-\frac{1}{2n}\frac{{\sum }_{j=1}^n{h}^2\left(x_j\right)e^{2{\beta }_{n}h\left(x_j\right)}}{{\sum }_{j=1}^n{h}^2\left(x_j\right)e^{{\beta }_{n}h\left(x_j\right)}}$$

For more than one h, let $h={(h_1,\dots ,h_t)}^{\prime }$ represent t constraints.

(i) For equal sample sizes, recall ${\mathit{x}}_j={(x_{1j},\dots ,x_{kj})}^{\prime },1\le j\le n$.

Recall $\mathsf{\Sigma },{\mathsf{\Sigma }}^{*}$ given as $\hat{\mathsf{\Sigma }}=({\sigma }_{nrs}\left({\mathbf{\beta }}_n\right);1\le r,s\le t),{\hat{\mathsf{\Sigma }}}^{*}=({\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right);1\le r,s\le t)$.

$${\sigma }_{nrs}\left({\mathbf{\beta }}_n\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n{h}_r\left({\mathit{x}}_j\right)h_s\left({\mathit{x}}_j\right)e^{{\sum }_{l=1}^t{\beta }_{ni}{h}_i\left(x_j\right)}}{\frac{1}{n}{\sum }_{j=1}^n{e}^{{\sum }_{l=1}^t{\beta }_{ni}{h}_i\left(x_j\right)}},$$

$${\sigma }_{nrs}^{*}\left({\mathbf{\beta }}_n\right)=\frac{\frac{1}{n}{\sum }_{j=1}^n{h}_r\left({\mathit{x}}_j\right)h_s\left({\mathit{x}}_j\right)e^{2{\sum }_{l=1}^t{\beta }_{ni}{h}_i\left(x_j\right)}}{{\left(\frac{1}{n}{\sum }_{j=1}^n{e}^{{\sum }_{l=1}^t{\beta }_{ni}{h}_i\left(x_j\right)}\right)}^2}.$$

Then, the IC is

$$\mathrm{I}\mathrm{C}=\frac{1}{n}\sum _{j=1}^n{e}^{{\sum }_{l=1}^t{\beta }_{ni}{h}_i\left({\mathit{x}}_j\right)}-\frac{1}{2n}\sum _{l=1}^t{\hat{\lambda }}_i$$

where ${\hat{\lambda }}_i$ is the ith eigen value of ${\hat{\mathsf{\Sigma }}}^{-\frac{1}{2}}{\hat{\mathsf{\Sigma }}}^{*}{\hat{\mathsf{\Sigma }}}^{-\frac{1}{2}}$. Find ${\hat{\mathsf{\Sigma }}}^{-\frac{1}{2}}$ by Cholesky decomposition.

(ii) For different sample sizes, define ${\mathsf{\Sigma }}_i,{\mathsf{\Sigma }}_i^{*}$ given as ${\hat{\mathsf{\Sigma }}}_i=({\sigma }_{n_{i}rs}\left({\mathit{\eta }}_{n_i}\right);1\le r,s\le t),{\hat{\mathsf{\Sigma }}}_i^{*}=({\sigma }_{n_{i}rs}^{*}\left({\mathit{\eta }}_{n_i}\right);1\le r,s\le t)$.

$${\sigma }_{n_{i}rs}\left({\mathbf{\eta }}_{n_i}\right)=\frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{h}_r\left(x_{ij}\right)h_s\left(x_{ij}\right)e^{{\sum }_{l=1}^t{\eta }_{nl}{h}_l\left(x_{ij}\right)}}{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{{\sum }_{l=1}^t{\eta }_{nl}{h}_l\left(x_{ij}\right)}},$$

$${\sigma }_{n_{i}rs}^{*}\left({\mathbf{\eta }}_{n_i}\right)=\frac{\frac{1}{n_i}{\sum }_{j=1}^{n_i}{h}_r\left(x_{ij}\right)h_s\left(x_{ij}\right)e^{2{\sum }_{l=1}^t{\eta }_{nl}{h}_l\left(x_{ij}\right)}}{{\left(\frac{1}{n_i}{\sum }_{j=1}^{n_i}{e}^{{\sum }_{l=1}^t{\eta }_{nl}{h}_l\left(x_{ij}\right)}\right)}^2}.$$

Then, the IC is

$$\mathrm{I}\mathrm{C}=\frac{1}{k-1}\sum _{i=1}^{k-1}{\mathrm{I}\mathrm{C}}_i$$

$${\mathrm{I}\mathrm{C}}_i=\frac{1}{n_i}\sum _{j=1}^{n_i}{e}^{{\sum }_{l=1}^t{\eta }_{nl}{h}_l\left(x_{ij}\right)}-\frac{1}{2n_i}\sum _{l=1}^t{\hat{\lambda }}_{il}$$

where ${\hat{\lambda }}_{il}$ is the lth eigen value of ${\hat{\mathsf{\Sigma }}}_{il}^{-\frac{1}{2}}{\hat{\mathsf{\Sigma }}}_{il}^{*}{\hat{\mathsf{\Sigma }}}_{il}^{-\frac{1}{2}}$. Find ${\hat{\mathsf{\Sigma }}}_{il}^{-\frac{1}{2}}$ by Cholesky decomposition.

4. Simulation Studies

In this section, we study the effect of violations of the assumptions of the one-way ANOVA on the performances of the beta parameters using Mean Square Errors. We also compare powers of the proposed methods with existing parametric and nonparametric methods. We used the R program for all simulation studies. The proposed methods are applicable to any type of data including under-dispersed or over-dispersed or data containing outliers as we do not assume any distribution $g\left(x\right)$ in our analysis.

4.1. Mean Square Errors of $\mathbf{\beta }$

Although the one-way ANOVA does not have a $\beta$-parameter as is, (14) with ${\hat{\beta }}_i$ replaced by ${\beta }_i$ may be considered as the one-way ANOVA model, which shows the $\beta$-parameters when $k=3$. To implement the assumption of equal variance, we used ${\sigma }^2=\frac{{\sigma }_1^2+{\sigma }_2^2+{\sigma }_3^2}{3}$ as the common variance in ANOVA. The true ${\beta }_1={\hat{\beta }}_1,{\beta }_2={\hat{\beta }}_2$ are in (17). In the simulation, the true ${\beta }_1,{\beta }_2$ are not changed when the population distribution is any different from normal to maintain the normality assumption of the one-way ANOVA.

There are four beta parameters (${\beta }_1,\dots ,{\beta }_4$) under the simultaneous constraints of equality of means and equality of variances. For one-way ANOVA, we set ${\beta }_3=0$ and ${\beta }_4=0$ as there are no variance constraints. In the TO model (10), however, the true $\mathbf{\beta }$ changes depending on the population and constraints as shown in Section 3 for selected distributions as solutions of the corresponding score equations.

For the simulation in

Table 1. Mean square errors of the $\mathbf{\beta }$ parameters.

	$\mathit{E}\left({\mathit{X}}_1\right)=\mathit{E}\left({\mathit{X}}_2\right)=\mathit{E}\left({\mathit{X}}_3\right)$
Components of			Sample	Parameters			MSE			TMSE
$\mathit{g}({\mathit{x}}_1,{\mathit{x}}_2,{\mathit{x}}_3)$	Mean	Var	Sizes	TO	ANOVA		TO	ANOVA		TO	ANOVA
$X_1\sim$ N(0,1)	0	1	200	${\beta }_1:$ 0.3094	0.3030		0.0044	0.0041		0.0109	0.0125
$X_2\sim$ N(0.2,1.1)	0.2	1.1	200	${\beta }_2:$ 0.4088	0.4242		0.0065	0.0084
$X_3\sim$ N(0.8,1.2)	0.8	1.2	200
$X_1\sim$ Beta(1,2)	0.33	0.06	200	${\beta }_1:$ 1.3433	1.4000		0.0726	0.0787		0.1442	0.1516
$X_2\sim$ Beta(1.33,1.99)	0.4	0.06	200	${\beta }_2:$ 1.5871	1.6000		0.0716	0.0729
$X_3\sim$ Beta(1.75,1.75)	0.5	0.06	200
$X_1\sim$ Beta(1,2)	0.33	0.06	200	${\beta }_1:$ 1.2424	1.2245		0.0642	0.0639		0.1205	0.1245
$X_2\sim$ Beta(1.08,1.62)	0.4	0.065	200	${\beta }_2:$ 1.3543	1.3994		0.0562	0.0607
$X_3\sim$ Beta(1.29,1.29)	0.5	0.07	200
	$E\left(X_1\right)=E\left(X_2\right)=E\left(X_3\right),V\left(X_1\right)=V\left(X_2\right)=V\left(X_3\right)$
$X_1\sim$ N(0,1)	0	1	200	${\beta }_1:$ 0.2828	0.3030		0.0059	0.0081		0.0217	0.0408
$X_2\sim$ N(0.2,1.1)	0.2	1.1	200	${\beta }_2:$ 0.3838	0.4242		0.0094	0.0166
$X_3\sim$ N(0.8,1.2)	0.8	1.2	200	${\beta }_3:$ 0.0429	0		0.0038	0.0091
				${\beta }_4:$ 0.0404	0		0.0026	0.0069
$X_1\sim$ Beta(1,2)	0.33	0.067	200	${\beta }_1:$ 2.8090	1.4000		0.9763	2.1442		4.5389	9.7559
$X_2\sim$ Beta(1.33,1.99)	0.4	0.067	200	${\beta }_2:$ 2.9256	1.6000		1.1376	2.1325
$X_3\sim$ Beta(1.75,1.75)	.5	0.067	200	${\beta }_3:$ −1.6512	0		1.2599	2.9440
				${\beta }_4:$ −1.4902	0		1.1651	2.5353
$X_1\sim$ Beta(1.33,1.99)	0.4	0.067	500	${\beta }_1:$ −1.4640	0		0.6133	2.3176		3.9619	12.9005
$X_2\sim$ Beta(1.2,1.8)	0.4	0.06	200	${\beta }_2:$ −1.7934	0		1.1961	3.3782
$X_3\sim$ Beta(.97,1.46)	0.4	0.07	100	${\beta }_3:$ 1.6611	0		0.7408	2.9568
				${\beta }_4:$ 2.0318	0		1.4117	4.2478

Here TO = Proposed optimality model, ANOVA = Analysis of variance, MSE = Mean square errors, TMSE = Total mean square errors.

, we considered independent normal, uniform and beta populations and formed $g\left(x\right)$ by multiplying the pdf’s of independent components as listed, which were chosen with varying means and/or variances. Various sample sizes of 50, 200 and 500 with 1000 runs for each case were considered of which only selected results are reported in Table 1 (see more details [6]). We found that the total mean square error of our method decreased for larger sample sizes.

As a consequence of the way the ANOVA model is defined in (14) using the $\mathbf{\beta }$, both ANOVA and TO solutions (or methods) become identical when populations are identical or when the means are the same, and thus they produce the same values for true $\mathbf{\beta }$’s, solutions and MSEs. Both methods have a relatively higher TMSE (the total of all MSEs) for beta reference distributions compared to the normal reference distributions. For the cases considered, the TO method has a smaller MSE compared with the one-way ANOVA method and more so when the mean and variance constraints are simultaneously considered.

4.2. Power Comparison: Equal Sample Sizes

When testing the equality of means with equal sample sizes, we conducted simulation studies to compare our proposed test statistic ${\chi }_1$’s (in (27)) performance with ${\chi }_2$ in (42) developed for different sample sizes, ($\chi$) [3], Hotelling’s tests ($T_1^2$ or $T_2^2$ depending on known (chi-square) or unknown (F-distribution)) variances, classical ANOVA and the nonparametric Kruskal–Wallis (KW) test statistics.

For $k=3$, with $\mathbf{\mu }=(0,{\mu }_2,{\mu }_3)$, we considered either $g\left(x\right)\sim N_3(\mathbf{\mu },\mathsf{\Sigma }=I_3)$, or $g\left(x\right)\sim gamma(3,1)\times gamma(3+{\mu }_2,1)\times gamma(3+{\mu }_3,1)$. The hypothesis of equality of means becomes equivalent to testing $H_0:{\mu }_2={\mu }_3=0$. Equal sample sizes are taken as $n=30$ or $n=50$. For all simulations, 1000 runs were used with a nominal level of $\alpha =0.05$.

The power results as a function of ${\mu }_2,{\mu }_3$ are shown in

Table 2. Power comparison with equal sample sizes.

						Normal										Gamma
$\mathit{n}$	${\mathit{\mu }}_{\mathbf{2}}$	${\mathit{\mu }}_{\mathbf{3}}$		$\mathit{\chi }$	${\mathit{\chi }}_{\mathbf{1}}$	${\mathit{\chi }}_{\mathbf{2}}$	${\mathit{T}}_{\mathbf{1}}^{\mathbf{2}}$	${\mathit{T}}_{\mathbf{2}}^{\mathbf{2}}$	ANOVA	KW				$\mathit{\chi }$	${\mathit{\chi }}_{\mathbf{1}}$	${\mathit{\chi }}_{\mathbf{2}}$	${\mathit{T}}_{\mathbf{1}}^{\mathbf{2}}$	${\mathit{T}}_{\mathbf{2}}^{\mathbf{2}}$	ANOVA	KW
30	0	0		0.054	0.080	0.096	0.043	0.058	0.040	0.042				0.069	0.087	0.126	0.045	0.068	0.041	0.038
	0.2	0.2		0.126	0.184	0.368	0.101	0.145	0.109	0.099				0.057	0.140	0.104	0.074	0.101	0.060	0.059
	0.1	0.4		0.309	0.381	0.587	0.267	0.330	0.284	0.278				0.083	0.182	0.175	0.107	0.145	0.120	0.110
	0.2	0.5		0.428	0.495	0.805	0.379	0.440	0.398	0.377				0.103	0.239	0.250	0.145	0.188	0.143	0.138
	0.5	0.5		0.527	0.598	0.963	0.474	0.546	0.499	0.480				0.115	0.267	0.418	0.161	0.218	0.170	0.183
	0.7	0.5		0.730	0.774	0.992	0.664	0.730	0.703	0.663				0.158	0.346	0.573	0.229	0.281	0.241	0.259
50	0	0		0.056	0.084	0.083	0.050	0.060	0.046	0.047				0.063	0.091	0.112	0.051	0.070	0.054	0.051
	0.2	0.2		0.161	0.198	0.483	0.139	0.164	0.150	0.153				0.058	0.113	0.116	0.075	0.095	0.078	0.080
	0.1	0.4		0.457	0.508	0.769	0.430	0.463	0.438	0.397				0.117	0.216	0.254	0.151	0.179	0.156	0.173
	0.2	0.5		0.621	0.648	0.938	0.577	0.618	0.600	0.565				0.144	0.268	0.398	0.202	0.231	0.203	0.217
	0.5	0.5		0.739	0.768	0.995	0.717	0.741	0.730	0.689				0.183	0.345	0.635	0.246	0.278	0.254	0.261
	0.7	0.5		0.897	0.917	1.000	0.882	0.903	0.896	0.887				0.306	0.482	0.831	0.383	0.424	0.386	0.422

Here $\mathbf{\chi }$ = Test statistic [3], ${\mathbf{\chi }}_{\mathbf{1}}$ = Proposed test statistic (27) for equal sample sizes, ${\mathbf{\chi }}_{\mathbf{2}}$ = Proposed test statistic (42) for different sample sizes, ${\mathit{T}}_{\mathbf{1}}^{\mathbf{2}}$ or ${\mathit{T}}_{\mathbf{2}}^{\mathbf{2}}$ = Hotelling’s tests depending on known (chi-square) or unknown (F-distribution) variances, ANOVA = F test statistic of analysis of variance, KW = Kruskal-Wallis test statistic.

. Both ${\chi }_1$ and ${\chi }_2$ tests have higher powers than other tests under the gamma distribution, including the nonparametric test with the ${\chi }_2$ test performing slightly better than the ${\chi }_1$ test. However, none of the $\chi ,{\chi }_1,{\chi }_2$ tests was dominated by the ANOVA test under the normal case as expected (also observed by [3] for the $\chi$ test).

4.3. Power Comparison: Different Sample Sizes

We compared the performance of the proposed semi-parametric test statistic ${\chi }_2$ in (42) with the test statistic $\chi$ [3], classical ANOVA and the Kruskal–Wallis test (KW).

We used the same $g\left(x\right)$, $H_0$, number of runs and nominal level as in the equal sample size case. The power results as a function of ${\mu }_2,{\mu }_3$ are shown in

Table 3. Power comparison with different sample sizes.

				Normal								Gamma
Sample Sizes	${\mu }_2$	${\mu }_3$	${\chi }_2$	$\chi$	ANOVA	KW				${\chi }_2$	$\chi$	ANOV	KW
$n_1=200$	0.0	0.0	0.068	0.050	0.048	0.048				0.085	0.055	0.040	0.040
$n_2=100$	0.2	0.2	0.594	0.333	0.330	0.306				0.182	0.113	0.161	0.159
$n_3=40$	0.1	0.4	0.719	0.531	0.520	0.498				0.244	0.120	0.202	0.208
	0.2	0.5	0.916	0.785	0.772	0.755				0.382	0.222	0.337	0.327
	0.5	0.5	1.000	0.987	0.988	0.981				0.807	0.514	0.602	0.629
	0.7	0.5	1.000	1.000	1.000	1.000				0.968	0.766	0.839	0.854
$n_1=200$	0.0	0.0	0.071	0.055	0.054	0.055				0.078	0.053	0.046	0.043
$n_2=100$	0.2	0.2	0.714	0.370	0.363	0.349				0.219	0.105	0.141	0.153
$n_3=100$	0.1	0.4	0.960	0.835	0.828	0.813				0.475	0.269	0.346	0.347
	0.2	0.5	0.999	0.960	0.959	0.962				0.735	0.443	0.542	0.569
	0.5	0.5	1.000	0.998	0.998	0.997				0.933	0.606	0.673	0.703
	0.7	0.5	1.000	1.000	1.000	1.000				0.990	0.819	0.867	0.893

Here ${\mathbf{\chi }}_{\mathbf{2}}$ = Proposed test statistic (42) for different sample sizes, $\mathbf{\chi }$ = Test Statistic [3], ANOVA = F test statistic of analysis of variance, KW = Kruskal-Wallis test statistic.

with different sample sizes $n_1,n_2,n_3$. For the calculation of ${\chi }_2$, the value of the sample mean with the largest sample size is taken as fixed in all cases.

It can be seen that the ${\chi }_2$ test performed better than the other tests. However, it is not dominated by the ANOVA test as expected under the normal distribution.

5. Applications

We considered two applications. Equal sample sizes were used in the radar meteorology example, and the sample sizes were different in the credit limit example.

5.1. Radar Meteorology

We considered radar reflectivity data [1] from two independent radars deployed during NASA’s Tropical Rainfall Measuring Mission Kwajalein Experiment. An S-band radar was located on Kwajalein Island at the southern end of the Kwajalein Atoll, and a C-band radar was on board the NOAA ship Ronald H. Brown. Different calibrations were applied to two radars, and their spherical data were recorded to the same 1 km cube Cartesian grid.

Histograms of the samples were identical if sample 1 and sample 2 both are taken from the C-band population (Figure 1). Histograms of sample 1 and 3 illustrate that the shapes of the distributions are different for sample 1 taken from the C-band population and sample 2 taken from the S-band population (Figure 1).

First, two independent samples were taken from the C-band radar population to test whether they were from the same population. The population size was around 3987, and the sample sizes were 500 each. To test that the populations were equal, the method of Section 4 (in particular, the test statistic ${\chi }_1$ in (27) was used with $k=2$), and the results are given in

Table 4. Hypothesis testing when samples are coming from the same or different populations.

Populations	${\mathit{H}}_0$	Transformation	Dual Solutions	${\mathit{\chi }}_1$	p-Value
C-band, C-band	$H_0:\beta =0$	$h\left(x\right)=x_1-x_2$	$\hat{\beta }$ = −0.00086	0.1382	0.71
	$H_0:{\beta }_1=0,$	$h_1\left(x\right)=x_1-x_2,$	$\widehat{{\beta }_1}=-0.0175,$	3.3577	0.18
	${\beta }_2=0$	$h_2\left(x\right)=x_1^2-x_2^2$	$\widehat{{\beta }_2}=0.0003$
	$H_0:\beta =0$	$h\left(x\right)=ln{x}_1-ln{x}_2$	$\hat{\beta }$ = −0.08541	−0.74671	0.4552
C-band, S-band	$H_0:\beta =0$	$h\left(x\right)=x_1-x_2$	$\hat{\beta }$ = −0.0087	13.24	0.0003
	$H_0:{\beta }_1=0,$	$h_1\left(x\right)=x_1-x_2,$	$\widehat{{\beta }_1}=0.0048$,	23.4	0.0
	${\beta }_2=0$	$h_2\left(x\right)=x_1^2-x_2^2$	$\widehat{{\beta }_2}=-0.0002$
	$H_0:\beta =0$	$h\left(x\right)=ln{x}_1-ln{x}_2$	$\hat{\beta }$ = 0.4414	4.8307	0.0

. Using only equality of means constraints, the data fail to reject $H_0:\beta =0$ with the p-value = 0.71. Using simultaneous equality of means and equality of variance constraints, the data again fail to reject $H_0:{\beta }_1=0,{\beta }_2=0$ with p-value = 0.18, and thus we concluded that both samples are from the same population (with respect to the means and variances).

Next, we considered two independent samples of size 500 each from C- and S-band populations to test if those were from the same population. With C-band radar data as the reference and when using only equality of means constraints, the data rejected $H_0:\beta =0$ with p-value =0.0003. Using equality of means and equality of variance constraints simultaneously, the data again rejected $H_0:{\beta }_1=0,{\beta }_2=0$ with p-value $\approx 0$; therefore, we concluded that the samples are from different populations (with respect to the means and variances).

From the histograms, the data appear to be skewed to the left (as was noted by [3]). Thus, we considered the log transformation of the data and repeated the analysis for equality of means as reported in Table 4; however, our conclusions remain unchanged.

5.2. Credit Limit Data

Previous study considered credit limit information from three different education levels, namely, graduate school, university and high school [2]. We want to investigate whether the populations are identical using the constraints that the means of the credit limit between three different education levels are equal. The sample sizes from the three populations are 10,585, 14,030 and 4917, respectively.

The credit limit data of each groups follows positively skewed distributions (Figure 2).

To test that the populations were identical with respect to their means, the method of Section 4 for different sample sizes, in particular, the test statistic ${\chi }_2$ in (42) was used with $k=3$. As the ‘university’ group had the largest sample size, the sample mean from that group was taken as fixed to be used in (29). While comparing the other groups with the university group, we found ${\widehat{\eta }}_1=-0.4435\times {10}^{-5}$ and ${\widehat{\eta }}_2=0.1396\times {10}^{-5}$. The test statistic value was ${\chi }_2=1272.92$ with p-value $\approx 0$. Thus, we reject $H_0:{\eta }_1={\eta }_2=0$, and conclude that the populations are different. As the data were skewed, a log transformation was, again, considered. Then, the dual problem solutions were ${\widehat{\eta }}_1=-0.563,{\widehat{\eta }}_2=0.1988$, and the test statistic was ${\chi }_2=928.75$ (p-value $\approx 0$) rejecting $H_0$.

6. Discussions

Often prior information is known about the population. However, the sample collected may not reveal this information due to the sampling variability. Hence, it is worthwhile to build a model that satisfies the prior information and is the closest to the observed data. In Theorem 1, $g\left(x\right)$ serves as the observed data, $\mathbb{C}$ serves as the prior information, and the distance between $g\left(x\right)$ and $\mathbb{C}$ is measured using the KL distance.

This paper proposed a method to compare between different populations based on a set of restrictions specified by the investigator. The restrictions were set in the form of moment constraints through one or more functions h. Setting different types of h compares different aspects of the distributions under consideration, e.g., $h\left(x\right)=x-c_1$ in (4) compares $g\left(x\right)$ and $g_i\left(x\right)$ regarding their means. When ${\beta }_i=0$ in (4), then $g_i\left(x\right)=g\left(x\right),\forall x$. However, when ${\mathbf{\beta }}_i\ne 0$, then $g_i\left(x\right)$ and $g\left(x\right)$ might differ in aspects other than only their values of $E\left(h\right(X\left)\right)$.

For real data, one can obtain basic information from the data, including the shape. If any of the distributions under consideration are known to be approximately symmetric, using $h_1\left(x\right)=x-c_1$ and/or $h_2\left(x\right)=x^2-c_2$ may be the first steps to determine if the distributions are different regarding their means and/or variances. However, if the distributions under consideration are known to be approximately skewed, then using $h\left(x\right)=lnx$ would be more appropriate. In general, the reference distribution in (4) may be any of the k distributions, leaving the exponential distortion intact but with shifted parameters. When using the one-at-a-time method for different sample sizes, we chose the sample with the largest size as the reference, considering it to be the most trusted, and used its mean as ${\mu }_k$.