Replacing Histogram with Smooth Empirical Probability Density Function Estimated by K-Moments

Abstract

Whilst several methods exist to provide sample estimates of the probability distribution function at several points, for the probability density of continuous stochastic variables, only a gross representation through the histogram is typically used. It is shown that the newly introduced concept of knowable moments (K-moments) can provide smooth empirical representations of the distribution function, which in turn can yield point and interval estimates of the density function at a large number of points or even at any arbitrary point within the range of the available observations. The proposed framework is simple to apply and is illustrated with several applications for a variety of distribution functions.

1. Introduction

The concepts of distribution function ($F\left(x\right)$) and probability density ($f\left(x\right)$) of a stochastic (or random) variable ($\underset{\_}{x}$ with realizations $x$; notice the notational convention to underline stochastic variables—the Dutch convention) are central in Kolmogorov’s foundation of probability in 1933 [1,2] and its applications. (Here, we adhere to the original Kolmogorov’s terms, noting that in the English literature $F\left(x\right)$ is also known as the cumulative distribution function.) Their estimation, based on a sample of $\underset{\_}{x}$, is crucial in most applications.

The standard estimate of $F\left(x\right)$ was again introduced by Kolmogorov [3], who termed it the empirical distribution, and is still in common use [4], sometimes under the name sample distribution function. Denoting ${\underset{\_}{x}}_{\left(i:n\right)}$ the ith order statistic in a sample of size $n$, i.e., the ith smallest of the $n$ variables arranged in increasing order (${\underset{\_}{x}}_{\left(1:n\right)}\le {\underset{\_}{x}}_{\left(2:n\right)}\le \dots \le {\underset{\_}{x}}_{\left(n:n\right)}$), the standard estimate is:

$$\widehat{F}\left(x\right)=\frac{n_x}{n}, x_{\left(n_x-1:n\right)}<x\le x_{\left(n_x:n\right)}$$

(1)

where $n_x$ is the number of the values in the observed sample that do not exceed $x$ (an integer). The function $\widehat{F}\left(x\right)$ has a staircase form with discontinuities at each of the observations $x_i$ (see Appendix A for an illustration and more details). If $\underset{\_}{x}$ is a continuous variable, almost certainly the values $x_i$ are distinct and the jump of $\widehat{F}\left(x\right)$ at $x_i$ equals $1/n$. Apparently then, the representation of the continuous $F\left(x\right)$ by the discontinuous $\widehat{F}\left(x\right)$ is not ideal. This representation has two additional problems, i.e., (a) for $x<{\underset{\_}{x}}_{\left(1:n\right)}$ the resulting $\widehat{F}\left(x\right)$ is 0, and (b) for $x\ge {\underset{\_}{x}}_{\left(n:n\right)}$ the resulting $\widehat{F}\left(x\right)$ is 1. Both these are problematic results if the variable $\underset{\_}{x}$ is unbounded.

Both the latter problems are usually tackled with the notion of so-called plotting positions (e.g., [5])—a rather unsatisfactory name. There is a variety of plotting position formulae, each of which is a modification of Equation (1) by adding one or two constants in the numerator and/or the denominator. A review of these formulae and their justification is provided by Koutsoyiannis (2022) [6], along with a set of new proposed ones, derived using the theoretical properties of order statistics.

One could also think to tackle the discontinuity problem by replacing the staircase function by some type of interpolation (e.g., linear or logarithmic). However, the thus resulting function would again be too rough for practical applications. In particular, a rough function, where the roughness is a statistical sampling effect rather than an intrinsic property of the distribution function, cannot support reliable estimation of derivatives. Since the density $f\left(x\right)$ is the first derivative of the distribution $F\left(x\right)$, the above framework cannot be used to estimate the former. An illustration of the roughness of $F\left(x\right)$ and $f\left(x\right)$, if the former is estimated from Equation (1) and the latter by the numerical derivative of the former, is provided in Appendix A (in particular, in Figure A1).

Methods for a detailed estimation of the probability density at different points $x$ are lacking. Instead, a gross estimation based on the histogram constitutes the standard representation of the density. Yet, the estimation of the density is necessary for several tasks, e.g., those involving hazard (where the hazard function is defined as $f\left(x\right)/\left(1-F\left(x\right)\right)$; see [7]) or entropy (where entropy is the expectation of $–\ln f\left(\underset{\_}{x}\right)$ properly standardized; see Section 3.3).

The histogram representation is constructed by choosing (a) an interval $\left[a,b\right]$ that contains all observations (often called the range) and (b) a number $k$ of equally spaced bins, so that the width of the bins is

$$w=\frac{b-a}{k}$$

(2)

The number $k$ is usually small, typically chosen by the old Sturges’ rule [8]:

$$k={\log }_2\left(2n\right)=1+\frac{\ln n}{\ln 2}$$

(3)

For example, for a sample size $n=100$, this results in $k\approx 8$. The underlying rationale for the rule and comparisons with additional rules are provided by Scott [9]. Once $a,b,k$ are chosen, the density estimate is

$$\widehat{f}\left(x\right)=\frac{n_{x_{i+1}}-n_{x_i}}{n w}, i=1,\dots ,k, n_{x_i}<x\le n_{x_{i+1}}, x_i=a+\left(i-1\right)w$$

(4)

The entire framework for the construction of histogram entails subjectivity, and lacks detail and accuracy.

As an alternative to constructing a smooth empirical density function, Rosenblatt [10] and Parzen [7] proposed the use of kernel smoothing. They have been followed by several researchers who developed the method further [11,12,13,14]. This is regarded as a non-parametric method, but it uses a specified kernel function which contains parameters, where both the function expression and its parameters are arbitrarily chosen by the user. Typical kernel functions include uniform, triangular, quadratic (Epanechnikov), biweight, triweight, normal, and even atomic kernels [15,16]. While the methods of this type may provide a smooth function, their reliability is questionable, and the results are affected by a great deal of subjectivity owing to user choices. These characteristics are illustrated in Appendix A (in particular, in Figure A2).

Here, we propose a new method of estimation of the probability density based on the concept of knowable moments, abbreviated as K-moments. As shown in [6] (pp. 147–249) and summarized in the next section, the noncentral K-moment of order $p$, $K_p^{’}$, can: (a) yield a reliable and unbiased estimate ${\widehat{K}}_p^{’}$ from a sample, up to order $p$ equal to the sample size $n$, and (b) be assigned a value of return period, or equivalently distribution function estimate, $\widehat{F}\left({\widehat{K}}_p^{’}\right)$. In addition, the sequence of estimates $\widehat{F}\left({\widehat{K}}_p^{\prime }\right)$ form a smooth function which can be used in estimating the probability density at a large number of points (as opposed to the roughness implied by Equation (1)—or other versions thereof based on plotting positions and eventually on order statistics—and as illustrated in Figure A1 of Appendix A). The estimate ${\widehat{K}}_p^{\prime }$ is based on the notion of order statistics but, contrary to their standard use, where only one sample value is used at a time, the K-moment framework combines many order statistics simultaneously, thus converting a rough arrangement to smooth, without involving any subjectively chosen kernel function.

2. K-Moments and Their Relevance

2.1. Definition and Interpretation

We recall that the noncentral (or raw) moment of order p of a stochastic variable $\underset{\_}{x}$ is defined as the expectation:

$${\mu }_p^{\prime } :=\mathrm{E}\left[{\underset{\_}{x}}^p\right]=\underset{-\infty }{\overset{\infty }{\int }}{x}^{p}f\left(x\right)\mathrm{d}x$$

(5)

with ${\mu }_1^{’}=\mathrm{E}\left[\underset{\_}{x}\right]=:\mu$ representing the mean. Let ${\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p$ be independent copies of $\underset{\_}{x}$, forming a sample. Then, the standard estimator of ${\mu }_p^{’}$ from the sample is

$${\underset{\_}{\widehat{\mu }}}_p^{\prime }=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\underset{\_}{x}}_i^p$$

(6)

It is well known that the estimator of the noncentral moment is unbiased, i.e.,

$$\mathrm{E}\left[{\underset{\_}{\widehat{\mu }}}_p^{\prime }\right]={\mu }_p^{\prime }$$

(7)

While unbiasedness is theoretically guaranteed, the convergence of ${\widehat{\mu }}_p^{\prime }$ to ${\mu }_p^{’}$ is extraordinarily slow if $p$ is not very low [17]. In practice, for large or moderate $p$ (greater than 2 or 3, depending on the sample size), what we actually calculate by applying the standard estimator, is an estimate of some extreme quantity, rather than an estimate of the moment ${\mu }_p^{’}$. To see this, we recall that for positive $x_i$ and for large (or even modest) p the approximation $\sum _{i=1}^n{x}_i^p\approx {\left(\underset{1\le i\le n}{\max }\left(x_i\right)\right)}^p$ holds, so that

$${\widehat{\mu }}_p^{\prime }=\frac{1}{n}\sum _{i=1}^n{x}_i^p\approx \frac{1}{n}{\left(\underset{1\le i\le n}{\max }\left(x_i\right)\right)}^p$$

(8)

Thus, unless p is very small, we cannot infer the value of ${\mu }_p^{’}$ from data, i.e., it is unknowable (Koutsoyiannis, 2019 [18]; see also [6], pp. 125–126). This is the case even if the sample size n is extraordinarily large [6,18]. This reduces the power of the concept of moments for statistical inference, which is the formal probabilistic induction (the modern version of the Aristotelian epagoge/ἐπαγωγή) and is based on expectations, estimated from samples by virtue of stationarity and ergodicity.

To obtain a knowable moment of order p, Koutsoyiannis [18] raised $x$ to a low power, $q<p$ and for the remaining $\left(p-q\right)$ multiplicative terms, replaced $x$ with $F\left(x\right)$, hence defining the noncentral knowable moment (or noncentral K-moment) of orders (p, q) as:

$$K_{pq}^{\prime } :=\left(p-q+1\right)\mathrm{E}\left[{\left(F\left(\underset{\_}{x}\right)\right)}^{p-q}{\underset{\_}{x}}^q\right], p\ge q$$

(9)

with the most interesting special case obtained for $q=1$:

$$K_p^{\prime } :=p\mathrm{E}\left[{\left(F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right], p\ge 1$$

(10)

Koutsoyiannis [6,18] also introduced other types of K-moments, among which here we will use, in addition to $K_p^{\prime }$, the tail-based (noncentral) moments:

$${\overline{K}}_p^{\prime } :=p\mathrm{E}\left[{\left(1-F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]=p\mathrm{E}\left[{\left(\overline{F}\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right], p\ge 1$$

(11)

where $\overline{F}\left(\underset{\_}{x}\right) :=1-F\left(\underset{\_}{x}\right)$ is the tail function.

For an interpretation of K-moments we consider the maximum of $p$ stochastic variables ${\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p$, which is the largest (pth) order statistic:

$${\underset{\_}{x}}_{\left(p\right)} :={\underset{\_}{x}}_{\left(p:p\right)}=\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p\right)$$

(12)

It is readily obtained that if $F\left(x\right)$ is the distribution function of $\underset{\_}{x}$ and $f\left(x\right)$ its probability density function, then those of ${\underset{\_}{x}}_{\left(p\right)}$ are:

$$F^{\left(p\right)}\left(x\right)={\left(F\left(x\right)\right)}^p, f^{\left(p\right)}\left(x\right)=pf\left(x\right){\left(F\left(x\right)\right)}^{p-1}$$

(13)

where the former is the product of p instances of $F\left(x\right)$ (justified by the fact that the variables ${\underset{\_}{x}}_{\mathrm{i}}$ are independent copies of $\underset{\_}{x}$, by definition of the sample concept), while the latter is the derivative of $F^{\left(p\right)}\left(x\right)$ with respect to $x$. The expected maximum of order p of $\underset{\_}{x}$, i.e., the expected value of ${\underset{\_}{x}}_{\left(p\right)}$, is therefore:

$$\mathrm{E}\left[{\underset{\_}{x}}_{\left(p\right)}\right]=\mathrm{E}\left[\max \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p\right)\right]=p\mathrm{E}\left[{\left(F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]$$

(14)

and this is precisely the noncentral K-moment $K_p^{\prime }$. Likewise, the minimum of the p variables

$${\underset{\_}{x}}_{\left(1:p\right)}=\min \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p\right)$$

(15)

has expectation:

$$\mathrm{E}\left[\min \left({\underset{\_}{x}}_1,{\underset{\_}{x}}_2, \dots , {\underset{\_}{x}}_p\right)\right]=p\mathrm{E}\left[{\left(1-F\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]=p\mathrm{E}\left[{\left(\overline{F}\left(\underset{\_}{x}\right)\right)}^{p-1}\underset{\_}{x}\right]$$

(16)

which is the tail K-moment ${\overline{K}}_p^{\prime }$.

It is thus easy to see that the sequence of $K_p^{\prime }$ is non-decreasing and that of ${\overline{K}}_p^{\prime }$ is non-increasing as p increases.

2.2. Estimation of K-Moments

Several estimators of the K-moments have been developed in [6], among which here we use the unbiased ones [6] (pp. 193–196 and 229–231):

$${\widehat{\underset{\_}{K}}}_p^{\prime }={\displaystyle \sum }_{i=1}^n{b}_{inp} {\underset{\_}{x}}_{\left(i:n\right)}, {\widehat{\overline{K}}}_p^{\prime }={\displaystyle \sum }_{i=1}^n{b}_{inp} {\underset{\_}{x}}_{\left(n-i+1:n\right)}={\displaystyle \sum }_{j=1}^n{b}_{n-j+1,n,p} {\underset{\_}{x}}_{\left(j:n\right)}$$

(17)

where

$$b_{inp} :=\{\begin{array}{ll}0,\hfill & i<p\hfill \\ \frac{p}{n} \frac{\mathsf{\Gamma }\left(n-p+1\right)}{\mathsf{\Gamma }\left(n\right)} \frac{\mathsf{\Gamma }\left(i\right)}{\mathsf{\Gamma }\left(i-p+1\right)},\hfill & i\ge p>0\hfill \end{array}$$

(18)

and $\mathsf{\Gamma }\left(a\right) :=\underset{0}{\overset{\infty }{\int }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t$ is the gamma function. This allows estimation of the K-moments for any order p from 1 to $n$. Thus, from a sample of size $n$ we can estimate a number of noncentral and tail moments equal to $2n-1$ (notice that ${\widehat{\underset{\_}{K}}}_1^{\prime }={\widehat{\overline{K}}}_1^{\prime }=\widehat{\mu }$), with high reliability and low uncertainty (see [6], p. 195). It can be easily verified that for any order p,

$${\displaystyle \sum }_{i=1}^n{b}_{inp}=1$$

(19)

which is a necessary condition for unbiasedness. Special cases of K-moment estimator coefficients $b_{inp}$ are shown in

Table 1. Special cases of K-moment estimator coefficients (adapted from [6], p. 194).

Case	${\mathit{b}}_{\mathit{i}\mathit{n}\mathit{p}}$	Case	${\mathit{b}}_{\mathit{i}\mathit{n}\mathit{p}}$
$p=1$	$b_{in1}=\frac{1}{n}$	$p=n-1$	$b_{n-1,n,n-1}=\frac{1}{n},b_{n,n,n-1}=1-\frac{1}{n}$
$p=2$	$b_{in2}=\frac{2}{n}\frac{i-1}{n-1}$	$p=n$	$b_{nnn}=1$
$p=3$	$b_{in2}=\frac{3}{n}\frac{i-1}{n-1}\frac{i-2}{n-2}$	$i=n$	$b_{nnp}=\frac{p}{n}$
$p=4$	$b_{in4}=\frac{4}{n}\frac{i-1}{n-1}\frac{i-2}{n-2}\frac{i-3}{n-3}$	$i=p$	$b_{pnp}=p\mathrm{B}\left(p,n-p+1\right)$ * symmetry: $b_{pnp}=b_{n-p,n,n-p}$ (minimum at $p=n/2$)

* $\mathrm{B}(a,b$) is the beta function.

.

The fact that $b_{inp}=0$ for $i<p$ suggests that, as the moment order increases, progressively fewer data values determine the moment estimate, until it remains only one (the maximum for the noncentral moment and the minimum for the tail moment), when $p=n$, with $b_{nnn}=1$. Furthermore, if $p>n$ then $b_{inp}=0$ for all $i, 1\le i\le n$, and therefore estimation becomes impossible.

2.3. Estimation of the Distribution Function at K-Moment Values

Order statistics have an important advantage over other statistics, as to each of them we can assign a value of the distribution function, or equivalently, the return period. We recall that for a specific event $A$ the return period, $T$, is defined to be the mean time between consecutive occurrences of the event $A$. Assuming that the event is the exceedance of a certain level $x$, i.e., $A :=\left\{\underset{\_}{x}>x\right\}$, the return period is related to the distribution function by

$$\frac{T}{D}=\frac{1}{\overline{F}\left(x\right)}=\frac{1}{1-F\left(x\right)}$$

(20)

where $D$ is a time window width (time scale or time step) used to define the metric $x$ (e.g., $D=1$ year if $x$ is the annual rainfall total). This is known as the return period of maxima.

Likewise, for the non-exceedance of the value $x$, i.e., $A :=\left\{\underset{\_}{x}\le x\right\}$, the return period (of minima) is

$$\frac{\overline{T}}{D}=\frac{1}{F\left(x\right)}$$

(21)

As seen above, the K-moments are closely related to order statistics and therefore it becomes possible to assign return periods to K-moment values. Intuitively, we can expect that the pth noncentral K-moment (the value $x=K_p^{’}$) will correspond to a return period of about 2pD. Τhis is precise for a symmetric distribution and for p = 1, as $K_1^{\prime }$ is the mean value which has return period 2D. (For instance, the mean of annual rainfall is exceeded, or non-exceeded, on the average, every two years). As we will see below, the return period cannot be much lower than 2pD for any p and for any distribution.

Generally, we can expect that the return period is an increasing function of the moment order p, with a relationship of approximate proportionality:

$$\frac{T\left(K_p^{\prime }\right)}{D}\underset{˜}{\propto }p$$

(22)

As justified above, a coefficient of proportionality equal to 2 can be used as a first rough approximation (rule of thumb that helps intuition). However, more precisely, the coefficient of proportionality, call it ${\Lambda }_p$, depends on the distribution function and the order p, but its variation is not wide. The precise definition of ${\Lambda }_p$ is:

$${\Lambda }_p :=\frac{1}{p\left(1-F\left(K_p^{\prime }\right)\right)}$$

(23)

For given p and distribution function $F\left(x\right)$, $K_p^{\prime }$, $T\left(K_p^{\prime }\right)$, $F\left(K_p^{\prime }\right)$ and ${\Lambda }_p$ are analytically or numerically determined from their definitions, but this might be complicated. However, the small variation of ${\Lambda }_p$ with p makes possible a very good approximation if we first accurately determine (a) the value ${\Lambda }_1$ for $p=1$, and (b) the asymptotic value ${\Lambda }_{\infty }$. The value ${\Lambda }_1$ is very easy to determine, as it refers to the return period of the mean:

$${\Lambda }_1=\frac{1}{1-F\left(\mu \right)}$$

(24)

and can also be reliably estimated from a sample by Equation (1).

Furthermore, in a number of customary distributions, specifically those belonging to the domain of attraction of the Extreme Value Type I distribution (see [6] (pp. 76–79 and 235–236), ${\Lambda }_{\infty }$ has a constant value, independent of the distribution:

$${\Lambda }_{\infty }={\mathrm{e}}^{\mathsf{\gamma }}=1.781$$

(25)

where γ is the Euler constant. For heavy tailed distributions ${\Lambda }_{\infty }$ depends on the higher tail index only. Details are given in [6] (pp. 208–215), while for the distributions examined in this study, which are listed along with their main characteristics in

Table 2. Main characteristics of the distribution functions used in the illustrations.

Name, Parameters *, Domain	$\mathbf{Probability} \mathbf{Density} \mathbf{Function}, \mathbf{f}\left(\mathit{x}\right)$	$\mathbf{Mean}, {\mathit{\mu }}_1^{\prime }$	$\mathbf{Variance}, {\mathit{\mu }}_2$	$\mathbf{Entropy} \mathit{\Phi }$
Exponential $\mu >0, x\ge 0$	${\mathrm{e}}^{–x/\mu }/\mu$	$\mu$	${\mu }^2$	$\ln \mathrm{e}\mu$
Normal $\mu \in ℝ, \sigma >0, x\in ℝ$	$\frac{\exp \left(-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}\right)}{\sqrt{2\mathsf{\pi }}\sigma }$	$\mu$	${\sigma }^2$	$\ln \left(\sqrt{2e\mathsf{\pi }} \sigma \right)$
Lognormal $\varsigma >0, \lambda >0, x\ge 0$	$\frac{\exp \left(-\frac{1}{2{\varsigma }^2}{\left(\ln \left(\frac{x}{\lambda }\right)\right)}^2\right)}{\sqrt{2\mathsf{\pi }} \varsigma x}$	${\mathrm{e}}^{\frac{{\varsigma }^2}{2}}\lambda$	${\mathrm{e}}^{{\varsigma }^2}\left({\mathrm{e}}^{{\varsigma }^2}-1\right){\lambda }^2$	$\ln \left(\sqrt{2e\mathsf{\pi }} \lambda \varsigma \right)$
Pareto $\xi >0, \lambda >0, x\ge 0$	$\frac{1}{\lambda }{\left(1+\xi \frac{x}{\lambda }\right)}^{-1-\frac{1}{\xi }}$	$\frac{\lambda }{1-\xi }$	$\frac{{\lambda }^2}{{\left(1-\xi \right)}^2\left(1-2\xi \right)}$	$\xi +\ln \mathrm{e}\lambda$

* Parameter notation: $\mu$ mean; $\sigma$ standard deviation; $\lambda$ scale parameter, $\xi$ upper tail index; $\varsigma$ shape parameter other than tail index.

, they are shown in

Table 3. Characteristic $\Lambda$ values for the distributions of Table 2.

Distribution	${\mathit{\Lambda }}_1$	${\mathit{\Lambda }}_{\mathit{\infty }}$	${\overline{\mathit{\Lambda }}}_1$	${\overline{\mathit{\Lambda }}}_{\mathit{\infty }}$
Exponential	$\mathrm{e}=2.718$	${\mathrm{e}}^{\mathsf{\gamma }}=1.781$ *	$\frac{\mathrm{e}}{\mathrm{e}-1}=1.582$	1
Normal	2	${\mathrm{e}}^{\mathsf{\gamma }}=1.781$	$2$	${\mathrm{e}}^{\mathsf{\gamma }}=1.781$
Lognormal	$\frac{2}{\mathrm{erfc}\left(\varsigma /2^{3/2}\right)}$	${\mathrm{e}}^{\mathsf{\gamma }}=1.781$	$\frac{2}{2-\mathrm{erfc}\left(\varsigma /2^{3/2}\right)}$	1 †
Pareto	${\left(1-\xi \right)}^{-1/\xi }$	$\mathsf{\Gamma }{\left(1-\xi \right)}^{1/\xi }$	$\frac{1}{1-{\left(1-\xi \right)}^{1/\xi }}$	1

* $\mathsf{\gamma }=0.577$ is the Euler’s constant. ^† The theoretically consistent value is ${\mathrm{e}}^{\mathsf{\gamma }}$, but the convergence to the limit is very slow and thus the value 1 (like in the exponential and Pareto distribution) provides more accurate numerical results for typical sample sizes.

.

Given ${\Lambda }_1$ and ${\Lambda }_{\infty }$, the coefficient ${\Lambda }_p$ for any order $p$ can be satisfactorily (see [6], (pp. 216–218)) approximated with the following simple relationship:

$${\Lambda }_p\approx {\Lambda }_{\infty }+\frac{\left({\Lambda }_1-{\Lambda }_{\infty }\right)}{p}$$

(26)

while more accurate approximations are given in [6] (pp. 208–215) and also discussed in Section 4 below. Equation (26) yields a linear relationship between the return period T and p:

$$\frac{T\left(K_p^{\prime }\right)}{D}=p{\Lambda }_p\approx {\Lambda }_{\infty }p+\left({\Lambda }_1-{\Lambda }_{\infty }\right)$$

(27)

from which we find

$$F\left(K_p^{\prime }\right)=1-\frac{1}{{\Lambda }_{\infty }p+{\Lambda }_1-{\Lambda }_{\infty }}$$

(28)

Likewise, for the tail moments we have:

$${\overline{\Lambda }}_p :=\frac{1}{p F\left({\overline{K}}_p^{\prime }\right)}$$

(29)

with

$${\overline{\Lambda }}_1=\frac{1}{F\left(\mu \right)}=\frac{{\Lambda }_1}{{\Lambda }_1-1}$$

(30)

while the limiting value ${\overline{\Lambda }}_{\infty }$ depends only on the lower tail index and is ${\overline{\Lambda }}_{\infty }={\Lambda }_{\infty }={\mathrm{e}}^{\mathsf{\gamma }}$ for the normal and other symmetric distributions, and ${\overline{\Lambda }}_{\infty }=1$ for distributions with lower-tail index equal to one such as the exponential and Pareto distributions (see Table 3).

Again we can use the approximation:

$${\overline{\Lambda }}_p\approx {\overline{\Lambda }}_{\infty }+\frac{{\overline{\Lambda }}_1-{\overline{\Lambda }}_{\infty }}{p}$$

(31)

and find

$$F\left({\overline{K}}_p^{\prime }\right)=\frac{1}{{\overline{\Lambda }}_{\infty }p+\left({\overline{\Lambda }}_1-{\overline{\Lambda }}_{\infty }\right)}$$

(32)

3. Results

3.1. Estimation of Probability Density

Based on the above framework, we can estimate $2n-1$ values of K-moments ${\widehat{K}}_p^{\prime }$ and ${\widehat{\overline{K}}}_p^{\prime }$, namely the ordered values ${\widehat{\overline{K}}}_n^{\prime }\le {\widehat{\overline{K}}}_{n-1}^{\prime }\le \dots \le {\widehat{\overline{K}}}_1^{\prime }\equiv {\widehat{K}}_1^{\prime }\le {\widehat{K}}_2^{\prime }\le \dots {\widehat{K}}_{n-1}^{\prime }\le {\widehat{K}}_n^{\prime }$. To each one of them we can assign an empirical estimate of the distribution function $\widehat{F}\left({\widehat{K}}_p^{\prime }\right)$ and $\widehat{F}\left({\widehat{\overline{K}}}_p^{\prime }\right)$. This is similar to the distribution function values assigned via the order statistics (the plotting positions) except that (a) the number of values in the K-moments framework is twice that of the case of order statistics, and (b) the arrangement of point estimates in the former case is smooth, while in the latter is rough. The smooth arrangement allows a direct estimate of the probability density for $1\le p\le n-1$. As at points $x={\widehat{K}}_p^{\prime }$ and $x={\widehat{\overline{K}}}_p^{\prime }$, the values of the distribution function have been estimated from Equations (28) and (32), i.e., $F\left(x\right)=\widehat{F}\left({\widehat{K}}_p^{\prime }\right)$ and $F\left(x\right)=\widehat{F}\left({\widehat{\overline{K}}}_p^{\prime }\right)$, it is then straightforward to approximate the derivative $f\left(x\right)=\mathrm{d}F\left(x\right)/\mathrm{d}x$ by its discrete version:

$$\widehat{f}\left(x\right)=\{\begin{array}{ll}\frac{\widehat{F}\left({\widehat{K}}_{p+1}^{\prime }\right)-\widehat{F}\left({\widehat{K}}_p^{\prime }\right)}{{\widehat{K}}_{p+1}^{\prime }-{\widehat{K}}_p^{\prime }},\hfill & {\widehat{K}}_p^{\prime }\le x<{\widehat{K}}_{p+1}^{\prime }\hfill \\ \frac{\widehat{F}\left({\widehat{\overline{K}}}_p^{\prime }\right)-\widehat{F}\left({\widehat{\overline{K}}}_{p+1}^{\prime }\right)}{{\widehat{\overline{K}}}_p^{\prime }-{\widehat{\overline{K}}}_{p+1}^{\prime }}, \hfill & {\widehat{\overline{K}}}_{p+1}^{\prime }\le x<{\widehat{\overline{K}}}_p^{\prime }\hfill \end{array}$$

(33)

This procedure will result in $2n-2$ different values of the density $\widehat{f}\left(x\right)$. In Section 4 we will see that it is possible to expand the number of estimation points by using non-integer orders p, but in general the number $2n-2$ is more than enough. This is illustrated in Figure 1 for a synthetic sample of size 100, generated from a lognormal distribution, also in comparison with a histogram of $k=10$ bins (slightly more than the number resulting from Sturges’ rule, $k\approx 8$).

Clearly the histogram representation of the true density (also shown in in Figure 1) is poor: five of its ten bins are empty, its shape is rough and the increasing limb of the density (for small $x$) is not captured at all. In contrast, the proposed method results in a very faithful representation of the true density.

3.2. Uncertainty Assessment

A more systematic illustration is made by means of Monte Carlo distribution with 100 realizations of samples of 100 items each, for all four distributions listed in Table 2. The Monte Carlo simulations allow assessing the estimation uncertainty in terms of prediction limits. The results of this investigation are shown in Figure 2 for the exponential distribution, Figure 3 for the normal distribution, Figure 4 for the lognormal distribution and Figure 5 for the Pareto distribution.

In all cases, the estimated probability density, expressed in terms of the median of the simulations (or the average thereof, which is very close to the median and was not plotted in the figures), harmonizes with the true shape of the probability density. This applies to the body of the distribution, as well as to the right and left tails, which are better discerned in the double logarithmic plots of the figures (middle panels). Slight discrepancies appear in the normal distribution (Figure 3), which will be discussed in Section 4. For comparison, in each of the figures, double logarithmic plots are also provided for the classical histograms (lower panels). In these, the simulated medians harmonize well with the true densities, yet the points are too few and the left tails of the distributions are not captured. As per the uncertainty, the proposed method clearly outperforms the histogram framework as the zones defined by prediction limits are quite narrower in the former case than in the latter.

3.3. Entropy Estimation

Most of the statistical estimators (e.g., of moments) do not involve the probability density and thus they do not require the estimation thereof. Entropy is an exception, because its very definition relies on the logarithm of the density function. Namely, the entropy $\Phi \left[\underset{\_}{x}\right]$ of the stochastic variable $\underset{\_}{x}$ is defined as:

$$\Phi \left[\underset{\_}{x}\right] :=\mathrm{E}\left[–\ln \frac{f\left(\underset{\_}{x}\right)}{\beta \left(\underset{\_}{x}\right)}\right]=-\underset{-\infty }{\overset{\infty }{{\displaystyle \int }}}\ln \frac{f\left(x\right)}{\beta \left(x\right)}f\left(x\right)\mathrm{d}x$$

(34)

where $\beta \left(x\right)$ is a background measure, which can be any probability density, proper (with integral equal to 1) or improper (meaning that its integral diverges). Typically, it is an (improper) Lebesgue density, i.e., a constant with dimensions $\left[\beta \left(x\right)\right]=\left[f\left(x\right)\right]=\left[x^{-1}\right]$, so that the argument of the logarithm function be dimensionless. Here, we assume $\beta \left(x\right)=1$.

The common practice is to estimate $f\left(x\right)$ through a histogram. However, this technique is not perfect because of the rough shape of the estimate and the subjective choices about the bins. These may result in a distorted estimate. To illustrate this, we consider the following example, based on the exponential distribution. For a sample of size $n$ from this distribution, the expected values of the highest of the $n$ values is $K_n^{\prime }=\mu H_n$, where $H_n$ is the nth harmonic number, and that of the lowest is ${\overline{K}}_n^{\prime }=\mu /n$ (see Koutsoyiannis [6], pp. 182–183). Therefore, the expected value of the range is $r :=K_n^{\prime }-{\overline{K}}_n^{\prime }=\mu H_{n-1}\approx \mu \left(\mathsf{\gamma }+\ln n\right)$. In the limiting case of only one bin ($k=1$), the entropy will be estimated as ${\widehat{\Phi }}_1\left[\underset{\_}{x}\right]=\ln r=\ln \left(\mu \left(\mathsf{\gamma }+\ln n\right)\right)$. At the opposite end, if we choose too many bins, $k>n$, so that each one contains either one or zero elements, then the probability density estimate will be either 0 (for the bins containing no element) or $1/\left(nw\right)$ where $w=r/k$. The former case does not contribute to entropy, so that the summation to estimate entropy (by converting the integral in (34)) is made on the bins containing elements. The entropy in this case will be ${\widehat{\Phi }}_k\left[\underset{\_}{x}\right]=-{\displaystyle \sum }_i\ln {\widehat{f}}_i{\widehat{f}}_{i}w={\displaystyle \sum }_i\ln \left(nw\right)\left(1/nw\right)w=\ln \left(nw\right)=\ln \left(nr/k\right)=\ln r+\ln (n/k).$ This depends on k and by choosing a large k it can become arbitrarily small (even negative) as $\ln (n/k)<0$. The true entropy is $\Phi \left[\underset{\_}{x}\right]=\ln \mathrm{e}\mu$ (Table 2). Now, assuming, for instance, $\mu =1,n=100,k=500$ the true entropy is $\Phi \left[\underset{\_}{x}\right]=1$, while the estimates are ${\widehat{\Phi }}_1\left[\underset{\_}{x}\right]=1.65>1$ and ${\widehat{\Phi }}_{500}\left[\underset{\_}{x}\right]=0.04<1$. These values indicate the subjectivity of the estimates. A good choice of $k$ will result in good estimate, yet we cannot be certain about its reliability as we cannot control the factors producing the estimation errors.

The more detailed representation of the probability density allows for a better estimation of entropy. However, we must have in mind that in unbounded stochastic variables there is uncertainty beyond the maximum (or the minimum) observed value in the sample, which we denote as $c$. For a variable bounded by zero from below and unbounded from above, we can proceed with the correction proposed by Koutsoyiannis and Sargentis [19] to take into account the contribution of probability density for $x>c$.

Specifically, the expectation of any function $g\left(\underset{\_}{x}\right)$ can be calculated as

$$\mathrm{E}\left[g\left(\underset{\_}{x}\right)\right] :=\underset{0}{\overset{\infty }{\int }}g\left(x\right)f\left(x\right)\mathrm{d}x=A^g+B^g,A^g :=\underset{0}{\overset{c}{\int }}g\left(x\right)f\left(x\right)\mathrm{d}x,B^g :=\underset{c}{\overset{\infty }{\int }}g\left(x\right)f\left(x\right)\mathrm{d}x$$

(35)

The quantity $A^g$ can directly be estimated from the available data, by approximating the integral with a sum. Assuming that the data are given in terms of density estimates ${\widehat{f}}_i$at points $x_i$, with $i=1,\dots m$and $x_m\equiv c$, we have:

$${\widehat{A}}^g={\displaystyle \sum }_{i=1}^{n}g\left(\frac{x_{i-1}+x_i}{2}\right){\widehat{f}}_i \left(x_i-x_{i-1}\right)={\displaystyle \sum }_{i=1}^{n}g\left(\frac{x_{i-1}+x_i}{2}\right)\left({\widehat{F}}_i-{\widehat{F}}_{i-1}\right)$$

(36)

To estimate the quantity$B^g$ we assume that beyond $c$ an exponential approximation is sufficient for the purpose:

$$f\left(x\right)={\mathrm{e}}^{-x/\kappa +\mathsf{\beta }}\Rightarrow \overline{F}\left(x\right)=\kappa {\mathrm{e}}^{-x/\kappa +\mathsf{\beta }}=\kappa f\left(x\right),x\ge c$$

(37)

where $\beta$ and $\kappa$ are parameters to be estimated. For the moment of order $p$ of the distribution we have:

$$B_p :=\underset{c}{\overset{\infty }{\int }}{x}^p{\mathrm{e}}^{-x/\kappa +\beta } \mathrm{d}x={\mathrm{e}}^{\beta } \mathsf{\Gamma }\left(p+1,\frac{c}{\kappa }\right){\kappa }^{p+1}$$

(38)

where $\mathsf{\Gamma }\left(a,x\right) :=\underset{x}{\overset{\infty }{\int }}{t}^{a-1}{\mathrm{e}}^{-t}\mathrm{d}t$ is the incomplete gamma function. In particular for $p=0,1,2$ we have

$$B_0=\overline{F}\left(c\right),B_1=B_0\left(c+\kappa \right),B_2=B_0\left({\kappa }^2+{\left(c+\kappa \right)}^2\right)$$

(39)

with

$$\kappa =\frac{\overline{F}\left(c\right)}{f\left(c\right)}$$

(40)

Furthermore, for the entropy we have

$$B^{\Phi }=B_0\left(1-\ln f\left(c\right)\right)=B_0\left(1-\ln B_0+\ln \kappa \right)$$

(41)

We assume that $B_0=\overline{F}\left(c\right)$ is known from the data, estimated as ${\widehat{B}}_0=\widehat{\overline{F}}\left(c\right)$. In the case that, in addition, $f\left(c\right)$ can reliably be estimated as $\widehat{f}\left(c\right)$, the sought parameter $\kappa$ is estimated as $\widehat{\kappa }=\widehat{\overline{F}}\left(c\right)/\widehat{f}\left(c\right)$. However, in the outmost available point of the tail, $f\left(c\right)$ is not adequately reliable. An alternative option is to estimate from the data the quantity $B_2$

$${\widehat{B}}_2={\widehat{\mu }}_2^{\prime }-{\widehat{A}}_2$$

(42)

where ${\widehat{\mu }}_2^{\prime }$ is determined by the standard sample estimator. In this case, solving the rightmost of Equation (39) for $\kappa$ we find

$$\widehat{\kappa }=\frac{1}{2}\left(\sqrt{\frac{{\widehat{B}}_2}{{\widehat{B}}_0}-c^2}-c\right)$$

(43)

This allows estimation of $B^g$ for the expectation of any function $g\left(\underset{\_}{x}\right)$. We note that the correction is applied only if the quantity within the square root turns out to be positive (which is usually the case).

If the distribution is unbounded from both below and above (e.g., in the normal distribution), then we apply the procedure twice for the lower and upper tail. In this case, we have $B_{0_{\mathrm{L}}}=F\left(c_{\mathrm{L}}\right)$ and $B_{0_{\mathrm{R}}}=\overline{F}\left(c_{\mathrm{R}}\right)$ where the subscripts L and R refer to left (below) and right (above), respectively. In this case, Equation (42) automatically includes both corrections and in Equation (43) we should replace ${\widehat{B}}_0$ with the sum ${\widehat{B}}_0{}_{\mathrm{L}}+{\widehat{B}}_0{}_{\mathrm{R}}$.

Application of this technique for the Monte Carlo simulations described above are given in

Table 4. Estimated means, standard deviations and entropies: averages from the 100 Monte Carlo simulations performed in the study with sample size 100.

Distribution, Parameters	Mean			Standard Deviation			Entropy
	True	Est. 1 *	Est. 2 *	True	Est. 1	Est. 2	True	Est. 2
Exponential, $\mu =1$	1	0.99	1.00	1	0.98	0.98	1	0.96
Normal, $\mu =0, \sigma =1$	0	0.01	0.01	1	0.99	0.99	1.42	1.37
Lognormal, $\varsigma =\lambda =1$	1.65	1.69	1.80	2.16	2.26	2.31	1.42	1.42
Pareto, $\xi =0.2, \lambda =1$	1.25	1.26	1.27	1.61	1.56	1.57	1.20	1.18

* Est. 1: Estimate from the typical sample statistics (without involving the probability density function); Est 2: Estimate from the empirical density function.

, in which we see that the method works well.

4. Discussion

The method is described above in its minimal configuration. Improvements are possible in several ways, e.g., to take into account possible dependence of the consecutive variables (i.e., when we do not have an observed sample but a time series), or to use more accurate representations of the relationship between K-moments and their corresponding values of distribution functions. These issues are studied in Koutsoyiannis [6] (pp. 147–249)—but not for the density estimation.

Here, we discuss the case of improvement of the distribution function estimation for the normal distribution, for which the simulation results (Figure 3) showed slight discrepancies. As shown in [6] (pp. 209–213), the more accurate representation of the Λ-coefficients for the normal distribution are

$${\Lambda }_p\approx {\Lambda }_{\infty }+\frac{A}{p}-B\ln \left(1+\frac{1}{\ln \left(p+1\right)}\right),A={\Lambda }_1-{\Lambda }_{\infty }+B\ln \left(1+\frac{1}{\ln 2}\right)$$

(44)

$${\overline{\Lambda }}_p\approx {\overline{\Lambda }}_{\infty }+\frac{\overline{A}}{p}+\overline{B}\ln \left(1+\frac{1}{\ln \left(p+1\right)}\right),\overline{A}={\overline{\Lambda }}_1-{\overline{\Lambda }}_{\infty }-\overline{B}\ln \left(1+\frac{1}{\ln 2}\right)$$

(45)

where for the normal distribution $B=\overline{B}=0.73$. In this case, we find:

$$F\left(K_p^{\prime }\right)=1-\frac{1}{\left({\Lambda }_{\infty }+B\ln \left(1+\frac{1}{\ln \left(p+1\right)}\right)\right)p+{\Lambda }_1-{\Lambda }_{\infty }-B\ln \left(1+\frac{1}{\ln 2}\right)}$$

(46)

$$F\left({\overline{K}}_p^{\prime }\right)=\frac{1}{\left({\overline{\Lambda }}_{\infty }-\overline{B}\ln \left(1+\frac{1}{\ln \left(p+1\right)}\right)\right)p+{\overline{\Lambda }}_1-{\overline{\Lambda }}_{\infty }+\overline{B}\ln \left(1+\frac{1}{\ln 2}\right)}$$

(47)

In addition, noticing that the points plotted around $p=1$ are at greater distances to each other than the other points, we can use non-integer values of $p$ between 1 and 2. Repeating the Monte Carlo simulation with these two modifications, we get the results shown in Figure 6 (lower panel), which are in better agreement with the true density than those of the minimal version, also reproduced in Figure 6 (upper panel).

Even if we use the minimal version, it is possible and sometimes useful to calculate $\widehat{f}\left({\widehat{K}}_p^{\prime }\right)$ for non-integer order $p$. In this case, we can even find (see derivation in Appendix B) an analytical expression for the probability density estimate, which for the noncentral K-moments is

$$\widehat{f}\left({\widehat{K}}_p^{\prime }\right)=\frac{{\Lambda }_{\infty } \widehat{\overline{F}}{\left({\widehat{K}}_p^{\prime }\right)}^2}{{{\displaystyle \sum }}_{i=\lceil p\rceil }^n{b}_{inp}{c}_{inp}{x}_{\left(i:n\right)}}$$

(48)

where

$${\widehat{K}}_p^{\prime }={\displaystyle \sum }_{i=\lceil p\rceil }^n{b}_{inp} x_{\left(i:n\right)}, \widehat{\overline{F}}\left({\widehat{K}}_p^{\prime }\right)=\frac{1}{{\Lambda }_{\infty }p+{\Lambda }_1-{\Lambda }_{\infty }}, c_{inp} :=\frac{1}{p}+H_{i-p}-H_{n-p}$$

(49)

and $b_{inp}$ is given by Equation (18). It is reminded that $H_n$ denotes the nth harmonic number. Similar equations can be developed for the tail K-moments.

Notice in Equations (48) and (49) that the lower limit of the sum is not the non-integer p but its floor $\lceil p\rceil$. This means that the functions $\widehat{F}\left({\widehat{K}}_p^{\prime }\right)$ and $\widehat{f}\left({\widehat{K}}_p^{\prime }\right)$ will not be fully continuous, but only left continuous. However, the discontinuities are practically negligible for $p<n-1$.

A final note is that the proposed method, in addition to providing point estimates $\widehat{f}\left({\widehat{K}}_p^{\prime }\right)$, can also produce interval estimates of $f\left({\widehat{K}}_p^{\prime }\right)$ by means of confidence limits determined by Monte Carlo simulation. These differ from the prediction limits of Section 3.2, which were constructed for a known true distribution function that was used to generate several (in our case 100) realizations of samples. For the confidence limits, the true distribution is assumed unknown and the simulation is made from the estimate $\widehat{F}\left(x\right)$, which is determined at the points ${\widehat{K}}_p^{\prime }$ and ${\widehat{\overline{K}}}_p^{\prime }.$ The generation from this distribution is easy: the values are generated as $x={\widehat{F}}^{-1}\left(u\right)$, where $u$ is a random number from the uniform distribution in [0, 1]. The inverse function ${\widehat{F}}^{-1}$ is determined by interpolation (and occasionally extrapolation) from the sequence of points $\left(x_i,\widehat{F}\left(x_i\right)\right)$, with the sequence of $x_i$ being $\left( {\widehat{\overline{K}}}_n^{\prime },{\widehat{\overline{K}}}_{n-1}^{\prime },\dots ,{\widehat{\overline{K}}}_1^{\prime }\equiv {\widehat{K}}_1^{\prime }, {\widehat{K}}_2^{\prime },\dots {\widehat{K}}_{n-1}^{\prime },{\widehat{K}}_n^{\prime }\right)$. The interpolation is better made in terms of the quantity $\ln \left(\widehat{F}/\left(1-\widehat{F}\right)\right)$ instead of $\widehat{F}$.

An illustration for the example of Figure 1 (lognormal distribution) is shown in Figure 7 (lower panel), where the produced confidence band is also compared with uncertainty band defined by the prediction limits of the lognormal distribution (upper panel of Figure 7). We observe that: (a) the confidence band (lower panel) has about the same width as the uncertainty band (upper panel), and (b) the true distribution, which was not used in the Monte Carlo simulation of the lower panel, is contained within the produced confidence band—as it should.

5. Conclusions

For continuous stochastic variables, the proposed framework is an improved, detailed and smooth alternative to the widely used concept of the histogram, which provides only a gross representation of the probability density. The proposed framework is based on the concept of knowable moments (K-moments) and its characteristics that make possible the estimate of the probability density are:

• The ability to reliably estimate from a sample, moments of high order, up to the sample size.
• The ability to assign values of the distribution function to each estimated value of K-moment.
• The smoothness of the estimated values, which are linear combinations of a number of observations, rather than based on a single observation as in other approaches.

The latter characteristic is crucial in estimating the probability density, which is the first derivative of the distribution function.

Prominent characteristics of the proposed method of density estimation, confirmed by several applications of the method for a variety of distribution functions, are:

• The faithful representation of the true density, both in the body and the tails of the distribution.
• The dense and smooth shape, owing to the ability to estimate values of the density at very many points (even for any arbitrary point) within the range of the available observations.
• The low uncertainty of estimates.
• The ability to provide both point and interval estimates (confidence limits), with the latter becoming possible by Monte Carlo simulation.
• The simplicity of the calculations, which can be made in a typical spreadsheet environment.

Specifically, the calculations include the following steps, which summarize the numerical part of the proposed method.

• We sort the observed sample in ascending order.
• We calculate the estimates ${\widehat{\underset{\_}{K}}}_p^{\prime }, {\widehat{\overline{K}}}_p^{\prime }, i=1,\dots ,n$ from Equations (17) and (18).
• We estimate the coefficient ${\Lambda }_1$ from Equation (24) and calculate ${\overline{\Lambda }}_1$ from Equation (30).
• We assume the tail indices of the distribution and choose values of ${\Lambda }_{\infty }$ and ${\overline{\Lambda }}_{\infty }$ from Table 3 (see [6] (pp. 209–213), for additional distributions).
• We calculate the estimates $\widehat{F}\left({\widehat{K}}_p^{\prime }\right)$ and $\widehat{F}\left({\widehat{\overline{K}}}_p^{\prime }\right)$ from Equations (28) and (32) for all ${\widehat{\underset{\_}{K}}}_p^{\prime }, {\widehat{\overline{K}}}_p^{\prime }$ derived in step 2. (By plotting the tails $\widehat{\overline{F}}\left({\widehat{K}}_p^{\prime }\right)$ and $\widehat{F}\left({\widehat{\overline{K}}}_p^{\prime }\right)$ in double logarithmic graphs, we check whether the empirically estimated tail indices agree with those assumed in Step 4 and, if not, we repeat steps 4 and 5 with new estimates).
• We calculate the estimates of $f\left(x\right)$ from Equation (33).

These calculations are illustrated in the Supporting Information, which contains a spreadsheet accompanying this paper. This gives a full-scale application of the method, related to the construction of Figure 1.

A problem of the method is that it requires one to have an idea of the type of the true distribution, in terms of its upper- and lower-tail indices, in order to estimate the asymptotic Λ-coefficients (${\Lambda }_{\infty },{\overline{\Lambda }}_{\infty }$). If the sample size is large, the K-moments can support the estimation of these tail indices (cf. points steps 4 and 5 above). However, for small samples ($n<~100$) their estimation becomes problematic and higher uncertainty is induced. This issue requires further investigation.