Previous Article in Journal
Method for Human Ear Localization in Controlled and Uncontrolled Environments
Previous Article in Special Issue
5th-Order Multivariate Edgeworth Expansions for Parametric Estimates

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

# The Law of the Iterated Logarithm for Lp-Norms of Kernel Estimators of Cumulative Distribution Functions

by
Fuxia Cheng
Department of Mathematics, Illinois State University, Normal, IL 61790, USA
Mathematics 2024, 12(7), 1063; https://doi.org/10.3390/math12071063
Submission received: 20 February 2024 / Revised: 23 March 2024 / Accepted: 28 March 2024 / Published: 1 April 2024

## Abstract

:
In this paper, we consider the strong convergence of $L p$-norms ($p ≥ 1$) of a kernel estimator of a cumulative distribution function (CDF). Under some mild conditions, the law of the iterated logarithm (LIL) for the $L p$-norms of empirical processes is extended to the kernel estimator of the CDF.
MSC:
60F15; 62G05

## 1. Introduction

Consider an independent identically distributed random sample $X 1 , X 2 , ⋯ , X n$ from a population with an unknown cumulative distribution function (CDF). For the empirical distribution function $F n$, defined as follows:
$F n ( x ) = 1 n ∑ i = 1 n I ( X i ≤ x ) , ∀ x ∈ R 1 ,$
with I denoting the indicator function, the classical Glivenko–Cantelli theorem states that $F n ( x )$ converges almost surely (a.s.) to $F ( x )$ uniformly in $x ∈ R 1$, i.e.,
$sup x ∈ R 1 | F n ( x ) − F ( x ) | → 0 , a . s .$
The extended Glivenko–Cantelli lemma (in Fabian and Hannan 1985, pp. 80–83 [1]) provides the strong uniform convergence rate as follows:
$sup x ∈ R n α | F n ( x ) − F ( x ) | → 0 a . s . , f o r a n y 0 < α < 1 / 2 .$
The law of the iterated logarithm (LIL) for $F n ( t )$, i.e.,
$lim sup n → ∞ n 2 log ( log n ) sup x | F n ( x ) − F ( x ) | = 1 2 a . s .$
was proven by Smirnov (1944) [2] and, independently, Chung (1949) [3].
Finkelstein (1971) [4] obtained the $L 2$-version of the law of iterated logarithm,
$lim sup n → ∞ n 2 log ( log n ) [ ∫ − ∞ ∞ ( F n ( x ) − F ( x ) ) 2 d F ( x ) ] 1 / 2 = 1 π a . s .$
For any $p ≥ 1$, setting
$C ( p ) = 1 2 ( p ( p + 2 ) π ) 1 / 2 ( 2 p + 2 ) 1 / p Γ ( 1 / p + 1 2 ) Γ ( 1 / p ) ,$
the law of the iterated logarithm for $L p$-norm of $F n ( x )$,
$lim sup n → ∞ n 2 log ( log n ) ∫ − ∞ ∞ | F n ( x ) − F ( x ) | p d F ( x ) 1 / p = C ( p ) a . s .$
was developped by Gajek, Kahszka, and Lenic (1996) [5]. It is easy to verify that
$C ( 1 ) = 3 6 a n d C ( 2 ) = 1 π .$
And (3) is a special case of (5) corresponding to $p = 2$.
Notice that there is one serious discontinuity drawback of $F n$, regardless of F being continuous or discrete. To treat this deficiency of $F n$, Yamato (1973) [6] proposed the following kernel distribution estimator:
$F ^ x = ∫ − ∞ x n − 1 ∑ i = 1 n k h u − X i d u , x ∈ R ,$
in which $h = h n$ is the usual band width sequence of positive numbers tending to zero, k is a probability density function(PDF) called kernel, and $k h u = k u / h / h$.
The aim of this paper is to provide certain conditions to guarantee the LIL of $L p$-norm of $F ^$. Some asymptotic properties of the smooth estimator $F ^$ have been established. For example, in Yamato (1973) [6], the asymptotic normality and uniform strong consistency of $F ^$ were obtained. In more general contexts, Winter (1979) [7] considered the convergence rate of perturbed empirical distribution functions. Wang, Cheng, and Yang (2013) [8] developed simultaneous confidence bands for F based on $F ^$. The strong convergence rate of $F ^$ was considered by Cheng (2017) [9], which extended the extended Glivenko–Cantelli Lemma (1) to the kernel estimator $F ^$.
Here, we shall continue to consider the strong convergence of a smooth estimator $F ^$ for F. More specifically, we are interested in extending the LIL of $L p$-norm in (5) for $F n ( t )$ to the kernel estimator $F ^$.
The outline of this paper is as follows: Section 2 describes the basic assumptions and main results: the strong uniform closeness between $F n$ and $F ^$, and the LIL of $L p$-norm of $F ^$. Detailed proofs are provided in Section 3.
Note that for the proof of the strong uniform closeness between $F n$ and $F ^$, we use the Kiefer type approximation for the empirical process (see Csörgő and Révész (1981) [10]).
Throughout the following all limits are taken as the sample size n tending to .

## 2. Assumptions and the Main Results

In this section, we start with the assumptions for the kernel function k.
Assumption A1.
k: Functions $k ( x )$, $x k ( x )$ and $x 2 k ( x )$ are integrable on the whole real line and satisfy the following properties:
$k ( x ) ≥ 0 , ∫ − ∞ + ∞ k ( x ) d x = 1 , ∫ − ∞ + ∞ x k ( x ) d x = 0 a n d ∫ − ∞ + ∞ x 2 k ( x ) d x < ∞ .$
About the band width h, we assume
$h 3 / 2 log ( log n ) → 0 a n d n h 4 / log ( log n ) → 0 ,$
which are stronger than the assumption $n h 4 → 0$ used in Cheng (2017) [9].
Under the above assumptions, we first state the result for evaluating the uniform closeness between $F ^$ and $F n$, which improves Theorem 2.1 in Cheng (2017) [9].
Theorem 1.
Assume thatAssumption kand (7) hold. Then, for the continuous CDF F with bounded second order derivative, we have
$sup x ∈ R n log ( log n ) | F ^ ( x ) − F n ( x ) | → 0 , a . s .$
Together with LIL in (2), the LIL can be extended to $F ^$, as follows:
Corollary 1.
Under the assumptions of Theorem 1, for the continuous CDF F with bounded second order derivative, we have
$lim sup n → ∞ n 2 log ( log n ) sup x | F ^ ( x ) − F ( x ) | = 1 2 a . s .$
Remark 1.
Using a different approach, (9) was verified in Winter (1979) [7].
Combining (8) with (5), the LIL for $L p$-norm of $F n$ can be extended to $F ^$.
Theorem 2.
Under the assumptions of Theorem 1, for any $p ≥ 1$ and the continuous CDF F with bounded second order derivative, we have
$lim sup n → ∞ n 2 log ( log n ) ∫ − ∞ ∞ | F ^ ( x ) − F ( x ) | p d F ( x ) 1 / p = C ( p ) a . s . ,$
where $C ( p )$ is defined in (4).
Remark 2.
Applying the facts $C ( 1 ) = 3 6$ and $C ( 2 ) = 1 π$, Theorem 2 can result in the following corollary:
Corollary 2.
Under the assumptions of Theorem 1, for the continuous CDF F with bounded second order derivative, we have
$lim sup n → ∞ n 2 log ( log n ) ∫ − ∞ ∞ | F ^ n ( x ) − F ( x ) | d F ( x ) = 3 6 a . s .$
and
$lim sup n → ∞ n 2 log ( log n ) ∫ − ∞ ∞ | F ^ n ( x ) − F ( x ) | 2 d F ( x ) 1 / 2 = 1 π a . s .$
Detailed proofs of the above results are given below.

## 3. Proof

Set
$U n ( x ) : = 1 n ∑ i = 1 n { I ( X i ≤ x ) − F ( x ) } , x ∈ R .$
Therefore, (2) guarantees that
$lim sup n → ∞ n log ( log n ) sup x | U n ( x ) | = O ( 1 ) a . s .$
For independent uniform $[ 0 , 1 ]$ random variables: $ξ 1 , ξ 2 , ⋯ , ξ n$, we define
$V n ( v ) : = 1 n ∑ i = 1 n I ( ξ i ≤ v ) − v , ∀ v ∈ [ 0 , 1 ] .$
Then, $V n ( v )$ is a standardized uniform $[ 0 , 1 ]$ empirical process, and $U n ( x )$ has the same distribution as $V n ( F ( x ) )$. Using Theorem 4.4.3 and Theorem 1.15.2 in Csörgő and Révész (1981) [10], applying the Kiefer type approximation of the empirical process, there exists a Kiefer process ${ K ( s ; t ) : 0 ≤ s ≤ 1 , 0 ≤ t < ∞ }$ such that
$sup x | n U n ( x ) − K F ( x ) , n | = O ( ( log n ) 2 ) a . s . ,$
with $B n ( v ) = K ( v , n ) / n , 0 ≤ v ≤ 1$ being a Brownian bridge.
The Proof of Theorem 1 involves three parts: (i) applying the the triangular inequality to the distribution functions, (ii) using the the Kiefer approximation of the empirical process, and (iii) applying the the Taylor expansion. See below for details.
Proof of Theorem 1.
Rewrite $F ^ ( x )$,
$F ^ x = n − 1 ∑ i = 1 n ∫ − ∞ x k h u − ε i d u = n − 1 ∑ i = 1 n G x − ε i h ,$
where $G x = ∫ − ∞ x k u d u .$ By the definition of $F n x$, performing integration by parts and a change of variable $u = x − t h$, we can continue to rewrite $F ^ ( x )$, as follows:
$F ^ x = ∫ − ∞ + ∞ G x − t h d F n t = G x − t h F n t | − ∞ + ∞ + ∫ − ∞ + ∞ F n t k x − t h 1 h d t = ∫ − ∞ + ∞ F n t 1 h k x − t h d t = ∫ − ∞ + ∞ F n x − h u k u d u .$
Combining (14) with the properties $k ( u ) ≥ 0$ and $∫ − ∞ + ∞ k u d u = 1$, and applying the triangular inequality, we have that
$| F ^ x − F n x | = | ∫ − ∞ + ∞ F n x − h u − F n x k u d u | ≤ | ∫ − ∞ + ∞ [ F n x − h u − F x − h u ] − [ F n x − F x ] k u d u | + ∫ − ∞ + ∞ F ( x − h u ) − F ( x ) k u d u = | ∫ − ∞ + ∞ [ U n x − h u − U n x ] k u d u | + ∫ − ∞ + ∞ F ( x − h u ) − F ( x ) k u d u .$
Moreover, this results in
$sup x | F ^ x − F n x | ≤ sup x | ∫ − ∞ + ∞ [ U n x − h u − U n x ] k u d u | + sup x ∫ − ∞ + ∞ F ( x − h u ) − F ( x ) k u d u = : D 1 n + D 2 n , s a y .$
Thus, to show Theorem 1, it is sufficient to verify that
$lim sup n → ∞ n log ( log n ) D 1 n = 0 a n d lim n → ∞ n log ( log n ) D 2 n = 0 a . s .$
By $l o g ( l o g n ) → ∞$ and the integrability of $k ( u )$, it follows that
$∫ l o g ( l o g n ) + ∞ + ∫ − ∞ − l o g ( l o g n ) k ( u ) d u = o ( 1 ) .$
Partitioning the integral in $D 1 n$ into three parts, and using the triangular inequality, we can obtain
$D 1 n = sup x | ∫ − ∞ − l o g ( l o g n ) + ∫ l o g ( l o g n ) + ∞ + ∫ − l o g ( l o g n ) − l o g ( l o g n ) [ U n x − h u − U n x ] k u d u | ≤ sup x ∫ − ∞ − l o g ( l o g n ) + ∫ l o g ( l o g n ) + ∞ + ∫ − l o g ( l o g n ) l o g ( l o g n ) | U n x − h u − U n t | k u d u ≤ 2 sup x | U n ( x ) | ∫ − ∞ − l o g ( l o g n ) + ∫ l o g ( l o g n ) + ∞ k u d u + sup x ∫ − l o g ( l o g n ) l o g ( l o g n ) | U n x − h u − U n x | k u d u = : D 11 n + D 12 n , s a y .$
It is easy to see that (11) and (16) imply that
$lim sup n → ∞ n log ( log n ) D 11 n = 0 a . s .$
As for $D 12 n$, with the triangular inequality, (12), $∫ − l o g ( l o g n ) l o g ( l o g n ) k ( u ) d u ≤ 1$ and the continuity of modulus of $B n ( F ( x ) ) = K ( F ( x ) , n ) / n$, we have
$D 12 n ≤ sup x 1 n | ∫ − l o g ( l o g n ) l o g ( l o g n ) [ n U n x − h u − K ( F ( x − h u ) , n ) ] − [ n U n ( x ) − K ( F ( x ) , n ) ] k ( u ) d u | + sup x 1 n | ∫ − l o g ( l o g n ) l o g ( l o g n ) [ K ( F ( x − h u ) , n ) − K ( F ( x ) , n ) ] k ( u ) d u | ≤ 2 n sup x | n U n ( x ) − K ( F ( x ) , n ) | + O ( h log ( log n ) h log ( log n ) / n = O ( ( log n ) 2 n ) + O h log ( log n ) h log ( log n ) / n a . s .$
Hence, combining the above bound with $( log n ) 2 n log ( l o g n ) → 0$ and the assumption $h 3 / 2 log ( log n ) → 0$, it follows that
$lim sup n → ∞ n log ( log n ) D 12 n = 0 . a . s .$
Next, we proceed to evaluate $D 2 n$. Using the Taylor expansion with integral remainder, the properties $∫ − ∞ + ∞ u k ( u ) d u = 0$, $∫ − ∞ + ∞ u 2 k ( u ) d u < ∞$, $sup t | f ′ ( t ) | < ∞$ and $n h 4 / log ( log n ) → 0$, we obtain
$F ( x − h u ) − F ( x ) = − h u f ( x ) + ∫ x x − h u f ′ ( s ) ( x − h u − s ) d s = − h u f ( x ) + ∫ 0 − h u f ′ ( x − h u − t ) t d t , D 2 n = sup x ∫ − ∞ + ∞ [ ∫ 0 − h u f ′ ( x − h u − t ) t d t ] k ( u ) d u ≤ sup x | f ′ ( x ) | ∫ − ∞ + ∞ 1 2 h 2 u 2 k ( u ) d u = O ( h 2 ) , n log ( log n ) D 2 n = O ( n h 4 log ( log n ) ) → 0 .$
Therefore, (15) can be produced from (17)–(20). We have completed the proof of Theorem 1. □
Proof of Corollary 1.
Decomposing $F ^ ( x ) − F ( x )$ into two parts and using the triangular inequality, we have
$| F ^ ( x ) − F ( x ) | = | F ^ ( x ) − F n ( x ) + F n ( x ) − F ( x ) | ≤ | F ^ ( x ) − F n ( x ) | + | F n ( x ) − F ( x ) | , | F ^ ( x ) − F ( x ) | ≥ − | F ^ ( x ) − F n ( x ) | + | F n ( x ) − F ( x ) | .$
Then, it follows that
$sup x ∈ R | F ^ ( x ) − F ( x ) | ≤ sup x ∈ R | F ^ ( x ) − F n ( x ) | + sup x ∈ R | F n ( x ) − F ( x ) |$
and
$sup x ∈ R | F ^ ( x ) − F ( x ) | ≥ − sup x ∈ R | F ^ ( x ) − F n ( x ) | + sup x ∈ R | F n ( x ) − F ( x ) | .$
Combining the above inequalities with (8) and (2), this guarantees that
$lim sup n → ∞ n 2 log ( log n ) sup x | F ^ ( x ) − F ( x ) | = 1 2 a . s .$
Thus, we have finished the proof of Theorem 1. □
Proof of Theorem 2.
For any $p ≥ 1$, using the triangular inequality and the fact that $∫ − ∞ ∞ 1 d F ( x ) = 1$, we have that
$∫ − ∞ ∞ | F ^ ( x ) − F ( x ) | p d F ( x ) 1 / p ≤ ∫ − ∞ ∞ | F ^ ( x ) − F n ( x ) | p d F ( x ) 1 / p + ∫ − ∞ ∞ | F n ( x ) − F ( x ) | p d F ( x ) 1 / p ≤ sup x ∈ R | F ^ ( x ) − F n ( x ) | + ∫ − ∞ ∞ | F n ( x ) − F ( x ) | p d F ( x ) 1 / p$
and
$∫ − ∞ ∞ | F ^ ( x ) − F ( x ) | p d F ( x ) 1 / p ≥ − ∫ − ∞ ∞ | F ^ ( x ) − F n ( x ) | p d F ( x ) 1 / p + ∫ − ∞ ∞ | F n ( x ) − F ( x ) | p d F ( x ) 1 / p ≥ − sup x ∈ R | F ^ ( x ) − F n ( x ) | + ∫ − ∞ ∞ | F n ( x ) − F ( x ) | p d F ( x ) 1 / p .$
Note that we have proven
$sup x ∈ R n log ( log n ) | F ^ ( x ) − F n ( x ) | → 0 , a . s .$
in Theorem 1. Thus, combining it with (21), (22) and the law of the iterated logarithm for $L p$-norm of $F n ( x )$ in (5), it follows that
$lim sup n → ∞ n 2 log ( log n ) ∫ − ∞ ∞ | F ^ ( x ) − F ( x ) | p d F ( x ) 1 / p = C ( p ) a . s .$
Therefore, we have finished the proof of Theorem 2. □
Proof of Corollary 2.
Corollary 2 is the special case of results of Theorem 2 with $p = 1 , 2$. □

## Funding

This research received no external funding.

## Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

## Acknowledgments

The author is grateful to the Editor and two referees for their helpful comments and suggestions, which greatly improved the presentation of this article.

## Conflicts of Interest

The author declares no conflict of interest.

## References

1. Fabian, V.; Hannan, J. Introduction to Probability and Mathematical Statistics; Cengage Learning: Belmont, CA, USA, 1985. [Google Scholar]
2. Smirnov, N.V. An approximation to the distribution laws of random quantities deter- mined by empirical data. Uspehi. Mat. Nauk. 1944, 10, 179–206. [Google Scholar]
3. Chung, K.L. An estimate concerning the Kolmogorov limit distribution. Trans. Am. Math. Soc. 1949, 67, 36–50. [Google Scholar]
4. Finkelstein, H. The law of the iterated logarithm for empirical distributions. Ann. Math. Statist. 1971, 42, 607–615. [Google Scholar] [CrossRef]
5. Gajek, L.; Kahszka, M.; Lenic, A. The law of the iterated logarithm for Lp-norms of empirical processes. Statist. Probab. Lett. 1996, 28, 107–110. [Google Scholar] [CrossRef]
6. Yamato, H. Uniform convergence of an estimator of a distribution function. Bull. Math. Statist. 1973, 15, 69–78. [Google Scholar] [CrossRef] [PubMed]
7. Winter, B.B. Convergence Rate of Perturbed Empirical Distribution Functions. J. Appl. Probab. 1979, 16, 163–173. [Google Scholar] [CrossRef]
8. Wang, J.; Cheng, F.; Yang, L. Smooth simultaneous confidence band for cumulative distribution function. J. Nonparametr. Stat. 2013, 25, 395–407. [Google Scholar] [CrossRef]
9. Cheng, F. Strong uniform consistency rates of kernel estimators of cumulative distribution functions. Commun. Stat.—Theory Methods 2017, 46, 6803–6807. [Google Scholar] [CrossRef]
10. Csörgő, M.; Révész, P. Strong Approximation in Probability and Statistics; Academic Press: New York, NY, USA, 1981. [Google Scholar]
 Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

## Share and Cite

MDPI and ACS Style

Cheng, F. The Law of the Iterated Logarithm for Lp-Norms of Kernel Estimators of Cumulative Distribution Functions. Mathematics 2024, 12, 1063. https://doi.org/10.3390/math12071063

AMA Style

Cheng F. The Law of the Iterated Logarithm for Lp-Norms of Kernel Estimators of Cumulative Distribution Functions. Mathematics. 2024; 12(7):1063. https://doi.org/10.3390/math12071063

Chicago/Turabian Style

Cheng, Fuxia. 2024. "The Law of the Iterated Logarithm for Lp-Norms of Kernel Estimators of Cumulative Distribution Functions" Mathematics 12, no. 7: 1063. https://doi.org/10.3390/math12071063

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.