# Sampling Theorems for Stochastic Signals. Appraisal of Paul L. Butzer’s Work

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

## 2. Paul Butzer’s Stochastic Sampling Theoretical Research

- [A] Butzer, P.L. A survey of the Whittaker-Shannon sampling theorem and some of its extensions. J. Math. Res. Exposition
**1983**, 3, 185–212. - [B] Butzer, P.L., Splettstösser, W. Stens, R.L. The sampling theorem and linear prediction in signal analysis. Jahresber. Deutsch. Math.-Verein.
**1988**, 90, 1–70.

- [C] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. I. General theorems on rates of convergence. Math. Methods Appl. Sci.
**1979**, 1, 241–264. - [D] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. II. Models of the general random noise and pulse train processes. Math. Methods Appl. Sci.
**1979**, 1, 346–353.

- [E] Butzer, P.L., Stens R.L. Prediction of nonbandlimited signals from past samples in terms of splines of low degree. Math. Nachr.
**1987**, 132, 115–130.

## 3. The Results

**boldface**). Let $f\in \mathrm{C}(\mathbb{R})$. The

**modulus of continuity**with respect to the difference order r is defined by

**stochastic process**$\{X(t,\omega )\equiv X(t):t\in \mathbb{R}\}$ defined on a standard probability space $(\Omega ,\mathcal{A},\mathsf{P})$ is an $\mathcal{A}$–measurable function, that is $X({t}_{0},\omega )$ becomes a random variable for any fixed $t={t}_{0}\in \mathbb{R}$, while $X(t,{\omega}_{0}),{\omega}_{0}\in \Omega $ is a deterministic function called trajectory of X. The process $X(t)$ belongs to the class of all finite second moment random variables ${\mathrm{L}}^{2}(\Omega ):={\mathrm{L}}^{2}(\Omega ,\mathcal{A},\mathsf{P})$. This class possesses a Hilbert space structure with scalar product defined by the expectation operator $\mathsf{E}$, that is $\langle X,Y\rangle =\mathsf{E}X{Y}^{\ast}$ and with the norm ${\u2225X\u2225}_{{\mathrm{L}}^{2}}=\sqrt{{\mathsf{E}\left|X\right|}^{2}}$ equipped.

**Karhunen–Cramér theorem**, which characterization theorem can also be called generalized harmonizability theorem [25,26,27], reads as follows [5] (p. 156). Assume that the correlation function ${R}_{X}(t,s)$ of a stochastic process $X(t)$ possesses a double integral representation in the form

**modulus of continuity for stochastic signal X**is defined by

**convolution of a process**$X\in {\mathrm{L}}^{2}(\Omega )$ with a (deterministic) function $g\in {\mathrm{L}}^{1}(\mathbb{R})$ that turns out to be the (stochastic) integral

**Theorem**

**1**([B] (p. 57, Theorem 6.11.))

**.**

**Theorem**

**2**([B] (p. 57, Theorem 6.12.))

**.**

**generalized sampling expansion**[B] (pp. 24–25) when the sinc kernel is replaced with another suitable kernel $\phi $:

**(a)**- there holds true $f(t)={lim}_{w\to \infty}({S}_{w}^{\phi}f)(t)$, for all $f\in \mathrm{C}(\mathbb{R})$, when $t\in \mathbb{R}$;
**(b)**- ${\left\{{S}_{w}^{\phi}\right\}}_{w>0}$ defines a family of bounded linear operators from $\mathrm{C}(\mathbb{R})$ into itself, so that$${\u2225{S}_{w}^{\phi}f\u2225}_{[\mathrm{C},\mathrm{C}]}={m}_{0}(\phi ),\phantom{\rule{1.em}{0ex}}\mathrm{and}\phantom{\rule{1.em}{0ex}}\underset{w\to \infty}{lim}{\u2225f-{S}_{w}^{\phi}f\u2225}_{\mathrm{C}}=0.$$

**Theorem**

**3**([B] (p. 58, Theorem 6.14))

**.**

**Theorem**

**4**([B] (p. 59, Theorem 6.16))

**.**

**(c)**- for r–fold differentiable WSS process X, we have$${\u2225X(t)-({S}_{w}^{\phi}X)(t)\u2225}_{{\mathrm{L}}^{2}}^{2}\le {\displaystyle \frac{({m}_{0}(\phi )+3){m}_{2r}(\phi )}{(2\phantom{\rule{0.166667em}{0ex}}r)!\phantom{\rule{0.166667em}{0ex}}{w}^{2r}}}\phantom{\rule{0.166667em}{0ex}}{R}_{X}^{(2r)}(0)\phantom{\rule{0.166667em}{0ex}};$$
**(d)**- moreover, there exists an absolute constant $K>0$, such that for WSS process $X\in {\mathrm{L}}^{2}(\Omega )$ there holds$${\u2225X(t)-({S}_{w}^{\phi}X)(t)\u2225}_{{\mathrm{L}}^{2}}\le K\phantom{\rule{0.166667em}{0ex}}{\omega}_{r}({w}^{-1};X;{\mathrm{L}}^{2}(\Omega )),\phantom{\rule{2.em}{0ex}}t\in \mathbb{R},w>0\phantom{\rule{0.166667em}{0ex}}.$$

**(d)**in Theorem 4, gives an excellent insight into analysis background of probabilistic setting of the aliasing approximation error’s magnitude in function of the bandwidth $w>0$ of the input stochastic process by its generalized sampling expansion series.

## 4. Discussion

**(ii)**studying the resulting truncated series; now, of the correlation function, the authors describe the final behavior of this double sum. Actually, Butzer and co-authors follow the similar derivation steps as the great probabilists Kari Karhunen and Harald Cramér when they establish the above mentioned, so-called Karhunen–Cramér theorem. Karhunen’s and Cramér’s proving method was based on the isometric isomorphy between the so-called Hilbert space of the stochastic process $X(t)$, $\mathcal{H}(X):={\mathcal{L}}^{2}\overline{\{X(t):t\in \mathbb{R}\}}$, say, which is the in medio closure of the linear span of the set $\{X(t):t\in \mathbb{R}\}$, and the H–space

## Funding

## Acknowledgments

**Harmonic Analysis and Applications - Strobl’18**conference in Strobl, Austria, held June 4-8, 2018, to invent and organize a special section

**’Sampling Theory after Paul Butzer’ to honor the 90th birthday of Paul Leo Butzer**. Thanking to this initiative the author has the excellent opportunity to expose his homage talk entitled On generalized derivative sampling series expansion in which several parts of this work were presented. In addition, the author is indebted to all three anonymous referees for their careful reading, constructive suggestions and reorganization requests of the first draft of the manuscript. By their kind guiding process, this survey note encompasses its final desired form.

## Conflicts of Interest

## References

- Kotel’nikov, V.A. On the carrying capacity of the “ether” and wire in telecommunications. In Material for the First USSR, All-Union Conference and Questions of Communications; Izdanie Red. Upr. Svyazi RKKA: Moscow, Russia, 1933; pp. 1–19, Reprint in Benedetto, J.J., Ferreira, P.J.S.G. (Eds.) Modern Sampling Theory: Mathematics and Applications; Birkhäuser, Boston, MA, USA, 2000. [Google Scholar]
- Shannon, C.E. A mathematical theory of communication. Bell Sys. Tech. J.
**1948**, 27, 623–657. [Google Scholar] [CrossRef] - Shannon, C.E. Communication in the presence of noise. Proc. IRE
**1949**, 37, 10–21. [Google Scholar] [CrossRef] - Kolmogorov, A.N. On the Shannon theory of information transmission in the case of continuous signals. IRE Trans. Inf. Theory
**1956**, 2, 102–108. [Google Scholar] [CrossRef] - Yaglom, A.M. Correlation Theory of Stationary and Related Random Functions II. Supplementary Notes and References; Springer Series in Statistics; Springer: New York, NY, USA, 1987. [Google Scholar]
- Belyaev, Y.K. Analytic random processes. Teor. Veroyat. Primenen.
**1959**, III, 437–444. (In Russian) [Google Scholar] [CrossRef] - Ville, J. Signaux analytiques a spectre borne. Bull. Tech. Inf. SOTELEC
**1949**, 4, 1–18. [Google Scholar] - Ville, J. Signaux analitiques a spectre borne. Câbles et Transm.
**1950**, 4, 9–23. [Google Scholar] - Ville, J. Signaux analitiques a spectre borne II. Câbles et Transm.
**1953**, 7, 44–53. [Google Scholar] - Oswald, J. Signaux aletoires a spectre limite. Câbles et Transm.
**1951**, 5, 158–177. [Google Scholar] - Lloyd, S.P. A sampling theorem for stationary (wide sense) stochastic processes. Trans. Amer. Math. Soc.
**1959**, 92, 1–12. [Google Scholar] - Parzen, E. A Simple Proof and Some Extensions of the Sampling Theorem; Tech. Rep. No. 7; Stanford University: Stanford, CA, USA, 1956. [Google Scholar]
- Balakrishnan, A.V. A note on the sampling principle for continuous signals. IRE Trans. Inf. Theory
**1957**, 3, 143–146. [Google Scholar] [CrossRef] - Draščić, B. Sampling reconstruction of stochastic signals—The roots in the fifties. Austrian J. Statist.
**2007**, 36, 65–72. [Google Scholar] [CrossRef] - Jerri, J.A. The Shannon sampling theorem—Its various extensions and applications: A tutorial review. Proc. IEEE
**1977**, 65, 1565–1596. [Google Scholar] [CrossRef] - Khurgin, Y.I.; Yakovlev, V.P. Progress in the Soviet Union on the theory and applications of bandlimited functions. Proc. IEEE
**1977**, 65, 1005–1028. [Google Scholar] [CrossRef] - Higgins, J.R. Five short stories about the cardinal series. Bull. Am. Math. Soc. (N.S.)
**1985**, 12, 45–89. [Google Scholar] [CrossRef] [Green Version] - Unser, M. Sampling-50 years after Shannon. Proc. IEEE
**2000**, 88, 569–587. [Google Scholar] [CrossRef] [Green Version] - Pogány, T. Almost sure sampling restoration of bandlimited stochastic signals. In Sampling Theory in Fourier and Signal Analysis: Advanced Topics; Higgins, J.R., Stens, R.L., Eds.; Oxford University Press: Oxford, UK, 1999; pp. 203–232, 284–286. [Google Scholar]
- Piranašvili, Z.A. The problem of interpolation of random processes. Teor. Verojatnost. Primenen.
**1967**, 12, 708–717. (In Russian) [Google Scholar] - Honda, I. On the rate of the central limit theorem for a class of linear processes. Rep. Statist. Appl. Res. Jpn. Sci. Eng.
**1979**, 26, 18–28. [Google Scholar] - Pogány, T. On the perfect linear prediction of bandlimited stationary stochastic processes. In Proceedings of the KOI’91; Neralić, L., Ed.; Ekonomski Fakultet: Zagreb, Croatia, 1991; pp. 259–264. [Google Scholar]
- Yaglom, A.M. An Introduction to the Theory of Stationary Random Functions, Revised English ed.; Silverman, R.A., Ed.; Prentice-Hall, Inc.: Englewood Cliffs, NJ, USA, 1962. [Google Scholar]
- Pogány, T. An approach to the sampling theorem for continuous time processes. Aust. J. Statist.
**1989**, 31, 427–432. [Google Scholar] [CrossRef] - Karhunen, K. Über lineare Methoden in der Wahrscheinlichkeitsrechnung. Ann. Acad. Sci. Fennicae Ser. A
**1947**, 37, 3–79. [Google Scholar] - Cramér, H. A contribution to the theory of stochastic processes. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability; University of California Press: Berkeley/Los Angeles, CA, USA, 1951; pp. 329–339. [Google Scholar]
- Yaglom, A.M. Correlation Theory of Stationary and Related Random Functions I. Basic Results; Springer Series in Statistics; Springer: New York, NY, USA, 1987. [Google Scholar]
- de la Vallée Poussin, C. Sur la meilleure approximation des fonctions d’une variable réelle par des expressions d’ordre donné. C. R. Acad. Sci. Paris
**1918**, 166, 799–802. [Google Scholar] - Brown, J.R., Jr. On mean-square aliasing error bound in cardinal sampling expansion of random processes. IEEE Trans. Inf. Theory
**1978**, IT-24, 254–256. [Google Scholar] [CrossRef] - Pogány, T. On the aliasing error upper bound for homogeneous random fields. Singal Process.
**1993**, 33, 127–129. [Google Scholar] [CrossRef] - Görlich, E.; Krieg, A.; Nessel, R.J.; Stens, R.L.; Paul, L. Butzer–Five years as professor emeritus. Results Math.
**1998**, 34, 20–31. [Google Scholar]

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pogány, T.K.
Sampling Theorems for Stochastic Signals. Appraisal of Paul L. Butzer’s Work. *Axioms* **2019**, *8*, 91.
https://doi.org/10.3390/axioms8030091

**AMA Style**

Pogány TK.
Sampling Theorems for Stochastic Signals. Appraisal of Paul L. Butzer’s Work. *Axioms*. 2019; 8(3):91.
https://doi.org/10.3390/axioms8030091

**Chicago/Turabian Style**

Pogány, Tibor K.
2019. "Sampling Theorems for Stochastic Signals. Appraisal of Paul L. Butzer’s Work" *Axioms* 8, no. 3: 91.
https://doi.org/10.3390/axioms8030091