# A Concise Tutorial on Functional Analysis for Applications to Signal Processing

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Banach Spaces for Signal Analysis

#### 2.1. Introduction to Banach Spaces

**Definition**

**1**

**.**Let a vector space X be defined over a field $\mathbb{K}$, where examples of $\mathbb{K}$ are the field of real numbers $(\mathbb{R},+,\xb7)$ and the field of complex numbers $(\mathbb{C},+,\xb7)$. Let a function $\parallel \u2022\parallel :X\to \mathbb{R}$, called norm and denoted as $x\mapsto \parallel x\parallel $, have the following properties:

- (positivity) $\forall x\in X,\phantom{\rule{4pt}{0ex}}\parallel x\parallel \ge 0,\phantom{\rule{0.222222em}{0ex}}\forall x\in X$, and $\parallel x\parallel =0$ if and only if $x=0$.
- (homogeneity) $\forall x\in X$ and $\forall c\in \mathbb{K},\phantom{\rule{4pt}{0ex}}\parallel cx\parallel =\left|c\right|\parallel x\parallel $.
- (triangular inequality) $\forall x,y\in X,\phantom{\rule{4pt}{0ex}}\parallel x+y\parallel \le \parallel x\parallel +\parallel y\parallel $.

**Example**

**1.**

**Theorem**

**1**

**.**Let $1<p<\infty $ and $\frac{1}{p}+\frac{1}{q}=1$. If $f\in {\ell}^{p}$ and $g\in {\ell}^{q}$, then $f\xb7g\in {\ell}^{1}$ and ${\parallel f\xb7g\parallel}_{{\ell}^{1}}\le {\parallel f\parallel}_{{\ell}^{p}}{\parallel g\parallel}_{{l}^{q}}$.

**Proof.**

**Theorem**

**2**

**.**If $1\le p\le \infty $ and $f,g\in {\ell}^{p}$, then ${\parallel f+g\parallel}_{{\ell}^{p}}\le {\parallel f\parallel}_{{\ell}^{p}}+{\parallel g\parallel}_{{\ell}^{p}}$.

**Proof.**

**Theorem**

**3**

**.**For the sequences $\mathbf{u}\in {\ell}^{p}$ for $p\in [1,\infty ]$ and $\mathbf{h}\in {\ell}^{1}$, the convolution product $\mathbf{h}\ast \mathbf{u}\in {\ell}^{p}$ and ${\parallel \mathbf{h}\ast \mathbf{u}\parallel}_{{\ell}^{p}}\le {\parallel \mathbf{h}\parallel}_{{\ell}^{1}}{\parallel \mathbf{u}\parallel}_{{\ell}^{p}}$.

**Proof.**

**Lemma**

**1**

**.**If $\{{x}_{n}\}\in {l}^{p}$ for some $p\in [1,\infty )$, then ${lim}_{n\to \infty}\left|{x}_{n}\right|=0$.

**Proof.**

**Definition**

**2**

**.**A system is said to be bounded-input-bounded-output (BIBO)-stable if every $\mathbf{u}\in {\ell}^{\infty}\Rightarrow \mathbf{y}\in {\ell}^{\infty}$. More generally, the system is called ${\ell}^{p}$-stable if $\mathbf{u}\in {\ell}^{p}\Rightarrow \mathbf{y}\in {\ell}^{p}$, where $p\in [1,\infty ]$.

**Example**

**2**

**.**In a general setting, let us consider an adaptive filtering problem in Figure 2, where a measurement vector $\mathbf{x}\left[n\right]\triangleq {\left[{x}_{1}\left[n\right],{x}_{2}\left[n\right],\dots ,{x}_{N}\left[n\right]\right]}^{T}$ is used to construct an estimate, $\widehat{\mathbf{d}}\left[n\right]\triangleq (\mathbf{h}\star \mathbf{x})\left[n\right]$, of the desired signal $\mathbf{d}\left[n\right]$ by a linear shift-variant filter $\mathbf{h}\left[n\right]$ [7]. Then, the task is to synthesize an adaptive algorithm to update the filter $\mathbf{h}\left[n\right]$ such that the estimation error $\mathbf{e}\left[n\right]\triangleq \left(\mathbf{d}\left[n\right]-\widehat{\mathbf{d}}\left[n\right]\right)\to \mathbf{0}$ as $n\to \infty $. Using Lemma 1, this could be achieved if $\mathbf{e}\in {\ell}^{p}$ for some $p\in [1,\infty )$ in the adaptive algorithm.

#### 2.2. Hardy Spaces and Spectral Factorization for Signal Processing

**Definition**

**3**

**.**Let ${D}_{r}\left({z}_{0}\right)\triangleq \{z\in \mathbb{C}:|z-{z}_{0}|<r\}$ be the open disc of radius $r>0$ with center at ${z}_{0}\in \mathbb{C}$. A complex-valued function $f\left(r{e}^{i\theta}\right)$, where $\theta \in [0,2\pi )$, is said to be analytic in ${D}_{r}\left({z}_{0}\right)$ if the derivative of $f\left(r{e}^{i\theta}\right)$ exists at each point of ${D}_{r}\left({z}_{0}\right)$.

**Theorem**

**4**

**.**Let $S\left(z\right)$ be a complex-valued function of the complex variable z. If $ln\left(S\right)\in {H}^{1}$, then there exists a real positive constant ${K}_{0}$ and a complex-valued function ${H}_{ca}\left(z\right)$ corresponding to a causal stable system with a causal stable inverse such that

**Proof.**

**Remark**

**1.**

**Example**

**3**

**.**Consider a random sequence $x\left[n\right]$ with a complex spectral density function:

**Example**

**4**

**.**Let a random sequence $x\left[n\right]$ have a complex spectral density ${S}_{x}\left(z\right)={e}^{z+{z}^{-1}}$.

**Theorem**

**5**

**.**A general random sequence $\mathbf{x}\left[n\right]$ can be written as the sum of two processes as:

**Proof.**

#### 2.3. Weak Topology in a Banach Space

**Definition**

**4**

**.**Let ${T}^{k}\in BL(V,V)$ be a bounded linear operator from V into V. Then, the sequence $\{{T}^{k}\}$ converges to some $T\in BL(V,V)$ in the operator norm (also called uniform convergence) if the induced norm $\parallel (T-{T}^{k}){\parallel}_{ind}\triangleq {lim}_{k\to \infty}{sup}_{{\parallel x\parallel}_{V}=1}{\parallel (T-{T}^{k})x\parallel}_{V}=0$, which is denoted as: ${T}^{k}\stackrel{u\phantom{\rule{4pt}{0ex}}}{\to}T$.

**Definition**

**5**

**.**Let ${T}^{k}\in BL(V,V)$ be a bounded linear operator from V into V. Then, the sequence $\{{T}^{k}\}$ converges strongly to some $T\in BL(V,V)$ if ${lim}_{k\to \infty}{\parallel (T-{T}^{k})x\parallel}_{V}=0\phantom{\rule{4pt}{0ex}}\forall x\in V$, which is denoted as ${T}^{k}\stackrel{s\phantom{\rule{4pt}{0ex}}}{\to}T$.

**Definition**

**6**

**.**Let ${T}^{k}\in BL(V,V)$ be a bounded linear operator from V into V. Then, the sequence $\{{T}^{k}\}$ converges weakly to some $T\in BL(V,V)$ if

**Remark**

**2**

**.**⇒ (Strong convergence) ⇒ (Weak Convergence). The converse is not true, in general.

**Remark**

**3.**

In a finite-dimensional Banach space V, the weak topology generated by ${V}^{\star}$ is the same as the strong topology generated by V.

**Definition**

**7**

**.**Given a Banach space X, let there be a class of bounded linear functionals $\mathcal{F}\subseteq {X}^{\star}$, and let $\Im \left(\mathcal{F}\right)$ be the topology in X generated by $\mathcal{F}$. Then, for a given vector/function $g\in X$, a sequence $\{{f}_{n}\}\subset X$ is said to converge to g in the weak topology $\Im \left(\mathcal{F}\right)$, denoted as ${f}_{n}\stackrel{w}{\to}g$ in $\Im \left(\mathcal{F}\right)$, provided that ${F}_{\alpha}\left({f}_{n}\right)$ converges strongly to ${F}_{\alpha}\left(g\right)$, denoted as ${F}_{\alpha}\left({f}_{n}\right)\stackrel{s}{\to}{F}_{\alpha}\left(g\right)\phantom{\rule{4pt}{0ex}}\forall {F}_{\alpha}\in \mathcal{F}$.

**Remark**

**4.**

## 3. Hilbert Spaces for Signal Processing

**Definition**

**8**

**.**Let a vector space X be defined over a field $\mathbb{K}$, which is either $\mathbb{R}$ or $\mathbb{C}$. A function $\langle \u2022,\u2022\rangle :X\times X\to \mathbb{K}$ is called an inner product if, for $\forall x,y,z\in X$ and $\forall \alpha \in \mathbb{K}$, the following conditions hold:

- 1.
- (positive definiteness) $\langle x,x\rangle >0$ when $x\ne \mathbf{0}$;
- 2.
- (additivity) $\langle (x+y),z\rangle =\langle x,z\rangle +\langle y,z\rangle $;
- 3.
- (homogeneity) $\langle \alpha x,y\rangle =\alpha \langle x,y\rangle $;
- 4.
- (symmetry) $\langle x,y\rangle =\overline{\langle y,x\rangle}$

- $\langle x,(y+z)\rangle =\langle x,y\rangle +\langle x,z\rangle $;
- $\langle x,\alpha y\rangle =\overline{\alpha}\langle x,y\rangle $;

**Example**

**5.**

**Theorem**

**6**

**Proof.**

**Remark**

**5.**

**Theorem**

**7.**

**Theorem**

**8**

**.**Let H be a Hilbert space, and let $V\subset H$ be a closed subspace of H, implying that V is also a Hilbert space. Then, it follows that

**Proof.**

**Remark**

**6.**

**Theorem**

**9**

**.**Let H be a Hilbert space and let $V=span\{{e}_{1},{e}_{2},\dots \}$ be a subspace of H. If ${P}_{V}:H\to V$ denotes the orthogonal projection of elements in H into V, then, the Bessel inequality

#### 3.1. Fourier Series Expansion in a Hilbert Space

**Theorem**

**10**

**.**Let $\{{\phi}_{n}\}$ be an orthonormal set in a Hilbert space H. Then, the following statements are equivalent:

- 1.
- $\{{\phi}_{n}\}$ is an orthonormal basis of H, i.e., $\{{\phi}_{n}\}$ is a complete orthonormal set in H.
- 2.
- (Fourier series expansion) Any vector $x\in H$ can be expanded as: $x\stackrel{ms}{=}\phantom{\rule{4pt}{0ex}}{\sum}_{n\in \mathbb{N}}\phantom{\rule{4pt}{0ex}}\langle x,{\phi}_{n}\rangle {\phi}_{n}$. Note: The inner products $\langle x,{\phi}_{n}\rangle $ are called Fourier coefficients of the vector x.
- 3.
- (Parseval Equality) For any two vectors $x,y\in H$, the inner product: $\langle x,y\rangle ={\sum}_{n\in \mathbb{N}}\langle x,{\phi}_{n}\rangle $ $\overline{\langle y,{\phi}_{n}\rangle}$
- 4.
- The norm: ${\parallel x\parallel}^{2}={\sum}_{n\in \mathbb{N}}{\left|\langle x,{\phi}_{n}\rangle \right|}^{2}\phantom{\rule{4pt}{0ex}}\forall x\in H$
- 5.
- Let U be a subspace of H such that U contains the sequence $\{{\phi}_{n}\}$. Then, U is dense in H, i.e., $\overline{U}=H$.

**Proof.**

#### 3.2. Fourier Transform and Inverse Fourier Transform

#### 3.3. Windowed Fourier Transform in a Hilbert Space

**Example**

**6.**

#### 3.4. Wavelet Transform in a Hilbert Space

**Example**

**7.**

**Example**

**8.**

- $\varphi \left(u\right)\ge 0\phantom{\rule{4pt}{0ex}}\forall u\in \mathbb{R}$;
- ${\int}_{-\infty}^{\infty}du\phantom{\rule{4pt}{0ex}}\varphi \left(u\right)=1$;
- ${\int}_{-\infty}^{\infty}du\phantom{\rule{4pt}{0ex}}\varphi \left(u\right)\phantom{\rule{4pt}{0ex}}u=0$;
- ${\int}_{-\infty}^{\infty}du\phantom{\rule{4pt}{0ex}}\varphi \left(u\right)\phantom{\rule{4pt}{0ex}}{u}^{2}=1$;

#### 3.5. Karhunen-Loéve Expansion of Random Signals

**Theorem**

**11**

**.**Let $X\left(t\right)$ be a zero-mean, second-order random process, defined over $[-T/2,T/2]$ where $T\in (0,\infty )$, with a continuous covariance function ${K}_{XX}(t,\tau )$. Then, it follows that

**Remark**

**7.**

**Example**

**9**

**.**Let the covariance function of zero-mean stationary white noise $w\left(t\right)$ be ${K}_{ww}(t,\tau )={\sigma}^{2}\delta (t-\tau )$. Then, the orthonormal functions ${\varphi}_{n}\left(t\right)$ satisfy the K-L integral equation, for all $n\in \mathbb{N}$, as:

**Example**

**10**

**.**Let us assume that a waveform $X\left(t\right)$ is observed over a finite time interval $[-T/2,T/2]$ to decide whether it contains a recoverable signal buried in noise, or the signal is completely noise-corrupted (i.e., the signal cannot be recovered). In this regard, we formulate a binary hypothesis testing problem with the hypothesis ${H}_{1}$ of having a recoverable signal and the hypothesis ${H}_{0}$ of complete noise capture, i.e.,

#### 3.6. Reproducing Kernel Hilbert Spaces

**Definition**

**9**

**.**Let T be an arbitrary non-empty set (e.g., the time domain or the spatial domain of a function) and let H be a Hilbert space of real-valued (resp. complex-valued) functions on T, equipped with pointwise vector addition and pointwise scalar multiplication, and the continuous functions in H are evaluated at each point $t\in T$. Then, H is defined to be a reproducing kernel Hilbert space (RKHS) if there exist a positive real ${M}_{t}$ and a continuous linear functional ${\mathcal{L}}_{t}$ on H such that $|{\mathcal{L}}_{t}\left(f\right)|=|f\left(t\right)|\le {M}_{t}{\parallel f\parallel}_{H}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\forall t\in T\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\forall f\in H$. [Note: Although ${M}_{t}$ is constrained to be a positive real, it is possible that ${sup}_{t\in T}{M}_{t}=\infty $.]

**Remark**

**8.**

**Example**

**11**

**.**Let us consider the space of continuous signals that are also band-limited with frequencies under the compact support, i.e., in the range of $[-2\pi \mathsf{\Omega},\phantom{\rule{4pt}{0ex}}2\pi \mathsf{\Omega}]$, where the cutoff frequency $\mathsf{\Omega}\in (0,\infty )$. It is noted that ${K}_{t}(\u2022)$ is a bandlimited version of the Dirac delta function, because ${K}_{t}\left(\tau \right)$ converges to the delta distribution, expressed as $\delta (\tau -t)$ in the weak sense, as the cutoff frequency Ω tends to infinity.

## 4. Summary and Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Preliminary Concepts

#### Appendix A.1. Metric Spaces and Topological Spaces

**Definition**

**A1**

**.**Let X be a non-empty set. A function $\rho :X\times X\to \mathbb{R}$, where $\mathbb{R}$ is the space of real numbers, is called a metric (or a distance function) on X if the following conditions hold for all $x,y,z\in X$:

- (i)
- Positivity: $\rho (x,y)\ge 0$, and $\rho (x,y)=0$ iff $x=y$;
- (ii)
- Symmetry: $\rho (x,y)=\rho (y,x)$;
- (iii)
- Triangular Inequality: $\rho (x,z)\le \rho (x,y)+\rho (y,z)$.

**Example**

**A1.**

**Remark**

**A1.**

**Definition**

**A2**

**.**A set $E\subseteq X$ in a metric space $(X,\rho )$ is called open if, for all $y\in E$, there exists an open ball ${B}_{\epsilon}\left(y\right)\triangleq \{x\in E:\phantom{\rule{4pt}{0ex}}\rho (x,y)<\epsilon \}$, which is of radius $\epsilon >0$ with center at y. A set $F\subseteq X$ is called closed if the complement $X\setminus F$ is open in $(X,\rho )$.

**Definition**

**A3**

**.**A sequence $\{{x}_{n}\}$ in a metric space $(X,\rho )$ is called a Cauchy sequence if $\forall \epsilon >0\phantom{\rule{4pt}{0ex}}\exists n\left(\epsilon \right)\in \mathbb{N}\phantom{\rule{4pt}{0ex}}\mathrm{such}\phantom{\rule{4pt}{0ex}}\mathrm{that}\phantom{\rule{4pt}{0ex}}\rho ({x}_{k},{x}_{\ell})<\epsilon \phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\forall k,\ell >n$. In other words, $\rho ({x}_{k},{x}_{\ell})\to 0$ as $k,\ell \to \infty $.

**Definition**

**A4**

**.**A metric space is called complete if every Cauchy sequence converges in the metric space,

**Definition**

**A5**

**.**A metric space $(X,\rho )$ is said to be sequentially compact if every sequence of points $\{{x}_{1},{x}_{2},\dots \}$ in $(X,\rho )$ contains a convergent subsequence $\{{{x}_{n}}_{1},{{x}_{n}}_{2},\dots \}$, and it is called sequentially precompact if every sequence of points $\{{x}_{1},{x}_{2},\dots \}$ in $(X,\rho )$ contains a Cauchy subsequence $\{{{x}_{n}}_{1},{{x}_{n}}_{2},\dots \}$ (see Definition A3 and also see [29]).

**Definition**

**A6**

**.**A topology on a nonempty set X is a collection ℑ of subsets of X, which has the following properties:

- 1.
- ℑ must contain the empty set ∅ and the set X.
- 2.
- Any union of the members in ℑ, belonging to an arbitrary (i.e., finite, countable, or uncountable) (A set is defined to be countable if it is bijective to the set, $\mathbb{N}$, of positive integers; and a finite or a countably infinite set is often called at most countable. An infinite set, which is not countable, is called uncountable. For example, the set of integers, $\mathbb{Z}\u228a\mathbb{R}$, is countable, while an interval $(a,b)\triangleq \{x\in \mathbb{R}:\phantom{\rule{4pt}{0ex}}a<x<b\}$ is uncountable. These concepts lead to the fundamental difference between “continuous-time (CT) analog" and “discrete-time (DT) digital” signal processing.) subcollection of sets must be contained in ℑ.
- 3.
- The intersection of the members of any finite subcollection of ℑ must be contained in ℑ.

**Definition**

**A7**

**.**A basis $\mathcal{B}$ for a topology $(X,\Im )$ is a collection of open sets in ℑ, called basis elements, if the following two conditions hold:

- 1.
- For each $x\in X$, there exists at least one basis element $B\in \mathcal{B}$ such that $x\in B$.
- 2.
- If $x\in {B}_{1}\cap {B}_{2}$, where ${B}_{1},{B}_{2}\in \mathcal{B}$, then there exists a basis element ${B}_{3}\in \mathcal{B}$ such that $x\in {B}_{3}$ and ${B}_{3}\subseteq {B}_{1}\cap {B}_{2}$.

#### Appendix A.2. Random Variables and Stochastic Processes

**Definition**

**A8**

**.**Let $\mathcal{F}$ be a (non-empty) collection of subsets of a (non-empty) set Ω having some or all of the following properties.

- (a)
- $\mathsf{\Omega}\in \mathcal{F}$.
- (b)
- If $A\in \mathcal{F}$ then ${A}^{c}\in \mathcal{F}$, where ${A}^{c}\triangleq \mathsf{\Omega}\setminus A$.
- (c)
- If ${A}_{1},{A}_{2},\dots ,{A}_{n}\in \mathcal{F}$ then ${\bigcup}_{i=1}^{n}{A}_{i}\in \mathcal{F}$.
- (d)
- If ${A}_{1},{A}_{2},\dots \in \mathcal{F}$ then ${\bigcup}_{i=1}^{\infty}{A}_{i}\in \mathcal{F}$,

**Remark**

**A2.**

**Definition**

**A9**

**.**Given a non-empty collection $\mathcal{D}$ of subsets of Ω, the smallest σ-algebra containing $\mathcal{D}$ is called the σ-algebra generated by $\mathcal{D}$. The Borel $\sigma $-algebra $\mathcal{B}\left(\mathbb{R}\right)$ is the σ-algebra generated by the collection of all open intervals $\{(a,b):a,b\in \mathbb{R}\}$ in the usual topology of $\mathbb{R}$. Members of $\mathcal{B}\left(\mathbb{R}\right)$ are called Borel sets.

**Definition**

**A10**

**.**A countably additive measure μ on a σ-algebra $\mathcal{F}$ is a non-negative, extended real-valued function on $\mathcal{F}$ such that if $\{{A}_{1},{A}_{2},\dots \}$ forms an at most countable (i.e., finite or countably infinite) collection of disjoint sets in $\mathcal{F}$, then $\mu \left({\bigcup}_{n}{A}_{n}\right)={\sum}_{n}\mu \left({A}_{n}\right)$. A measurable space is a pair $(\mathsf{\Omega},\mathcal{F}$), and a measure space is a triple $(\mathsf{\Omega},\mathcal{F},\mu )$, where Ω is a non-empty set, $\mathcal{F}$ is a σ-algebra of subsets of Ω, and μ is a measure on $\mathcal{F}$. The sets in $\mathcal{F}$ are called measurable sets.

**Example**

**A2.**

**Definition**

**A11**

**.**If $\mu \left(\mathsf{\Omega}\right)=1$, then μ is called a probability measure, usually denoted by P, and the triplet $(\mathsf{\Omega},\mathcal{F},P)$ is called a probability space.

**Definition**

**A12**

**.**Let $({\mathsf{\Omega}}_{1},{\mathcal{F}}_{1})$ and $({\mathsf{\Omega}}_{2},{\mathcal{F}}_{2})$ be two measurable spaces. A function $f:({\mathsf{\Omega}}_{1},{\mathcal{F}}_{1})\to ({\mathsf{\Omega}}_{2},{\mathcal{F}}_{2})$ is called $({\mathcal{F}}_{1}-{\mathcal{F}}_{2})$ measurable if the inverse image ${f}^{-1}\left(A\right)\in {\mathcal{F}}_{1}$ $\forall A\in {\mathcal{F}}_{2}$. If ${\mathsf{\Omega}}_{2}=\mathbb{R}$ and ${\mathcal{F}}_{2}=\mathcal{B}\left(\mathbb{R}\right)$ then f is said to be Borel measurable.

**Definition**

**A13**

**.**A random variable X on a probability space $(\mathsf{\Omega},\mathcal{F},P)$ is a Borel measurable function from Ω to $\mathbb{R}$. Similarly, a sequence of random variables $\{{X}_{1},{X}_{2},\dots \}$ is called a discrete random process.

**Remark**

**A3.**

- The expected value $E\left[x\right(t\left)\right]$ is a constant for all t;
- The autocorrelation ${r}_{x}(t,\tau )$ depends only on the difference $(t-\tau )$, not explicitly on both t and $\tau $.

## References

- Bachman, G.; Narici, L. Functional Analysis; Academic Press: New York, NY, USA, 1966. [Google Scholar]
- Naylor, A.; Sell, G. Linear Operator Theory in Engineering and Science, 2nd ed.; Springer-Verlag: New York, NY, USA, 1982. [Google Scholar]
- Rudin, W. Real and Complex Analysis; McGraw-Hill: Boston, MA, USA, 1987. [Google Scholar]
- Royden, H. Real Analysis, 3rd ed.; Macmillan: New York, NY, USA, 1989. [Google Scholar]
- Kreyszig, E. Introductory Functional Analysis with Applications; John Wiley & Sons: Hoboken, NJ, USA, 1978. [Google Scholar]
- Bobrowski, A. Functional Analysis for Probability and Stochastic Processes; Cambridge University Press: Cambridge, UK, 2005. [Google Scholar]
- Hayes, M. Statistical Digital Signal Processing and Modeling, 1st ed.; Wiley: Hoboken, NJ, USA, 1996. [Google Scholar]
- Haykin, S. Adaptive Filter Theory, 4th ed.; Prentice Hall: Englewood Cliffs, NJ, USA, 2002. [Google Scholar]
- Farhang-Boroujeny, B. Adaptive Filters Theory and Applications; John Wiley & Sons: Hoboken, NJ, USA, 2003. [Google Scholar]
- Bressan, A. Lecture Notes on Functional Analysis with Applications to Linear Partial Differential Equations; American Mathematical Society: Providence, RI, USA, 2013. [Google Scholar]
- Reed, M.; Simon, B. Methods of Modern Mathematical Physics Part 1: Functional Analysis; Academic Press: Cambridge, MA, USA, 1980. [Google Scholar]
- Luenberger, D. Optimization by Vector Space Methods; John Wiley & Sons: Hoboken, NJ, USA, 1969. [Google Scholar]
- Desoer, C.; Vidyasagar, M. Feedback Systems: Input-Output Properties; Academic Press: Cambridge, MA, USA, 1975. [Google Scholar]
- Therrien, C. Discrete Random Signals and Statistical Signal Processing; Prentice Hall: Englewood Cliffs, NJ, USA, 1992. [Google Scholar]
- Proakis, J.; Manolakis, D. Digital Signal Processing: Principles, Algorithms, and Applications, 3rd ed.; Macmillan Publishing Company: New York, NY, USA, 1998. [Google Scholar]
- Oppenheim, A.; Schafer, R. Discrete-Time Signal Processing; Prentice Hall: Englewood Cliffs, NJ, USA, 1989. [Google Scholar]
- Vapnik, V.; Izmailov, R. Rethinking statistical learning theory: Learning using statistical invariants. Mach. Learn.
**2019**, 108, 381–423. [Google Scholar] [CrossRef][Green Version] - Ghalyan, N.F.; Ray, A. Symbolic Time Series Analysis for Anomaly Detection in Measure-invariant Ergodic Systems. J. Dyn. Syst. Meas. Control.
**2020**, 142, 061003. [Google Scholar] [CrossRef] - Ghalyan, N.F.; Ray, A. Measure invariance of symbolic systems for low-delay detection of anomalous events. Mech. Syst. Signal Process.
**2021**, 159, 107746. [Google Scholar] [CrossRef] - Lorch, E. Spectral Analysis; Oxford University Press: New York, NY, USA, 1962. [Google Scholar]
- Kaiser, G. A Friendly Guide to Wavelets; Birkhauser: Boston, MA, USA, 1994. [Google Scholar]
- Mallat, S. A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed.; Academic Press: Amsterdam, The Netherlands, 2009. [Google Scholar]
- Ray, A. On State-space Modeling and Signal Localization in Dynamical Systems. ASME Lett. Dyn. Syst. Control.
**2022**, 2, 011006. [Google Scholar] [CrossRef] - Vetterli, M.; Kovacevic, J. Wavelets and Subband Coding; Prentice-Hall, Inc.: Hoboken, NJ, USA, 1995. [Google Scholar]
- Stark, H.; Woods, J. Probability and Random Processes with Applications to Signal Processing; Prentice-Hall: Upper Saddle River, NJ, USA, 2002. [Google Scholar]
- Helstrom, C. Elements of Signal Detection and Estimation; Prentice Hall: Englewood Cliffs, NJ, USA, 1995. [Google Scholar]
- Ash, R. Real Analysis and Probability; Academic Press: Boston, MA, USA, 1972. [Google Scholar]
- Munkres, J. Topology, 2nd ed.; Prentice-Hall: Upper Saddle River, NJ, USA, 2000. [Google Scholar]
- Shilov, G. Elementary Real and Complex Analysis; Dover Publication Inc.: Mineola, NY, USA, 1996. [Google Scholar]
- Papoulis, A. Probability, Random Variables, and Stochastic Processes, 2nd ed.; McGraw-Hill, Inc.: Boston, MA, USA, 1984. [Google Scholar]

**Figure 2.**An adaptive filter consisting of a shift-variant filter

**h**with an adaptive algorithm for updating the filter coefficients.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ghalyan, N.F.; Ray, A.; Jenkins, W.K.
A Concise Tutorial on Functional Analysis for Applications to Signal Processing. *Sci* **2022**, *4*, 40.
https://doi.org/10.3390/sci4040040

**AMA Style**

Ghalyan NF, Ray A, Jenkins WK.
A Concise Tutorial on Functional Analysis for Applications to Signal Processing. *Sci*. 2022; 4(4):40.
https://doi.org/10.3390/sci4040040

**Chicago/Turabian Style**

Ghalyan, Najah F., Asok Ray, and William Kenneth Jenkins.
2022. "A Concise Tutorial on Functional Analysis for Applications to Signal Processing" *Sci* 4, no. 4: 40.
https://doi.org/10.3390/sci4040040