Extended Divergence on a Foliation by Deformed Probability Simplexes

Abstract

This study considers a new decomposition of an extended divergence on a foliation by deformed probability simplexes from the information geometry perspective. In particular, we treat the case where each deformed probability simplex corresponds to a set of q-escort distributions. For the foliation, different q-parameters and the corresponding $\alpha$-parameters of dualistic structures are defined on each of the various leaves. We propose the divergence decomposition theorem that guides the proximity of q-escort distributions with different q-parameters and compare the new theorem to the previous theorem of the standard divergence on a Hessian manifold with a fixed $\alpha$-parameter.

1. Introduction

In the field of nonextensive statistics, q-normal distributions and the generalization, q-exponential families, play an important role [1,2,3]. Since Ohara first pointed out the correspondence between the q-parameter of nonextensive statistics and the $\alpha$-parameter of information geometry [4,5], the information geometric structure of q-exponential families has been investigated [6,7,8,9,10,11,12,13,14].

On a set of probability distributions, divergences are usually defined for a fixed $\alpha$-parameter of the dualistic structure. Using those results, we defined an extended divergence on a foliation by sets of probability distributions, setting different $\alpha$-parameters on each leaf. In particular, we treated a foliation by deformed probability simplexes [15].

In this paper, we also study deformed probability simplexes corresponding to sets of escort distributions with q-parameters, which satisfy $q=(1-\alpha )/2$ for $\alpha$-parameters of information geometry. We clarify the relationship among affine spaces, affine immersions and the extended divergence more than in our previous paper. A comparison with the extended divergence and the duo Bregman divergence used in machine learning is also described [16].

First, we explain the dualistic structures, $\alpha$-divergences, and the Tsallis relative entropy on the probability simplex, using the concept of affine geometry and information geometry. The relationship between an $\alpha$-parameter and the Tsallis q-parameter is stated. Next, we describe the dualistic structures and the divergences generated by affine immersions on the deformed probability simplexes corresponding to sets of escort distributions. It also includes topics about Hessian manifolds and their level surfaces. We then define an extended divergence on a foliation by deformed probability simplexes. Finally, we propose a new decomposition of an extended divergence on the foliation.

2. The Tsallis Relative Entropy and the Kullback–Leibler Divergence on the Probability Simplex

In this section, we explain dualistic structures, $\alpha$-divergences, and the Tsallis relative entropy on the probability simplex [4,5,12].

Let ${\mathbf{A}}^{n+1}$ be an $(n+1)$-dimensional real affine space and $\{x^1,\cdots ,x^{n+1}\}$ be the canonical affine coordinate system on ${\mathbf{A}}^{n+1}$, i.e., $\tilde{D}dx=0$, where $\tilde{D}$ is the canonical flat affine connection on ${\mathbf{A}}^{n+1}$. Let ${\mathcal{S}}^n$ be a simplex in ${\mathbf{A}}_{+}^{n+1}$ defined by

$${\mathcal{S}}^n=\{p|p\in {\mathbf{A}}_{+}^{n+1},\sum _{i=1}^{n+1}{x}^i\left(p\right)=1\}.$$

(1)

If $x^1\left(p\right),\cdots ,x^{n+1}\left(p\right)$ are regarded as probabilities of $n+1$ states, ${\mathcal{S}}^n$ is called the n-dimensional probability simplex. Let $\{{\overline{p}}^1,\cdots ,{\overline{p}}^n\}$ be an affine coordinate system on ${\mathcal{S}}^n$ defined by ${\overline{p}}^i\left(p\right)=x^i\left(p\right)-x^{n+1}\left(p\right)$ for $i=1,\cdots ,n$, and

$$\{{\partial }_1,\cdots ,{\partial }_n\},where{\partial }_i{|}_p=\left(\frac{\partial }{\partial x^i}-\frac{\partial }{\partial x^{n+1}}\right){|}_p,p\in {\mathcal{S}}^n,$$

(2)

be a frame of a tangent vector field on ${\mathcal{S}}^n$.

The Fisher metric $g=\left(g_{ij}\right)$ on ${\mathcal{S}}^n$ is defined by

$$g_{ij}\left(p\right)\equiv g({\partial }_i,{\partial }_j){|}_p=\sum _{k=1}^{n+1}{x}^k\left(p\right)\frac{\partial log{x}^k}{\partial x^i}{|}_p\frac{\partial log{x}^k}{\partial x^j}{|}_p=\frac{1}{x^i\left(p\right)}{\delta }_{ij}+\frac{1}{x^{n+1}\left(p\right)},$$

(3)

$$p\in {\mathcal{S}}^n,i,j=1,\cdots ,n,$$

where ${\delta }_{ij}$ is the Kronecker’s delta. We define an $\alpha$-connection ${\nabla }^{\left(\alpha \right)}$ on ${\mathcal{S}}^n$ by

$${\nabla }_{{\partial }_i}^{\left(\alpha \right)}{\partial }_j=\sum _{k=1}^n{\mathrm{\Gamma }}_{ij}^{\left(\alpha \right)k}{\partial }_k,$$

(4)

$${\mathrm{\Gamma }}_{ij}^{\left(\alpha \right)k}{|}_p=\frac{1+\alpha }{2}(-\frac{1}{x^k(p)}{\delta }_{ij}^k+x^k(p)g_{ij}(p)),i,j,k=1,\cdots ,n,$$

(5)

where ${\delta }_{ij}^k=1$ if $i=j=k$, and ${\delta }_{ij}^k=0$ if others. Then, the Levi–Civita connection ∇ of g coincides with ${\nabla }^{\left(0\right)}$. For $\alpha \in \mathbf{R}$, we have

$$Xg(Y,Z)=g({\nabla }_X^{\left(\alpha \right)}Y,Z)+g(Y,{\nabla }_X^{(-\alpha )}Z)\mathrm{for}X,Y,Z\in \mathcal{X}\left({\mathcal{S}}^n\right),$$

(6)

where $\mathcal{X}\left({\mathcal{S}}^n\right)$ is the set of all smooth tangent vector fields on ${\mathcal{S}}^n$. Then, ${\nabla }^{(-\alpha )}$ is called the dual connection of ${\nabla }^{\left(\alpha \right)}$. For each $\alpha$, ${\nabla }^{\left(\alpha \right)}$ is torsion-free and ${\nabla }^{\left(\alpha \right)}g$ is symmetric. Therefore, the triple $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ is a statistical manifold, and $({\mathcal{S}}^n,{\nabla }^{(-\alpha )},g)$ the dual statistical manifold of it.

Note that affine connections ${\nabla }^{\left(1\right)}$ and ${\nabla }^{(-1)}$ in Equations (4)–(6) are the dual connection and the canonical connection, respectively.

It is known that when $n\ge 2$, the curvature of the statistical manifold $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ is a constant value

$$\kappa =\frac{1+\alpha }{2}\frac{1-\alpha }{2}=\frac{1-{\alpha }^2}{4}.$$

Therefore, the curvature of the dual statistical manifold $({\mathcal{S}}^n,{\nabla }^{(-\alpha )},g)$ is also $\kappa =(1-{\alpha }^2)/4$. Iff $\alpha =\pm 1$, the curvature of $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ is zero, and $({\nabla }^{\left(\alpha \right)},{\nabla }^{(-\alpha )},g)$ is called the dually flat structure.

For $\alpha \ne \pm 1$, an $\alpha$-divergence ${\mathbf{D}}^{\left(\alpha \right)}$ on ${\mathbf{A}}_{+}^{n+1}$ is often defined by

$${\mathbf{D}}^{(\alpha )}(p,r)=\frac{4}{1-{\alpha }^2}\{\frac{1-\alpha }{2}\sum _{i=1}^{n+1}{x}^i\left(p\right)+\frac{1+\alpha }{2}\sum _{i=1}^{n+1}{x}^i\left(r\right)-\sum _{i=1}^{n+1}{x}^i{\left(p\right)}^{\frac{1-\alpha }{2}}{x}^i{\left(r\right)}^{\frac{1+\alpha }{2}}\},$$

(7)

$$p,r\in {\mathbf{A}}_{+}^{n+1}.$$

If $q=(1-\alpha )/2$, it holds that

$${\mathbf{D}}^{\left(\alpha \right)}(p,r)=\frac{1}{q}{K}_q(p,r),p,r\in {\mathcal{S}}^n,$$

(8)

for the Tsallis relative entropy $K_q$ on ${\mathcal{S}}^n$ defined by

$$K_q(p,r)\equiv -\sum _{i=1}^{n+1}{x}^i\left(p\right){ln}_q\frac{x^i\left(r\right)}{x^i\left(p\right)}=\frac{1}{1-q}\{1-\sum _{i=1}^{n+1}{x}^i{\left(p\right)}^q{x}^i{\left(r\right)}^{1-q}\},p,r\in {\mathcal{S}}^n,$$

(9)

where ${ln}_q$ is the q-logarithmic function defined by

$${ln}_{q}x\equiv \frac{x^{1-q}-1}{1-q},q\ne 1,x>0$$

(10)

Refs. [1,2]. The Tsallis relative entropy $K_q$ converges to the Kullback–Leibler divergence as $q\to 1$, because ${lim}_{q\to 1}{ln}_{q}x=logx$. In the information geometric view, the $\alpha$-divergence ${\mathbf{D}}^{\left(\alpha \right)}$ converges to the Kullback–Leibler divergence as $\alpha \to -1$.

For the Tsallis q-parameter, the curvature of the statistical manifold $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ is $\kappa =q(1-q)$.

3. Divergences Generated by Affine Immersions as Level Surfaces

In this section, we describe the general theory of affine immersions and divergences related to level surfaces of the Hessian domain.

If the Hessian $\tilde{D}d\phi ={\sum }_{i,j}\left({\partial }^2\phi \right)/(\partial x^i\partial x^j)d{x}^{i}d{x}^j$ of a function $\phi$ on a domain $\mathrm{\Omega }\subseteq {\mathbf{A}}^{n+1}$ is non-degenerate, the triple $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ is called a Hessian domain. A statistical manifold is said to be flat if the curvature tensor of its affine connection vanishes. A flat statistical manifold is locally a Hessian domain. Conversely, a Hessian domain is a flat statistical manifold [12,17].

In a previous study, we show the following theorem on the level surfaces of a Hessian function.

Theorem 1

([18]). Let M be a simply connected n-dimensional level surface of φ on an $(n+1)$-dimensional Hessian domain $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ with a Riemannian metric $\tilde{g}$ and suppose that $n\ge 2$. If we consider $(\mathrm{\Omega },\tilde{D},\tilde{g})$ a flat statistical manifold, $(M,D,g)$ is a 1-conformally flat statistical submanifold of $(\mathrm{\Omega },\tilde{D},\tilde{g})$, where D and g denote the connection and the Riemannian metric on M induced by $\tilde{D}$ and $\tilde{g}$, respectively.

Here, “1-conformally flat” represents the characterization of surfaces projected by a flat statistical manifold along dual coordinates. We continue to explain the terms used in Theorem 1 and the outline of the proof.

For $\alpha \in \mathbf{R}$, statistical manifolds $(N,\nabla ,h)$ and $(N,\overline{\nabla },\overline{h})$ are $\alpha$-conformally equivalent if there exists a function $\varphi$ on N such that

$$\begin{array}{ccc}\hfill \overline{h}(X,Y)& =& e^{\varphi }h(X,Y),\hfill \\ \hfill h({\overline{\nabla }}_{X}Y,Z)& =& h({\nabla }_{X}Y,Z)-\frac{1+\alpha }{2}d\varphi \left(Z\right)h(X,Y)\hfill \\ & & +\frac{1-\alpha }{2}\{d\varphi \left(X\right)h(Y,Z)+d\varphi \left(Y\right)h(X,Z)\},X,Y,Z\in \mathcal{X}\left(N\right).\hfill \end{array}$$

If $(N,\overline{\nabla },\overline{h})$ is 1-conformally equivalent to a flat statistical manifold $(N,\nabla ,h)$, $(N,\overline{\nabla },\overline{h})$ is called a 1-conformally flat statistical manifold. A statistical manifold $(N,\nabla ,h)$ is 1-conformally flat iff the dual statistical manifold $(N,{\nabla }^{\prime },h)$ is $(-1)$-conformally flat [19].

In terms of affine geometry, $(N,{\nabla }^{\prime },h)$ and $(N,{\overline{\nabla }}^{\prime },h)$ are $(-1)$-conformally equivalent if and only if ${\nabla }^{\prime }$ and ${\overline{\nabla }}^{\prime }$ are projectively equivalent [20,21].

For an $(n+1)$-dimensional Hessian domain $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$, an n-dimensional level surface of $\phi$ has the dualistic structure as the statistical submanifold structure. On the other hand, the level surface also has the structure induced by the affine immersion. It is essential for Theorem 1 that the statistical submanifold structure coincides with the dualistic structure by the affine immersion on a level surface of $\phi$.

For $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$, let x be the canonical immersion of an n-dimensional level surface M into $\mathrm{\Omega }$. Let E be a transversal vector field on M defined by

$$E=-d\phi {\left(\tilde{E}\right)}^{-1}\tilde{E},$$

(11)

where $\tilde{E}$ is the gradient vector field of $\phi$ on $\mathrm{\Omega }$ defined by

$$\tilde{g}(\tilde{X},\tilde{E})=d\phi \left(\tilde{X}\right),\tilde{X}\in \mathcal{X}\left(\mathrm{\Omega }\right).$$

(12)

For an affine immersion $(x,E)$ and the canonical flat affine connection $\tilde{D}$ on $\mathrm{\Omega }\subseteq {\mathbf{A}}^{n+1}$, the induced affine connection $D^E$, the affine fundamental form $g^E$, the shape operator ${\mathcal{S}}^E$ and the transversal connection form ${\tau }^E$ on M are defined by

$$\begin{array}{ccc}\hfill D_{X}Y& =& D_X^{E}Y+g^E(X,Y)E,\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill D_{X}E& =& {\mathcal{S}}^E\left(X\right)+{\tau }^E\left(X\right)E,X,Y\in \mathcal{X}\left(M\right).\hfill \end{array}$$

(14)

See [21,22]. Then, $D^E$ and $g^E$ coincide with the restricted affine connection of $\tilde{D}$ and the restricted Riemannian metric of $\tilde{g}$, respectively. For the level surface M, the transversal connection form satisfies that ${\tau }^E\equiv 0$. Therefore, $(x,E)$ it is called the equiaffine immersion. It is known that a simply connected statistical manifold can be realized in ${\mathbf{A}}^{n+1}$ by a non-degenerate equiaffine immersion iff it is 1-conformally flat [19]. Thus, Theorem 1 holds.

Next, we introduce a divergence on a Hessian domain, treating it as a flat statistical manifold.

The canonical divergence $\rho$ of a Hessian domain $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ is defined by

$$\rho (p,r)=\phi \left(p\right)+{\phi }^{*}\left(\tilde{\iota }\left(r\right)\right)+\sum _{i=1}^{n+1}{x}^i\left(p\right)x_i^{\prime }\left(r\right)forp,r\in \mathrm{\Omega },$$

(15)

where $\tilde{\iota }$ is the gradient mapping from $\mathrm{\Omega }$ to the dual affine space ${\mathbf{A}}_{n+1}^{*}$, i.e.,

$$x_i^{\prime }=x_i^{*}\circ \tilde{\iota }=-\frac{\partial \phi }{\partial x^i},$$

(16)

and {$x_1^{*},\dots ,x_{n+1}^{*}$} is the dual affine coordinate system of {$x^1,\dots ,x^{n+1}$}. The Legendre transform ${\phi }^{*}$ of $\phi$ is defined by

$${\phi }^{*}\circ \tilde{\iota }=-\sum _{i=1}^{n+1}{x}^i{x}_i^{\prime }-\phi .$$

(17)

See [12].

Let $\iota$ be the conormal immersion for the affine immersion $(x,E)$ defined by Equation (11), 12. By the definition of a conormal immersion, $\iota$ satisfies that

$$\langle \iota \left(p\right),Y_p\rangle =0,Y_p\in T_{p}M,\langle \iota \left(p\right),E_p\rangle =1\mathrm{for}p\in M,$$

where $\langle a,b\rangle$ is the pairing of $a\in {\mathbf{A}}_{n+1}^{*}$ and $b\in {\mathbf{A}}^{n+1}$. It is known that the conormal immersion $\iota$ coincides with the restriction of the gradient mapping $\tilde{\iota }$ to the level surface M.

The next definition is given in relation to affine immersions and divergences.

Definition 1

([19]).Let $(N,\nabla ,h)$ be a 1-conformally flat statistical manifold realized by a non-degenerate affine immersion $(v,\xi )$ into ${\mathbf{A}}^{n+1}$, and w the conormal immersion for v. Then the divergence ${\rho }_{\mathrm{c}onf}$ of $(N,\nabla ,h)$ is defined by

$${\rho }_{\mathrm{c}onf}(p,r)=\langle w\left(r\right),v\left(p\right)-v\left(r\right)\rangle forp,r\in N.$$

The ${\rho }_{\mathrm{c}onf}$ definition is independent of the choice of a realization of $(N,\nabla ,h)$.

The divergence ${\rho }_{\mathrm{c}onf}$ is referred to as Kurose geometric divergence in affine geometry and as Fenchel–Young divergence in the machine learning community [23,24]. Since an n-dimensional level surface M of $(\mathrm{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ is a 1-conformally flat statistical manifold realized by a non-degenerate affine immersion $(x,E)$, ${\rho }_{\mathrm{c}onf}$ on M is as follows:

$${\rho }_{\mathrm{c}onf}(p,r)=\langle \iota \left(r\right),x\left(p\right)-x\left(r\right)\rangle \mathrm{for}p,r\in M.$$

(18)

Let ${\rho }_{\mathrm{s}ub}$ be the restriction of the canonical divergence $\rho$ to $(M,D,g)$ as a statistical submanifold of $(\mathrm{\Omega },\tilde{D},\tilde{g})$. From Equations (15), (17) and (18), the next theorem holds.

Theorem 2

([20]). For a 1-conformally flat statistical submanifold $(M,D,g)$ of $(\mathrm{\Omega },\tilde{D},\tilde{g})$, two divergences ${\rho }_{\mathrm{c}onf}$ and ${\rho }_{\mathrm{s}ub}$ coincide.

4. Deformed Probability Simplexes and Escort Distributions Generated by Affine Immersions

In this section, we explain dualistic structures on deformed probability simplexes, which correspond to sets of escort distributions via affine immersion.

We set $p_i=x^i\left(p\right)$, $i=1,\cdots ,n+1$ for $p\in {\mathcal{S}}^n$, where ${\mathcal{S}}^n$ and $\{x^1,\cdots ,x^{n+1}\}$ be the probability simplex and the canonical affine coordinate system on ${\mathbf{A}}^{n+1}$, respectively. For $n+1$ states $p_1,\cdots ,p_{n+1}$ on ${\mathcal{S}}^n$ and $0<q<1$, if each probability $\mathbf{P}\left(p_i\right)$ satisfies

$$\mathbf{P}\left(p_i\right)=\frac{{\left(p_i\right)}^q}{{\sum }_{i=1}^{n+1}{\left(p_i\right)}^q},i=1,\cdots ,n+1,$$

(19)

the probability distribution $\mathbf{P}$ is called the escort distribution [1,2], where ${\left(p_i\right)}^q$ is $p_i$ powered by q.

It realizes the dualistic structure of a set of escort distributions via the affine immersion into ${\mathbf{A}}_{+}^{n+1}$ [4,5]. For $0<q<1$, let $f_q$ be the affine immersion of ${\mathcal{S}}^n$ into ${\mathbf{A}}_{+}^{n+1}$ defined by

$$x^i\left(f_q\left(p\right)\right)=\frac{1}{q}{\left(x^i\left(p\right)\right)}^q,i=1,\cdots ,n+1,forp\in {\mathcal{S}}^n.$$

(20)

Then, the escort distribution $\mathbf{P}$ is also represented as follows:

$$\mathbf{P}\left(p_i\right)=\frac{{\theta }^i}{{\sum }_{i=1}^{n+1}{\theta }^i},{\theta }^i=\frac{1}{q}{\left(p_i\right)}^q,i=1,\cdots ,n+1.$$

(21)

For a function ${\psi }_q$ on ${\mathbf{A}}_{+}^{n+1}$ defined by

$${\psi }_q=\frac{1}{1-q}\sum _{i=1}^{n+1}{\left(q{x}^i\right)}^{\frac{1}{q}},$$

(22)

the image $f_q\left({\mathcal{S}}^n\right)$ is a level surface of ${\psi }_q$ satisfying ${\psi }_q=1/(1-q)$. For $0<q<1$, the Hessian matrix of the function ${\psi }_q$ is positive definite on ${\mathbf{A}}_{+}^{n+1}$. Then, ${\psi }_q$ induces the Hessian structure $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{g}}_q\equiv ({\partial }^2{\psi }_q/\partial x^i\partial x^j))$. By definition

$${\tilde{\mathrm{\Gamma }}}_{ijk}=\sum _{l=1}^{n+1}{\tilde{g}}_{qkl}{\tilde{\mathrm{\Gamma }}}_{ij}^l=\frac{{\partial }^3\psi }{\partial x^i\partial x^j\partial x^k},i,j,k=1,\cdots ,n,$$

(23)

$${\tilde{D}}_{\frac{\partial }{\partial x_i}}^{\left(\alpha \right)}\frac{\partial }{\partial x_j}=\frac{1-\alpha }{2}\sum _{k=1}^{n+1}{\tilde{\mathrm{\Gamma }}}_{ij}^k\frac{\partial }{\partial x_k},\alpha =1-2q,$$

(24)

the tetrad $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{D}}^{(-1)},{\tilde{g}}_q)$ is the dually flat structure. The connection ${\tilde{D}}^{\left(0\right)}$ coincides with the Levi–Civita connection of the Riemannian metric ${\tilde{g}}_q$.

We denote by D and $g_q$ the restricted $\tilde{D}$ and $\widetilde{g_q}$ on $f_q\left({\mathcal{S}}^n\right)$, and induce the dualistic structure of $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ as the submanifold structure of $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{g}}_q)$. From the discussion in Section 3, $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ coincides with the dualistic structure induced by the equiaffine immersion $(f_q,E_q)$, where

$$E_q\equiv -d{\psi }_q{\left(\widetilde{E_q}\right)}^{-1}\widetilde{E_q}$$

(25)

for the gradient vector field $\widetilde{E_q}$ of ${\psi }_q$ on ${\mathbf{A}}_{+}^{n+1}$ defined by

$${\tilde{g}}_q(\tilde{X},\widetilde{E_q})=d{\psi }_q\left(\tilde{X}\right)\mathrm{for}\tilde{X}\in \mathcal{X}\left({\mathbf{A}}_{+}^{n+1}\right).$$

(26)

The pullback of $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ to ${\mathcal{S}}^n$ is $(-1)$-conformally equivalent to $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ defined by Equations (3)–(5). In addition, $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ has a constant curvature $\kappa =q(1-q)=(1-{\alpha }^2)/4$ [5].

On $(f_q\left({\mathcal{S}}^n\right),D,g_q)$, the restricted divergence ${\rho }_q$ from the canonical divergence of $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{g}}_q)$ coincides with the geometric divergence by Equation (18) from the affine immersion $(f_q,E_q)$. For an affine coordinate system $\{x_1^{\prime },\cdots ,x_{n+1}^{\prime }\}$ on ${\mathbf{A}}^{n+1}$ defined by

$$x_i^{\prime }=-\frac{\partial {\psi }_q}{\partial x^i}=-\frac{1}{1-q}{\left(q{x}^i\right)}^{\frac{1-q}{q}},$$

(27)

the divergence ${\rho }_q$ of $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ is described as

$${\rho }_q(a,b)=\sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right)),a,b\in f_q\left({\mathcal{S}}^n\right).$$

(28)

In addition, the pullback divergence of ${\rho }_q$ to ${\mathcal{S}}^n$ coincides with ${\mathbf{D}}^{\left(\alpha \right)}$ and the Tsallis relative entropy $K_q$ [4].

At the end of this section, we mention the divergence of $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{g}}_q)$. By Equation (17), the Legendre transform ${\psi }_q^{*}$ of ${\psi }_q$ is

$${\psi }_q^{*}\left(x^{\prime }\left(a\right)\right)=-{\psi }_q\left(a\right)+\sum _{i=1}^{n+1}{x}^i\left(a\right)x_i^{\prime }\left(a\right),a\in {\mathbf{A}}_{+}^{n+1}.$$

(29)

By Equations (15) and (16), the canonical divergence ${\rho }_q$ of $({\mathbf{A}}_{+}^{n+1},\tilde{D},{\tilde{g}}_q)$ is defined by

$${\rho }_q(a,b)={\psi }_q\left(a\right)-{\psi }_q\left(b\right)+\sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right)),a,b\in {\mathbf{A}}_{+}^{n+1},$$

(30)

represented by the same symbol ${\rho }_q$ of $(f_q\left({\mathcal{S}}^n\right),D,g_q)$.

5. Extended Divergence on a Foliation by Deformed Probability Simplexes

Previous sections described the divergence for each fixed q and each fixed $\alpha$. This section defines an extended divergence on a foliation by deformed probability simplexes $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ for all $0<q<1$, and shows the divergence decomposition theorem. The contents of our paper [15] are included but are explained in detail by the setting of affine geometry.

To give the proximity of q-escort distributions with different q-parameters, we define an extended divergence on a foliation by deformed probability simplexes as follows.

Definition 2.

Let $S_{fol}={\cup }_{0<q<1}{f}_q\left({\mathcal{S}}^n\right)=\{p|p\in {\mathbf{A}}_{+}^{n+1},{\sum }_{i=1}^{n+1}{x}^i\left(p\right)>1\}$, which corresponds to a foliation $\mathcal{F}=\left\{f_q\left({\mathcal{S}}^n\right)\right|0<q<1\}$. We call a function ${\rho }_{fol}$ on $S_{fol}\times S_{fol}$ defined by Equation (31) an extended divergence on a foliation by deformed probability simplexes.

$${\rho }_{fol}(a,b)\equiv {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(b\right)}\left(b\right)+\sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right))$$

(31)

$$fora\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right),0<q\left(a\right)<1,0<q\left(b\right)<1.$$

The i-th component of the conormal immersion of $(f_q,E_q)$ is $-\partial {\psi }_q/\partial x^i$. By the right-hand side of Equation (27), the dual coordinate of b, denoted by $x^{\prime }\left(b\right)$, satisfies that

$$-x^{\prime }\left(b\right)\equiv (-x_1^{\prime }\left(b\right),\cdots ,-x_{n+1}^{\prime }\left(b\right))\in f_{1-q\left(b\right)}\left({\mathcal{S}}^n\right).$$

Therefore, we consider $f_{1-q}\left({\mathcal{S}}^n\right)$ as the dual simplex of $f_q\left({\mathcal{S}}^n\right)$ for $0<q<1$. As $q=1/2$, $f_q\left({\mathcal{S}}^n\right)$ is self dual [4]. Note that the i-th component of the dual coordinate of b is denoted by ${\eta }_i\left(b\right)=-x_i^{\prime }\left(b\right)=(\partial {\psi }_q/\partial x^i){|}_b$ in [15].

On the extended divergence, the next proposition holds.

Proposition 1.

An extended divergence ${\rho }_{fol}$ on $S_{fol}$ of satisfies that:

(i) If $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$,

$${\rho }_{fol}(a,b)={\rho }_{q\left(a\right)}(a,b)={\mathbf{D}}^{\left(\alpha \right(a\left)\right)}(f_{q\left(a\right)}^{-1}\left(a\right),f_{q\left(a\right)}^{-1}\left(b\right)),$$

where${\rho }_q$is the divergence of$(f_q\left({\mathcal{S}}^n\right),D,g_q)$by Equation (28),${\mathbf{D}}^{\left(\alpha \right)}$is an α-divergence defined by Equation (7), and $\alpha \left(a\right)=1-2q\left(a\right)$.

(ii) In the case of $q\left(a\right)\ge q\left(b\right)$,

$${\rho }_{fol}(a,b)\ge 0for(a,b)\in S_{fol}\times S_{fol},$$

and if and only if $a=b$,

$${\rho }_{fol}(a,b)=0.$$

Proof. 

If $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, ${\psi }_{q\left(a\right)}\left(a\right)={\psi }_{q\left(b\right)}\left(a\right)={\psi }_{q\left(b\right)}\left(b\right)$. By Equations (28) and (31),

$${\rho }_{fol}(a,b)=\sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right))={\rho }_{q\left(a\right)}(a,b).$$

Then, (i) holds. If $1>q\left(a\right)\ge q\left(b\right)>0$, it holds that ${\psi }_{q\left(a\right)}\left(a\right)\ge {\psi }_{q\left(b\right)}\left(b\right)$ because

$${\psi }_{q\left(a\right)}\left(a\right)=\frac{1}{1-q\left(a\right)},{\psi }_{q\left(b\right)}\left(b\right)=\frac{1}{1-q\left(b\right)}$$

(32)

are induced by the definition of $f_q\left({\mathcal{S}}^n\right)$. In addition, $f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$ and $f_{q\left(b\right)}\left({\mathcal{S}}^n\right)$ are convex surfaces centered on the origini of ${\mathbf{A}}_{+}^{n+1}$, and the surfaces $f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$ closer to the origin than $f_{q\left(b\right)}\left({\mathcal{S}}^n\right)$. Then, ${\sum }_{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right))\ge 0$. Thus, (ii) holds. □

We define the extended dual divergence ${\rho }_{fol}^{*}$ of ${\rho }_{fol}$ as follows;

$${\rho }_{fol}^{*}(a,b)\equiv {\psi }_{q\left(a\right)}^{*}\left(x^{\prime }\left(a\right)\right)-{\psi }_{q\left(b\right)}^{*}\left(x^{\prime }\left(b\right)\right)+\sum _{i=1}^{n+1}{x}^i\left(b\right)(x_i^{\prime }\left(a\right)-x_i^{\prime }\left(b\right))$$

(33)

$$fora\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right),0<q\left(a\right)<1,0<q\left(b\right)<1,$$

where ${\psi }_q^{*}$ is the Legendre transform of ${\psi }_q$ for $0<q<1$. Then, the following holds.

Proposition 2.

The functions ${\rho }_{fol}$ and ${\rho }_{fol}^{*}$ satisfy that

$${\rho }_{fol}^{*}(b,a)={\rho }_{fol}(a,b)fora\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right).$$

(34)

Proof. 

By the definition of the Legendre transform, we have

$$\begin{array}{ccc}\hfill {\rho }_{fol}^{*}(b,a)& =& {\psi }_{q\left(b\right)}^{*}\left(x^{\prime }\left(b\right)\right)-{\psi }_{q\left(a\right)}^{*}\left(x^{\prime }\left(a\right)\right)+\sum _{i=1}^{n+1}{x}^i\left(a\right)(x_i^{\prime }\left(b\right)-x_i^{\prime }\left(a\right))\hfill \\ & =& -{\psi }_{q\left(b\right)}\left(b\right)-\sum _{i=1}^{n+1}{x}^i\left(b\right)x_i^{\prime }\left(b\right)\hfill \\ & & +{\psi }_{q\left(a\right)}\left(a\right)+\sum _{i=1}^{n+1}{x}^i\left(a\right)x_i^{\prime }\left(a\right)+\sum _{i=1}^{n+1}{x}^i\left(a\right)(x_i^{\prime }\left(b\right)-x_i^{\prime }\left(a\right))\hfill \\ & =& {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(b\right)}\left(b\right)+\sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right))\hfill \\ & =& {\rho }_{fol}(a,b).\hfill \end{array}$$

□

The extended divergence is related to the duo Bregman (pseudo-)divergence, where the parameters also define the convex functions [16]. To work with the entire parametrized probability distribution families and to explore the application of divergences, we must investigate their relationship.

6. Decomposition of an Extended Divergence

In this section, we explain the orthogonal foliation of $\mathcal{F}$. Next, we give a decomposition of an extended divergence along the orthogonal leaf and the original leaf.

For the foliation $\mathcal{F}=\left\{f_q\left({\mathcal{S}}^n\right)\right|0<q<1\}$, we consider the flow on $S_{fol}$ defined using the following equation.

$$\frac{d{x}_i^{\prime }}{dt}=x_i^{\prime },i=1,\cdots ,n+1,$$

(35)

where a function $x_i^{\prime }$ on $S_{fol}$ takes the i-th component of the dual coordinate on $f_q\left({\mathcal{S}}^n\right)$ as Equation (27) for each $0<q<1$. An integral curve of Equation (35) is orthogonal to $f_q\left({\mathcal{S}}^n\right)$ for each q with respect to the pairing $\langle ,\rangle$. The set of integral curves becomes the orthogonal foliation of $\mathcal{F}$. We denote it by ${\mathcal{F}}^{\perp }$.

Translating into the primal coordinate system, we have the next equation.

$$\frac{d{x}^i}{dt}={\tilde{E}}^i,i=1,\cdots ,n+1,\mathrm{on}{\mathcal{S}}_{fol},$$

(36)

$$\tilde{E}={\tilde{E}}_q^i=\sum _{j=1}^{n+1}{g}_q^{ij}\frac{\partial {\psi }_q}{\partial x^j}\mathrm{on}{f}_q\left({\mathcal{S}}^n\right),$$

(37)

where $(g_q^{ij})$ is the inverse matrix of $\left(g_{qij}\right)$. The right-hand side of Equation (37) is calculated using Equations (11) and (12) for ${\psi }_q$. A leaf of ${\mathcal{F}}^{\perp }$ is an integral curve of the vector field $\tilde{E}$ that takes the value ${\tilde{E}}_q$ on $f_q\left({\mathcal{S}}^n\right)$ for each q.

The following theorem is on the decomposition of the extended divergence.

Theorem 3.

Let ${\mathcal{S}}^n$ be the probability simplex, and $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ the 1-conformally flat statistical manifold generated by the affine immersion $(f_q,E_q)$, where $f_q$ is defined as

$$x^i\left(f_q\left(p\right)\right)=\frac{1}{q}{\left(x^i\left(p\right)\right)}^q,i=1,\cdots ,n+1,forp\in {\mathcal{S}}^n,$$

(38)

${\psi }_q\equiv 1/(1-q){\sum }_{i=1}^{n+1}{\left(q{x}^i\right)}^{1/q}$, $E_q\equiv -d\psi {\left({\tilde{E}}_q\right)}^{-1}{\tilde{E}}_q$, ${\tilde{E}}_q^i\equiv {\sum }_{j=1}^{n+1}{g}_q^{ij}\partial {\psi }_q/\partial x^j$, and $g_q$ is the restriction of $\left(g_{qij}\right)=Dd{\psi }_q$ to $f_q\left({\mathcal{S}}^n\right)$. Let $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, $0<q\left(a\right)<1$, and $c\in {\mathcal{S}}_{fol}\equiv {\cup }_{0<q<1}{f}_q\left({\mathcal{S}}^n\right)$. If there exists an orthogonal leaf $L^{\perp }\in {\mathcal{F}}^{\perp }$ that includes b and c, we have

$${\rho }_{fol}(a,c)=\mu {\rho }_{fol}(a,b)+{\rho }_{fol}(b,c),x^{\prime }\left(c\right)=\mu x^{\prime }\left(b\right),\mu >0,$$

(39)

where $x^{\prime }(·)$ is the dual coordinate of $f_q\left({\mathcal{S}}^n\right)$ for each q.

Proof. 

From $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, it holds that ${\psi }_{q\left(a\right)}\left(a\right)={\psi }_{q\left(b\right)}\left(b\right)$, where $q\left(b\right)=q\left(a\right)$. By the definition in Equations (22) and (23), we have

$$\begin{array}{ccc}\hfill {\rho }_{fol}(a,c)& =& {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(c\right)}\left(c\right)+\sum _{i=1}^{n+1}{x}_i^{\prime }\left(c\right)(x^i\left(a\right)-x^i\left(c\right))\hfill \\ & =& {\psi }_{q\left(b\right)}\left(b\right)-{\psi }_{q\left(c\right)}\left(c\right)+\sum _{i=1}^{n+1}\{x_i^{\prime }\left(c\right)(x^i\left(a\right)-x^i\left(b\right))+x_i^{\prime }\left(c\right)(x^i\left(b\right)-x^i\left(c\right))\}\hfill \\ & =& +\mu \sum _{i=1}^{n+1}{x}_i^{\prime }\left(b\right)(x^i\left(a\right)-x^i\left(b\right))\hfill \\ & & +\{{\psi }_{q\left(b\right)}\left(b\right)-{\psi }_{q\left(c\right)}\left(c\right)+\sum _{i=1}^{n+1}{x}_i^{\prime }\left(c\right)(x^i\left(b\right)-x^i\left(c\right))\}\hfill \\ & =& \mu {\rho }_{fol}(a,b)+{\rho }_{fol}(b,c).\hfill \end{array}$$

□

See Figure 1 for a decomposition of extended divergence and graphs of deformed simplexes $f_q\left({\mathcal{S}}^n\right)$.

A decomposition similar to Equation (39) is also available on a foliation by Hessian level surfaces of one Hessian manifold [20]. Theorem 3 generalizes the previous decomposition.

Finally, we describe the gradient flow on a leaf $f_q\left({\mathcal{S}}^n\right)$ using the extended divergence.

Theorem 4.

For a submanifold $(f_q\left({\mathcal{S}}^n\right),D,g_q)$ of ${\mathcal{S}}_{fol}$, we denote by $\{x^1,\cdots ,x^n\}$ an affine coordinate system on $f_q\left({\mathcal{S}}^n\right)$ such that $Dd{x}^i=0$, $i=1,\cdots ,n$, and set $g_{qij}=g_q(\partial /\partial x^i,\partial /\partial x^j)$, $\left(g_q^{ij}\right)={\left(g_{qij}\right)}^{-1}$. For a fixed point $c\in L^{\perp }$, the gradient flow on $f_q\left({\mathcal{S}}^n\right)$ defined by

$$\frac{d{x}^i}{dt}=-\sum _{j=1}^n{g}_q^{ij}\frac{\partial }{\partial x^j}{\rho }_{fol}(a_x,c),i=1,\cdots ,n,a_x\in f_q\left({\mathcal{S}}^n\right)$$

(40)

converges to the unique point $b\in L^{\perp }\cap f_q\left({\mathcal{S}}^n\right)$, where $a_x$ is a variable point parametrized as $\{x^1\left(t\right),\cdots ,x^n\left(t\right)\}$.

Proof. 

By Theorem 3, for any $a_x\in f_q\left({\mathcal{S}}^n\right)$, there exists $\mu >0$ such that

$${\rho }_{fol}(a_x,c)=\mu {\rho }_{fol}(a_x,b)+{\rho }_{fol}(b,c),x^{\prime }\left(c\right)=\mu x^{\prime }\left(b\right).$$

Equation (40) is described by the dual coordinate system $\{x_1^{\prime },\cdots ,x_n^{\prime }\}$ on $f_q\left({\mathcal{S}}^n\right)$ as follows;

$$\frac{d{x}_i^{\prime }}{dt}=-\mu \frac{\partial }{\partial x^j}{\rho }_{fol}(a_x,b),i=1,\cdots ,n.$$

(41)

On $f_q\left({\mathcal{S}}^n\right)$, from Prop. 1.(i), ${\rho }_{fol}$ coincides with the geometric divergence ${\rho }_q$, generated by the affine immersion $(f_q,E_q)$. The geometric divergence generates the dual coordinate $x_i^{\prime }$ such that $D^{*}d{x}_i^{\prime }=0$, $i=1,\cdots ,n$, to be derived by $x^i$ [19]. Then, it holds that

$$\frac{d{x}_i^{\prime }}{dt}=-\mu (x_i^{\prime }\left(a_x\right)-x_i^{\prime }\left(b\right)),i=1,\cdots ,n,$$

(42)

and that

$$x_i^{\prime }=x_i^{\prime }\left(b\right)+(x_i^{\prime }\left(a{|}_{t=0}\right)-x_i^{\prime }\left(b\right))e^{-\mu t},i=1,\cdots ,n,$$

(43)

where ${a|}_{t=0}$ is an initial point of Equation (40). Then, the gradient flow of Equation (40) converges to $b\in L^{\perp }\cap f_q\left({\mathcal{S}}^n\right)$ following a geodesic for the dual coordinate system. □

The gradient flow similar to Equation (40) has been provided on a flat statistical submanifold [25]. The similar one on a Hessian level surface, i.e., a 1-conformally statistical submanifold, has been given in [20]. Theorem 4 generalizes the previous theorems on gradient flows.

7. Conclusions

This study considers a foliation of deformed probability simplexes corresponding to sets of escort distributions with q-parameters, for the continuous transition of $\alpha$-parameters on information geometry. Since these are typical q-exponential families, we still need to provide details on the extended divergence and natural definition of the foliation of q-exponential families.

The extended divergence guides the proximity of q-exponential distributions with different q-parameters. Therefore, our theory guarantees the mathematical basis for generalizing methods of machine learning and statistical mechanics to the case of the q-distribution families when different q-parameters are mixed. The decomposition theorem is applied to the problem of the optimal choice of q-parameter. The application methods are open to consideration. It also remains to investigate the relationship with a new $\lambda$-duality on nonextensive statistical mechanics with mixed parameters [26,27].