Private Key and Decoder Side Information for Secure and Private Source Coding ^†

Abstract

We extend the problem of secure source coding by considering a remote source whose noisy measurements are correlated random variables used for secure source reconstruction. The main additions to the problem are as follows: $\left(1\right)$ all terminals noncausally observe a noisy measurement of the remote source; $\left(2\right)$ a private key is available to all legitimate terminals; $\left(3\right)$ the public communication link between the encoder and decoder is rate-limited; and $\left(4\right)$ the secrecy leakage to the eavesdropper is measured with respect to the encoder input, whereas the privacy leakage is measured with respect to the remote source. Exact rate regions are characterized for a lossy source coding problem with a private key, remote source, and decoder side information under security, privacy, communication, and distortion constraints. By replacing the distortion constraint with a reliability constraint, we obtain the exact rate region for the lossless case as well. Furthermore, the lossy rate region for scalar discrete-time Gaussian sources and measurement channels is established. An achievable lossy rate region that can be numerically computed is also provided for binary-input multiple additive discrete-time Gaussian noise measurement channels.

1. Introduction

Consider multiple terminals that observe correlated random sequences and wish to reconstruct these sequences at another terminal, called a decoder, by sending messages through noiseless communication links, i.e., the distributed source coding problem [1]. A sensor network where each node observes a correlated random sequence that needs to be reconstructed at a distant node is a classic example of this problem [2] (p. 258). Similarly, function computation problems in which a fusion center observes messages sent by other nodes to compute a function are closely related problems that can be used to model various recent applications [3,4]. Since messages sent over communication links can be public, security constraints are imposed on these messages against an eavesdropper in the same network [5]. If all sent messages are available to the eavesdropper, it is necessary to provide an advantage to the decoder over the eavesdropper to enable secure source coding. Providing side information that is correlated with the sequences that should be reconstructed to the decoder can provide such an advantage over the eavesdropper that can also have side information, as in [6,7,8]. Allowing for the eavesdropper to access only a strict subset of all messages is also a method to enable secure distributed source coding, which was considered in [9,10,11]; see also [12], in which a similar method was applied to enable secure remote source reconstruction. Similarly, a private key that is shared by legitimate terminals and hidden from the eavesdropper can also provide such an advantage, as in [13,14].

Source coding models in the literature commonly assume that dependent multiletter random variables are available and should be compressed. For secret-key agreement [15,16] and secure function computation problems [17,18], which are instances of the source coding with the side information problem [19] (Section IV-B), the correlation between these multiletter random variables was posited in [20,21] to stem from an underlying ground truth that is a remote source, such that its noisy measurements are these dependent random variables. Such a remote source allows one to model the cause of correlation in a network, so we also posit that there is a remote source whose noisy measurements are used in the source coding problems discussed below, which is similar to the models in [22] (p. 78) and [23] (Figure 9). Furthermore, in the chief executive officer (CEO) problem [24], there is a remote source whose noisy measurements are encoded, such that a decoder can reconstruct the remote source by using encoder outputs. Our model is different from the model in the CEO problem, since in our model, the decoder aims to recover encoder observations rather than the remote source that is considered mainly to describe the cause of correlation between encoder observations. Thus, we define the secrecy leakage as the amount of information leaked to an eavesdropper about encoder observations. Since the remote source is common for all observations in the same network, we impose a privacy leakage constraint on the remote source because each encoder output observed by an eavesdropper leaks information about unused encoder observations, which might later cause secrecy leakage when the unused encoder observations are employed [25,26,27]; see [28,29,30] for joint secrecy and joint privacy constraints imposed due to multiple uses of the same source.

1.1. Summary of Contributions

We extend the lossless and lossy source coding rate region analyses by considering a remote source that should be kept private, decoder and eavesdropper side information, and a private key shared by the encoder and decoder. Considering that one encoder provides insights with enough richness to extend the results to multiple encoders [31], in this work, we consider the single encoder case. A summary of the main contributions is as follows.

• We characterize the lossy secure and private source coding region when noisy measurements of a remote source are observed by all terminals, and there is one private key available.
• Requiring reliable source reconstruction, we also characterize the rate region for the lossless secure and private source coding problem.
• A Gaussian remote source and independent additive Gaussian noise measurement channels are considered to establish their lossy rate region under squared error distortion.
• We provide an achievable lossy secure and private source coding region for a binary remote source and its measurements through additive Gaussian noise channels, which includes computable differential entropy terms.

1.2. Organization

This paper is organized as follows. In Section 2, we introduce the lossless and lossy secure and private source coding problems with decoder and eavesdropper side information and a private key under storage, secrecy, privacy, and reliability or distortion constraints. In Section 3, we characterize the rate regions for the introduced problems, which include three parts that correspond to different private key rate regimes. In Section 4, we evaluate the lossy rate region for Gaussian sources and channels with squared error distortion. In Section 5, we consider a binary modulated remote source measured through additive Gaussian noise channels and provide an inner bound for the lossy rate region with Hamming distortion. In Section 6, we provide the proof for the lossy secure and and private source coding region.

1.3. Notation

Uppercase X represents random variables and lowercase x their realizations from a set ${\displaystyle \mathcal{X}}$, denoted by calligraphic letters. A discrete random variable X has probability distribution ${\displaystyle P_X}$ and a continuous random variable X probability density function (pdf) ${\displaystyle p_X}$. A subscript i denotes the position of a variable in a length-n sequence ${\displaystyle X^n=X_1,X_2,\dots ,X_i,\dots ,X_n}$. Boldface uppercase $\mathbf{X}={[X_1,X_2,\dots ]}^T$ represent vector random variables, where T denotes the transpose. $[1:m]$ denotes the set $\{1,2,\dots ,m\}$ for an integer $m\ge 1$. Define ${\left[a\right]}^{-}=min\{a,0\}$ for $a\in \mathbb{R}$. Function $H_b\left(x\right)=-xlogx-(1-x)log(1-x)$ is the binary entropy function, where logarithms are to the base 2. A binary symmetric channel (BSC) with crossover probability $ϵ$ is denoted by BSC($ϵ$). $X\sim \mathrm{Bern}\left(\beta \right)$ with $\mathcal{X}=\{0,1\}$ is a binary random variable with $Pr[X=1]=\beta$. The ∗-operator represents $p\ast q=(1-2q)p+q$. Function $Q(·)$ denotes the complementary cumulative distribution function of the standard Gaussian distribution. The function $sgn(·)$ represents the signum function.

2. System Model

We consider the lossy source coding model with one encoder, one decoder, and an eavesdropper (Eve), depicted in Figure 1. The encoder $\mathsf{Enc}(·,·)$ observes a noisy measurement ${\tilde{X}}^n$ of an i.i.d. remote source $X^n\sim P_X^n$ through a memoryless channel $P_{\tilde{X}|X}$ in addition to a private key $K\in [1:2^{n{R}_0}]$. The encoder output is an index W that is sent over a link with limited communication rate. Decoder $\mathsf{Dec}(·,·,·)$ observes index W, private key K, and another noisy measurement $Y^n$ of the same remote source $X^n$ through another memoryless channel $P_{YZ|X}$ in order to estimate ${\tilde{X}}^n$ as $\widehat{{\tilde{X}}^n}$. The other noisy output $Z^n$ of $P_{YZ|X}$ is observed by Eve in addition to index W. Assume K is uniformly distributed, hidden from Eve, and independent of the source output and its noisy measurements. The source and measurement alphabets are finite sets.

We next define the rate region for the lossy secure and private source coding problem defined above.

Definition 1.

A lossy tuple $(R_w,R_s,R_{\ell },D)\in {\mathbb{R}}_{\ge 0}^4$ is achievable given a private key with rate $R_0\ge 0$, if for any $\delta >0$ there exist $n\ge 1$, an encoder, and a decoder, such that

$$\begin{array}{ccc}\frac{1}{n}log\left|\mathcal{W}\right|\le R_w+\delta \hfill & & \hfill \left(\mathrm{storage}\right)\end{array}$$

(1)

$$\begin{array}{ccc}\frac{1}{n}I({\tilde{X}}^n;W|Z^n)\le R_s+\delta \hfill & & \hfill \left(\sec \mathrm{recy}\right)\end{array}$$

(2)

$$\begin{array}{ccc}\frac{1}{n}I(X^n;W|Z^n)\le R_{\ell }+\delta \hfill & & \hfill \left(\mathrm{privacy}\right)\end{array}$$

(3)

$$\begin{array}{ccc}\mathbb{E}\left[d\left({\tilde{X}}^n,\widehat{{\tilde{X}}^n}(Y^n,W,K)\right)\right]\le D+\delta \hfill & & \hfill \left(\mathrm{distortion}\right)\end{array}$$

(4)

where $d({\tilde{x}}^n,\widehat{{\tilde{x}}^n})=\frac{1}{n}{\sum }_{i=1}^{n}d({\tilde{x}}_i,\widehat{{\tilde{x}}_i})$ is a per-letter bounded distortion metric. The lossy secure and private source coding region ${\mathcal{R}}_D$ is the closure of the set of all achievable lossy tuples. ◊

In (2) and (3), we consider conditional mutual information terms to take account of unavoidable secrecy and privacy leakages due to Eve’s side information, i.e., $I({\tilde{X}}^n;Z^n)$ and $I(X^n;Z^n)$, respectively; see also [21,32]. Furthermore, we consider conditional mutual information terms rather than corresponding conditional entropy terms, the latter of which is used in [6,14,33,34,35], to characterize the secrecy and privacy leakages simplifies our analysis.

We next define the rate region for the lossless secure and private source coding problem.

Definition 2.

A lossless tuple $(R_{\mathrm{w}},R_s,R_{\ell })\in {\mathbb{R}}_{\ge 0}^3$ is achievable given a private key with rate $R_0\ge 0$, if for any $\delta >0$ there exist $n\ge 1$, an encoder, and a decoder, such that we have (1)–(3) and

$$\begin{array}{ccc}Pr\left[{\tilde{X}}^n\ne \widehat{{\tilde{X}}^n}(Y^n,W,K)\right]\le \delta \hfill & \hfill & \hfill \left(\mathrm{reliability}\right).\end{array}$$

(5)

The lossless secure and private source coding region $\mathcal{R}$ is the closure of the set of all achievable lossless tuples. ◊

3. Secure and Private Source Coding Regions

3.1. Lossy Source Coding

The lossy secure and and private source coding region ${\mathcal{R}}_D$ is characterized below; see Section 6 for its proof.

Define ${\left[a\right]}^{-}=min\{a,0\}$ for $a\in \mathbb{R}$.

Theorem 1.

For given $P_X$, $P_{\tilde{X}|X}$, $P_{YZ|X}$, and $R_0$, the region ${\mathcal{R}}_D$ is the set of all rate tuples $(R_w,R_s,R_{\ell },D)$ satisfying

$$\begin{array}{cc}\hfill & R_w\ge I(U;\tilde{X}|Y)\hfill \end{array}$$

(6)

and if $R_0<I(U;\tilde{X}|Y,V)$, then

$$\begin{array}{cc}\hfill & R_s\ge I(U;\tilde{X}|Z)+R^{\prime }-R_0\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & R_{\ell }\ge I(U;X|Z)+R^{\prime }-R_0\hfill \end{array}$$

(8)

where we have

$$\begin{array}{c}\hfill R^{\prime }={[I(U;Z|V,Q)-I(U;Y|V,Q)]}^{-}\end{array}$$

(9)

and if $I(U;\tilde{X}|Y,V)\le R_0<I(U;\tilde{X}|Y)$, then

$$\begin{array}{cc}\hfill & R_s\ge I(V;\tilde{X}|Z)\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & R_{\ell }\ge I(V;X|Z)\hfill \end{array}$$

(11)

and if $R_0\ge I(U;\tilde{X}|Y)$, then

$$\begin{array}{cc}\hfill & R_s\ge 0\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & R_{\ell }\ge 0\hfill \end{array}$$

(13)

for some

$$\begin{array}{c}\hfill P_{QVU\tilde{X}XYZ}=P_{Q|V}{P}_{V|U}{P}_{U|\tilde{X}}{P}_{\tilde{X}|X}{P}_X{P}_{YZ|X}\end{array}$$

(14)

such that $\mathbb{E}\left[d\left(\tilde{X},\widehat{\tilde{X}}(U,Y)\right)\right]\le D$ for some reconstruction function $\widehat{\tilde{X}}(U,Y)$. The region ${\mathcal{R}}_D$ is convexified by using the time-sharing random variable Q, required due to the ${[·]}^{-}$ operation. One can limit the cardinalities to $\left|\mathcal{Q}\right|\le 2$, $\left|\mathcal{V}\right|\le |\tilde{X}|+3$, and $\left|\mathcal{U}\right|\le \left(\right|\tilde{X}{|+3)}^2$.

We remark that (12) and (13) show that one can simultaneously achieve strong secrecy and strong privacy, i.e., the conditional mutual information terms in (2) and (3), respectively, are negligible, by using a large private key K, which is a result missing in some recent works on secure source coding with a private key.

3.2. Lossless Source Coding

The lossless secure and and private source coding region $\mathcal{R}$ is characterized next; see below for a proof sketch.

Proposition 1.

For given $P_X$, $P_{\tilde{X}|X}$, $P_{YZ|X}$, and $R_0$, the region $\mathcal{R}$ is the set of all rate tuples $(R_w,R_s,R_{\ell })$ satisfying

$$\begin{array}{cc}\hfill & R_w\ge H\left(\tilde{X}\right|Y)\hfill \end{array}$$

(15)

and if $R_0<H\left(\tilde{X}\right|Y,V)$, then

$$\begin{array}{cc}\hfill & R_s\ge H\left(\tilde{X}\right|Z)+R^{″}-R_0\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & R_{\ell }\ge I(\tilde{X};X|Z)+R^{″}-R_0\hfill \end{array}$$

(17)

where we have

$$\begin{array}{c}\hfill R^{″}={[I(\tilde{X};Z|V,Q)-I(\tilde{X};Y|V,Q)]}^{-}\end{array}$$

(18)

and if $H\left(\tilde{X}\right|Y,V)\le R_0<H\left(\tilde{X}\right|Y)$, then

$$\begin{array}{cc}\hfill & R_s\ge I(V;\tilde{X}|Z)\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & R_{\ell }\ge I(V;X|Z)\hfill \end{array}$$

(20)

and if $R_0\ge H\left(\tilde{X}\right|Y)$, then

$$\begin{array}{cc}\hfill & R_s\ge 0\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & R_{\ell }\ge 0\hfill \end{array}$$

(22)

for some

$$\begin{array}{c}\hfill P_{QV\tilde{X}XYZ}=P_{Q|V}{P}_{V|\tilde{X}}{P}_{\tilde{X}|X}{P}_X{P}_{YZ|X}.\end{array}$$

(23)

One can limit the cardinalities to $\left|\mathcal{Q}\right|\le 2$ and $\left|\mathcal{V}\right|\le |\tilde{X}|+2$.

Proof Sketch.

The proof for the lossless region $\mathcal{R}$ follows from the proof for the lossy region ${\mathcal{R}}_D$, given in Theorem 1 above, by choosing $U=\tilde{X}$, such that we have reconstruction function $\widehat{\tilde{X}}(\tilde{X},Y)=\tilde{X}$, so we achieve $D=0$. Thus, the reliability constraint in (5) is satisfied because $d(·,·)$ is a distortion metric. □

4. Gaussian Sources and Additive Gaussian Noise Channels

We evaluate the lossy rate region for a Gaussian example with squared error distortion by finding the optimal auxiliary random variable in the corresponding rate region. Consider a special lossy source coding case in which $\left(i\right)$ there is no private key; $\left(ii\right)$ the eavesdropper’s channel observation $Z^n$ is less noisy than the decoder’s channel observation $Y^n$, such that we obtain a lossy source coding region with a single auxiliary random variable that should be optimized.

We next define less noisy channels, considering $P_{YZ|X}$.

Definition 3

([36]). Z (or eavesdropper) is less noisy than Y (or decoder) if

$$\begin{array}{c}\hfill I(L;Z)\ge I(L;Y)\end{array}$$

(24)

holds for any random variable L, such that $L-X-(Y,Z)$ form a Markov chain. ◊

Corollary 1.

For given $P_X$, $P_{\tilde{X}|X}$, $P_{YZ|X}$, and $R_0=0$, the region ${\mathcal{R}}_D$ when the eavesdropper is less noisy than the decoder is the set of all rate tuples $(R_w,R_s,R_{\ell },D)$ satisfying

$$\begin{array}{cc}\hfill & R_w\ge I(U;\tilde{X}|Y)=I(U;\tilde{X})-I(U;Y)\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & R_{\mathrm{s}}\ge I(U;\tilde{X}|Z)=I(U;\tilde{X})-I(U;Z)\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill & R_{\ell }\ge I(U;X|Z)=I(U;X)-I(U;Z)\hfill \end{array}$$

(27)

for some

$$\begin{array}{c}\hfill P_{U\tilde{X}XYZ}=P_{U|\tilde{X}}{P}_{\tilde{X}|X}{P}_X{P}_{YZ|X}\end{array}$$

(28)

such that $\mathbb{E}\left[d\left(\tilde{X},\widehat{\tilde{X}}(U,Y)\right)\right]\le D$ for some reconstruction function $\widehat{\tilde{X}}(U,Y)$. One can limit the cardinality to $\left|\mathcal{U}\right|\le |\tilde{X}|+3$.

Proof Sketch.

The proof for Corollary 1 follows from the proof for Theorem 1 by considering the bounds in (6)–(8) since $R_0=0$. Furthermore, $R^{\prime }$ defined in (9) is 0 for the less noisy condition considered, which follows because $(Q,V)-U-X-(Y,Z)$ form a Markov chain. □

Suppose the following scalar discrete-time Gaussian source and channel model for the lossy source coding problem depicted in Figure 1

$$\begin{array}{cc}\hfill & X={\rho }_x\tilde{X}+N_x\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill & Y={\rho }_{y}X+N_y\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & Z={\rho }_{z}X+N_z\hfill \end{array}$$

(31)

where we have remote source $X\sim \mathcal{N}(0,1)$, fixed correlation coefficients ${\rho }_x,{\rho }_y,{\rho }_z\in (-1,1)$, and additive Gaussian noise random variables

$$\begin{array}{cc}\hfill & N_x\sim \mathcal{N}(0,1-{\rho }_x^2)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill & N_y\sim \mathcal{N}(0,1-{\rho }_y^2)\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill & N_z\sim \mathcal{N}(0,1-{\rho }_z^2)\hfill \end{array}$$

(34)

such that $(\tilde{X},N_x,N_y,N_z)$ are mutually independent, and we consider the squared error distortion, i.e., $d(\tilde{x},\widehat{\tilde{x}})={(\tilde{x}-\widehat{\tilde{x}})}^2$. Note that (29) is an inverse measurement channel $P_{X|\tilde{X}}$ that is a weighted sum of two independent Gaussian random variables, imposed to be able to apply the conditional entropy power inequality (EPI) [37] (Lemma II); see [20] (Theorem 3) and [38] (Section V) for binary symmetric inverse channel assumptions imposed to apply Mrs. Gerber’s lemma [39]. Suppose $|{\rho }_z|>|{\rho }_y|$, such that Y is less stochastically degraded than Z, since then there exists a random variable $\tilde{Y}$ such that $P_{\tilde{Y}|X}=P_{Y|X}$ and $P_{\tilde{Y}Z|X}=P_{Z|X}{P}_{\tilde{Y}|Z}$ [40] (Lemma 6), so Z is also less noisy than Y since less noisy channels constitute a strict superset of the set of stochastically-degraded channels and both channel sets consider only the conditional marginal probability distributions [2] (p. 121).

We next take the liberty to use the lossy rate region in Corollary 1, characterized for discrete memoryless channels, for the model in (29)–(31). This is common in the literature since there is a discretization procedure to extend the achievability proof to well-behaved continuous-alphabet random variables and the converse proof applies to arbitrary random variables; see [2] (Remark 3.8). For Gaussian sources and channels, we use differential entropy and eliminate the cardinality bound on the auxiliary random variable. The lossy source coding region for the model in (29)–(31) without a private key is given below.

Proposition 2.

For the model in (29)–(31), such that $|{\rho }_z|>|{\rho }_y|$ and $R_0=0$, the region ${\mathcal{R}}_{\mathrm{D}}$ with squared error distortion is the set of all rate tuples $(R_w,R_{\mathrm{s}},R_{\ell },D)$ satisfying, for $\alpha \in (0,1]$,

$$\begin{array}{cc}\hfill & R_w\ge \frac{1}{2}log\left(\frac{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}{\alpha }\right)\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill & R_{\mathrm{s}}\ge \frac{1}{2}log\left(\frac{1-{\rho }_x^2{\rho }_z^2(1-\alpha )}{\alpha }\right)\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill & R_{\ell }\ge \frac{1}{2}log\left(\frac{1-{\rho }_x^2{\rho }_z^2(1-\alpha )}{1-{\rho }_x^2(1-\alpha )}\right)\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill & D\ge \frac{\alpha (1-{\rho }_x^2{\rho }_y^2)}{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}.\hfill \end{array}$$

(38)

Proof Sketch.

For the achievability proof, let $U\sim \mathcal{N}(0,1-\alpha )$ and $\mathrm{\Theta }\sim \mathcal{N}(0,\alpha )$, as in [41] ([Equation (32)]) and [42] (Appendix B), be independent random variables for some $\alpha \in (0,1]$ such that $\tilde{X}=U+\mathrm{\Theta }$ and $U-\tilde{X}-X-(Y,Z)$ form a Markov chain. Choose the reconstruction function $\widehat{\tilde{X}}(U,Y)$ as the minimum mean square error (MMSE) estimator, and given any fixed $D>0$, auxiliary random variables are chosen such that the distortion constraint is satisfied. We then have, for the squared error distortion,

$$\begin{array}{c}\hfill D=\mathbb{E}\left[{\left(\tilde{X}-\widehat{\tilde{X}}(U,Y)\right)}^2\right]\stackrel{\left(a\right)}{=}\frac{1}{2\pi e}{e}^{2h\left(\tilde{X}\right|U,Y)}\end{array}$$

(39)

where equality in $\left(a\right)$ is achieved because $\tilde{X}$ is Gaussian and the reconstruction function is the MMSE estimator [43] (Theorem 8.6.6). Define the covariance matrix of the vector random variable $[\tilde{X},U,Y]$ as ${\mathbf{K}}_{\tilde{X}UY}$ and of $[U,Y]$ as ${\mathbf{K}}_{UY}$, respectively. We then have

$$\begin{array}{cc}\hfill & h\left(\tilde{X}\right|U,Y)=h(\tilde{X},U,Y)-h(U,Y)\hfill \\ \hfill & =\frac{1}{2}log\left(2\pi e\frac{\det \left({\mathbf{K}}_{\tilde{X}UY}\right)}{\det \left({\mathbf{K}}_{UY}\right)}\right)\hfill \end{array}$$

(40)

where $\det (·)$ is the determinant of a matrix; see also [12] (Section F). Combining (39) and (40), and calculating the determinants, we obtain

$$\begin{array}{c}\hfill D=\frac{\alpha (1-{\rho }_x^2{\rho }_y^2)}{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}.\end{array}$$

(41)

One can also show that

$$\begin{array}{cc}\hfill & I(U;\tilde{X})=h\left(\tilde{X}\right)-h\left(\tilde{X}\right|U)=\frac{1}{2}log\left(\frac{1}{\alpha }\right)\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill & I(U;X)=h\left(X\right)-h\left(X\right|U)=\frac{1}{2}log\left(\frac{1}{1-{\rho }_x^2(1-\alpha )}\right)\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill & I(U;Y)=h\left(Y\right)-h\left(Y\right|U)=\frac{1}{2}log\left(\frac{1}{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}\right)\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill & I(U;Z)=h\left(Z\right)-h\left(Z\right|U)=\frac{1}{2}log\left(\frac{1}{1-{\rho }_x^2{\rho }_z^2(1-\alpha )}\right).\hfill \end{array}$$

(45)

Thus, by calculating (25)–(27), the achievability proof follows.

For the converse proof, one can first show that

$$\begin{array}{cc}\hfill & I(U;\tilde{X})-I(U;Y)=h\left(Y\right|U)-h\left(\tilde{X}\right|U)\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill & I(U;\tilde{X})-I(U;Z)=h\left(Z\right|U)-h\left(\tilde{X}\right|U)\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill & I(U;X)-I(U;Z)=h\left(Z\right|U)-h(X\left|U\right)\hfill \end{array}$$

(48)

which follow since $h\left(\tilde{X}\right)=h\left(X\right)=h\left(Y\right)=h\left(Z\right)$. Suppose

$$\begin{array}{cc}\hfill & h\left(\tilde{X}\right|U)=\frac{1}{2}log\left(2\pi e\alpha \right)\hfill \end{array}$$

(49)

for any $\alpha \in (0,1]$ that represents the unique variance of a Gaussian random variable; see [20] (Lemma 2) for a similar result applied to binary random variables. Thus, by applying the conditional EPI, we obtain

$$\begin{array}{cc}\hfill & e^{2h\left(Y\right|U)}\stackrel{\left(a\right)}{=}{e}^{2h\left({\rho }_x{\rho }_y\tilde{X}\right|U)}+e^{2h({\rho }_y{N}_x+N_y)}\hfill \\ \hfill & =2\pi e\left({\rho }_x^2{\rho }_y^2\alpha +{\rho }_y^2(1-{\rho }_x^2)+1-{\rho }_y^2\right)\hfill \\ \hfill & =2\pi e\left(1-{\rho }_x^2{\rho }_y^2(1-\alpha )\right)\hfill \end{array}$$

(50)

where $\left(a\right)$ follows because $U-\tilde{X}-(N_x,N_y)$ form a Markov chain and $(N_x,N_y)$ are independent of $\tilde{X}$, so $(N_x,N_y)$ are independent of U, and equality is satisfied since, given U, ${\rho }_x{\rho }_y\tilde{X}$ and $({\rho }_y{N}_x+N_y)$ are conditionally independent and they are Gaussian random variables, as imposed in (49) above; see [20] (Lemma 1 and Equation (28)) for a similar result applied to binary random variables by extending Mrs. Gerber’s lemma. Similarly, we have

$$\begin{array}{cc}\hfill & e^{2h\left(Z\right|U)}=2\pi e\left(1-{\rho }_x^2{\rho }_z^2(1-\alpha )\right)\hfill \end{array}$$

(51)

which follows by replacing $(Y,{\rho }_y,N_y)$ with $(Z,{\rho }_z,N_z)$ in (50), respectively, because the channel $P_{Y|U}$ can be mapped to $P_{Z|U}$ with these changes due to (29)–(31) and the Markov chain relation $U-\tilde{X}-X-(Y,Z)$. Furthermore, we have

$$\begin{array}{cc}\hfill & e^{2h\left(X\right|U)}\stackrel{\left(a\right)}{=}{e}^{2h\left({\rho }_x\tilde{X}\right|U)}+e^{2h\left(N_x\right)}\hfill \\ \hfill & =2\pi e\left({\rho }_x^2\alpha +1-{\rho }_x^2\right)\hfill \\ \hfill & =2\pi e\left(1-{\rho }_x^2(1-\alpha )\right)\hfill \end{array}$$

(52)

where $\left(a\right)$ follows because $N_x$ is independent of U, and equality is achieved since, given U, ${\rho }_x\tilde{X}$ and $N_x$ are conditionally independent and are Gaussian random variables. Therefore, by applying (46)–(52) to (25)–(27), the converse proof for (35)–(37) follows.

Next, consider

$$\begin{array}{cc}\hfill & h\left(\tilde{X}\right|U,Y)=-I(U;\tilde{X}|Y)+h\left(\tilde{X}\right|Y)\hfill \\ \hfill & \stackrel{\left(a\right)}{=}-h\left(Y\right|U)+h\left(\tilde{X}\right|U)+h\left(Y\right|\tilde{X})\hfill \\ \hfill & \stackrel{\left(b\right)}{=}\frac{1}{2}log\left(\frac{\alpha }{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}\right)+h({\rho }_x{\rho }_y\tilde{X}+{\rho }_y{N}_x+N_y|\tilde{X})\hfill \\ \hfill & \stackrel{\left(c\right)}{=}\frac{1}{2}log\left(\frac{\alpha }{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}\right)+h({\rho }_y{N}_x+N_y)\hfill \\ \hfill & =\frac{1}{2}log\left(2\pi e\frac{\alpha ({\rho }_y^2(1-{\rho }_x^2)+(1-{\rho }_y^2))}{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}\right)\hfill \\ \hfill & =\frac{1}{2}log\left(2\pi e\frac{\alpha (1-{\rho }_x^2{\rho }_y^2)}{1-{\rho }_x^2{\rho }_y^2(1-\alpha )}\right)\hfill \end{array}$$

(53)

where $\left(a\right)$ follows by (25) and (46), and since $h\left(Y\right)=h\left(\tilde{X}\right)$, $\left(b\right)$ follows by (49) and (50), and $\left(c\right)$ follows because $(N_x,N_y)$ are independent of $\tilde{X}$. Furthermore, for any random variable $\tilde{X}$ and reconstruction function $\widehat{\tilde{X}}(U,Y)$, we have [43] (Theorem 8.6.6)

$$\begin{array}{c}\hfill \mathbb{E}\left[{\left(\tilde{X}-\widehat{\tilde{X}}(U,Y)\right)}^2\right]\ge \frac{1}{2\pi e}{e}^{2h\left(\tilde{X}\right|U,Y)}.\end{array}$$

(54)

Combining the distortion constraint given in Corollary 1 with (53) and (54), the converse proof for (38) follows. □

5. Multiple Binary-input Additive Gaussian Noise Channels

Consider next a binary remote source $X\in \{-1,1\}$ and its binary noisy measurement $\tilde{X}\in \{-1,1\}$ observed by the encoder, which represents a practical setting with binary quantizations. For instance, a static random-access memory (SRAM) start-up output at a nominal temperature is a binary value obtained by quantizing sums of Gaussian random variables [28,44]. Suppose the noisy channel $P_{YZ|X}$ outputs consist of a single discrete-time additive Gaussian noise channel output Y observed by the decoder and two independent discrete-time additive Gaussian noise channel outputs $\mathit{Z}={[Z_1,Z_2]}^T$ observed by the eavesdropper, in which the eavesdropper obtains more information by measuring the remote source twice. Furthermore, assume that X is uniformly distributed, the binary channel $P_{\tilde{X}|X}$ is symmetric such that $Pr[\tilde{X}\ne X]=p$ for $p\in [0,1]$, and we also have

$$\begin{array}{cc}\hfill Y=& {\rho }_{y}X+N_y\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill \mathit{Z}=& \left[\begin{array}{c}{Z}_1\\ Z_2\end{array}\right]={\rho }_{z}X\left[\begin{array}{c}1\\ 1\end{array}\right]+\left[\begin{array}{c}{N}_{z_1}\\ N_{z_2}\end{array}\right]\hfill \end{array}$$

(56)

where we have fixed correlation coefficients ${\rho }_y,{\rho }_z\in (-1,1)$ and additive Gaussian noise random variables

$$\begin{array}{cc}\hfill & N_y\sim \mathcal{N}(0,1-{\rho }_y^2)\hfill \end{array}$$

(57)

$$\begin{array}{cc}\hfill & N_{z_1}\sim \mathcal{N}(0,1-{\rho }_z^2)\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill & N_{z_2}\sim \mathcal{N}(0,1-{\rho }_z^2)\hfill \end{array}$$

(59)

such that $(X,N_y,N_{z_1},N_{z_2})$ are mutually independent. Consider the Hamming distortion, i.e., $d(\tilde{x},\widehat{\tilde{x}})=\mathbb{1}\{\tilde{x}\ne \widehat{\tilde{x}}\}$. Impose the condition $|{\rho }_z|>|{\rho }_y|$ such that $Z_1$ and $Z_2$ are less noisy than Y, so $\mathit{Z}$ is also less noisy than Y, which follows by applying similar steps as being applied in Section 4. Thus, for $R_0=0$, the region ${\mathcal{R}}_{\mathrm{D}}$ characterized in Corollary 1 is also valid for such binary-input additive Gaussian noise channels when one replaces Z with $\mathit{Z}$. A computable achievable lossy secure and private source coding region for such channels is given next.

Proposition 3.

For the setting with multiple binary-input additive Gaussian noise channels, defined above, such that $|{\rho }_z|>|{\rho }_y|$ and $R_0=0$, the region ${\mathcal{R}}_{\mathrm{D}}$ with Hamming distortion includes the set of all rate tuples $(R_w,R_{\mathrm{s}},R_{\ell },D)$ satisfying, for an independent random variable $C\sim \mathrm{Bern}(p\ast q)$ with any $q\in [0,0.5]$ and for any $\lambda \in [0,1]$,

$$\begin{array}{c}{R}_w\ge \lambda \left(1-H_b\left(q\right)-h\left({\rho }_{y}X+N_y\right)+h\left({\rho }_y(1-2C)+N_y\right)\right)\hfill \end{array}$$

(60)

$$\begin{array}{l}{R}_s\ge \lambda \left(1-H_b\left(q\right)-h\left(\left[\begin{array}{c}{\rho }_{z}X+N_{z_1}\\ {\rho }_{z}X+N_{z_2}\end{array}\right]\right)+h\left(\left[\begin{array}{c}{\rho }_z(1-2C)+N_{z_1}\\ {\rho }_z(1-2C)+N_{z_2}\end{array}\right]\right)\right)\end{array}$$

(61)

$$\begin{array}{l}{R}_{\ell }\ge \lambda \left(1-H_b(p\ast q)-h\left(\left[\begin{array}{c}{\rho }_{z}X+N_{z_1}\\ {\rho }_{z}X+N_{z_2}\end{array}\right]\right)+h\left(\left[\begin{array}{c}{\rho }_z(1-2C)+N_{z_1}\\ {\rho }_z(1-2C)+N_{z_2}\end{array}\right]\right)\right)\end{array}$$

(62)

$$\begin{array}{c}D\ge \lambda q+(1-\lambda )\left(p\ast Q\left(\frac{{\rho }_y}{\sqrt{1-{\rho }_y^2}}\right)\right)\hfill \end{array}$$

(63)

where random variable $Y=\left({\rho }_{y}X+N_y\right)$ has pdf

$$\begin{array}{c}\hfill \frac{1}{2}\frac{\left(e^{-\frac{{(y+{\rho }_y)}^2}{2(1-{\rho }_y^2)}}+e^{-\frac{{(y-{\rho }_y)}^2}{2(1-{\rho }_y^2)}}\right)}{\sqrt{2\pi (1-{\rho }_y^2)}}\end{array}$$

(64)

the random variable $\overline{Y}=\left({\rho }_y(1-2C)+N_y\right)$ has pdf

$$\begin{array}{c}\hfill (p\ast q)\frac{e^{-\frac{{(\overline{y}+{\rho }_y)}^2}{2(1-{\rho }_y^2)}}}{\sqrt{2\pi (1-{\rho }_y^2)}}+(1-(p\ast q))\frac{e^{-\frac{{(\overline{y}-{\rho }_y)}^2}{2(1-{\rho }_y^2)}}}{\sqrt{2\pi (1-{\rho }_y^2)}}\end{array}$$

(65)

the vector random variable $\left[\begin{array}{c}{Z}_1\\ Z_2\end{array}\right]=\left(\left[\begin{array}{c}{\rho }_{z}X+N_{z_1}\\ {\rho }_{z}X+N_{z_2}\end{array}\right]\right)$ has joint pdf

$$\begin{array}{c}\hfill \frac{1}{2}\frac{\left(e^{-\frac{\left({(z_1+{\rho }_z)}^2+{(z_2+{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}+e^{-\frac{\left({(z_1-{\rho }_z)}^2+{(z_2-{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}\right)}{2\pi (1-{\rho }_z^2)}\end{array}$$

(66)

and the vector random variable $\left[\begin{array}{c}{\overline{Z}}_1\\ {\overline{Z}}_1\end{array}\right]=\left(\left[\begin{array}{c}{\rho }_z(1-2C)+N_{z_1}\\ {\rho }_z(1-2C)+N_{z_2}\end{array}\right]\right)$ has joint pdf

$$\begin{array}{c}\hfill (p\ast q)\frac{e^{-\frac{\left({({\overline{z}}_1+{\rho }_z)}^2+{({\overline{z}}_2+{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}}{2\pi (1-{\rho }_z^2)}+(1-(p\ast q))\frac{e^{-\frac{\left({({\overline{z}}_1-{\rho }_z)}^2+{({\overline{z}}_2-{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}}{2\pi (1-{\rho }_z^2)}.\end{array}$$

(67)

Proof. 

We first evaluate (25)–(27) by choosing a binary uniformly distributed U and a channel $P_{\tilde{X}|U}$ such that $Pr[\tilde{X}\ne U]=q$ for any $q\in [0,0.5]$. We have

$$\begin{array}{cc}\hfill & I(U;\tilde{X})=H\left(\tilde{X}\right)-H\left(\tilde{X}\right|U)\stackrel{\left(a\right)}{=}1-H_b\left(q\right)\hfill \end{array}$$

(68)

$$\begin{array}{cc}\hfill & I(U;X)=H\left(X\right)-H\left(X\right|U)\stackrel{\left(b\right)}{=}1-H_b(p\ast q)\hfill \end{array}$$

(69)

where $\left(a\right)$ and $\left(b\right)$ follow by relabeling the input and output symbols to represent the channels $P_{\tilde{X}|U}$ and $P_{X|\tilde{X}}$ as BSC$\left(q\right)$ and BSC$\left(p\right)$, respectively, which follows since entropy is preserved under a bijective mapping for discrete random variables. For relabeled symbols, the channel $P_{X|U}$ is a BSC$(p\ast q)$ since it is a concatenation of two BSCs, so denote the independent random noise component in this channel as $C\sim \mathrm{Bern}(p\ast q)$. Then, we obtain

$$\begin{array}{c}\hfill h\left(Y\right|U)=h({\rho }_{y}X+N_y|U)\stackrel{\left(a\right)}{=}h({\rho }_y(1-2C)+N_y)=h\left(\overline{Y}\right)\end{array}$$

(70)

where $\left(a\right)$ follows since symbols $\{-1,1\}$ correspond to the antipodal modulation of binary symbols, and since $(C,N_y,U)$ are mutually independent. One can compute (70) numerically by using the pdf

$$\begin{array}{c}\hfill p_{\overline{Y}}\left(\overline{y}\right)=\sum _{c=0}^1{P}_C\left(c\right)p_{\overline{Y}|C}\left(\overline{y}\right|c)=(p\ast q)\frac{e^{-\frac{{(\overline{y}+{\rho }_y)}^2}{2(1-{\rho }_y^2)}}}{\sqrt{2\pi (1-{\rho }_y^2)}}+(1-(p\ast q))\frac{e^{-\frac{{(\overline{y}-{\rho }_y)}^2}{2(1-{\rho }_y^2)}}}{\sqrt{2\pi (1-{\rho }_y^2)}}.\end{array}$$

(71)

Similarly, we can compute

$$\begin{array}{c}\hfill h\left(Y\right)=h({\rho }_{y}X+N_y)\end{array}$$

(72)

numerically by using the pdf

$$\begin{array}{c}\hfill p_Y\left(y\right)=\sum _{x\in \{-1,1\}}{P}_X\left(x\right)p_{Y|X}\left(y\right|x)=\frac{1}{2}\frac{\left(e^{-\frac{{(y+{\rho }_y)}^2}{2(1-{\rho }_y^2)}}+e^{-\frac{{(y-{\rho }_y)}^2}{2(1-{\rho }_y^2)}}\right)}{\sqrt{2\pi (1-{\rho }_y^2)}}.\end{array}$$

(73)

Next, consider

$$\begin{array}{c}h\left(\mathit{Z}\right|U)=h\left(\left({\rho }_{z}X\left[\begin{array}{c}1\\ 1\end{array}\right]+\left[\begin{array}{c}{N}_{z_1}\\ N_{z_2}\end{array}\right]\right)|U\right)\stackrel{\left(a\right)}{=}h\left(\left[\begin{array}{c}{\rho }_z(1-2C)+N_{z_1}\\ {\rho }_z(1-2C)+N_{z_2}\end{array}\right]\right)=h\left(\left[\begin{array}{c}{\overline{Z}}_1\\ {\overline{Z}}_2\end{array}\right]\right)\end{array}$$

(74)

where $\left(a\right)$ follows since $(C,N_{z_1},N_{z_2},U)$ are mutually independent. Denote

$$\begin{array}{cc}\hfill & \overline{\mathit{Z}}={[{\overline{Z}}_1,{\overline{Z}}_2]}^T.\hfill \end{array}$$

(75)

We can compute (74) numerically by using the joint pdf

$$\begin{array}{cc}\hfill & p_{\overline{\mathit{Z}}}\left(\overline{\mathbf{z}}\right)=p_{{\overline{Z}}_1{\overline{Z}}_2}({\overline{z}}_1,{\overline{z}}_2)=\sum _{c=0}^1{P}_C\left(c\right)p_{{\overline{Z}}_1{\overline{Z}}_2|C}({\overline{z}}_1,{\overline{z}}_2|c)\hfill \\ \hfill & =(p\ast q)\frac{e^{-\frac{\left({({\overline{z}}_1+{\rho }_z)}^2+{({\overline{z}}_2+{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}}{2\pi (1-{\rho }_z^2)}+(1-(p\ast q))\frac{e^{-\frac{\left({({\overline{z}}_1-{\rho }_z)}^2+{({\overline{z}}_2-{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}}{2\pi (1-{\rho }_z^2)}\hfill \end{array}$$

(76)

which follows since $\overline{\mathit{Z}}|C$ is a jointly Gaussian vector random variable with independent components ${\overline{Z}}_1|C$ and ${\overline{Z}}_2|C$, since every scalar linear combination of the components is Gaussian; see [45] (Theorem 1). Similarly, we can compute

$$\begin{array}{c}\hfill h\left(\mathit{Z}\right)=h\left(\left[\begin{array}{c}{\rho }_{z}X+N_{z_1}\\ {\rho }_{z}X+N_{z_2}\end{array}\right]\right)\end{array}$$

(77)

numerically by using the joint pdf

$$\begin{array}{cc}\hfill & p_{\mathit{Z}}\left(\mathit{z}\right)=p_{Z_1{Z}_2}(z_1,z_2)=\sum _{x\in \{-1,1\}}{P}_X\left(x\right)p_{Z_1{Z}_2|X}(z_1,z_2|x)\hfill \\ \hfill & =\frac{1}{2}\frac{\left(e^{-\frac{\left({(z_1+{\rho }_z)}^2+{(z_2+{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}+e^{-\frac{\left({(z_1-{\rho }_z)}^2+{(z_2-{\rho }_z)}^2\right)}{2(1-{\rho }_z^2)}}\right)}{2\pi (1-{\rho }_z^2)}.\hfill \end{array}$$

(78)

Now, we consider the expected distortion. First, choose the reconstruction function

$$\begin{array}{c}\hfill {\widehat{\tilde{X}}}_1(U,Y)=U\end{array}$$

(79)

for the binary uniformly distributed U and the channel $P_{\tilde{X}|U}$ such that $Pr[\tilde{X}\ne U]=q$ for any $q\in [0,0.5]$, as considered above. For this reconstruction function and choices of U and $P_{\tilde{X}|U}$, we obtain the expected distortion

$$\begin{array}{c}\hfill \mathbb{E}\left[d\left(\tilde{X},{\widehat{\tilde{X}}}_1(U,Y)\right)\right]=q.\end{array}$$

(80)

Second, choose the reconstruction function

$$\begin{array}{c}\hfill {\widehat{\tilde{X}}}_2(U,Y)=sgn\left(Y\right)\end{array}$$

(81)

and consider U. We then obtain

$$\begin{array}{c}\hfill \mathbb{E}\left[d\left(\tilde{X},{\widehat{\tilde{X}}}_2(U,Y)\right)\right]=p\ast Q\left(\frac{{\rho }_y}{\sqrt{1-{\rho }_y^2}}\right)\end{array}$$

(82)

which follows since the channel $P_{sgn\left(Y\right)|\tilde{X}}$ can be considered as a concatenation of two BSCs with crossover probabilities p and $Q\left(\frac{{\displaystyle {\rho }_y}}{\sqrt{{\displaystyle 1-{\rho }_y^2}}}\right)$, where the former follows since $Pr[\tilde{X}\ne X]=p$ and the latter because $X\in \{-1,1\}$ and

$$\begin{array}{c}\hfill Pr[X\ne sgn\left(Y\right)]=Pr[X\ne sgn({\rho }_{y}X+N_y)]=Pr[N_y>{\rho }_y].\end{array}$$

(83)

Therefore, the proof for the achievable lossy secure and private source coding region follows by combining (68)–(70), (72), (74), (77), (80), and (82) by applying time sharing, with time-sharing parameter $\lambda \in [0,1]$, between the two reconstruction functions in (79) and (81) with corresponding U and $P_{\tilde{X}|U}$, since for constant U the terms in (25)–(27) are zero. □

Remark 1.

The proof of Proposition 3 follows similar steps as those in [46] (Section II) and it seems that the achievable lossy secure and private source coding region given in Proposition 3 is optimal. Considering $(R_w,R_s,R_{\ell })$, one can apply Mrs. Gerber’s lemma to show that the choice of U such that $P_{\tilde{X}|U}$ is a BSC$\left(q\right)$ after relabeling the input and output symbols is optimal, since Mrs. Gerber’s lemma is valid for all binary-input symmetric memoryless channels with discrete or continuous outputs [47]. This result follows because convexity is preserved; see also [48] (Appendix B) for an alternative proof of convexity preservation for independent BSC measurements. However, it is not entirely clear how to prove that the sign operation used for estimation suffices for the rate region.

6. Proof for Theorem 1

6.1. Achievability Proof for Theorem 1

Proof Sketch.

We leverage the output statistics of random binning (OSRB) method [16,49,50] for the achievability proof by following the steps described in [51] (Section 1.6).

Let $(V^n,U^n,{\tilde{X}}^n,X^n,Y^n,Z^n)$ be i.i.d. according to $P_{VU\tilde{X}XYZ}$ that can be obtained from (14) by fixing $P_{U|\tilde{X}}$ and $P_{V|U}$, such that $\mathbb{E}\left[d\right(\tilde{X},\widehat{\tilde{X}})]\le (D+ϵ)$ for any $ϵ>0$. To each $v^n$ assign two random bin indices $F_{\mathrm{v}}\in [1:2^{n{\tilde{R}}_{\mathrm{v}}}]$ and $W_{\mathrm{v}}\in [1:2^{n{R}_{\mathrm{v}}}]$. Furthermore, to each $u^n$ assign three random bin indices $F_{\mathrm{u}}\in [1:2^{n{\tilde{R}}_{\mathrm{u}}}]$, $W_{\mathrm{u}}\in [1:2^{n{R}_{\mathrm{u}}}]$, and $K_{\mathrm{u}}\in [1:2^{n{R}_0}]$, where $R_0$ is the private key rate defined in Section 2. Public indices $F=(F_{\mathrm{v}},F_{\mathrm{u}})$ represent the choice of a source encoder and decoder pair. Furthermore, we impose that the messages sent by the source encoder $\mathsf{Enc}(·,·)$ to the source decoder $\mathsf{Dec}(·,·,·)$ are

$$\begin{array}{cc}\hfill & W=(W_{\mathrm{v}},W_{\mathrm{u}},K+K_{\mathrm{u}})\hfill \end{array}$$

(84)

where the summation with the private key is in modulo- $2^{n{R}_0}$, i.e., one-time padding.

The public index $F_{\mathrm{v}}$ is almost independent of $({\tilde{X}}^n,X^n,Y^n,Z^n)$ if we have [49] (Theorem 1)

$$\begin{array}{c}\hfill {\tilde{R}}_{\mathrm{v}}<H\left(V\right|\tilde{X},X,Y,Z)\stackrel{\left(a\right)}{=}H\left(V\right|\tilde{X})\end{array}$$

(85)

where $\left(a\right)$ follows since $(X,Y,Z)-\tilde{X}-V$ form a Markov chain. The constraint in (85) suggests that the expected value, taken over the random bin assignments, of the variational distance between the joint probability distributions $\mathrm{Unif}[1:2^{n{\tilde{R}}_{\mathrm{v}}}]·P_{{\tilde{X}}^n}$ and $P_{F_{\mathrm{v}}{\tilde{X}}^n}$ vanishes when $n\to \infty$. Moreover, the public index $F_{\mathrm{u}}$ is almost independent of $(V^n,{\tilde{X}}^n,X^n,Y^n,Z^n)$ if we have

$$\begin{array}{c}\hfill {\tilde{R}}_{\mathrm{u}}<H\left(U\right|V,\tilde{X},X,Y,Z)\stackrel{\left(a\right)}{=}H\left(U\right|V,\tilde{X})\end{array}$$

(86)

where $\left(a\right)$ follows from the Markov chain relation $(X,Y,Z)-\tilde{X}-(U,V)$.

Using a Slepian–Wolf (SW) [1] decoder that observes $(Y^n,F_{\mathrm{v}},W_{\mathrm{v}})$, one can reliably estimate $V^n$ if we have [49] (Lemma 1)

$$\begin{array}{c}\hfill {\tilde{R}}_{\mathrm{v}}+R_{\mathrm{v}}>H\left(V\right|Y)\end{array}$$

(87)

since then the expected error probability, taken over random bin assignments, vanishes when $n\to \infty$. Furthermore, one can reliably estimate $U^n$ by using a SW decoder that observes $(K,V^n,Y^n,F_{\mathrm{u}},W_{\mathrm{u}},K+K_{\mathrm{u}})$ if we have

$$\begin{array}{cc}\hfill & R_0+{\tilde{R}}_{\mathrm{u}}+R_{\mathrm{u}}>H\left(U\right|V,Y).\hfill \end{array}$$

(88)

To satisfy (85)–(88), for any $ϵ>0$ we fix

$$\begin{array}{cc}\hfill & {\tilde{R}}_{\mathrm{v}}=H\left(V\right|\tilde{X})-ϵ\hfill \end{array}$$

(89)

$$\begin{array}{cc}\hfill & R_{\mathrm{v}}=I(V;\tilde{X})-I(V;Y)+2ϵ\hfill \end{array}$$

(90)

$$\begin{array}{cc}\hfill & {\tilde{R}}_{\mathrm{u}}=H\left(U\right|V,\tilde{X})-ϵ\hfill \end{array}$$

(91)

$$\begin{array}{cc}\hfill & R_0+R_{\mathrm{u}}=I(U;\tilde{X}|V)-I(U;Y|V)+2ϵ.\hfill \end{array}$$

(92)

Since all tuples $(v^n,u^n,{\tilde{x}}^n,x^n,y^n,z^n)$ are in the jointly typical set with high probability, by the typical average lemma [2] (p. 26), the distortion constraint (4) is satisfied.

Communication Rate: (90) and (92) result in a communication (storage) rate of

$$\begin{array}{cc}\hfill & R_{\mathrm{w}}=R_0+R_{\mathrm{v}}+R_{\mathrm{u}}\stackrel{\left(a\right)}{=}I(U;\tilde{X}|Y)+4ϵ\hfill \end{array}$$

(93)

where $\left(a\right)$ follows since $V-U-\tilde{X}-Y$ form a Markov chain.

Privacy Leakage Rate: Since private key K is uniformly distributed, and is independent of source and channel random variables, we can consider the following virtual scenario to calculate the leakage. We first assume for the virtual scenario that there is no private key such that the encoder output for the virtual scenario is

$$\begin{array}{cc}\hfill & \overline{W}=(W_{\mathrm{v}},W_{\mathrm{u}},K_{\mathrm{u}}).\hfill \end{array}$$

(94)

We calculate the leakage for the virtual scenario. Then, given the mentioned properties of the private key and due to the one-time padding step in (84), we can subtract $H\left(K\right)=n{R}_0$ from the leakage calculated for the virtual scenario to obtain the leakage for the original problem, which follows from the sum of (91) and (92) if $ϵ\to 0$ when $n\to \infty$. Thus, we have the privacy leakage

$$\begin{array}{cc}\hfill & I(X^n;W,F|Z^n)=I(X^n;\overline{W},F|Z^n)-n{R}_0\hfill \\ \hfill & \stackrel{\left(a\right)}{=}H(\overline{W},F|Z^n)-H(\overline{W},F|X^n)-n{R}_0\hfill \\ \hfill & \stackrel{\left(b\right)}{=}H(\overline{W},F|Z^n)-H(U^n,V^n|X^n)+H\left(V^n\right|\overline{W},F,X^n)+H\left(U^n\right|V^n,\overline{W},F,X^n)-n{R}_0\hfill \\ \hfill & \stackrel{\left(c\right)}{\le }H(\overline{W},F|Z^n)-nH(U,V|X)+2n{ϵ}_n-n{R}_0\hfill \end{array}$$

(95)

where $\left(a\right)$ follows because $(\overline{W},F)-X^n-Z^n$ form a Markov chain, $\left(b\right)$ follows since $(U^n,V^n)$ determine $(F_{\mathrm{u}},W_{\mathrm{u}},K_{\mathrm{u}},F_{\mathrm{v}},W_{\mathrm{v}})$, and $\left(c\right)$ follows since $(U^n,V^n,X^n)$ is i.i.d. and for some ${ϵ}_n>0$ such that ${ϵ}_n\to 0$ when $n\to \infty$ because $(F_{\mathrm{v}},W_{\mathrm{v}},X^n)$ can reliably recover $V^n$ by (87) because of the Markov chain relation $V^n-X^n-Y^n$ and, similarly, $(F_{\mathrm{u}},W_{\mathrm{u}},K_{\mathrm{u}},V^n,X^n)$ can reliably recover $U^n$ by (88) because of $H\left(U\right|V,Y)\ge H(U|V,X)$ that is proved in [21] (Equation (55)) for the Markov chain relation $(V,U)-X-Y$.

Next, we consider the term $H(\overline{W},F|Z^n)$ in (95) and provide single letter bounds on it by applying the six different decodability results given in [21] (Section V-A) that are applied to an entirely similar conditional entropy term in [21] (Equation (54)) that measures the uncertainty in indices conditioned on an i.i.d. multi-letter random variable. Thus, combining the six decodability results in [21] (Section V-A) with (95) we obtain

$$\begin{array}{cc}\hfill & I(X^n;W,F|Z^n)\le n\left({[I(U;Z|V)-I(U;Y|V)+ϵ]}^{-}+I(U;X|Z)+3{ϵ}_n-R_0\right).\hfill \end{array}$$

(96)

The equation (92) implicitly assumes that private key rate $R_0$ is less than $(I(U;\tilde{X}|V)-I(U;Y|V)+2ϵ)$$=(I(U;\tilde{X}|Y,V)+2ϵ)$, where the equality follows from the Markov chain relation $(V,U)-\tilde{X}-Y$. The communication rate results are not affected by this assumption, since ${\tilde{X}}^n$ should be reconstructed by the decoder. However, if the private key rate $R_0$ is greater than or equal to $(I(U;\tilde{X}|Y,V)+2ϵ)$, then we can remove the bin index $K_{\mathrm{u}}$ from the code construction above and apply one-time padding to the bin index $W_{\mathrm{u}}$, such that we have the encoder output

$$\begin{array}{cc}\hfill & \stackrel{=}{W}=(W_{\mathrm{v}},W_{\mathrm{u}}+K)\hfill \end{array}$$

(97)

where the summation with the private key is in modulo- $2^{n{R}_{\mathrm{u}}}=2^{n(I(U;\tilde{X}|Y,V)+2ϵ)}$. Thus, one then does not leak any information about $W_{\mathrm{u}}$ to the eavesdropper because of the one-time padding step in (97). We then have privacy leakage

$$\begin{array}{cc}\hfill & I(X^n;\stackrel{=}{W},F|Z^n)=I(X^n;W_{\mathrm{v}},F|Z^n)\hfill \\ \hfill & \stackrel{\left(a\right)}{\le }H\left(X^n\right|Z^n)-H\left(X^n\right|Z^n,W_{\mathrm{v}},F_{\mathrm{v}})+{ϵ}_n^{\prime }\hfill \\ \hfill & \stackrel{\left(b\right)}{\le }H\left(X^n\right|Z^n)-H\left(X^n\right|Z^n,V^n)+{ϵ}_n^{\prime }\hfill \\ \hfill & \stackrel{\left(c\right)}{=}nI(V;X|Z)+{ϵ}_n^{\prime }\hfill \end{array}$$

(98)

where $\left(a\right)$ follows for some ${ϵ}_n^{\prime }$ such that ${ϵ}_n^{\prime }\to 0$ when $n\to \infty$ since by (86) $F_{\mathrm{u}}$ is almost independent of $(V^n,X^n,Z^n)$; see also [52] (Theorem 1), $\left(b\right)$ follows since $V^n$ determines $(F_{\mathrm{v}},W_{\mathrm{v}})$, and $\left(c\right)$ follows because $(X^n,Z^n,V^n)$ are i.i.d.

Note we can reduce the privacy leakage given in (98) if $R_0\ge (I(U;\tilde{X})-I(U;Y)+4ϵ)=(I(U;\tilde{X}|Y)+4ϵ)$, where the equality follows from the Markov chain relation $U-\tilde{X}-Y$, since then we can apply one-time padding to both bin indices $W_{\mathrm{v}}$ and $W_{\mathrm{u}}$ with the sum rate

$$\begin{array}{cc}\hfill & R_{\mathrm{v}}+R_{\mathrm{u}}\stackrel{\left(a\right)}{=}I(V;\tilde{X})-I(V;Y)+2ϵ+I(U;\tilde{X}|V)-I(U;Y|V)+2ϵ\hfill \\ \hfill & \stackrel{\left(b\right)}{=}I(U;\tilde{X})-I(U;Y)+4ϵ\hfill \end{array}$$

(99)

where $\left(a\right)$ follows by (90) and (92), and $\left(b\right)$ follows from the Markov chain relation $V-U-\tilde{X}-Y$. Thus, one then does not leak any information about $(W_{\mathrm{v}},W_{\mathrm{u}})$ to the eavesdropper because of the one-time padding step, so we then obtain the privacy leakage of

$$\begin{array}{cc}\hfill & I(X^n;F|Z^n)=I(X^n;F_{\mathrm{v}}|Z^n)+I(X^n;F_{\mathrm{u}}|Z^n,F_{\mathrm{v}})\stackrel{\left(a\right)}{\le }2{ϵ}_n^{\prime }\hfill \end{array}$$

(100)

where $\left(a\right)$ follows since by (85) $F_{\mathrm{v}}$ is almost independent of $(X^n,Z^n)$ and by (86) $F_{\mathrm{u}}$ is almost independent of $(V^n,X^n,Z^n)$.

Secrecy Leakage Rate: Similar to the privacy leakage analysis above, we first consider the virtual scenario with the encoder output given in (94), and then calculate the leakage for the original problem by subtracting $H\left(K\right)=n{R}_0$ from the leakage calculated for the virtual scenario. Thus, we obtain

$$\begin{array}{cc}\hfill & I({\tilde{X}}^n;W,F|Z^n)=I({\tilde{X}}^n;\overline{W},F|Z^n)-n{R}_0\hfill \\ \hfill & \stackrel{\left(a\right)}{=}H(\overline{W},F|Z^n)-H(\overline{W},F|{\tilde{X}}^n)-n{R}_0\hfill \\ \hfill & \stackrel{\left(b\right)}{=}H(\overline{W},F|Z^n)-H(U^n,V^n|{\tilde{X}}^n)+H\left(V^n\right|\overline{W},F,{\tilde{X}}^n)+H\left(U^n\right|V^n,\overline{W},F,{\tilde{X}}^n)\hfill \\ \hfill & \stackrel{\left(c\right)}{\le }H(\overline{W},F|Z^n)-nH(U,V|\tilde{X})+2n{ϵ}_n^{\prime }-n{R}_0\hfill \\ \hfill & \stackrel{\left(d\right)}{\le }n\left({[I(U;Z|V)-I(U;Y|V)+ϵ]}^{-}+I(U;\tilde{X}|Z)+3{ϵ}_n^{\prime }-R_0\right)\hfill \end{array}$$

(101)

where $\left(a\right)$ follows from the Markov chain relation $(\overline{W},F)-{\tilde{X}}^n-Z^n$, $\left(b\right)$ follows since $(U^n,V^n)$ determine $(\overline{W},F)$, $\left(c\right)$ follows because $(V^n,U^n,{\tilde{X}}^n)$ are i.i.d. and because $(F_{\mathrm{v}},W_{\mathrm{v}},{\tilde{X}}^n)$ can reliably recover $V^n$ by (87) due to the Markov chain relation $V^n-{\tilde{X}}^n-Y^n$ and, similarly, $(F_{\mathrm{u}},W_{\mathrm{u}},K_{\mathrm{u}},V^n,{\tilde{X}}^n)$ can reliably recover $U^n$ by (88) due to $H\left(U\right|V,Y)\ge H\left(U\right|V,\tilde{X})$ that can be proved as in [21] (Equation (55)) for the Markov chain relation $(V,U)-\tilde{X}-Y$, and $\left(d\right)$ follows by applying the six decodability results in [21] (Section V-A) that are applied to (95) with the final result in (96) by replacing X with $\tilde{X}$.

Similar to the privacy leakage analysis above, if we have $R_0\ge (I(U;\tilde{X}|Y,V)+2ϵ)$, then we can eliminate $K_{\mathrm{u}}$ and apply one-time padding as in (97), such that no information about $W_{\mathrm{u}}$ is leaked to the eavesdropper, we have

$$\begin{array}{cc}\hfill & I({\tilde{X}}^n;\stackrel{=}{W},F|Z^n)=I({\tilde{X}}^n;W_{\mathrm{v}},F|Z^n)\hfill \\ \hfill & \stackrel{\left(a\right)}{\le }H\left({\tilde{X}}^n\right|Z^n)-H\left({\tilde{X}}^n\right|Z^n,W_{\mathrm{v}},F_{\mathrm{v}})+{ϵ}_n^{\prime }\hfill \\ \hfill & \stackrel{\left(b\right)}{\le }H\left({\tilde{X}}^n\right|Z^n)-H\left({\tilde{X}}^n\right|Z^n,V^n)+{ϵ}_n^{\prime }\hfill \\ \hfill & \stackrel{\left(c\right)}{=}nI(V;\tilde{X}|Z)+{ϵ}_n^{\prime }\hfill \end{array}$$

(102)

where $\left(a\right)$ follows because by (86) $F_{\mathrm{u}}$ is almost independent of $(V^n,{\tilde{X}}^n,Z^n)$, $\left(b\right)$ follows since $V^n$ determines $(F_{\mathrm{v}},W_{\mathrm{v}})$, and $\left(c\right)$ follows because $({\tilde{X}}^n,Z^n,V^n)$ are i.i.d.

If $R_0\ge (I(U;\tilde{X}|Y)+4ϵ)$, we can apply one-time padding to hide $(W_{\mathrm{v}},W_{\mathrm{u}})$, as in the privacy leakage analysis above. We then have the secrecy leakage of

$$\begin{array}{cc}\hfill & I({\tilde{X}}^n;F|Z^n)=I({\tilde{X}}^n;F_{\mathrm{v}}|Z^n)+I({\tilde{X}}^n;F_{\mathrm{u}}|Z^n,F_{\mathrm{v}})\stackrel{\left(a\right)}{\le }2{ϵ}_n^{\prime }\hfill \end{array}$$

(103)

where $\left(a\right)$ follows since by (85) $F_{\mathrm{v}}$ is almost independent of $({\tilde{X}}^n,Z^n)$ and by (86) $F_{\mathrm{u}}$ is almost independent of $(V^n,{\tilde{X}}^n,Z^n)$.

Suppose that public indices F are generated uniformly at random, and the encoder generates $(V^n,U^n)$ according to $P_{V^n{U}^n|{\tilde{X}}^n{F}_{\mathrm{v}}{F}_{\mathrm{u}}}$ that can be obtained from the proposed binning scheme above to compute the bins $W_{\mathrm{v}}$ from $V^n$ and $W_{\mathrm{u}}$ from $U^n$, respectively. Such a procedure results in a joint probability distribution almost equal to $P_{VU\tilde{X}XYZ}$ fixed above [51] (Section 1.6). The privacy and secrecy leakage metrics above are expectations over all possible public index realizations $F=f$. Therefore, using a time-sharing random variable Q for convexification and applying the selection lemma [53] (Lemma 2.2) to each decodability case separately, the achievability for Theorem 1 follows by choosing an $ϵ>0$ such that $ϵ\to 0$ when $n\to \infty$. □

6.2. Converse Proof for Theorem 1

Proof Sketch.

Assume that for some ${\delta }_n>0$ and $n\ge 1$, there exist an encoder and a decoder, such that (1)–(4) are satisfied for some tuple $(R_{\mathrm{w}},R_s,R_{\ell },D)$ given a private key with rate $R_0$.

Define $V_i\triangleq (W,Y_{i+1}^n,Z^{i-1})$ and $U_i\triangleq (W,Y_{i+1}^n,Z^{i-1},X^{i-1},K)$ that satisfy the Markov chain relation $V_i-U_i-{\tilde{X}}_i-X_i-(Y_i,Z_i)$ by definition of the source statistics. We have

$$\begin{array}{cc}\hfill & D+{\delta }_n\stackrel{\left(a\right)}{\ge }\mathbb{E}\left[d\left({\tilde{X}}^n,\widehat{{\tilde{X}}^n}(Y^n,W,K)\right)\right]\hfill \\ \hfill & \stackrel{\left(b\right)}{\ge }\mathbb{E}\left[d\left({\tilde{X}}^n,\widehat{{\tilde{X}}^n}(Y^n,W,K,X^{i-1},Z^{i-1})\right)\right]\hfill \\ \hfill & \stackrel{\left(c\right)}{=}\mathbb{E}\left[d\left({\tilde{X}}^n,\widehat{{\tilde{X}}^n}(Y_i^n,W,K,X^{i-1},Z^{i-1})\right)\right]\hfill \\ \hfill & \stackrel{\left(d\right)}{=}\frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[d\left({\tilde{X}}_i,\widehat{{\tilde{X}}_i}(U_i,Y_i)\right)\right]\hfill \end{array}$$

(104)

where $\left(a\right)$ follows by (4), $\left(b\right)$ follows since providing more information to the reconstruction function does not increase expected distortion, $\left(c\right)$ follows from the Markov chain relation

$$\begin{array}{c}\hfill Y^{i-1}-(Y_i^n,X^{i-1},Z^{i-1},W,K)-{\tilde{X}}^n\end{array}$$

(105)

and $\left(d\right)$ follows from the definition of $U_i$.

Communication Rate: For any $R_0\ge 0$, we have

$$\begin{array}{cc}\hfill & n(R_{\mathrm{w}}+{\delta }_n)\stackrel{\left(a\right)}{\ge }log\left|\mathcal{W}\right|\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \ge H\left(W\right|Y^n,K)-H\left(W\right|Y^n,K,{\tilde{X}}^n)\hfill \end{array}$$

(106)

$$\begin{array}{cc}\hfill & \stackrel{\left(b\right)}{=}\sum _{i=1}^{n}I(W;{\tilde{X}}_i|{\tilde{X}}^{i-1},Y_{i+1}^n,Z^{i-1},K,Y_i)\hfill \\ \hfill & \stackrel{\left(c\right)}{=}\sum _{i=1}^{n}I({\tilde{X}}^{i-1},Y_{i+1}^n,Z^{i-1},K,W;{\tilde{X}}_i|Y_i)\hfill \\ \hfill & \stackrel{\left(d\right)}{\ge }\sum _{i=1}^{n}I(X^{i-1},Y_{i+1}^n,Z^{i-1},K,W;{\tilde{X}}_i|Y_i)\hfill \end{array}$$

(107)

$$\begin{array}{cc}\hfill & \stackrel{\left(e\right)}{=}\sum _{i=1}^{n}I(U_i;{\tilde{X}}_i|Y_i)\hfill \end{array}$$

(108)

where $\left(a\right)$ follows by (1), $\left(b\right)$ follows from the Markov chain relation

$$\begin{array}{c}\hfill (Y^{i-1},X^{i-1},Z^{i-1})-({\tilde{X}}^{i-1},Y_i^n,K)-({\tilde{X}}_i,W)\end{array}$$

(109)

$\left(c\right)$ follows because $({\tilde{X}}_i,Y_i)$ are independent of $({\tilde{X}}^{i-1},Y_{i+1}^n,Z^{i-1},K)$, $\left(d\right)$ follows by applying the data processing inequality to the Markov chain relation in (109), and $\left(e\right)$ follows from the definition of $U_i$.

Privacy Leakage Rate: We obtain

$$\begin{array}{cc}\hfill & n(R_{\ell }+{\delta }_n)\hfill \\ \hfill & \stackrel{\left(a\right)}{\ge }[I(W;Y^n)-I(W;Z^n)]+[I(W;X^n)-I(W;Y^n)]\hfill \\ \hfill & \stackrel{\left(b\right)}{=}[I(W;Y^n)-I(W;Z^n)]+I(W;X^n|K)-I(K;X^n|W)-I(W;Y^n|K)+I(K;Y^n|W)\hfill \\ \hfill & \stackrel{\left(c\right)}{=}[I(W;Y^n)-I(W;Z^n)]+[I(W;X^n|K)-I(W;Y^n|K)]-I(K;X^n|W,Y^n)\hfill \\ \hfill & \ge \sum _{i=1}^n\left[I(W;Y_i|Y_{i+1}^n)-I(W;Z_i|Z^{i-1})\right]\hfill \\ \hfill & +\sum _{i=1}^n\left[I(W;X_i|X^{i-1},K)-I(W;Y_i|Y_{i+1}^n,K)\right]-H\left(K\right)\hfill \\ \hfill & \stackrel{\left(d\right)}{=}\sum _{i=1}^n\left[I(W;Y_i|Y_{i+1}^n,Z^{i-1})-I(W;Z_i|Z^{i-1},Y_{i+1}^n)-R_0\right]\hfill \\ \hfill & +\sum _{i=1}^n\left[I(W;X_i|X^{i-1},Y_{i+1}^n,K)-I(W;Y_i|Y_{i+1}^n,X^{i-1},K)\right]\hfill \\ \hfill & \stackrel{\left(e\right)}{=}\sum _{i=1}^n\left[I(W;Y_i|Y_{i+1}^n,Z^{i-1})-I(W;Z_i|Z^{i-1},Y_{i+1}^n)-R_0\right]\hfill \\ \hfill & +\sum _{i=1}^n\left[I(W;X_i|X^{i-1},Y_{i+1}^n,Z^{i-1},K)-I(W;Y_i|Y_{i+1}^n,X^{i-1},Z^{i-1},K)\right]\hfill \\ \hfill & \stackrel{\left(f\right)}{=}\sum _{i=1}^n\left[I(W,Y_{i+1}^n,Z^{i-1};Y_i)-I(W,Z^{i-1},Y_{i+1}^n;Z_i)-R_0\right]\hfill \\ \hfill & +\sum _{i=1}^n\left[I(W,X^{i-1},Y_{i+1}^n,Z^{i-1},K;X_i)-I(W,Y_{i+1}^n,X^{i-1},Z^{i-1},K;Y_i)\right]\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \stackrel{\left(g\right)}{=}\sum _{i=1}^n\left[I(V_i;Y_i)-I(V_i;Z_i)-R_0+I(U_i,V_i;X_i)-I(U_i,V_i;Y_i)\right]\hfill \\ \hfill & =\sum _{i=1}^n\left[-I(U_i,V_i;Z_i)-R_0+I(U_i,V_i;X_i)+\left(I(U_i;Z_i|V_i)-I(U_i;Y_i|V_i)\right)\right]\hfill \\ \hfill & \stackrel{\left(h\right)}{\ge }\sum _{i=1}^n\left[I(U_i;X_i|Z_i)-R_0+{[I(U_i;Z_i|V_i)-I(U_i;Y_i|V_i)]}^{-}\right]\hfill \end{array}$$

(110)

where $\left(a\right)$ follows by (3) and from the Markov chain relation $W-X^n-Z^n$, $\left(b\right)$ follows since K is independent of $(X^n,Y^n)$, $\left(c\right)$ follows from the Markov chain relation $(W,K)-X^n-Y^n$, $\left(d\right)$ follows because $H\left(K\right)=n{R}_0$ and from Csiszár’s sum identity [54], $\left(e\right)$ follows from the Markov chain relations

$$\begin{array}{cc}\hfill & Z^{i-1}-(X^{i-1},Y_{i+1}^n,K)-(X_i,W)\hfill \end{array}$$

(111)

$$\begin{array}{cc}\hfill & Z^{i-1}-(X^{i-1},Y_{i+1}^n,K)-(Y_i,W)\hfill \end{array}$$

(112)

$\left(f\right)$ follows because $(X^n,Y^n,Z^n)$ are i.i.d. and K is independent of $(X^n,Y^n,Z^n)$, $\left(g\right)$ follows from the definitions of $V_i$ and $U_i$, and $\left(h\right)$ follows from the Markov chain relation $V_i-U_i-X_i-Z_i$.

Next, we provide the matching converse for the privacy leakage rate in (98), which is achieved when $R_0\ge I(U;\tilde{X}|Y,V)$. We have

$$\begin{array}{cc}\hfill & n(R_{\ell }+{\delta }_n)\stackrel{\left(a\right)}{\ge }H\left(X^n\right|Z^n)-H\left(X^n\right|Z^n,W)\hfill \\ \hfill & \stackrel{\left(b\right)}{=}H\left(X^n\right|Z^n)-\sum _{i=1}^{n}H\left(X_i\right|Z_i,Z^{i-1},X_{i+1}^n,W,Y_{i+1}^n)\hfill \\ \hfill & \stackrel{\left(c\right)}{=}H\left(X^n\right|Z^n)-\sum _{i=1}^{n}H\left(X_i\right|Z_i,V_i,X_{i+1}^n)\hfill \\ \hfill & \stackrel{\left(d\right)}{\ge }\sum _{i=1}^n[H\left(X_i\right|Z_i)-H\left(X_i\right|Z_i,V_i)]\hfill \\ \hfill & =\sum _{i=1}^{n}I(V_i;X_i|Z_i)\hfill \end{array}$$

(113)

where $\left(a\right)$ follows by (3), $\left(b\right)$ follows from the Markov chain relation

$$\begin{array}{c}\hfill (Z_{i+1}^n,Y_{i+1}^n)-(X_{i+1}^n,W,Z^i)-X_i\end{array}$$

(114)

$\left(c\right)$ follows from the definition of $V_i$, and $\left(d\right)$ follows because $(X^n,Z^n)$ are i.i.d.

The matching converse for the privacy leakage rate in (100), achieved when $R_0\ge I(U;\tilde{X}|Y)$, follows from the fact that conditional mutual information is non-negative.

Secrecy Leakage Rate: We have

$$\begin{array}{cc}\hfill & n(R_s+{\delta }_n)\hfill \\ \hfill & \stackrel{\left(a\right)}{\ge }[I(W;Y^n)-I(W;Z^n)]+[I(W;{\tilde{X}}^n)-I(W;Y^n)]\hfill \\ \hfill & \stackrel{\left(b\right)}{=}[I(W;Y^n)-I(W;Z^n)]+I(W;{\tilde{X}}^n|K)-I(K;{\tilde{X}}^n|W)-I(W;Y^n|K)+I(K;Y^n|W)\hfill \\ \hfill & \stackrel{\left(c\right)}{=}[I(W;Y^n)-I(W;Z^n)]+[I(W;{\tilde{X}}^n|K)-I(W;Y^n|K)]-I(K;{\tilde{X}}^n|W,Y^n)\hfill \\ \hfill & \stackrel{\left(d\right)}{\ge }\sum _{i=1}^n\left[I(W;Y_i|Y_{i+1}^n)-I(W;Z_i|Z^{i-1})\right]+I(W;{\tilde{X}}^n|Y^n,K)-H\left(K\right)\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \stackrel{\left(e\right)}{=}\sum _{i=1}^n\left[I(W;Y_i|Y_{i+1}^n,Z^{i-1})-I(W;Z_i|Z^{i-1},Y_{i+1}^n)-R_0\right]\hfill \\ \hfill & +nH\left(\tilde{X}\right|Y)-\sum _{i=1}^{n}H\left({\tilde{X}}_i\right|Y_i,Y_{i+1}^n,W,K,{\tilde{X}}^{i-1})\hfill \\ \hfill & \stackrel{\left(f\right)}{\ge }\sum _{i=1}^n\left[I(W,Y_{i+1}^n,Z^{i-1};Y_i)-I(W,Z^{i-1},Y_{i+1}^n;Z_i)-R_0\right]\hfill \\ \hfill & +nH\left(\tilde{X}\right|Y)-\sum _{i=1}^{n}H\left({\tilde{X}}_i\right|Y_i,Y_{i+1}^n,W,K,X^{i-1},Z^{i-1})\hfill \\ \hfill & \stackrel{\left(g\right)}{=}\sum _{i=1}^n\left[I(V_i;Y_i)-I(V_i;Z_i)-R_0\right]+nH\left(\tilde{X}\right|Y)-\sum _{i=1}^{n}H\left({\tilde{X}}_i\right|Y_i,U_i,V_i)\hfill \\ \hfill & \stackrel{\left(h\right)}{=}\sum _{i=1}^n\left[I(V_i;Y_i)-I(V_i;Z_i)-R_0\right]+\sum _{i=1}^n\left[I(U_i,V_i;{\tilde{X}}_i)-I(U_i,V_i;Y_i)\right]\hfill \\ \hfill & =\sum _{i=1}^n\left[-I(U_i,V_i;Z_i)-R_0+I(U_i,V_i;{\tilde{X}}_i)+\left(I(U_i;Z_i|V_i)-I(U_i;Y_i|V_i)\right)\right]\hfill \\ \hfill & \stackrel{\left(i\right)}{\ge }\sum _{i=1}^n\left[I(U_i;{\tilde{X}}_i|Z_i)-R_0+{[I(U_i;Z_i|V_i)-I(U_i;Y_i|V_i)]}^{-}\right]\hfill \end{array}$$

(115)

where $\left(a\right)$ follows by (2) and from the Markov chain relation $W-{\tilde{X}}^n-Z^n$, $\left(b\right)$ follows because K is independent of $({\tilde{X}}^n,Y^n)$, $\left(c\right)$ and $\left(d\right)$ follow from the Markov chain relation $(W,K)-{\tilde{X}}^n-Y^n$, $\left(e\right)$ follows because $H\left(K\right)=n{R}_0$ and $({\tilde{X}}^n,Y^n)$ are i.i.d. and independent of K, and from the Csiszár’s sum identity and the Markov chain relation

$$\begin{array}{c}\hfill Y^{i-1}-({\tilde{X}}^{i-1},W,K,Y_{i+1}^n,Y_i)-{\tilde{X}}_i\end{array}$$

(116)

$\left(f\right)$ follows since $(Y^n,Z^n)$ are i.i.d. and from the data processing inequality applied to the Markov chain relation

$$\begin{array}{c}\hfill (X^{i-1},Z^{i-1})-({\tilde{X}}^{i-1},W,K,Y_{i+1}^n,Y_i)-{\tilde{X}}_i\end{array}$$

(117)

$\left(g\right)$ follows from the definitions of $V_i$ and $U_i$, $\left(h\right)$ follows from the Markov chain relation $(V_i,U_i)-{\tilde{X}}_i-Y_i$, and $\left(i\right)$ follows from the Markov chain relation $V_i-U_i-{\tilde{X}}_i-Z_i$.

Next, the matching converse for the secrecy leakage rate in (102), achieved when $R_0\ge I(U;\tilde{X}|Y,V)$, is provided.

$$\begin{array}{cc}\hfill & n(R_s+{\delta }_n)\stackrel{\left(a\right)}{\ge }H\left({\tilde{X}}^n\right|Z^n)-H\left({\tilde{X}}^n\right|Z^n,W)\hfill \\ \hfill & \stackrel{\left(b\right)}{\ge }H\left({\tilde{X}}^n\right|Z^n)-\sum _{i=1}^{n}H\left({\tilde{X}}_i\right|Z_i,Z^{i-1},{\tilde{X}}_{i+1}^n,W,Y_{i+1}^n)\hfill \\ \hfill & \stackrel{\left(c\right)}{=}H\left({\tilde{X}}^n\right|Z^n)-\sum _{i=1}^{n}H\left({\tilde{X}}_i\right|Z_i,V_i,{\tilde{X}}_{i+1}^n)\hfill \\ \hfill & \stackrel{\left(d\right)}{\ge }\sum _{i=1}^n[H\left({\tilde{X}}_i\right|Z_i)-H\left({\tilde{X}}_i\right|Z_i,V_i)]=\sum _{i=1}^{n}I(V_i;{\tilde{X}}_i|Z_i)\hfill \end{array}$$

(118)

where $\left(a\right)$ follows by (2), $\left(b\right)$ follows from the Markov chain relation

$$\begin{array}{c}\hfill (Z_{i+1}^n,Y_{i+1}^n)-({\tilde{X}}_{i+1}^n,W,Z^i)-{\tilde{X}}_i\end{array}$$

(119)

$\left(c\right)$ follows from the definition of $V_i$, and $\left(d\right)$ follows because $({\tilde{X}}^n,Z^n)$ are i.i.d.

Similar to the privacy leakage analysis above, the matching converse for the secrecy leakage rate in (103), achieved when $R_0\ge I(U;\tilde{X}|Y)$, follows from the fact that conditional mutual information is non-negative. □

Introduce a uniformly distributed time-sharing random variable ${\displaystyle Q\sim \mathrm{Unif}[1:n]}$ that is independent of other random variables, and define $X=X_Q$, ${\displaystyle \tilde{X}={\tilde{X}}_Q}$, ${\displaystyle Y=Y_Q}$, ${\displaystyle Z=Z_Q}$, $V=V_Q$, and $U=(U_Q,Q)$, so

$$\begin{array}{cc}\hfill & (Q,V)-U-\tilde{X}-X-(Y,Z)\hfill \end{array}$$

(120)

form a Markov chain. The converse proof follows by letting ${\delta }_n\to 0$.

Cardinality Bounds: We use the support lemma [54] (Lemma 15.4) for the cardinality bound proofs, which is a standard step, so we omit the proof.