Information Rates for Channels with Fading, Side Information and Adaptive Codewords

Abstract

Generalized mutual information (GMI) is used to compute achievable rates for fading channels with various types of channel state information at the transmitter (CSIT) and receiver (CSIR). The GMI is based on variations of auxiliary channel models with additive white Gaussian noise (AWGN) and circularly-symmetric complex Gaussian inputs. One variation uses reverse channel models with minimum mean square error (MMSE) estimates that give the largest rates but are challenging to optimize. A second variation uses forward channel models with linear MMSE estimates that are easier to optimize. Both model classes are applied to channels where the receiver is unaware of the CSIT and for which adaptive codewords achieve capacity. The forward model inputs are chosen as linear functions of the adaptive codeword’s entries to simplify the analysis. For scalar channels, the maximum GMI is then achieved by a conventional codebook, where the amplitude and phase of each channel symbol are modified based on the CSIT. The GMI increases by partitioning the channel output alphabet and using a different auxiliary model for each partition subset. The partitioning also helps to determine the capacity scaling at high and low signal-to-noise ratios. A class of power control policies is described for partial CSIR, including a MMSE policy for full CSIT. Several examples of fading channels with AWGN illustrate the theory, focusing on on-off fading and Rayleigh fading. The capacity results generalize to block fading channels with in-block feedback, including capacity expressions in terms of mutual and directed information.

1. Introduction

The capacity of fading channels is a topic of interest in wireless communications [1,2,3,4]. Fading refers to model variations over time, frequency, and space. A common approach to track fading is to insert pilot symbols into transmit symbol strings, have receivers estimate fading parameters via the pilot symbols, and have the receivers share their estimated channel state information (CSI) with the transmitters. The CSI available at the receiver (CSIR) and transmitter (CSIT) may be different and imperfect.

Information-theoretic studies on fading channels distinguish between average (ergodic) and outage capacity, causal and non-causal CSI, symbol and rate-limited CSI, and different qualities of CSIR and CSIT that are coarsely categorized as no, perfect, or partial. We refer to [5] for a review of the literature up to 2008. We here focus exclusively on average capacity and causal CSIT as introduced in [6]. Codes for such CSIT, or more generally for noisy feedback [7], are based on Shannon strategies, also called codetrees ([8], Chapter 9.4), or adaptive codewords ([9], Section 4.1). (The term “adaptive codeword” was suggested to the author by J. L. Massey.) Adaptive codewords are usually implemented by a conventional codebook and by modifying the codeword symbols as a function of the CSIT. This approach is optimal for some channels [10] and will be our main interest.

1.1. Block Fading

A model that accounts for the different time scales of data transmission (e.g., nanoseconds) and channel variations (e.g., milliseconds) is block fading [11,12]. Such fading has the channel parameters constant within blocks of L symbols and varying across blocks. A basic setup is as follows.

• The fading is described by a state process $S_{H1},S_{H2},\dots$ independent of the transmitter messages and channel noise. The subscript “H” emphasizes that the states $S_{Hi}$ may be hidden from the transceivers.
• Each receiver sees a state process $S_{R1},S_{R2},\dots$ where $S_{Ri}$ is a noisy function of $S_{Hi}$ for all i.
• Each transmitter sees a state process $S_{T1},S_{T2},\dots$ where $S_{Ti}$ is a noisy function of $S_{Hi}$ for all i.

The state processes may be modeled as memoryless [11,12] or governed by a Markov chain [13,14,15,16,17,18,19,20,21]. The memoryless models are particular cases of Shannon’s model [6]. For scalar channels, $S_{Hi}$ is usually a complex number $H_i$. Similarly, for vector or multi-input, multi-output (MIMO) channels with M- and N-dimensional inputs and outputs, respectively, $S_{Hi}$ is a $N\times M$ matrix ${\mathbf{H}}_i$.

Consider, for example, a point-to-point channel with block-fading and complex-alphabet inputs $X_{i\ell }$ and outputs

$$\begin{array}{c}\hfill Y_{i\ell }=H_i{X}_{i\ell }+Z_{i\ell }\end{array}$$

(1)

where the index i, $i=1,\dots ,n$, enumerates the blocks and the index ℓ, $\ell =1,\dots ,L$, enumerates the symbols of each block. The additive white Gaussian noise (AWGN) $Z_{11},Z_{12},\dots$ is a sequence of independent and identically distributed (i.i.d.) random variables that have a common circularly-symmetric complex Gaussian (CSCG) distribution.

1.2. CSI and In-Block Feedback

The motivation for modeling CSI as independent of the messages is simplicity. If one uses only pilot symbols to estimate the $H_i$ in (1), for example, then the independence is valid, and the capacity analysis may be tractable. However, to improve performance, one can implement data and parameter estimation jointly, and one can actively adjust the transmit symbols $X_{i\ell }$ using past received symbols $Y_{ik}$, $k=1,\dots ,\ell -1$, if in-block feedback is available. (Across-block feedback does not increase capacity if the state processes are memoryless; see ([22], Remark 16).) An information theory for such feedback was developed in [22], where a challenge is that code design is based on adaptive codewords that are more sophisticated than conventional codewords.

For example, suppose the CSIR is $S_{Ri}=H_i$. Then, one might expect that CSCG signaling is optimal, and the capacity is an average of $log(1+\mathrm{SNR})$ terms, where SNR is a signal-to-noise ratio. However, this simplification is based on constraints, e.g., that the CSIT is a function of the CSIR and that the $X_{i\ell }$ cannot influence the CSIT. The former constraint can be realistic, e.g., if the receiver quantizes a pilot-based estimate of $H_i$ and sends the quantization bits to the transmitter via a low-latency and reliable feedback link. On the other hand, the latter constraint is unrealistic in general.

1.3. Auxiliary Models

This paper’s primary motivation is to further develop information theory for adaptive codewords. To gain insight, it is helpful to have achievable rates with $log(1+\mathrm{SNR})$ terms. A common approach to obtain such expressions is to lower bound the channel mutual information $I(X;Y)$ as follows.

Suppose X is continuous and consider two conditional densities: the density $p(x|y)$ and an auxiliary density $q(x|y)$. We will refer to such densities as reverse models; similarly, $p(y|x)$ and $q(y|x)$ are called forward models. One may write the differential entropy of X given Y as

$$\begin{array}{cc}\hfill h(X|Y)& =\mathrm{E}\left[-logp(X|Y)\right]=\underset{\mathrm{average}\mathrm{cross}-\mathrm{entropy}}{\underbrace{\mathrm{E}\left[-logq(X|Y)\right]}}-\underset{\mathrm{average}\mathrm{divergence}\ge 0}{\underbrace{\mathrm{E}\left[log\frac{p(X|Y)}{q(X|Y)}\right]}}\hfill \end{array}$$

(2)

where the first expectation in (2) is an average cross-entropy, and the second is an average informational divergence, which is non-negative. Several criteria affect the choice of $q(x|y)$: the cross-entropy should be simple enough to admit theoretical or numerical analysis, e.g., by Monte Carlo simulation; the cross-entropy should be close to $h(X|Y)$; and the cross-entropy should suggest suitable transmitter and receiver structures.

We illustrate how reverse and forward auxiliary models have been applied to bound mutual information. Assume that $\mathrm{E}\left[X\right]=\mathrm{E}\left[Y\right]=0$ for simplicity.

Reverse Model: Consider the reverse density that models $X,Y$ as jointly CSCG:

$$\begin{array}{c}\hfill q(x|y)=\frac{1}{\pi {\sigma }_L^2}exp\left(-{\left|x-{\widehat{x}}_L\right|}^2/{\sigma }_L^2\right)\end{array}$$

(3)

where ${\widehat{X}}_L=\left(\mathrm{E}\left[X{Y}^{*}\right]/\mathrm{E}\left[{\left|Y\right|}^2\right]\right)Y$ and

$$\begin{array}{c}\hfill {\sigma }_L^2=\mathrm{E}\left[{\left|X-{\widehat{X}}_L\right|}^2\right]=\mathrm{E}\left[{\left|X\right|}^2\right]-\frac{|\mathrm{E}\left[X{Y}^{*}\right]{|}^2}{\mathrm{E}\left[{\left|Y\right|}^2\right]}\end{array}$$

(4)

is the mean square error (MSE) of the estimate ${\widehat{X}}_L$. In fact, ${\widehat{X}}_L$ is the linear estimate with the minimum MSE (MMSE), and ${\sigma }_L^2$ is the linear MMSE (LMMSE) which is independent of $Y=y$; see Section 2.5. The bound in (2) gives

$$\begin{array}{c}\hfill h(X|Y)\le log\left(\pi e{\sigma }_L^2\right).\end{array}$$

(5)

Thus, if X is CSCG, then we have the desired form

$$\begin{array}{c}\hfill I(X;Y)=h(X)-h(X|Y)\ge log\left(1+\frac{{\left|h\right|}^2\mathrm{E}\left[{\left|X\right|}^2\right]}{{\sigma }^2}\right)\end{array}$$

(6)

where the parameters h and ${\sigma }^2$ are

$$\begin{array}{c}\hfill h=\frac{\mathrm{E}\left[Y{X}^{*}\right]}{\mathrm{E}\left[{\left|X\right|}^2\right]},{\sigma }^2=\mathrm{E}\left[{|Y-hX|}^2\right].\end{array}$$

(7)

The bound (6) is apparently due to Pinsker [23,24,25] and is widely used in the literature; see e.g., [18,26,27,28,29,30,31,32,33,34,35,36,37,38]. The bound is usually related to channels $p(y|x)$ with additive noise but (2)–(6) show that it applies generally. The extension to vector channels is given in Section 2.7 below.

Forward Model: A more flexible approach is to choose the reverse density as

$$\begin{array}{c}\hfill q(x|y)=\frac{p(x)q{(y|x)}^s}{q(y)}\end{array}$$

(8)

where $q(y|x)$ is a forward auxiliary model (not necessarily a density), $s\ge 0$ is a parameter to be optimized, and

$$\begin{array}{c}\hfill q(y)={\int }_{\mathbb{C}}p(x)q{(y|x)}^{s}dx.\end{array}$$

(9)

Inserting (8) into (2) we compute

$$\begin{array}{c}\hfill I(X;Y)\ge \underset{s\ge 0}{max}\mathrm{E}\left[log\frac{q{(Y|X)}^s}{q(Y)}\right].\end{array}$$

(10)

The right-hand side (RHS) of (10) is called a generalized mutual information (GMI) [39,40] and has been applied to problems in information theory [41], wireless communications [42,43,44,45,46,47,48,49,50,51], and fiber-optic communications [52,53,54,55,56,57,58,59,60,61]. For example, the bounds (6) and (10) are the same if $s=1$ and

$$\begin{array}{c}\hfill q(y|x)=exp\left({-|y-hx|}^2/{\sigma }^2\right)\end{array}$$

(11)

where h and ${\sigma }^2$ are given by (7). Note that (11) is not a density unless ${\sigma }^2=1/\pi$ but $q(x|y)$ is a density. (We require $q(x|y)$ to be a density to apply the divergence bound in (2).)

We compare the two approaches. The bound (5) is simple to apply and works well since the choices (7) give the maximal GMI for CSCG X; see Proposition 1 below. However, there are limitations: one must use continuous X, the auxiliary model $q(y|x)$ is fixed as (11), and the bound does not show how to design the receiver. Instead, the GMI applies to continuous/discrete/mixed X and has an operational interpretation: the receiver uses $q(y|x)$ rather than $p(y|x)$ to decode. The framework of such mismatched receivers appeared in ([62], Exercise 5.22); see also [63].

1.4. Refined Auxiliary Models

The two approaches above can be refined in several ways, and we review selected variations in the literature.

Reverse Models: The model $q(x|y)$ can be different for each $Y=y$, e.g., on may choose X as Gaussian with mean $\mathrm{E}\left[X|Y=y\right]$ and variance

$$\begin{array}{c}\hfill \mathrm{Var}\left[X|Y=y\right]=\mathrm{E}\left[{\left|X\right|}^2|Y=y\right]-|\mathrm{E}\left[X|Y=y\right]{|}^2\end{array}$$

(12)

and where

$$\begin{array}{c}\hfill q(x|y)=\frac{1}{\pi \mathrm{Var}\left[X|Y=y\right]}exp\left(-\frac{{\left|x-\mathrm{E}\left[X|Y=y\right]\right|}^2}{\mathrm{Var}\left[X|Y=y\right]}\right).\end{array}$$

(13)

Inserting (13) in (2) we have the bound

$$\begin{array}{c}\hfill h(X|Y)\le \mathrm{E}\left[log\left(\pi e\mathrm{Var}\left[X|Y\right]\right)\right]\end{array}$$

(14)

which improves (5) in general, since $\mathrm{Var}\left[X|Y=y\right]$ is the MMSE of X given the event $Y=y$. In other words, we have $\mathrm{Var}\left[X|Y=y\right]\le {\sigma }_L^2$ for all $Y=y$ and the following bound improves (6) for CSCG X:

$$\begin{array}{c}\hfill I(X;Y)\ge \mathrm{E}\left[log\frac{\mathrm{E}\left[{\left|X\right|}^2\right]}{\mathrm{Var}\left[X|Y\right]}\right].\end{array}$$

(15)

In fact, the bound (15) was derived in ([50], Section III.B) by optimizing the GMI in (10) over all forward models of the form

$$\begin{array}{c}\hfill q(y|x)=exp\left(-{\left|{\tilde{g}}_y-{\tilde{f}}_{y}x\right|}^2\right)\end{array}$$

(16)

where ${\tilde{f}}_y$, ${\tilde{g}}_y$ depend on y; see also [47,48,49]. We provide a simple proof. By inserting (16) into (8) and (9), absorbing the s parameter in ${\tilde{f}}_y$ and ${\tilde{g}}_y$, and completing squares, one can equivalently optimize over all reverse densities of the form

$$\begin{array}{cc}\hfill & q(x|y)=exp\left(-{\left|g_y-f_{y}x\right|}^2+h_y\right)\hfill \end{array}$$

(17)

where $|f_y{|}^2=\pi e^{h_y}$ so that $q(x|y)$ is a density. We next bound the cross-entropy as

$$\begin{array}{cc}\mathrm{E}\left[-logq(X|Y=y)\right]\hfill & =\mathrm{E}\left[{\left|g_y/f_y-X\right|}^2\right]{\left|f_y\right|}^2-h_y\hfill \\ \hfill & \ge \mathrm{Var}\left[X|Y=y\right]\pi e^{h_y}-h_y\hfill \end{array}$$

(18)

with equality if $g_y/f_y=\mathrm{E}\left[X|Y=y\right]$; see Section 2.5. The RHS of (18) is minimized by $\mathrm{Var}\left[X|Y=y\right]\pi e^{h_y}=1$, so the best choice for $f_y$, $g_y$, $h_y$ gives the bound (14).

Remark 1.

The model (16) uses generalized nearest-neighbor decoding, improving the rules proposed in [42,43,44]. The authors of [50] pointed out that (6) and (15) use the LMMSE and MMSE, respectively; see ([50], Equation (87)).

Remark 2.

A corresponding forward model can be based on (8) and (13), namely

$$\begin{array}{c}\hfill q{(y|x)}^s=\frac{q(x|y)}{p(x)}\Rightarrow q(y)=1.\end{array}$$

(19)

Remark 3.

The RHS of (15) has a more complicated form than the RHS of (6) due to the outer expectation and conditional variance, and this makes optimizing X challenging when there is CSIR and CSIT. Also, if $p(y|x)$ is known, then it seems sensible to numerically compute $p(y)$ and $I(X;Y)$ directly, e.g., via Monte Carlo or numerical integration.

Remark 4.

Decoding rules for discrete X can be based on decision theory as well as estimation theory; see ([64], Equation (11)).

Forward Models: Refinements of (11) appear in the optical fiber literature where the non-linear Schrödinger equation describes wave propagation [52]. Such channels exhibit complicated interactions of attenuation, dispersion, nonlinearity, and noise, and the channel density is too challenging to compute. One thus resorts to capacity lower bounds based on GMI and Monte Carlo simulation. The simplest models are memoryless, and they work well if chosen carefully. For example, the paper [52] used auxiliary models of the form

$$\begin{array}{c}\hfill q(y|x)=exp\left({-|y-hx|}^2/{\sigma }_{\left|x\right|}^2\right)\end{array}$$

(20)

where h accounts for attenuation and self-phase modulation, and where the noise variance ${\sigma }_{\left|x\right|}^2$ depends on $\left|x\right|$. Also, X was chosen to have concentric rings rather than a CSCG density. Subsequent papers applied progressively more sophisticated models with memory to better approximate the actual channel; see [53,54,55,56,57,58,59]. However, the rate gains over the model (20) are minor (≈12%) for 1000 km links, and the newer models do not suggest practical receiver structures.

A related application is short-reach fiber-optic systems that use direct detection (DD) receivers [65] with photodiodes. The paper [60] showed that sampling faster than the symbol rate increases the DD capacity. However, spectrally efficient filtering gives the channel a long memory, motivating auxiliary models $q(y|x)$ with reduced memory to simplify GMI computations [61,66]. More generally, one may use channel-shortening filters [67,68,69] to increase the GMI.

Remark 5.

The ultimate GMI is $I(X;Y)$, and one can compute this quantity numerically for the channels considered in this paper. We are motivated to focus on forward auxiliary models $q(y|x)$ to understand how to improve information rates for more complex channels. For instance, simple $q(y|x)$ let one understand properties of optimal codes, see Lemma 3, and they suggest explicit power control policies, see Theorem 2.

Remark 6.

The paper [37] (see also ([2], Equation (3.3.45)) and ([70], Equation (6))) derives two capacity lower bounds for massive MIMO channels. These bounds are designed for problems where the fading parameters have small variance so that, in effect, ${\sigma }^2$ in (7) is small. We will instead encounter cases where ${\sigma }^2$ grows in proportion to $\mathrm{E}\left[{\left|X\right|}^2\right]$ and the RHS of (6) quickly saturates as $\mathrm{E}\left[{\left|X\right|}^2\right]$ grows; see Remark 20.

1.5. Organization

This paper is organized as follows. Section 2 defines notation and reviews basic results. Section 3 develops two results for the GMI of scalar auxiliary models with AWGN:

• Proposition 1 in Section 3.1 states a known result, namely that the RHS of (6) is the maximum GMI for the AWGN auxiliary model (11) and a CSCG X.
• Lemma 1 in Section 3.2 generalizes Proposition 1 by partitioning the channel output alphabet into K subsets, $K\ge 1$. We use $K=2$ to establish capacity properties at high and low SNR.

Section 4 and Section 5 apply the GMI to channels with CSIT and CSIR.

• Section 4.3 treats adaptive codewords and develops structural properties of their optimal distribution.
• Lemma 2 in Section 4.4 generalizes Proposition 1 to MIMO channels and adaptive codewords. The receiver models each transmit symbol as a weighted sum of the entries of the corresponding adaptive symbol.
• Lemma 3 in Section 4.5 states that the maximum GMI for scalar channels, an AWGN auxiliary model, adaptive codewords with jointly CSCG entries, and $K=1$ is achieved by using a conventional codebook where each symbol is modified based on the CSIT.
• Lemma 4 in Section 4.6 extends Lemma 3 to MIMO channels, including diagonal or parallel channels.
• Theorem 1 in Section 5.1 generalizes Lemma 3 to include CSIR; we use this result several times in Section 6.
• Lemma 5 in Section 5.3 generalizes Lemmas 1 and 2 by partitioning the channel output alphabet.

Section 6, Section 7 and Section 8 apply the GMI to fading channels with AWGN and illustrate the theory for on-off and Rayleigh fading.

• Lemma 6 in Section 6 gives a general capacity upper bound.
• Section 6.5 introduces a class of power control policies for full CSIT. Theorem 2 develops the optimal policy with an MMSE form.
• Theorem 3 in Section 6.6 provides a quadratic waterfilling expression for the GMI with partial CSIR.

Section 9 develops theory for block fading channels with in-block feedback (or in-block CSIT) that is a function of the CSIR and past channel inputs and outputs.

• Theorem 4 in Section 9.2 generalizes Lemma 4 to MIMO block fading channels;
• Section 9.3 develops capacity expressions in terms of directed information;
• Section 9.4 specializes the capacity to fading channels with AWGN and delayed CSIR;
• Proposition 3 generalizes Proposition 2 to channels with special CSIR and CSIT.

Section 10 concludes the paper. Finally, Appendix A, Appendix B, Appendix C, Appendix D, Appendix E, Appendix F and Appendix G provide results on special functions, GMI calculations, and proofs.

2. Preliminaries

2.1. Basic Notation

Let $1(·)$ be the indicator function that takes on the value 1 if its argument is true and 0 otherwise. Let $\delta (.)$ be the Dirac generalized function with ${\int }_{\mathcal{X}}\delta (x)f(x)dx=f(0)·1(0\in \mathcal{X})$. For $x\in \mathbb{R}$, define ${(x)}^{+}=max(0,x)$. The complex-conjugate, absolute value, and phase of $x\in \mathbb{C}$ are written as $x^{*}$, $\left|x\right|$, and $arg(x)$, respectively. We write $j=\sqrt{-1}$ and $\overline{ϵ}=1-ϵ$.

Sets are written with calligraphic font, e.g., $\mathcal{S}=\{1,\dots ,n\}$ and the cardinality of $\mathcal{S}$ is $\left|\mathcal{S}\right|$. The complement of $\mathcal{S}$ in $\mathcal{T}$ is ${\mathcal{S}}^c$ where $\mathcal{T}$ is understood from the context.

2.2. Vectors and Matrices

Column vectors are written as $\underline{x}={[x_1,\dots ,x_M]}^T$ where M is the dimension, and T denotes transposition. The complex-conjugate transpose (or Hermitian) of $\underline{x}$ is written as ${\underline{x}}^{†}$. The Euclidean norm of $\underline{x}$ is $\parallel \underline{x}\parallel$. Matrices are written with bold letters such as $\mathbf{A}$. The letter $\mathbf{I}$ denotes the identity matrix. The determinant and trace of a square matrix $\mathbf{A}$ are written as $det\mathbf{A}$ and $\mathrm{tr}\mathbf{A}$, respectively.

A singular value decomposition (SVD) is $\mathbf{A}=\mathbf{U}\Sigma {\mathbf{V}}^{†}$ where $\mathbf{U}$ and $\mathbf{V}$ are unitary matrices and $\Sigma$ is a rectangular diagonal matrix with the singular values of $\mathbf{A}$ on the diagonal. The square matrix $\mathbf{A}$ is positive semi-definite if ${\underline{x}}^{†}\mathbf{A}\underline{x}\ge 0$ for all $\underline{x}$. The notation $\mathbf{A}⪯\mathbf{B}$ means that $\mathbf{B}-\mathbf{A}$ is positive semi-definite. Similarly, $\mathbf{A}$ is positive definite if ${\underline{x}}^{†}\mathbf{A}\underline{x}>0$ for all $\underline{x}$, and we write $\mathbf{A}\prec \mathbf{B}$ if $\mathbf{B}-\mathbf{A}$ is positive definite.

2.3. Random Variables

Random variables are written with uppercase letters, such as X, and their realizations with lowercase letters, such as x. We write the distribution of discrete X with alphabet $\mathcal{X}=\{0,\dots ,n-1\}$ as $P_X=[P_X(0),\dots ,P_X(n-1)]$. The density of a real- or complex-valued X is written as $p_X$. Mixed discrete-continuous distributions are written using mixtures of densities and Dirac-$\delta$ functions.

Conditional distributions and densities are written as $P_{X|Y}$ and $p_{X|Y}$, respectively. We usually drop subscripts if the argument is a lowercase version of the random variable, e.g., we write $p(y|x)$ for $p_{Y|X}(y|x)$. One exception is that we consistently write the distributions $P_{S_R}(.)$ and $P_{S_T}(.)$ of the CSIR and CSIT with the subscript to avoid confusion with power notation.

2.4. Second-Order Statistics

The expectation and variance of the complex-valued random variable X are $\mathrm{E}\left[X\right]$ and $\mathrm{Var}\left[X\right]=\mathrm{E}\left[|X-\mathrm{E}\left[X\right]{|}^2\right]$, respectively. The correlation coefficient of $X_1$ and $X_2$ is $\rho =\mathrm{E}\left[U_1{U}_2^{*}\right]$ where

$$\begin{array}{c}\hfill U_i=(X_i-\mathrm{E}\left[X_i\right])/\sqrt{\mathrm{Var}\left[X_i\right]}\end{array}$$

for $i=1,2$. We say that $X_1$ and $X_2$ are fully correlated if $\rho =e^{j\varphi }$ for some real $\varphi$. Conditional expectation and variance are written as $\mathrm{E}\left[X|A=a\right]$ and

$$\begin{array}{c}\hfill \mathrm{Var}\left[X|A=a\right]=\mathrm{E}\left[(X-\mathrm{E}\left[X\right]){(X-\mathrm{E}\left[X\right])}^{*}|A=a\right].\end{array}$$

The expressions $\mathrm{E}\left[X|A\right]$, $\mathrm{Var}\left[X|A\right]$ are random variables that take on the values $\mathrm{E}\left[X|A=a\right]$, $\mathrm{Var}\left[X|A=a\right]$ if $A=a$.

The expectation and covariance matrix of the random column vector $\underline{X}={[X_1,\dots ,X_M]}^T$ are $\mathrm{E}\left[\underline{X}\right]$ and ${\mathbf{Q}}_{\underline{X}}=\mathrm{E}\left[(\underline{X}-\mathrm{E}\left[\underline{X}\right]){(\underline{X}-\mathrm{E}\left[\underline{X}\right])}^{†}\right]$, respectively. We write ${\mathbf{Q}}_{\underline{X},\underline{Y}}$ for the covariance matrix of the stacked vector ${\left[{\underline{X}}^T{\underline{Y}}^T\right]}^T$. We write ${\mathbf{Q}}_{\underline{X}|\underline{Y}=\underline{y}}$ for the covariance matrix of $\underline{X}$ conditioned on the event $\underline{Y}=\underline{y}$. ${\mathbf{Q}}_{\underline{X}|\underline{Y}}$ is a random matrix that takes on the matrix value ${\mathbf{Q}}_{\underline{X}|\underline{Y}=\underline{y}}$ when $\underline{Y}=\underline{y}$.

We often consider CSCG random variables and vectors. A CSCG $\underline{X}$ has density

$$\begin{array}{c}\hfill p(\underline{x})=\frac{exp\left(-{\underline{x}}^{†}{\mathbf{Q}}_{\underline{X}}^{-1}\underline{x}\right)}{{\pi }^{M}det{\mathbf{Q}}_{\underline{X}}}\end{array}$$

and we write $\underline{X}\sim \mathcal{CN}(\underline{0},{\mathbf{Q}}_{\underline{X}})$.

2.5. MMSE and LMMSE Estimation

Assume that $\mathrm{E}\left[\underline{X}\right]=\mathrm{E}\left[\underline{Y}\right]=\underline{0}$. The MMSE estimate of $\underline{X}$ given the event $\underline{Y}=\underline{y}$ is the vector $\widehat{\underline{X}}(\underline{y})$ that minimizes

$$\begin{array}{c}\hfill \mathrm{E}\left[\left.{∥\underline{X}-\widehat{\underline{X}}(\underline{y})∥}^2\right|\underline{Y}=\underline{y}\right].\end{array}$$

Direct analysis gives ([71], Chapter 4)

$$\begin{array}{cc}\hfill & \widehat{\underline{X}}(\underline{y})=\mathrm{E}\left[\underline{X}|\underline{Y}=\underline{y}\right]\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & \mathrm{E}\left[\parallel \underline{X}-\widehat{\underline{X}}{\parallel }^2\right]=\mathrm{E}\left[\parallel \underline{X}{\parallel }^2\right]-\mathrm{E}\left[\parallel \widehat{\underline{X}}{\parallel }^2\right]\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill & {\mathbf{Q}}_{\underline{X}-\widehat{\underline{X}}}={\mathbf{Q}}_{\underline{X}}-{\mathbf{Q}}_{\widehat{\underline{X}}}\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill & \mathrm{E}\left[\left(\underline{X}-\widehat{\underline{X}}\right){\underline{Y}}^{†}\right]=\mathbf{0}\hfill \end{array}$$

(24)

where the last identity is called the orthogonality principle.

The LMMSE estimate of $\underline{X}$ given $\underline{Y}$ with invertible ${\mathbf{Q}}_{\underline{Y}}$ is the vector ${\widehat{\underline{X}}}_L=\mathbf{C}\underline{Y}$ where $\mathbf{C}$ is chosen to minimize $\mathrm{E}\left[\parallel \underline{X}-{\widehat{\underline{X}}}_L{\parallel }^2\right]$. We compute

$$\begin{array}{cc}\hfill & {\widehat{\underline{X}}}_L=\mathrm{E}\left[\underline{X}{\underline{Y}}^{†}\right]{\mathbf{Q}}_{\underline{Y}}^{-1}\underline{Y}\hfill \end{array}$$

(25)

and we also have the properties (22)–(24) with $\widehat{\underline{X}}$ replaced by ${\widehat{\underline{X}}}_L$. Moreover, if $\underline{X}$ and $\underline{Y}$ are jointly CSCG, then the MMSE and LMMSE estimators coincide, and the orthogonality principle (24) implies that the error $\underline{X}-\widehat{\underline{X}}$ is independent of $\underline{Y}$, i.e., we have

$$\begin{array}{cc}\hfill \mathrm{E}\left[\left.\left(\underline{X}-\widehat{\underline{X}}\right){\left(\underline{X}-\widehat{\underline{X}}\right)}^{†}\right|\underline{Y}=\underline{y}\right]& =\mathrm{E}\left[\left.\underline{X}{\underline{X}}^{†}\right|\underline{Y}=\underline{y}\right]-\mathrm{E}\left[\underline{X}{\underline{Y}}^{†}\right]{\mathbf{Q}}_{\underline{Y}}^{-1}\underline{y}{\underline{y}}^{†}{\mathbf{Q}}_{\underline{Y}}^{-1}\mathrm{E}{\left[\underline{X}{\underline{Y}}^{†}\right]}^{†}\hfill \\ \hfill & ={\mathbf{Q}}_{\underline{X}}-{\mathbf{Q}}_{\widehat{\underline{X}}}.\hfill \end{array}$$

(26)

2.6. Entropy, Divergence, and Information

Entropies of random vectors with densities p are written as

$$\begin{array}{c}\hfill h(\underline{X})=\mathrm{E}\left[-logp(\underline{X})\right],h(\underline{X}|\underline{Y})=\mathrm{E}\left[-logp(\underline{X}|\underline{Y})\right]\end{array}$$

where we use logarithms to the base e for analysis. The informational divergence of the densities p and q is

$$\begin{array}{c}\hfill D\left(p\parallel q\right)=\mathrm{E}\left[log\frac{p(\underline{X})}{q(\underline{X})}\right]\end{array}$$

and $D(p\parallel q)\ge 0$ with equality if and only if $p=q$ almost everywhere. The mutual information of $\underline{X}$ and $\underline{Y}$ is

$$\begin{array}{c}\hfill I(\underline{X};\underline{Y})=D\left(p(\underline{X},\underline{Y})\parallel p(\underline{X})p(\underline{Y})\right)=\mathrm{E}\left[log\frac{p(\underline{Y}|\underline{X})}{p(\underline{Y})}\right].\end{array}$$

The average mutual information of $\underline{X}$ and $\underline{Y}$ conditioned on $\underline{Z}$ is $I(\underline{X};\underline{Y}|\underline{Z})$. We write strings as $X^L=(X_1,X_2,\dots ,X_L)$ and use the directed information notation (see [9,72])

$$\begin{array}{cc}\hfill & I(X^L\to Y^L|Z)=\sum _{\ell =1}^{L}I(X^{\ell };Y_{\ell }|Y^{\ell -1},Z)\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill & I(X^L\to Y^L\parallel Z^L|W)=\sum _{\ell =1}^{L}I(X^{\ell };Y_{\ell }|Y^{\ell -1},Z^{\ell },W)\hfill \end{array}$$

(28)

where $Y_0=0$.

2.7. Entropy and Information Bounds

The expression (2) applies to random vectors. Choosing $q(\underline{x}|\underline{y})$ as the conditional density where the $\underline{X},\underline{Y}$ are modeled as jointly CSCG we obtain a generalization of (5):

$$\begin{array}{cc}\hfill h(\underline{X}|\underline{Y})& \le log\frac{det\left(\pi e{\mathbf{Q}}_{\underline{X},\underline{Y}}\right)}{det\left(\pi e{\mathbf{Q}}_{\underline{Y}}\right)}=logdet\left(\pi e\left\{{\mathbf{Q}}_{\underline{X}}-\mathrm{E}\left[\underline{X}{\underline{Y}}^{†}\right]{\mathbf{Q}}_{\underline{Y}}^{-1}\mathrm{E}\left[\underline{Y}{\underline{X}}^{†}\right]\right\}\right).\hfill \end{array}$$

(29)

The vector generalization of (6) for CSCG $\underline{X}$ is

$$\begin{array}{cc}\hfill I(\underline{X};\underline{Y})& =h(\underline{X})-h(\underline{X}|\underline{Y})\hfill \\ \hfill & \ge logdet\left({\left({\mathbf{Q}}_{\underline{X}}-\mathrm{E}\left[\underline{X}{\underline{Y}}^{†}\right]{\mathbf{Q}}_{\underline{Y}}^{-1}\mathrm{E}\left[\underline{Y}{\underline{X}}^{†}\right]\right)}^{-1}{\mathbf{Q}}_{\underline{X}}\right)\hfill \\ \hfill & \stackrel{(a)}{=}logdet\left(\mathbf{I}+{\mathbf{Q}}_{\underline{Z}}^{-1}\mathbf{H}{\mathbf{Q}}_{\underline{X}}^{-1}{\mathbf{H}}^{†}\right)\hfill \end{array}$$

(30)

where (cf. (7))

$$\begin{array}{c}\hfill \mathbf{H}=\mathrm{E}\left[\underline{Y}{\underline{X}}^{†}\right]{\mathbf{Q}}_{\underline{\overline{X}}}^{-1},{\mathbf{Q}}_{\underline{Z}}={\mathbf{Q}}_{\underline{Y}}-\mathbf{H}{\mathbf{Q}}_{\underline{X}}{\mathbf{H}}^{†}\end{array}$$

(31)

and step $(a)$ in (30) follows by the Woodbury identity

$$\begin{array}{c}\hfill {\left(\mathbf{A}+\mathbf{B}\mathbf{C}\mathbf{D}\right)}^{-1}={\mathbf{A}}^{-1}-{\mathbf{A}}^{-1}\mathbf{B}{\left({\mathbf{C}}^{-1}+\mathbf{D}{\mathbf{A}}^{-1}\mathbf{B}\right)}^{-1}\mathbf{D}{\mathbf{A}}^{-1}\end{array}$$

(32)

and the Sylvester identity

$$\begin{array}{cc}\hfill & det\left(\mathbf{I}+\mathbf{A}\mathbf{B}\right)=det\left(\mathbf{I}+\mathbf{B}\mathbf{A}\right).\hfill \end{array}$$

(33)

We also have vector generalizations of (14) and (15):

$$\begin{array}{cc}\hfill h(\underline{X}|\underline{Y})& \le \mathrm{E}\left[logdet\left(\pi e{\mathbf{Q}}_{\underline{X}|\underline{Y}}\right)\right]\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill I(\underline{X};\underline{Y})& \ge \mathrm{E}\left[log\frac{det{\mathbf{Q}}_{\underline{X}}}{det{\mathbf{Q}}_{\underline{X}|\underline{Y}}}\right]\mathrm{for}\mathrm{CSC}\mathrm{G}\underline{X}.\hfill \end{array}$$

(35)

2.8. Capacity and Wideband Rates

Consider the complex-alphabet AWGN channel with output $Y=X+Z$ and noise $Z\sim \mathcal{CN}(0,1)$. The capacity with the block power constraint $\frac{1}{n}{\sum }_{i=1}^n{\left|X_i\right|}^2\le P$ is

$$\begin{array}{c}\hfill C(P)=\underset{\mathrm{E}\left[{\left|X\right|}^2\right]\le P}{max}I(X;Y)=log(1+P).\end{array}$$

(36)

The low SNR regime (small P) is known as the wideband regime [73]. For well-behaved channels such as AWGN channels, the minimum $E_b/N_0$ and the slope S of the capacity vs. $E_b/N_0$ in bits/(3 dB) at the minimum $E_b/N_0$ are (see ([73], Equation (35)) and ([73], Theorem 9))

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{C^{\prime }(0)},S=\frac{2{\left[C^{\prime }(0)\right]}^2}{-C^{″}(0)}\end{array}$$

(37)

where $C^{\prime }(P)$ and $C^{″}(P)$ are the first and second derivatives of $C(P)$ (measured in nats) with respect to P, respectively. For example, the wideband derivatives for (36) are $C^{\prime }(0)=1$ and $C^{″}(0)=-1$ so that the wideband values (37) are

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=log2,S=2.\end{array}$$

(38)

The minimal $E_b/N_0$ is usually stated in decibels, for example $10{log}_{10}(log2)=-1.59$ dB. An extension of the theory to general channels is described in ([74], Section III).

Remark 7.

A useful method is flash signaling, where one sends with zero energy most of the time. In particular, we will consider the CSCG flash density

$$\begin{array}{c}\hfill p(x)=(1-p)\delta (x)+p\frac{e^{-{\left|x\right|}^2/(P/p)}}{\pi (P/p)}\end{array}$$

(39)

where $0<p\le 1$ so that the average power is $\mathrm{E}\left[{\left|X\right|}^2\right]=P$. Note that flash signaling is defined in ([73], Definition 2) as a family of distributions satisfying a particular property as $P\to 0$. We use the terminology informally.

2.9. Uniformly-Spaced Quantizer

Consider a uniformly-spaced scalar quantizer $q_u(.)$ with B bits, domain $[0,\infty )$, and reconstruction points

$$\begin{array}{c}\hfill s\in \{\Delta /2,3\Delta /2,\dots ,\Delta /2+(2^B-1)\Delta \}\end{array}$$

where $\Delta >0$. The quantization intervals are

$$\begin{array}{c}\hfill \mathcal{I}(s)=\left\{\begin{array}{cc}\left[s-\frac{\Delta }{2},s+\frac{\Delta }{2}\right),\hfill & s\ne s_{\max }\hfill \\ \left[s-\frac{\Delta }{2},\infty \right),\hfill & s=s_{\max }\hfill \end{array}\right.\end{array}$$

where $s_{\max }=\Delta /2+(2^B-1)\Delta$. We will consider $B=0,1,\infty$. For $B=\infty$ we choose $q_u(x)=x$.

Suppose one applies the quantizer to the non-negative random variable G with density $p(g)$ to obtain $S_T=q_u(G)$. Let $P_{S_T}$ and $P_{S_T|G}$ be the probability mass functions of $S_T$ without and with conditioning on G, respectively. We have

$$\begin{array}{c}\hfill P_{S_T|G}(s|g)=1\left(g\in \mathcal{I}(s)\right),P_{S_T}(s)={\int }_{g\in \mathcal{I}(s)}p(g)dg\end{array}$$

(40)

and using Bayes’ rule, we obtain

$$\begin{array}{cc}\hfill p(g|s)& =\left\{\begin{array}{cc}p(g)/P_{S_T}(s),\hfill & g\in \mathcal{I}(s)\hfill \\ 0,\hfill & \mathrm{else}.\hfill \end{array}\right.\hfill \end{array}$$

(41)

3. Generalized Mutual Information

We re-derive the GMI in the usual way, where one starts with the forward model $q(y|x)$ rather than the reverse density $q(x|y)$ in (8). Consider the joint density $p(x,y)$ and define $q(y)$ as in (9) for $s\ge 0$. Note that neither $q(y|x)$ nor $q(y)$ must be densities. The GMI is defined in [39] to be ${max}_{s\ge 0}{I}_s(X;Y)$ where (see the RHS of (10))

$$\begin{array}{c}\hfill I_s(X;Y)=\mathrm{E}\left[log\frac{q{(Y|X)}^s}{q(Y)}\right]\end{array}$$

(42)

and where the expectation is with respect to $p(x,y)$. The GMI is a lower bound on the mutual information since

$$\begin{array}{c}\hfill I_s(X;Y)=I(X;Y)-D\left(p_{X,Y}∥p_Y{q}_{X|Y}\right).\end{array}$$

(43)

Moreover, by using Gallager’s derivation of error exponents, but without modifying his “s” variable, the GMI $I_s(X;Y)$ is achievable with a mismatched decoder that uses $q(y|x)$ for its decoding metric [39].

3.1. AWGN Forward Model with CSCG Inputs

A natural metric is based on the AWGN auxiliary channel $Y_a=hX+Z$ where h is a channel parameter and $Z\sim \mathcal{CN}(0,{\sigma }^2)$ is independent of X, i.e., we have the auxiliary model (here a density)

$$\begin{array}{c}\hfill q(y|x)=\frac{1}{\pi {\sigma }^2}exp\left({-|y-hx|}^2/{\sigma }^2\right)\end{array}$$

(44)

where h and ${\sigma }^2$ are to be optimized. A natural input is $X\sim \mathcal{CN}(0,P)$ so that (9) is

$$\begin{array}{c}\hfill q(y)=\frac{\pi {\sigma }^2/s}{{(\pi {\sigma }^2)}^s}·\frac{exp\left(\frac{-{\left|y\right|}^2}{{\sigma }^2/s+{\left|h\right|}^{2}P}\right)}{\pi ({\sigma }^2/s+|h{|}^{2}P)}.\end{array}$$

(45)

We have the following result, see [43] that considers channels of the form (1) and ([47], Proposition 1) that considers general $p(y|x)$.

Proposition 1.

The maximum GMI (42) for the channel $p(y|x)$, a CSCG input X with variance $P>0$, and the auxiliary model (44) with ${\sigma }^2>0$ is

$$\begin{array}{c}\hfill I_1(X;Y)=log\left(1+\frac{|\tilde{h}{|}^{2}P}{{\tilde{\sigma }}^2}\right)\end{array}$$

(46)

where $s=1$ and (cf. (7))

$$\begin{array}{cc}\hfill \tilde{h}& =\mathrm{E}\left[Y{X}^{*}\right]/P\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\tilde{\sigma }}^2& =\mathrm{E}\left[|Y-\tilde{h}{X|}^2\right]=\mathrm{E}\left[{\left|Y\right|}^2\right]-{\left|\tilde{h}\right|}^{2}P.\hfill \end{array}$$

(48)

The expectations are with respect to the actual density $p(x,y)$.

Proof. 

The GMI (42) for the model (44) is

$$\begin{array}{cc}\hfill I_s(X;Y)& =log\left(1+\frac{{\left|h\right|}^{2}P}{{\sigma }^2/s}\right)+\frac{\mathrm{E}\left[{\left|Y\right|}^2\right]}{{\sigma }^2/s+{\left|h\right|}^{2}P}-\frac{\mathrm{E}\left[{|Y-hX|}^2\right]}{{\sigma }^2/s}.\hfill \end{array}$$

(49)

Since (49) depends only on the ratio ${\sigma }^2/s$ one may as well set $s=1$. Thus, choosing $h=\tilde{h}$ and ${\sigma }^2={\tilde{\sigma }}^2$ gives (46).

Next, consider $Y_a=\tilde{h}X+\tilde{Z}$ where $\tilde{Z}\sim \mathcal{CN}(0,{\tilde{\sigma }}^2)$ is independent of X. We have

$$\begin{array}{cc}\hfill & \mathrm{E}\left[|Y_a{|}^2\right]=\mathrm{E}\left[{\left|Y\right|}^2\right]\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill & \mathrm{E}\left[|Y_a-\tilde{h}X{|}^2\right]=\mathrm{E}\left[|Y-\tilde{h}X{|}^2\right].\hfill \end{array}$$

(51)

In other words, the second-order statistics for the two channels with outputs Y (the actual channel output) and $Y_a$ are the same. But the GMI (46) is the mutual information $I(X;Y_a)$. Using (43) and (49), for any s, h and ${\sigma }^2$ we have

$$\begin{array}{c}\hfill I(X;Y_a)=log\left(1+\frac{|\tilde{h}{|}^{2}P}{{\tilde{\sigma }}^2}\right)\ge I_s(X;Y_a)=I_s(X;Y)\end{array}$$

(52)

and equality holds if $h=\tilde{h}$ and ${\sigma }^2/s={\tilde{\sigma }}^2$. □

Remark 8.

The rate (46) is the same as the RHS of (6).

Remark 9.

Proposition 1 generalizes to vector models and adaptive input symbols; see Section 4.4.

Remark 10.

The estimate $\tilde{h}$ is the MMSE estimate of h:

$$\begin{array}{c}\hfill \tilde{h}=arg\underset{h}{min}\mathrm{E}\left[{|Y-hX|}^2\right]\end{array}$$

(53)

and ${\tilde{\sigma }}^2$ is the variance of the error. To see this, expand

$$\begin{array}{c}\hfill \mathrm{E}\left[{|Y-hX|}^2\right]=\mathrm{E}\left[|(Y-\tilde{h}X)+(\tilde{h}-h){X|}^2\right]={\tilde{\sigma }}^2+{|\tilde{h}-h|}^{2}P\end{array}$$

(54)

where the final step follows by the definition of $\tilde{h}$ in (47).

Remark 11.

Suppose that h is an estimate other than (53). Then if $\mathrm{E}\left[{\left|Y\right|}^2\right]>\mathrm{E}\left[{\left|Y-hX\right|}^2\right]$ we may choose

$$\begin{array}{c}\hfill {\sigma }^2/s={\left|h\right|}^{2}P·\frac{\mathrm{E}\left[{\left|Y-hX\right|}^2\right]}{\mathrm{E}\left[{\left|Y\right|}^2\right]-\mathrm{E}\left[{\left|Y-hX\right|}^2\right]}\end{array}$$

(55)

and the GMI (49) simplifies to

$$\begin{array}{cc}\hfill I_s(X;Y)& =log\left(\frac{\mathrm{E}\left[{\left|Y\right|}^2\right]}{\mathrm{E}\left[{\left|Y-hX\right|}^2\right]}\right).\hfill \end{array}$$

(56)

Remark 12.

The LM rate (for “lower bound to the mismatch capacity”) improves the GMI for some $q(y|x)$ [40,75]. The LM rate replaces $q(y|x)$ with $q(y|x)e^{t(x)/s}$ for some function $t(.)$ and permits optimizing s and $t(.)$; see ([41], Section 2.3.2). For example, if $p(y|x)$ has the form $q{(y|x)}^s{e}^{t(x)}$ then the LM rate can be larger than the GMI; see [76,77].

3.2. CSIR and K-Partitions

We consider two generalizations of Proposition 1. The first is for channels with a state $S_R$ known at the receiver but not at the transmitter. The second expands the class of CSCG auxiliary models. The motivation is to obtain more precise models under partial CSIR, especially to better deal with channels at high SNR and with high rates. We here consider discrete $S_R$ and later extend to continuous $S_R$.

CSIR: Consider the average GMI

$$\begin{array}{c}\hfill I_1(X;Y|S_R)=\sum _{s_R}{P}_{S_R}(s_R)I_1(X;Y|S_R=s_R)\end{array}$$

(57)

where $I_1(X;Y|S_R=s_R)$ is the usual GMI where all densities are conditioned on $S_R=s_R$. The parameters (47) and (48) for the event $S_R=s_R$ are now

$$\begin{array}{cc}\hfill \tilde{h}(s_R)& =\frac{\mathrm{E}\left[\left.Y{X}^{*}\right|S_R=s_R\right]}{\mathrm{E}\left[\left.{\left|X\right|}^2\right|S_R=s_R\right]}\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill {\tilde{\sigma }}^2(s_R)& =\mathrm{E}\left[\left.|Y-\tilde{h}(s_R){X|}^2\right|S_R=s_R\right].\hfill \end{array}$$

(59)

The GMI (57) is thus

$$\begin{array}{c}\hfill I_1(X;Y|S_R)=\sum _{s_R}{P}_{S_R}(s_R)log\left(1+\frac{|\tilde{h}(s_R){|}^{2}P}{\tilde{\sigma }{(s_R)}^2}\right).\end{array}$$

(60)

K-Partitions: Let $\{{\mathcal{Y}}_k:k=1,\dots ,K\}$ be a K-partition of $\mathcal{Y}$ and define the auxiliary model

$$\begin{array}{c}\hfill q(y|x)=\frac{1}{\pi {\sigma }_k^2}{e}^{-|y-h_k{x|}^2/{\sigma }_k^2},y\in {\mathcal{Y}}_k.\end{array}$$

(61)

Observe that $q(y|x)$ is not necessarily a density. We choose $X\sim \mathcal{CN}(0,P)$ so that (9) becomes (cf. (45))

$$\begin{array}{c}\hfill q(y)=\frac{\pi {\sigma }_k^2/s}{{(\pi {\sigma }_k^2)}^s}·\frac{exp\left(\frac{-{\left|y\right|}^2}{{\sigma }_k^2/s+{\left|h_k\right|}^{2}P}\right)}{\pi ({\sigma }_k^2/s+|h_k{|}^{2}P)},y\in {\mathcal{Y}}_k.\end{array}$$

(62)

Define the events ${\mathcal{E}}_k=\{Y\in {\mathcal{Y}}_k\}$ for $k=1,\dots ,K$. We have

$$\begin{array}{cc}\hfill I_s(X;Y)& =\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]·\mathrm{E}\left[\left.log\frac{q{(Y|X)}^s}{q(Y)}\right|{\mathcal{E}}_k\right]\hfill \end{array}$$

(63)

and inserting (61) and (62) we have the following lemma.

Lemma 1.

The GMI (42) for the channel $p(y|x)$, $s=1$, a CSCG input X with variance P, and the auxiliary model (61) is (see (49))

$$\begin{array}{c}\hfill I_1(X;Y)=\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]\left[log\left(1+\frac{|h_k{|}^{2}P}{{\sigma }_k^2}\right)+\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_k\right]}{{\sigma }_k^2+{\left|h_k\right|}^{2}P}-\frac{\mathrm{E}\left[|Y-h_{k}X{|}^2|{\mathcal{E}}_k\right]}{{\sigma }_k^2}\right].\end{array}$$

(64)

Remark 13.

K-partitioning formally includes (57) as a special case by including $S_R$ as part of the receiver’s “overall” channel output $\tilde{Y}=[Y,S_R]$. For example, one can partition $\tilde{\mathcal{Y}}$ as $\{{\tilde{\mathcal{Y}}}_{s_R}:s_R\in {\mathcal{S}}_R\}$ where ${\tilde{\mathcal{Y}}}_{s_R}=\mathcal{Y}\times \left\{s_R\right\}$.

Remark 14.

The models (16) and (61) suggest building receivers based on adaptive Gaussian statistics. However, we are motivated to introduce (61) to prove capacity scaling results. For this purpose, we will use $K=2$ with the partition

$$\begin{array}{c}\hfill {\mathcal{E}}_1={\left\{\right|Y|}^2<t_R\},{\mathcal{E}}_2={\left\{\right|Y|}^2\ge t_R\}\end{array}$$

(65)

and $h_1=0$, ${\sigma }_1^2=1$. The GMI (64) thus has only the $k=2$ term and it remains to choose $h_2$, ${\sigma }_2^2$, and $t_R$.

Remark 15.

One can generalize Lemma 1 and partition $\mathcal{X}\times \mathcal{Y}$ rather than $\mathcal{Y}$ only. However, the $q(y)$ in (62) might not have a CSCG form.

Remark 16.

Define $P_k=\mathrm{E}\left[{\left|X\right|}^2|{\mathcal{E}}_k\right]$ and choose the LMMSE auxiliary models with

$$\begin{array}{cc}\hfill h_k& =\mathrm{E}\left[\left.Y{X}^{*}\right|{\mathcal{E}}_k\right]/P_k\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill {\sigma }_k^2& =\mathrm{E}\left[\left.|Y-h_k{X|}^2\right|{\mathcal{E}}_k\right]=\mathrm{E}\left[\left.{\left|Y\right|}^2\right|{\mathcal{E}}_k\right]-{\left|h_k\right|}^2{P}_k\hfill \end{array}$$

(67)

for $k=1,\dots ,K$. The expression (64) is then

$$\begin{array}{c}\hfill I_1(X;Y)=\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]\left[log\left(1+\frac{|h_k{|}^{2}P}{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_k\right]-{\left|h_k\right|}^2{P}_k}\right)-\frac{|h_k{|}^2(P-P_k)}{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_k\right]+{\left|h_k\right|}^2(P-P_k)}\right].\end{array}$$

(68)

Remark 17.

The LMMSE-based GMI (68) reduces to the GMI of Proposition 1 by choosing the trivial partition with $K=1$ and ${\mathcal{Y}}_1=\mathcal{Y}$. However, the GMI (68) may not be optimal for $K\ge 2$. What can be said is that the phase of $h_k$ in (64) should be the same as the phase of $\mathrm{E}\left[Y{X}^{*}|{\mathcal{E}}_k\right]$ for all k. We thus have K two-dimensional optimization problems, one for each pair $(|h_k|,{\sigma }_k^2)$, $k=1,\dots ,K$.

Remark 18.

Suppose we choose a different auxiliary model for each $Y=y$, i.e., consider $K\to \infty$. The reverse density GMI uses the auxiliary model (19) which gives the RHS of (15):

$$\begin{array}{c}\hfill I_1(X;Y)={\int }_{\mathbb{C}}p(y)log\frac{P}{\mathrm{Var}\left[X|Y=y\right]}dy.\end{array}$$

(69)

Instead, the suboptimal (68) is the complicated expression

$$\begin{array}{cc}\hfill I_1(X;Y)={\int }_{\mathbb{C}}p(y)& \left[log\left(1+\frac{|\mathrm{E}\left[X|Y=y\right]{|}^2(P/P_y)}{\mathrm{Var}\left[X|Y=y\right]}\right)\right.\hfill \\ \hfill & \left.-\frac{|\mathrm{E}\left[X|Y=y\right]{|}^2(P/P_y-1)}{\mathrm{Var}\left[X|Y=y\right]+{\left|\mathrm{E}\left[X|Y=y\right]\right|}^2(P/P_y)}\right]dy.\hfill \end{array}$$

(70)

where $P_y=\mathrm{E}\left[{\left|X\right|}^2|Y=y\right]$. We show how to compute these GMIs in Appendix C.

3.3. Example: On-Off Fading

Consider the channel $Y=HX+Z$ where $H,X,Z$ are mutually independent, $P_H(0)=P_H(\sqrt{2})=1/2$, and $Z\sim \mathcal{CN}(0,1)$. The channel exhibits particularly simple fading, giving basic insight into more realistic fading models. We consider two basic scenarios: full CSIR and no CSIR.

Full CSIR: Suppose $S_R=H$ and

$$\begin{array}{c}\hfill q(y|x,h)=p(y|x,h)=\frac{1}{\pi {\sigma }^2}{e}^{{-|y-hx|}^2/{\sigma }^2}\end{array}$$

(71)

which corresponds to having (58) and (59) as

$$\begin{array}{c}\hfill \tilde{h}(0)=0,\tilde{h}\left(\sqrt{2}\right)=\sqrt{2},{\tilde{\sigma }}^2(0)={\sigma }^2\left(\sqrt{2}\right)=1.\end{array}$$

(72)

The GMI (60) with $X\sim \mathcal{CN}(0,P)$ thus gives the capacity

$$\begin{array}{c}\hfill C(P)=\frac{1}{2}log\left(1+2P\right).\end{array}$$

(73)

The wideband values (37) are

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=log2,S=1.\end{array}$$

(74)

Compared with (38), the minimal $E_b/N_0$ is the same as without fading, namely $-1.59$ dB. However, fading reduces the capacity slope S; see the dashed curve in Figure 1.

No CSIR: Suppose $S_R=0$ and $X\sim \mathcal{CN}(0,P)$ and consider the densities

$$\begin{array}{cc}\hfill & p(y|x)=\frac{e^{-{\left|y\right|}^2}}{2\pi }+\frac{e^{-|y-\sqrt{2}{x|}^2}}{2\pi }\hfill \end{array}$$

(75)

$$\begin{array}{cc}\hfill & p(y)=\frac{e^{-{\left|y\right|}^2}}{2\pi }+\frac{e^{-{\left|y\right|}^2/(1+2P)}}{2\pi (1+2P)}.\hfill \end{array}$$

(76)

The mutual information can be computed by numerical integration or by Monte Carlo integration:

$$\begin{array}{c}\hfill I(X;Y)\approx \frac{1}{N}\sum _{i=1}^{N}log\frac{p_{Y|X}(y_i|x_i)}{p_Y(y_i)}\end{array}$$

(77)

where the RHS of (77) converges to $I(X;Y)$ for long strings $x^N,y^N$ sampled from $p(x,y)$. The results for $X\sim \mathcal{CN}(0,P)$ are shown in Figure 1 as the curve labeled “$I(X;Y)$ Gauss”.

Next, Proposition 1 gives $h=1/\sqrt{2}$, ${\sigma }^2=1+P/2$, and

$$\begin{array}{c}\hfill I_1(X;Y)=log\left(1+\frac{P}{2+P}\right).\end{array}$$

(78)

The wideband values (37) are

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=log4,S=2/3\end{array}$$

(79)

so the minimal $E_b/N_0$ is 1.42 dB and the capacity slope S has decreased further. Moreover, the rate saturates at large SNR at 1 bit per channel use.

The “$I(X;Y)$ Gauss” curve in Figure 1 suggests that the no-CSIR capacity approaches the full-CSIR capacity for large SNR. To prove this, consider the $K=2$ partition specified in Remark 14 with $h_1=0$, $h_2=\sqrt{2}$, and ${\sigma }_2^2=1$. Since we are not using LMMSE auxiliary models, we must compute the GMI using the general expression (64), which is

$$\begin{array}{c}\hfill I_1(X;Y)=\mathrm{Pr}\left[{\mathcal{E}}_2\right]\left[log(1+2P)+\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+2P}-\mathrm{E}\left[{\left|Y-\sqrt{2}X\right|}^2|{\mathcal{E}}_2\right]\right].\end{array}$$

(80)

In Appendix B.1, we show that choosing $t_R=P^{{\lambda }_R}+b$ where $0<{\lambda }_R<1$ and b is a real constant makes all terms behave as desired as P increases:

$$\begin{array}{c}\hfill \mathrm{Pr}\left[{\mathcal{E}}_2\right]\to 1/2,\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+2P}\to 1,\mathrm{E}\left[\left.{\left|Y-\sqrt{2}X\right|}^2\right|{\mathcal{E}}_2\right]\to 1.\end{array}$$

(81)

The GMI (80) of Lemma 1 thus gives the maximal value (73) for large P:

$$\begin{array}{c}\hfill \underset{P\to \infty }{lim}\left[\frac{1}{2}log(1+2P)-I_1(X;Y)\right]=0.\end{array}$$

(82)

Figure 1 shows the behavior of $I_1(X;Y)$ for $K=2$, ${\lambda }_R=0.4$, and $b=3$. Effectively, at large SNR, the receiver can estimate H accurately, and one approaches the full-CSIR capacity.

Remark 19.

For on-off fading, one may compute $I(X;Y)$ directly and use the densities (75) and (76) to decode. Nevertheless, the partitioning of Lemma 1 helps prove the capacity scaling (82).

Consider next the reverse density GMI (69) and the forward model GMI (70). Appendix C.1 shows how to compute $\mathrm{E}\left[X|Y=y\right]$, $\mathrm{E}\left[{\left|X\right|}^2|Y=y\right]$, and $\mathrm{Var}\left[X|Y=y\right]$, and Figure 1 plots the GMIs as the curves labeled “rGMI” and “GMI, K = ∞”, respectively. The rGMI curve gives the best possible rates for AWGN auxiliary models, as shown in Section 1.4. The results also show that the large-K GMI (70) is worse than the $K=1$ GMI at low SNR but better than the $K=2$ GMI of Remark 14.

Finally, the curve labeled “$I(X;Y)$ Gauss” in Figure 1 suggests that the minimal $E_b/N_0$ is 1.42 dB even for the capacity-achieving distribution. However, we know from ([73], Theorem 1) that flash signaling (39) can approach the minimal $E_b/N_0$ of $-1.59$ dB. For example, the flash rates $I(X;Y)$ with $p=0.05$ are plotted in Figure 1. Unfortunately, the wideband slope is $S=0$ ([73], Theorem 17), and one requires very large flash powers (very small p) to approach $-1.59$ dB.

Remark 20.

As stated in Remark 6, the paper [37] (see also [2,70]) derives two capacity lower bounds. These bounds are the same for our problem, and they are derived using the following steps (see ([37], Lemmas 3 and 4)):

$$\begin{array}{cc}\hfill I(X;Y)& =I(X,S_H;Y)-I(S_H;Y|X)\hfill \\ \hfill & \ge I(X;Y|S_H)-I(S_H;Y|X).\hfill \end{array}$$

(83)

Now consider $Y=HX+Z$ where $H,X,Z$ are mutually independent, $S_H=H$, $\mathrm{Var}\left[Z\right]=1$, and $X\sim \mathcal{CN}(0,P)$. We have

$$\begin{array}{cc}\hfill I(X;Y|H)& \ge \mathrm{E}\left[log(1+|H{|}^{2}P)\right]\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill I(H;Y|X)& =h(Y|X)-h(Z)\hfill \\ & \le log\left(\pi e(1+\mathrm{Var}\left[H\right]P)\right)-h(Z)\hfill \end{array}$$

(85)

where (84) and (85) follow by (5), in the latter case with the roles of X and Y reversed. The bound (85) works well if $\mathrm{Var}\left[H\right]$ is small, as for massive MIMO with “channel hardening”. However, for our on-off fading model, the bound (83) is

$$\begin{array}{cc}\hfill I(X;Y)& \ge \mathrm{E}\left[log\left(1+{\left|H\right|}^{2}P\right)\right]-log(1+\mathrm{Var}\left[H\right]P)\hfill \\ \hfill & =\frac{1}{2}log(1+2P)-log(1+P/2)\hfill \end{array}$$

(86)

which is worse than the $K=1$ and $K=\infty$ GMIs and is not shown in Figure 1.

4. Channels with CSIT

This section studies Shannon’s channel with side information, or state, known causally at the transmitter [5,6]. We begin by treating general channels and then focus mainly on complex-alphabet channels. The capacity expression has a random variable A that is either a list (for discrete-alphabet states) or a function (for continuous-alphabet states). We refer to A as an adaptive symbol of an adaptive codeword.

4.1. Model

The problem is specified by the functional dependence graph (FDG) in Figure 2. The model has a message M, a CSIT string $S_T^n$, and a noise string $Z^n$. The variables M, $S_T^n$, $Z^n$ are mutually statistically independent, and $S_T^n$ and $Z^n$ are strings of i.i.d. random variables with the same distributions as $S_T$ and Z, respectively. $S_T^n$ is available causally at the transmitter, i.e., the channel input $X_i$, $i=1,\dots ,n$, is a function of M and the sub-string $S_T^i$. The receiver sees the channel outputs

$$\begin{array}{c}\hfill Y_i=f(X_i,S_{Ti},Z_i)\end{array}$$

(87)

for some function $f(.)$ and $i=1,2,\dots ,n$.

Each $A_i$ represents a list of possible choices of $X_i$ at time i. More precisely, suppose that $S_T$ has alphabet ${\mathcal{S}}_T=\{0,1,\dots ,\nu -1\}$ and define the adaptive symbol

$$\begin{array}{c}\hfill A=\left(X(0),\dots ,X(\nu -1)\right)\end{array}$$

whose entries have alphabet $\mathcal{X}$. Here $S_T=s_T$ means that $X(s_T)$ is transmitted, i.e., we have $X=X(S_T)$. If $S_T$ has a continuous alphabet, we make A a function rather than a list, and we may again write $X=X(S_T)$. Some authors therefore write A as $X(.)$. (Shannon in [6] denoted our A and X as the respective X and x.)

Remark 21.

The conventional choice for A if $\mathcal{X}=\mathbb{C}$ is

$$\begin{array}{c}\hfill A=\left(\sqrt{P(0)}{e}^{j\varphi (0)},\dots ,\sqrt{P(\nu -1)}{e}^{j\varphi (\nu -1)}\right)·U\end{array}$$

(88)

where U has $\mathrm{E}\left[{\left|U\right|}^2\right]=1$, $P(s_T)=\mathrm{E}\left[|X(s_T){|}^2\right]$, and $\varphi (s_T)$ is a phase shift. The interpretation is that U represents the symbol of a conventional codebook without CSIT, and these symbols are scaled and rotated. In other words, one separates the message-carrying U from an adaptation due to $S_T$ via

$$\begin{array}{c}\hfill X=\sqrt{P(S_T)}{e}^{j\varphi (S_T)}U.\end{array}$$

(89)

Remark 22.

One may define the channel by the functional relation (87), by $p(y|a)$, or by $p(y|x,s_T)$; see Shannon’s emphasis in ([6], Theorem); see ([22], Remark 3). We generally prefer to use $p(y|a)$ since we interpret A as a channel input.

Remark 23.

One can add feedback and let $X_i$ be a function of $(M,S_T^i,Y^{i-1})$, but feedback does not increase the capacity if the state and noise processes are memoryless ([22], Section V).

Remark 24.

The model (87) permits block fading and MIMO transmission by choosing $X_i$ and $Y_i$ as vectors [11,78].

4.2. Capacity

The capacity of the model under study is (see [6])

$$\begin{array}{c}\hfill C=\underset{A}{max}I(A;Y)\end{array}$$

(90)

where $A-[S_T,X]-Y$ forms a Markov chain. One may limit attention to A with cardinality $\left|\mathcal{A}\right|$ satisfying (see ([22], Equation (56)), [79], ([80], Theorem 1))

$$\begin{array}{c}\hfill \left|\mathcal{A}\right|\le min\left(\left|\mathcal{Y}\right|,1+|{\mathcal{S}}_T|(\left|\mathcal{X}\right|-1)\right).\end{array}$$

(91)

As usual, for the cost function $c(x,y)$ and the average block cost constraint

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n\mathrm{E}\left[c(X_i,Y_i)\right]\le P\end{array}$$

(92)

the unconstrained maximization in (90) becomes a constrained maximization over the A for which $\mathrm{E}\left[c(X,Y)\right]\le P$. Also, a simple upper bound on the capacity is

$$\begin{array}{cc}\hfill C(P)& \le \underset{A:\mathrm{E}\left[c(X,Y)\right]\le P}{max}I(A;Y,S_T)\hfill \\ \hfill & \stackrel{(a)}{=}\underset{X(S_T):\mathrm{E}\left[c(X(S_T),Y)\right]\le P}{max}I(X;Y|S_T)\hfill \end{array}$$

(93)

where step $(a)$ follows by the independence of A and $S_T$. This bound is tight if the receiver knows $S_T$.

Remark 25.

The chain rule for mutual information gives

$$\begin{array}{cc}\hfill I(A;Y)& =I\left(X(0)\dots X(\nu -1);Y\right)\hfill \end{array}$$

(94)

$$\begin{array}{cc}\hfill & =\sum _{s_T=0}^{\nu -1}I\left(X(s_T);Y|X(0),\dots ,X(s_T-1)\right).\hfill \end{array}$$

(95)

The RHS of (94) suggests treating the channel as a multi-input, single-output (MISO) channel, and the expression (95) suggests using multi-level coding with multi-stage decoding [81]. For example, one may use polar coded modulation [82,83,84] with Honda-Yamamoto shaping [85,86].

Remark 26.

For $\mathcal{X}=\mathbb{C}$ and the conventional adaptive symbol (88), we compute $I(A;Y)=I(U;Y)$ and

$$\begin{array}{c}\hfill C(P)=\underset{P(S_T),\varphi (S_T):\mathrm{E}\left[c(X(S_T),Y)\right]\le P}{max}I(U;Y).\end{array}$$

(96)

4.3. Structure of the Optimal Input Distribution

Let $\mathcal{A}$ be the alphabet of A and let $\mathcal{X}=\mathbb{C}$, i.e., we have $\mathcal{A}={\mathbb{C}}^{\nu }$ for discrete $S_T$. Consider the expansions

$$\begin{array}{cc}\hfill p(y|a)& =\sum _{s_T}{P}_{S_T}(s_T)p(y|x(s_T),s_T)\hfill \\ \hfill p(y)& ={\int }_{\mathcal{A}}p(a)p(y|a)da\hfill \end{array}$$

(97)

$$\begin{array}{cc}\hfill & =\sum _{s_T}{P}_{S_T}(s_T){\int }_{\mathbb{C}}p(x(s_T))p(y|x(s_T),s_T)dx(s_T).\hfill \end{array}$$

(98)

Observe that $p(y)$, and hence $h(Y)$, depends only on the marginals $p(x(s_T))$ of A; see ([80], Section III). So define the set of densities having the same marginals as A:

$$\begin{array}{c}\hfill \mathcal{P}(A)=\left\{p(\tilde{a}):p(\tilde{x}(s_T))=p(x(s_T))\mathrm{for}\mathrm{all}{s}_T\in {\mathcal{S}}_T\right\}.\end{array}$$

This set is convex, since for any $p^{(1)}(a),p^{(2)}(a)\in \mathcal{P}(A)$ and $0\le \lambda \le 1$ we have

$$\begin{array}{c}\hfill \lambda p^{(1)}(a)+(1-\lambda )p^{(2)}(a)\in \mathcal{P}(A).\end{array}$$

(99)

Moreover, for fixed $p(y)$, the expression $I(A;Y)$ is a convex function of $p(a|y)$, and $p(a|y)=p(a)p(y|a)/p(y)$ is a linear function of $p(a)$. Maximizing $I(A;Y)$ over $\mathcal{P}(A)$ is thus the same as minimizing the concave function $h(Y|A)$ over the convex set $\mathcal{P}(A)$. An optimal $p(a)$ is thus an extreme of $\mathcal{P}(A)$. Some properties of such extremes are developed in [87,88].

For example, consider $|{\mathcal{S}}_T|=2$ and $\mathcal{X}={\mathcal{S}}_T=\{0,1\}$, for which (91) states that at most $\left|\mathcal{A}\right|=3$ adaptive symbols need have positive probability (and at most $\left|\mathcal{A}\right|=2$ adaptive symbols if $\left|\mathcal{Y}\right|=2$). Suppose the marginals have $P_{X(0)}(0)=1/2$, $P_{X(1)}(0)=3/4$ and consider the matrix notation

$$\begin{array}{c}\hfill P_A=\left[\begin{array}{cc}{P}_A(0,0)& P_A(0,1)\\ P_A(1,0)& P_A(1,1)\end{array}\right]\end{array}$$

where we write $P_A(x_1,x_2)$ for $P_A([x_1,x_2])$. The optimal $P_A$ must then be one of the two extremes

$$\begin{array}{c}\hfill \begin{array}{c}{P}_A=\left[\begin{array}{cc}1/2& 0\\ 1/4& 1/4\end{array}\right],P_A=\left[\begin{array}{cc}1/4& 1/4\\ 1/2& 0\end{array}\right].\end{array}\end{array}$$

(100)

For the first $P_A$, the codebook has the property that if $X(0)=0$ then $X(1)=0$ while if $X(0)=1$ then $X(1)$ is uniformly distributed over $\mathcal{X}=\{0,1\}$.

Next, consider $|{\mathcal{S}}_T|=2$ and marginals $P_{X(0)}$, $P_{X(1)}$ that are uniform over $\mathcal{X}=\{0,1,\dots ,|\mathcal{X}|-1\}$. This case was treated in detail in ([80], Section VI.A), see also [89], and we provide a different perspective. A classic theorem of Birkhoff [90] ensures that the extremes of $\mathcal{P}(A)$ are the $\left|\mathcal{X}\right|!$ distributions $P_A$ for which the $\left|\mathcal{X}\right|\times \left|\mathcal{X}\right|$ matrix

$$\begin{array}{c}\hfill P_A=\left[\begin{array}{ccc}{P}_A(0,0)& \dots & P_A(0,\left|\mathcal{X}\right|-1)\\ ⋮& \ddots & ⋮\\ P_A(\left|\mathcal{X}\right|-1,0)& \dots & P_A(\left|\mathcal{X}\right|-1,\left|\mathcal{X}\right|-1)\end{array}\right].\end{array}$$

is a permutation matrix multiplied by $1/\left|\mathcal{X}\right|$. For example, for $\left|\mathcal{X}\right|=2$ we have the two extremes

$$\begin{array}{c}\hfill \begin{array}{c}{P}_A=\frac{1}{2}\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],P_A=\frac{1}{2}\left[\begin{array}{cc}0& 1\\ 1& 0\end{array}\right].\end{array}\end{array}$$

(101)

The permutation property means that $X(s_T)$ is a function of $X(0)$, i.e., the encoding simplifies to a conventional codebook as in Remark 21 with uniformly-distributed U and a permutation ${\pi }_{s_T}(.)$ indexed by $s_T$ such that $X(S_T)={\pi }_{S_T}(U)$. For example, for the first $P_A$ in (101) we may choose $X(S_T)=U$, which is independent of $S_T$. On the other hand, for the second $P_A$ in (101) we may choose $X(S_T)=U\oplus S_T$ where ⊕ denotes addition modulo-2.

For $|{\mathcal{S}}_T|>2$, the geometry of $\mathcal{P}(A)$ is more complicated; see ([80], Section VI.B). For example, consider $\mathcal{X}=\{0,1\}$ and suppose the marginals $P_{X(s_T)}$, $s_T\in {\mathcal{S}}_T$, are all uniform. Then the extremes include $P_A$ related to linear codes and their cosets, e.g., two extremes for $|{\mathcal{S}}_T|=3$ are related to the repetition code and single parity check code:

$$\begin{array}{c}\hfill \begin{array}{c}{P}_A(a)=1/2,a\in \{[0,0,0],[1,1,1]\}\hfill \\ P_A(a)=1/4,a\in \{[0,0,0],[0,1,1],[1,0,1],[1,1,0]\}.\hfill \end{array}\end{array}$$

This observation motivates concatenated coding, where the message is first encoded by an outer encoder followed by an inner code that is the coset of a linear code. The transmitter then sends the entries at position $S_T$ of the inner codewords, which are vectors of dimension $|{\mathcal{S}}_T|$. We do not know if there are channels for which such codes are helpful.

4.4. Generalized Mutual Information

Consider the vector channel $p(\underline{y}|\underline{x})$ with input set $\mathcal{X}={\mathbb{C}}^M$ and output set $\mathcal{Y}={\mathbb{C}}^N$. The GMI for adaptive symbols is ${max}_{s\ge 0}{I}_s(A;\underline{Y})$ where

$$\begin{array}{c}\hfill I_s(A;\underline{Y})=\mathrm{E}\left[log\frac{q{(\underline{Y}|A)}^s}{q(\underline{Y})}\right]\end{array}$$

(102)

and the expectation is with respect to $p(a,\underline{y})$. Suppose the auxiliary model is $q(\underline{y}|a)$ and define

$$\begin{array}{c}\hfill q(\underline{y})={\int }_{\mathcal{A}}p(a)q{(\underline{y}|a)}^{s}da.\end{array}$$

(103)

The GMI again provides a lower bound on the mutual information since (cf. (43))

$$\begin{array}{c}\hfill I_s(A;\underline{Y})=I(A;\underline{Y})-D\left(p_{A,\underline{Y}}∥p_{\underline{Y}}{q}_{A|\underline{Y}}\right)\end{array}$$

(104)

where $q(a|\underline{y})=p(a)q{(\underline{y}|a)}^s/q(\underline{y})$ is a reverse channel density.

We next study reverse and forward models as in Section 1.3 and Section 1.4. Suppose the entries $\underline{X}(s_T)$ of A are jointly CSCG.

Reverse Model: We write $\underline{A}$ when we consider A to be a column vector that stacks the $\underline{X}(s_T)$. Consider the following reverse density motivated by (13):

$$\begin{array}{c}\hfill q(\underline{a}|\underline{y})=\frac{exp\left(-{(\underline{a}-\mathrm{E}\left[\underline{A}|\underline{Y}=\underline{y}\right])}^{†}{\mathbf{Q}}_{\underline{A}|\underline{Y}=\underline{y}}^{-1}(\underline{a}-\mathrm{E}\left[\underline{A}|\underline{Y}=\underline{y}\right])\right)}{{\pi }^{\nu M}det{\mathbf{Q}}_{\underline{A}|\underline{Y}=\underline{y}}}.\end{array}$$

(105)

A corresponding forward model is $q\left(\underline{y}|a\right)=q\left(a|\underline{y}\right)/p(a)$ and the GMI with $s=1$ becomes (cf. (35))

$$\begin{array}{c}\hfill I_1(A;\underline{Y})=\mathrm{E}\left[log\frac{det{\mathbf{Q}}_{\underline{A}}}{det{\mathbf{Q}}_{\underline{A}|\underline{Y}}}\right].\end{array}$$

(106)

To simplify, one may focus on adaptive symbols as in (89):

$$\begin{array}{c}\hfill \underline{X}={\mathbf{Q}}_{\underline{X}(S_T)}^{1/2}·\underline{U}\end{array}$$

(107)

where $\underline{U}\sim \mathcal{CN}(\underline{0},\mathbf{I})$ and the ${\mathbf{Q}}_{\underline{X}(s_T)}$ are covariance matrices. We thus have $I(A;\underline{Y})=I(\underline{U};\underline{Y})$ (cf. (96)) and using (105) but with $\underline{A}$ replaced with $\underline{U}$ we obtain

$$\begin{array}{c}\hfill I_1(A;\underline{Y})=\mathrm{E}\left[-logdet{\mathbf{Q}}_{\underline{U}|\underline{Y}}\right].\end{array}$$

(108)

Forward Model: Perhaps the simplest forward model is $q(\underline{y}|a)=p(\underline{y}|\underline{x}(s_T))$ for some fixed value $s_T\in {\mathcal{S}}_T$. One may interpret this model as having the receiver assume that $S_T=s_T$. A natural generalization of this idea is as follows: define the auxiliary vector

$$\begin{array}{c}\hfill \underline{\overline{X}}=\sum _{s_T}\mathbf{W}(s_T)\underline{X}(s_T)\end{array}$$

(109)

where the $\mathbf{W}(s_T)$ are $M\times M$ complex matrices, i.e., $\underline{\overline{X}}$ is a linear function of the entries of $A=[\underline{X}(s_T):s_T\in {\mathcal{S}}_T]$. For example, the matrices might be chosen based on $P_{S_T}(.)$. However, observe that $\underline{\overline{X}}$ is independent of $S_T$. Now define the auxiliary model

$$\begin{array}{c}\hfill q(\underline{y}|a)=q(\underline{y}|\underline{\overline{x}})\end{array}$$

where we abuse notation by using the same $q(.)$. The expression (103) becomes

$$\begin{array}{cc}\hfill q(\underline{y})& ={\int }_{\mathcal{A}}p(a)q{(\underline{y}|a)}^{s}da={\int }_{\mathbb{C}}p(\underline{\overline{x}})q{(\underline{y}|\underline{\overline{x}})}^{s}d\underline{\overline{x}}.\hfill \end{array}$$

(110)

Remark 27.

We often consider ${\mathcal{S}}_T$ to be a discrete set, but for CSCG channels we also consider ${\mathcal{S}}_T=\mathbb{C}$ so that the sum over ${\mathcal{S}}_T$ in (109) is replaced by an integral over $\mathbb{C}$.

We now specialize further by choosing the auxiliary channel ${\underline{Y}}_a=\mathbf{H}\underline{\overline{X}}+\underline{Z}$ where $\mathbf{H}$ is an $N\times M$ complex matrix, $\underline{Z}$ is an N-dimensional CSCG vector that is independent of $\underline{\overline{X}}$ and has invertible covariance matrix ${\mathbf{Q}}_{\underline{Z}}$, and $\mathbf{H}$ and ${\mathbf{Q}}_{\underline{Z}}$ are to be optimized. Further choose $A=[\underline{X}(s_T):s_T\in {\mathcal{S}}_T]$ whose entries are jointly CSCG with correlation matrices

$$\begin{array}{c}\hfill \mathbf{R}(s_{T1},s_{T2})=\mathrm{E}\left[\underline{X}(s_{T1})\underline{X}{(s_{T2})}^{†}\right].\end{array}$$

Since $\underline{\overline{X}}$ in (109) is independent of $S_T$, we have

$$\begin{array}{c}\hfill q(\underline{y}|a)=\frac{exp\left(-{\left(\underline{y}-\mathbf{H}\underline{\overline{x}}\right)}^{†}{\mathbf{Q}}_{\underline{Z}}^{-1}\left(\underline{y}-\mathbf{H}\underline{\overline{x}}\right)\right)}{{\pi }^{N}det{\mathbf{Q}}_{\underline{Z}}}.\end{array}$$

(111)

Moreover, $\underline{\overline{X}}$ is CSCG so (110) is

$$\begin{array}{c}\hfill q(\underline{y})=\frac{{\pi }^{N}det\left({\mathbf{Q}}_{\underline{Z}}/s\right)}{{\left({\pi }^{N}det{\mathbf{Q}}_{\underline{Z}}\right)}^s}·\frac{exp\left(-{\underline{y}}^{†}{\left({\mathbf{Q}}_{\underline{Z}}/s+\mathbf{H}{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}^{†}\right)}^{-1}\underline{y}\right)}{{\pi }^{N}det\left({\mathbf{Q}}_{\underline{Z}}/s+\mathbf{H}{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}^{†}\right)}\end{array}$$

where

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{\underline{\overline{X}}}& =\sum _{s_{T1},s_{T_2}}\mathbf{W}(s_{T1})\mathbf{R}(s_{T1},s_{T2})\mathbf{W}{(s_{T2})}^{†}.\hfill \end{array}$$

We have the following generalization of Proposition 1.

Lemma 2.

The maximum GMI (102) for the channel $p(\underline{y}|a)$, an adaptive vector $A=[\underline{X}(s_T):s_T\in {\mathcal{S}}_T]$ that has jointly CSCG entries, an $\underline{\overline{X}}$ as in (109) with ${\mathbf{Q}}_{\underline{\overline{X}}}\succ \mathbf{0}$, and the auxiliary model (111) with ${\mathbf{Q}}_{\underline{Z}}\succ \mathbf{0}$ is

$$\begin{array}{cc}\hfill & I_1(A;\underline{Y})=logdet\left(\mathbf{I}+{\mathbf{Q}}_{\underline{\tilde{Z}}}^{-1}\tilde{\mathbf{H}}{\mathbf{Q}}_{\underline{\overline{X}}}{\tilde{\mathbf{H}}}^{†}\right)\hfill \end{array}$$

(112)

where (cf. (31))

$$\begin{array}{cc}\hfill \tilde{\mathbf{H}}& =\mathrm{E}\left[\underline{Y}{\underline{\overline{X}}}^{†}\right]{\mathbf{Q}}_{\underline{\overline{X}}}^{-1}\hfill \end{array}$$

(113)

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{\underline{\tilde{Z}}}& ={\mathbf{Q}}_{\underline{Y}}-\tilde{\mathbf{H}}{\mathbf{Q}}_{\underline{\overline{X}}}{\tilde{\mathbf{H}}}^{†}.\hfill \end{array}$$

(114)

The expectation is with respect to the actual channel with joint distribution/density $p(a,\underline{y})$.

Proof. 

See Appendix D. □

Remark 28.

Since $\underline{\overline{X}}$ is a function of A, the rate (112) can alternatively be derived by using $I(A;\underline{Y})\ge I(\underline{\overline{X}};\underline{Y})$ and applying the bound (30) with $\underline{X}$ replaced with $\underline{\overline{X}}$.

Remark 29.

The estimate $\tilde{\mathbf{H}}$ is the MMSE estimate of $\mathbf{H}$:

$$\begin{array}{c}\hfill \tilde{\mathbf{H}}=arg\underset{\mathbf{H}}{min}\mathrm{E}\left[\parallel \underline{Y}-\mathbf{H}\underline{\overline{X}}{\parallel }^2\right]\end{array}$$

(115)

and ${\mathbf{Q}}_{\tilde{\underline{Z}}}$ is the resulting covariance matrix of the error. To see this, expand (cf. (54))

$$\begin{array}{cc}\hfill \mathrm{E}\left[\parallel \underline{Y}-\mathbf{H}\underline{\overline{X}}{\parallel }^2\right]& =\mathrm{E}\left[\parallel (\underline{Y}-\tilde{\mathbf{H}}\underline{\overline{X}})+(\tilde{\mathbf{H}}-\mathbf{H})\underline{\overline{X}}{\parallel }^2\right]\hfill \\ \hfill & =\mathrm{E}\left[\parallel \underline{Y}-\tilde{\mathbf{H}}\underline{\overline{X}}{\parallel }^2\right]+\mathrm{tr}\left((\tilde{\mathbf{H}}-\mathbf{H}){\mathbf{Q}}_{\underline{\overline{X}}}{(\tilde{\mathbf{H}}-\mathbf{H})}^{†}\right)\hfill \end{array}$$

(116)

where the final step follows by the definition of $\tilde{\mathbf{H}}$ in (113).

Remark 30.

Suppose that $\mathbf{H}$ is an estimate other than (115). Generalizing (55), if ${\mathbf{Q}}_{\underline{Y}}\succ {\mathbf{Q}}_{\underline{\overline{Z}}}$ we may choose

$$\begin{array}{c}\hfill {\mathbf{Q}}_{\underline{Z}}/s={\left(\mathbf{H}{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}^{†}\right)}^{1/2}{\left({\mathbf{Q}}_{\underline{Y}}-{\mathbf{Q}}_{\underline{\overline{Z}}}\right)}^{-1/2}{\mathbf{Q}}_{\underline{\overline{Z}}}{\left({\mathbf{Q}}_{\underline{Y}}-{\mathbf{Q}}_{\underline{\overline{Z}}}\right)}^{-1/2}{\left(\mathbf{H}{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}^{†}\right)}^{1/2}\end{array}$$

(117)

where

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{\underline{\overline{Z}}}& =\mathrm{E}\left[\left(\underline{Y}-\mathbf{H}\underline{\overline{X}}\right){\left(\underline{Y}-\mathbf{H}\underline{\overline{X}}\right)}^{†}\right].\hfill \end{array}$$

(118)

Appendix D shows that (102) then simplifies to (cf. (56))

$$\begin{array}{c}\hfill I_s(A;\underline{Y})=logdet\left({\mathbf{Q}}_{\underline{\overline{Z}}}^{-1}{\mathbf{Q}}_{\underline{Y}}\right).\end{array}$$

(119)

Remark 31.

The GMI (112) does not depend on the scaling of $\underline{\overline{X}}$ since this is absorbed in $\tilde{\mathbf{H}}$. For example, one can choose the weighting matrices in (109) so that $\mathrm{E}\left[\parallel \underline{\overline{X}}{\parallel }^2\right]=P$.

4.5. Optimal Codebooks for CSCG Forward Models

The following Lemma maximizes the GMI for scalar channels and A with CSCG entries without requiring A to have the form (89). Nevertheless, this form is optimal, and we refer to ([10], page 2013) and Section 6.4 for similar results. In the following, let $U(s_T)\sim \mathcal{CN}(0,1)$ for all $s_T$.

Lemma 3.

The maximum GMI (102) for the channel $p(y|a)$, an adaptive symbol A with jointly CSCG entries, the forward model (111), and with fixed $P(s_T)=\mathrm{E}\left[|X(s_T){|}^2\right]$ is

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{\tilde{P}}{\mathrm{E}\left[{\left|Y\right|}^2\right]-\tilde{P}}\right)\end{array}$$

(120)

where, writing $X(s_T)=\sqrt{P(s_T)}U(s_T)$ for all $s_T$, we have

$$\begin{array}{cc}\hfill \tilde{P}& =\mathrm{E}{\left[\left|\mathrm{E}\left[\left.YU{(S_T)}^{*}\right|S_T\right]\right|\right]}^2.\hfill \end{array}$$

(121)

This GMI is achieved by choosing fully-correlated symbols:

$$\begin{array}{c}\hfill X(s_T)=\sqrt{P(s_T)}{e}^{j\varphi (s_T)}U\end{array}$$

(122)

and $\overline{X}=cU$ for some non-zero constant c and a common $U\sim \mathcal{CN}(0,1)$, and where

$$\begin{array}{c}\hfill \varphi (s_T)=-arg\left(\mathrm{E}\left[\left.YU{(s_T)}^{*}\right|S_T=s_T\right]\right).\end{array}$$

(123)

Proof. 

See Appendix E. □

Remark 32.

The expression (121) is based on (A58) in Appendix E and can alternatively be written as $\tilde{P}=|\tilde{h}{|}^2\overline{P}$ where

$$\begin{array}{cc}\hfill \tilde{h}& =\mathrm{E}\left[Y{\overline{X}}^{*}\right]/\overline{P}.\hfill \end{array}$$

Remark 33.

The power levels $P(s_T)$ may be optimized, usually under a constraint such as $\mathrm{E}\left[P(S_T)\right]\le P$.

Remark 34.

By the Cauchy-Schwarz inequality, we have

$$\begin{array}{c}\hfill \mathrm{E}{\left[\left|\mathrm{E}\left[\left.YU{(S_T)}^{*}\right|S_T\right]\right|\right]}^2\le \mathrm{E}\left[{\left|Y\right|}^2\right].\end{array}$$

Furthermore, equality holds if and only if $\left|YU{(s_T)}^{*}\right|$ is a constant for each $s_T$, but this case is not interesting.

4.6. Forward Model GMI for MIMO Channels

The following lemma generalizes Lemma 3 to MIMO channels without claiming a closed-form expression for the optimal GMI. Let $\underline{U}(s_T)\sim \mathcal{CN}(\underline{0},\mathbf{I})$ for all $s_T$.

Lemma 4.

A GMI (102) for the channel $p(\underline{y}|a)$, an adaptive vector A with jointly CSCG entries, the auxiliary model (111), and with fixed ${\mathbf{Q}}_{\underline{X}(s_T)}$ is given by (112) that we write as

$$\begin{array}{c}\hfill I_1(A;\underline{Y})=log\left(\frac{det{\mathbf{Q}}_{\underline{Y}}}{det\left({\mathbf{Q}}_{\underline{Y}}-\tilde{\mathbf{D}}{\tilde{\mathbf{D}}}^{†}\right)}\right).\end{array}$$

(124)

where for $M\times M$ unitary ${\mathbf{V}}_R(s_T)$ we have

$$\begin{array}{c}\hfill \tilde{\mathbf{D}}=\mathrm{E}\left[{\mathbf{U}}_T(S_T)\Sigma (S_T){\mathbf{V}}_R{(S_T)}^{†}\right]\end{array}$$

(125)

and ${\mathbf{U}}_T(s_T)$ and $\Sigma (s_T)$ are $N\times N$ unitary and $N\times M$ rectangular diagonal matrices, respectively, of the SVD

$$\begin{array}{c}\hfill \mathrm{E}\left[\left.\underline{Y}{\underline{U}(s_T)}^{†}\right|S_T=s_T\right]={\mathbf{U}}_T(s_T)\Sigma (s_T){\mathbf{V}}_T{(s_T)}^{†}\end{array}$$

(126)

for all $s_T$, and the ${\mathbf{V}}_T(s_T)$ are $M\times M$ unitary matrices. The GMI (124) is achieved by choosing the symbols (cf. (122) and (A87) below):

$$\begin{array}{c}\hfill \underline{X}(s_T)={\mathbf{Q}}_{\underline{X}(s_T)}^{1/2}{\mathbf{V}}_T(s_T)\underline{U}\end{array}$$

(127)

and $\underline{\overline{X}}=\mathbf{C}\underline{U}$ for some invertible $M\times M$ matrix $\mathbf{C}$ and a common M-dimensional vector $\underline{U}\sim \mathcal{CN}(\underline{0},\mathbf{I})$. One may maximize (124) over the unitary ${\mathbf{V}}_R(s_T)$.

Proof. 

See Appendix G. □

Using Lemma 4, the theory for MISO channels with $N=1$ is similar to the scalar case of Lemma 3; see Remark 35 below. However, optimizing the GMI is more difficult for $N>1$ because one must optimize over the unitary matrices ${\mathbf{V}}_R(s_T)$ in (125); see Remark 36 below.

Remark 35.

Consider $N=1$ in which case one may set ${\mathbf{U}}_T(s_T)=1$ and (126) is a $1\times M$ vector where $\Sigma (s_T)$ has as the only non-zero singular value

$$\begin{array}{c}\hfill \sigma (s_T)=∥\mathrm{E}\left[\left.Y\underline{U}{(s_T)}^{†}\right|S_T=s_T\right]∥={\left(\sum _{m=1}^M{\left|\mathrm{E}\left[\left.Y{U}_m{(s_T)}^{*}\right|S_T=s_T\right]\right|}^2\right)}^{1/2}.\end{array}$$

(128)

The absolute value of the scalar (125) is maximized by choosing ${\mathbf{V}}_R(s_T)=\mathbf{I}$ for all $s_T$ to obtain (cf. (121))

$$\begin{array}{c}\hfill \tilde{\mathbf{D}}{\tilde{\mathbf{D}}}^{†}=\mathrm{E}{\left[\sigma (S_T)\right]}^2.\end{array}$$

(129)

Remark 36.

Consider $M=1$ in which case one may set ${\mathbf{V}}_T(s_T)=1$ and (126) is a $N\times 1$ vector where $\Sigma (s_T)$ has as the only non-zero singular value

$$\begin{array}{c}\hfill \sigma (s_T)=∥\mathrm{E}\left[\left.\underline{Y}U{(s_T)}^{†}\right|S_T=s_T\right]∥={\left(\sum _{n=1}^N{\left|\mathrm{E}\left[\left.Y_{n}U{(s_T)}^{*}\right|S_T=s_T\right]\right|}^2\right)}^{1/2}.\end{array}$$

(130)

We should now find the $V_R(s_T)=e^{j{\varphi }_R(s_T)}$ that minimize the determinant in the denominator of (124) where (see (125))

$$\begin{array}{c}\hfill \tilde{\mathbf{D}}=\mathrm{E}\left[{\underline{u}}_T(S_T)\sigma (S_T)e^{-j{\varphi }_R(S_T)}\right]\end{array}$$

(131)

and where each ${\underline{u}}_T(s_T)$ is one of the columns of the $N\times N$ unitary matrix ${\mathbf{U}}_T(s_T)$.

Remark 37.

Consider $M=N$ and the product channel

$$\begin{array}{c}\hfill p(\underline{y}|a)=\prod _{m=1}^{M}p\left(y_m|[x_m(s_T):s_T\in {\mathcal{S}}_T]\right)\end{array}$$

(132)

where $x_m(s_T)$ is the m’th entry of $\underline{x}(s_T)$. We choose ${\mathbf{Q}}_{\underline{X}(s_T)}$ as diagonal with diagonal entries $\sqrt{P_m(s_T)}$, $m=1,\dots ,M$. Also choosing ${\mathbf{V}}_R(s_T)=\mathbf{I}$ makes the matrix $\tilde{\mathbf{D}}{\tilde{\mathbf{D}}}^{†}$ diagonal with the diagonal entries (cf. (121) where $M=N=1$)

$$\begin{array}{c}\hfill {\left(\sum _{s_T}{P}_{S_T}(s_T)\left|\mathrm{E}\left[\left.Y_m{U}_m{(s_T)}^{*}\right|S_T=s_T\right]\right|\right)}^2\end{array}$$

(133)

for $m=1,\dots ,M$. The GMI (124) is thus (cf. (120))

$$\begin{array}{c}\hfill I_1(A;\underline{Y})=\sum _{m=1}^{M}log\left(\frac{\mathrm{E}\left[|Y_m{|}^2\right]}{\mathrm{E}\left[|Y_m{|}^2\right]-\mathrm{E}{\left[\left|\mathrm{E}\left[\left.Y_m{U}_m{(S_T)}^{*}\right|S_T\right]\right|\right]}^2}\right).\end{array}$$

(134)

Remark 38.

For general $p(\underline{y}|a)$, one might wish to choose diagonal ${\mathbf{Q}}_{\underline{X}(s_T)}$ and a product model

$$\begin{array}{c}\hfill q(\underline{y}|a)=\prod _{m=1}^M{q}_m(y_m|{\overline{x}}_m)\end{array}$$

where the $q_m(.)$ are scalar AWGN channels

$$\begin{array}{c}\hfill q_m(y|x)=\frac{1}{\pi {\sigma }_m^2}exp\left(-|y-h_m{x|}^2/{\sigma }_m^2\right)\end{array}$$

with possibly different $h_m$ and ${\sigma }_m^2$ for each m. Consider also

$$\begin{array}{c}\hfill {\overline{X}}_m=\sum _{s_T}{w}_m(s_T)X_m(s_T)\end{array}$$

for some complex weights $w_m(s_T)$, i.e., ${\overline{X}}_m$ is a weighted sum of entries from the list $[X_m(s_T):s_T\in {\mathcal{S}}_T]$. The maximum GMI is now the same as (134) but without requiring the actual channel to have the form (132).

Remark 39.

If the actual channel is $\underline{Y}=\mathbf{H}\underline{X}+\underline{Z}$ then

$$\begin{array}{c}\hfill \mathrm{E}\left[\underline{Y}\underline{U}{(s_T)}^{†}|S_T=s_T\right]=\mathrm{E}\left[\mathbf{H}\underline{X}(s_T)\underline{U}{(s_T)}^{†}|S_T=s_T\right]=\mathrm{E}\left[\mathbf{H}|S_T=s_T\right]{\mathbf{Q}}_{\underline{X}(s_T)}^{1/2}\end{array}$$

(135)

where the final step follows because $\underline{U}(S_T)-S_T-\mathbf{H}$ forms a Markov chain. The expression (135) is useful because it separates the effects of the channel and the transmitter.

Remark 40.

Combining Remarks 37 and 39, suppose the actual channel is $\underline{Y}=\mathbf{H}\underline{X}+\underline{Z}$ with $M=N$ and where $\mathbf{H}$ is diagonal with diagonal entries $H_m$, $m=1,\dots ,M$. The GMI (124) is then (cf. (134))

$$\begin{array}{c}\hfill I_1(A;\underline{Y})=\sum _{m=1}^{M}log\left(\frac{\mathrm{E}\left[|Y_m{|}^2\right]}{\mathrm{E}\left[|Y_m{|}^2\right]-\mathrm{E}{\left[\left|\mathrm{E}\left[\left.H_m\sqrt{P_m(S_T)}\right|S_T\right]\right|\right]}^2}\right)\end{array}$$

(136)

where $\mathrm{E}\left[|Y_m{|}^2\right]=1+\mathrm{E}\left[|H_m{|}^2{P}_m(S_T)\right]$.

5. Channels with CSIR and CSIT

Shannon’s model includes CSIR [11]. The FDG is shown in Figure 3 where there is a hidden state $S_H$, the CSIR $S_R$ and CSIT $S_T$ are functions of $S_H$, and the receiver sees the channel outputs

$$\begin{array}{c}\hfill [Y_i,S_{Ri}]=[f(X_i,S_{Hi},Z_i),S_{Ri}]\end{array}$$

(137)

for some function $f(.)$ and $i=1,2,\dots ,n$. (By defining $S_H=[S_{H1},Z_H]$ and calling $S_{H1}$ the hidden channel state we can include the case where $S_R$ and $S_T$ are noisy functions of $S_{H1}$.) As before, M, $S_H^n$, $Z^n$ are mutually statistically independent, and $S_H^n$ and $Z^n$ are i.i.d. strings of random variables with the same distributions as $S_T$ and Z, respectively. Observe that we have changed the notation by writing Y for only part of the channel output. The new Y (without the $S_R$) is usually called the “channel output”.

5.1. Capacity and GMI

We begin with scalar channels for which (90) is

$$\begin{array}{c}\hfill C=\underset{A}{max}I(A;Y,S_R)=\underset{A}{max}I(A;Y|S_R)\end{array}$$

(138)

where A and $S_R$ are independent.

Reverse Model: The expression (108) with the adaptive symbol (88) is

$$\begin{array}{c}\hfill I_1(A;Y,S_R)=\mathrm{E}\left[-log\mathrm{Var}\left[U|Y,S_R\right]\right].\end{array}$$

(139)

Forward Model: Consider the expansion

$$\begin{array}{c}\hfill I_1(A;Y|S_R)={\int }_{{\mathcal{S}}_R}p(s_R)I_1(A;Y|S_R=s_R)d{s}_R\end{array}$$

(140)

where $I_1(A;Y|S_R=s_R)$ is the GMI (102) with all densities conditioned on $S_R=s_R$. We choose the forward model

$$\begin{array}{c}\hfill q(y|a,s_R)=\frac{1}{\pi \sigma {(s_R)}^2}exp\left(-\frac{|y-h(s_R)\overline{x}(s_R){|}^2}{\sigma {(s_R)}^2}\right).\end{array}$$

(141)

where similar to (109) we define

$$\begin{array}{c}\hfill \overline{X}(s_R)=\sum _{s_T}w(s_T,s_R)X(s_T)\end{array}$$

(142)

for complex weights $w(s_T,s_R)$, i.e., $\overline{X}(s_R)$ is a weighted sum of entries from the list $A=[X(s_T):s_T\in {\mathcal{S}}_T]$. We have the following straightforward generalization of Lemma 3.

Theorem 1.

The maximum GMI (140) for the channel $p(y|a,s_R)$, an adaptive symbol A with jointly CSCG entries, the model (141), and with fixed $P(s_T)=\mathrm{E}\left[|X(s_T){|}^2\right]$ is

$$\begin{array}{c}\hfill I_1(A;Y|S_R)=\mathrm{E}\left[log\left(1+\frac{\tilde{P}(S_R)}{\mathrm{E}\left[{\left|Y\right|}^2|S_R\right]-\tilde{P}(S_R)}\right)\right]\end{array}$$

(143)

where for all $s_R\in {\mathcal{S}}_R$ we have

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& =\mathrm{E}{\left[\left|\mathrm{E}\left[\left.YU{(S_T)}^{*}\right|S_T,S_R=s_R\right]\right|\right]}^2.\hfill \end{array}$$

(144)

Remark 41.

To establish Theorem 1, the receiver may choose $\overline{X}=\sqrt{P}U$ to be independent of $s_R$. Alternatively, the receiver may choose $\overline{X}(s_R)=\sqrt{\mathrm{E}\left[{\left|X\right|}^2|S_R=s_R\right]}U$. Both choices give the same GMI since the expectation in (144) does not depend on the scaling of $\overline{X}$; see Remark 31.

Remark 42.

The partition idea of Lemmas 1 and 5 carries over to Theorem 1. We may generalize (143) as

$$\begin{array}{cc}\hfill I_1(A;Y|S_R)={\int }_{{\mathcal{S}}_R}p(s_R)& \sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k|S_R=s_R\right]\hfill \\ \hfill & \left[log\left(1+\frac{|h_k(s_R){|}^{2}P}{{\sigma }_k^2(s_R)}\right)+\frac{\mathrm{E}\left[\left.{\left|Y\right|}^2\right|{\mathcal{E}}_k,S_R=s_R\right]}{{\sigma }_k^2(s_R)+{\left|h_k(s_R)\right|}^{2}P}\right.\hfill \\ \hfill & \left.-\frac{\mathrm{E}\left[\left.|Y-h_k(s_R)\sqrt{P}{U|}^2\right|{\mathcal{E}}_k,S_R=s_R\right]}{{\sigma }_k^2(s_R)}\right]d{s}_R\hfill \end{array}$$

(145)

where the $X(s_T)$, $s_T\in {\mathcal{S}}_T$, are given by (122) and the $h_k(s_R)$ and ${\sigma }_k^2(s_R)$, $k=1,\dots ,K$, $s_R\in {\mathcal{S}}_R$, can be optimized.

Remark 43.

One is usually interested in the optimal power control policy $P(s_T)$ under the constraint $\mathrm{E}\left[P(S_T)\right]\le P$. Taking the derivative of (143) with respect to $\sqrt{P(s_T)}$ and setting to zero we obtain

$$\begin{array}{c}\hfill \mathrm{E}\left[\frac{\mathrm{E}\left[{\left|Y\right|}^2|S_R\right]\tilde{P}{(S_R)}^{\prime }-\tilde{P}(S_R)\mathrm{E}{\left[{\left|Y\right|}^2|S_R\right]}^{\prime }}{\mathrm{E}\left[{\left|Y\right|}^2|S_R\right]\left[\mathrm{E}\left[{\left|Y\right|}^2|S_R\right]-\tilde{P}(S_R)\right]}\right]=2\lambda \sqrt{P(s_T)}{P}_{S_T}(s_T)\end{array}$$

(146)

where $\tilde{P}{(S_R)}^{\prime }$ and $\mathrm{E}{\left[{\left|Y\right|}^2|S_R\right]}^{\prime }$ are derivatives with respect to $\sqrt{P(s_T)}$. We use (146) below to derive power control policies.

Remark 44.

A related model is a compound channel where $p(y|a,s_R)$ is indexed by the parameter $s_R$ ([91], Chapter 4). The problem is to find the maximum worst-case reliable rate if the transmitter does not know $s_R$. Alternatively, the transmitter must send its message to all $|{\mathcal{S}}_R|$ receivers indexed by $s_R\in {\mathcal{S}}_R$. A compound channel may thus be interpreted as a broadcast channel with a common message.

5.2. CSIT@ R

An interesting specialization of Shannon’s model is when the receiver knows $S_T$ and can determine $X(S_T)$. We refer to this scenario as CSIT@R. The model was considered in ([10], Section II) when $S_T$ is a function of $S_R$. More generally, suppose $S_T$ is a function of $[Y,S_R]$. The capacity (138) then simplifies to (see ([10], Proposition 1))

$$\begin{array}{cc}\hfill C& \stackrel{(a)}{=}\underset{A}{max}I(A;Y,S_T|S_R)\hfill \\ \hfill & \stackrel{(b)}{=}\underset{A}{max}I(X;Y|S_R,S_T)\hfill \\ \hfill & \stackrel{(c)}{=}\sum _{s_T}{P}_{S_T}(s_T)\left[\underset{X(s_T)}{max}I(X(s_T);Y|S_R,S_T=s_T)\right]\hfill \end{array}$$

(147)

where step $(a)$ follows because $S_T$ is a function of $[Y,S_R]$; step $(b)$ follows because A and $(S_R,S_T)$ are independent, X is a function of $[A,S_T]$, and $A-[S_T,X]-Y$ forms a Markov chain; and step $(c)$ follows because one may optimize $X(s_T)$ separately for each $s_T\in {\mathcal{S}}_T$.

As discussed in [10], a practical motivation for this model is when the CSIT is based on error-free feedback from the receiver to the transmitter. In this case, where $S_T$ is a function of $S_R$, the expression (144) becomes

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& ={\left|\mathrm{E}\left[\left.YU{(s_T)}^{*}\right|S_R=s_R\right]\right|}^2.\hfill \end{array}$$

(148)

Remark 45.

The insight that one can replace adaptive symbols A with channel inputs X when X is a function of A and past Y appeared for two-way channels in ([9], Section 4.2.3) and networks in ([22], Section V.A), ([72], Section IV.F).

5.3. MIMO Channels and K-Partitions

We consider generalizations to MIMO channels and K-partitions as in Section 3.2.

MIMO Channels: Consider the average GMI

$$\begin{array}{c}\hfill I_1(A;\underline{Y}|S_R)={\int }_{{\mathcal{S}}_R}p(s_R)I_1(A;\underline{Y}|S_R=s_R)d{s}_R\end{array}$$

(149)

and choose the parameters (113) and (114) for the event $S_R=s_R$. We have

$$\begin{array}{cc}\hfill \tilde{\mathbf{H}}(s_R)& =\mathrm{E}\left[\left.\underline{Y}{\underline{\overline{X}}}^{†}\right|S_R=s_R\right]\mathrm{E}{\left[\left.\underline{\overline{X}}{\underline{\overline{X}}}^{†}\right|S_R=s_R\right]}^{-1}\hfill \end{array}$$

(150)

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{\underline{\tilde{Z}}}(s_R)& =\mathrm{E}\left[\left.\underline{Y}{\underline{Y}}^{†}\right|S_R=s_R\right]-\tilde{\mathbf{H}}(s_R)\mathrm{E}\left[\left.\underline{\overline{X}}{\underline{\overline{X}}}^{†}\right|S_R=s_R\right]\tilde{\mathbf{H}}{(s_R)}^{†}\hfill \end{array}$$

(151)

and the GMI (149) is (cf. (60) and (112))

$$\begin{array}{c}\hfill I_1(A;\underline{Y}|S_R)=\mathrm{E}\left[logdet\left(\mathbf{I}+{\mathbf{Q}}_{\underline{\tilde{Z}}}{(S_R)}^{-1}\tilde{\mathbf{H}}(S_R){\mathbf{Q}}_{\underline{\overline{X}}}\tilde{\mathbf{H}}{(S_R)}^{†}\right)\right].\end{array}$$

(152)

K-Partitions: Let $\{{\underline{\mathcal{Y}}}_k:k=1,\dots ,K\}$ be a K-partition of $\underline{\mathcal{Y}}$ and define the events ${\mathcal{E}}_k=\{\underline{Y}\in {\underline{\mathcal{Y}}}_k\}$ for $k=1,\dots ,K$. As in Remark 13, K-partitioning formally includes (149) as a special case by including $S_R$ as part of the receiver’s “overall” channel output $\underline{\tilde{Y}}=[\underline{Y},S_R]$. The following lemma generalizes Lemma 1.

Lemma 5.

A GMI with $s=1$ for the channel $p(\underline{y}|a)$ is

$$\begin{array}{cc}\hfill I_1(A;\underline{Y})=\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]& \left\{logdet\left(\mathbf{I}+{\mathbf{Q}}_{{\underline{Z}}_k}^{-1}{\mathbf{H}}_k{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}_k^{†}\right)\right.\hfill \\ \hfill & +\mathrm{E}\left[\left.{\underline{Y}}^{†}{\left({\mathbf{Q}}_{{\underline{Z}}_k}+{\mathbf{H}}_k{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}_k^{†}\right)}^{-1}\underline{Y}\right|{\mathcal{E}}_k\right]\hfill \\ \hfill & \left.-\mathrm{E}\left[\left.{\left(\underline{Y}-{\mathbf{H}}_k\underline{\overline{X}}\right)}^{†}{\mathbf{Q}}_{{\underline{Z}}_k}^{-1}\left(\underline{Y}-{\mathbf{H}}_k\underline{\overline{X}}\right)\right|{\mathcal{E}}_k\right]\right\}\hfill \end{array}$$

(153)

where the ${\mathbf{H}}_k$ and ${\mathbf{Q}}_{{\underline{Z}}_k}$, $k=1,\dots ,K$, can be optimized.

Remark 46.

For scalars the GMI (153) is

$$\begin{array}{cc}\hfill I_1(A;Y)=\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]& \left[log\left(1+\frac{|h_k{|}^2\overline{P}}{{\sigma }_k^2}\right)+\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_k\right]}{{\sigma }_k^2+{\left|h_k\right|}^2\overline{P}}-\frac{\mathrm{E}\left[|Y-h_k\overline{X}{|}^2|{\mathcal{E}}_k\right]}{{\sigma }_k^2}\right]\hfill \end{array}$$

(154)

which is the same as (64) except that $\overline{X}$, $\overline{P}$ replace $X,P$. If we follow (66) and (67) then (154) becomes (68) but with

$$\begin{array}{c}\hfill h_k=\mathrm{E}\left[\left.Y{\overline{X}}^{*}\right|{\mathcal{E}}_k\right]/P_k,P_k=\mathrm{E}\left[\left.{\left|\overline{X}\right|}^2\right|{\mathcal{E}}_k\right].\end{array}$$

Remark 47.

Consider Remark 14 and choose $K=2$, $h_1=0$, ${\sigma }_1^2=1$. The GMI (154) then has only the $k=2$ term, and it again remains to select $h_2$, ${\sigma }_2^2$, and $t_R$.

Remark 48.

If we define

$$\begin{array}{c}\hfill {\mathbf{Q}}_{\underline{\overline{X}}}^{(k)}=\mathrm{E}\left[\left.\underline{\overline{X}}{\underline{\overline{X}}}^{†}\right|{\mathcal{E}}_k\right],{\mathbf{Q}}_{\underline{Y}}^{(k)}=\mathrm{E}\left[\left.\underline{Y}{\underline{Y}}^{†}\right|{\mathcal{E}}_k\right]\end{array}$$

(155)

and choose the LMMSE auxiliary models with

$$\begin{array}{cc}\hfill {\mathbf{H}}_k& =\mathrm{E}\left[\left.\underline{Y}{\underline{\overline{X}}}^{†}\right|{\mathcal{E}}_k\right]{\left({\mathbf{Q}}_{\underline{\overline{X}}}^{(k)}\right)}^{-1}\hfill \end{array}$$

(156)

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{{\underline{Z}}_k}& ={\mathbf{Q}}_{\underline{Y}}^{(k)}-{\mathbf{H}}_k{\mathbf{Q}}_{\underline{\overline{X}}}^{(k)}{\mathbf{H}}_k^{†}\hfill \end{array}$$

(157)

for $k=1,\dots ,K$ then the expression (153) is (cf. (68))

$$\begin{array}{cc}\hfill I_1(A;\underline{Y})=\sum _{k=1}^K\mathrm{Pr}\left[{\mathcal{E}}_k\right]& \left[logdet\left(\mathbf{I}+{\mathbf{Q}}_{{\underline{Z}}_k}^{-1}{\mathbf{H}}_k{\mathbf{Q}}_{\underline{\overline{X}}}{\mathbf{H}}_k^{†}\right)\right.\hfill \end{array}$$

$$\begin{array}{c}\hfill \left.-\mathrm{tr}\left({\left({\mathbf{Q}}_{\underline{Y}}^{(k)}+{\mathbf{H}}_k{\mathbf{D}}_{\underline{\overline{X}}}^{(k)}{\mathbf{H}}_k^{†}\right)}^{-1}{\mathbf{H}}_k{\mathbf{D}}_{\underline{\overline{X}}}^{(k)}{\mathbf{H}}_k^{†}\right)\right]\end{array}$$

(158)

where ${\mathbf{D}}_{\underline{\overline{X}}}^{(k)}={\mathbf{Q}}_{\underline{\overline{X}}}-{\mathbf{Q}}_{\underline{\overline{X}}}^{(k)}$.

Remark 49.

We may proceed as in Remark 18 and consider large K. These steps are given in Appendix F.

6. Fading Channels with AWGN

This section treats scalar, complex-alphabet, AWGN channels with CSIR for which the channel output is

$$\begin{array}{c}\hfill [Y,S_R]=[HX+Z,S_R]\end{array}$$

(159)

where $H,A,Z$ are mutually independent, $\mathrm{E}\left[{\left|H\right|}^2\right]=1$, and $Z\sim \mathcal{CN}(0,1)$. The capacity under the power constraint $\mathrm{E}\left[{\left|X\right|}^2\right]\le P$ is (cf. (138))

$$\begin{array}{c}\hfill C(P)=\underset{A:\mathrm{E}\left[{\left|X\right|}^2\right]\le P}{max}I(A;Y|S_R).\end{array}$$

(160)

However, the optimization in (160) is often intractable, and we desire expressions with $log(1+\mathrm{SNR})$ terms to gain insight. We develop three such expressions: an upper bound and two lower bounds. It will be convenient to write $G={\left|H\right|}^2$.

Capacity Upper Bound: We state this bound as a lemma since we use it to prove Proposition 2 below.

Lemma 6.

The capacity (160) is upper bounded as

$$\begin{array}{c}\hfill C(P)\le max\mathrm{E}\left[log\left(1+GP(S_T)\right)\right]\end{array}$$

(161)

where the maximization is over $P(S_T)$ with $\mathrm{E}\left[P(S_T)\right]=P$.

Proof. 

Consider the steps

$$\begin{array}{cc}\hfill I(A;Y|S_R)& \le I(A;Y,S_T,H|S_R)\hfill \\ \hfill & \stackrel{(a)}{=}I(A;Y|S_R,S_T,H)\hfill \\ \hfill & =h(Y|S_R,S_T,H)-h(Z)\hfill \\ \hfill & \stackrel{(b)}{\le }\mathrm{E}\left[log\mathrm{Var}\left[Y|S_R,S_T,H\right]\right]\hfill \end{array}$$

(162)

where step $(a)$ is because A and $[S_R,S_T,H]$ are independent, and step $(b)$ follows by the entropy bound

$$\begin{array}{c}\hfill h(Y|B=b)\le log\left(\pi e\mathrm{Var}\left[Y|B=b\right]\right)\end{array}$$

(163)

which we applied with $B=[S_R,S_T,H]$. Finally, we compute $\mathrm{Var}\left[Y|S_R,S_T,H\right]=1+GP(S_T)$. □

Reverse Model GMI: Consider the adaptive symbol (88) and the GMI (139). We expand the variances in (139) as

$$\begin{array}{c}\hfill \mathrm{Var}\left[U|Y=y,S_R=s_R\right]=\mathrm{E}\left[{\left|U\right|}^2|Y=y,S_R=s_R\right]-|\mathrm{E}\left[U|Y=y,S_R=s_R\right]{|}^2.\end{array}$$

Appendix C shows that one may write

$$\begin{array}{c}\hfill \mathrm{E}\left[U|Y=y,S_R=s_R\right]={\int }_{\mathbb{C}\times {\mathcal{S}}_T}p(h,s_T|y,s_R)\frac{h\sqrt{P(s_T)}{e}^{j\varphi (s_T)}y}{1+{\left|h\right|}^{2}P(s_T)}d{s}_{T}dh\end{array}$$

(164)

and

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left|U\right|}^2|Y=y,S_R=s_R\right]& ={\int }_{\mathbb{C}\times {\mathcal{S}}_T}p(h,s_T|y,s_R)\hfill \\ \hfill & \left(\frac{1}{1+{\left|h\right|}^{2}P(s_T)}+\frac{{\left|h\right|}^{2}P(s_T){\left|y\right|}^2}{{\left(1+{\left|h\right|}^{2}P(s_T)\right)}^2}\right)d{s}_{T}dh.\hfill \end{array}$$

(165)

We use the expressions (164) and (165) to compute achievable rates by numerical integration. For example, suppose that $S_T=0$ and $S_R=H$, i.e., we have full CSIR and no CSIT. The averaging density is then

$$\begin{array}{c}\hfill p(h,s_T|y,s_R)=\delta (h-s_R)\delta (s_T)\end{array}$$

and the variance simplifies to the capacity-achieving form

$$\begin{array}{c}\hfill \mathrm{Var}\left[U|Y=y,S_R=h\right]=\frac{1}{1+{\left|h\right|}^{2}P}.\end{array}$$

Forward Model GMI: A forward model GMI is given by Theorem 1 where

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& =\mathrm{E}{\left[\left|\mathrm{E}\left[\left.H\sqrt{P(S_T)}\right|S_T,S_R=s_R\right]\right|\right]}^2\hfill \end{array}$$

(166)

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]& =1+\mathrm{E}\left[GP(S_T)|S_R=s_R\right]\hfill \end{array}$$

(167)

so that (143) becomes

$$\begin{array}{c}\hfill I_1(A;Y|S_R)=\mathrm{E}\left[log\left(1+\frac{\tilde{P}(S_R)}{1+\mathrm{E}\left[GP(S_T)|S_R\right]-\tilde{P}(S_R)}\right)\right].\end{array}$$

(168)

Remark 50.

Jensen’s inequality implies that the denominator in (168) is greater than or equal to

$$\begin{array}{c}\hfill 1+\mathrm{Var}\left[\left.\sqrt{GP(S_T)}\right|S_R\right].\end{array}$$

(169)

Equality requires that for all $S_R=s_R$ we have

$$\begin{array}{c}\hfill \tilde{P}(s_R)=\mathrm{E}{\left[\left.\sqrt{GP(S_T)}\right|S_R=s_R\right]}^2\end{array}$$

(170)

which is valid if H is a function of $[S_R,S_T]$, for example. However, if there is channel uncertainty after conditioning on $[S_R,S_T]$ then $\tilde{P}(s_R)$ is usually smaller than the RHS of (170).

Remark 51.

Consider $S_R=H$ or $S_R=H\sqrt{P(S_T)}$. For both cases, H is a function of $[S_R,S_T]$ and the denominator in (168) is the variance (169). In fact, for $S_R=H\sqrt{P(S_T)}$, the expression (169) takes on the minimal value 1. This CSIR is thus the best possible; see Proposition 2.

Remark 52.

For MIMO channels we replace (159) with

$$\begin{array}{c}\hfill [\underline{Y},S_R]=[\mathbf{H}\underline{X}+\underline{Z},S_R]\end{array}$$

(171)

where $\mathbf{H},A,\underline{Z}$ are mutually independent and $\underline{Z}\sim \mathcal{CN}(\underline{0},\mathbf{I})$. One usually considers the constraint $\mathrm{E}\left[\parallel \underline{X}{\parallel }^2\right]\le P$.

Remark 53.

The model (171) includes block fading. For example, choosing $M=N$ and $\mathbf{H}=H\mathbf{I}$ gives scalar block fading. Moreover, the capacity per symbol without in-block feedback is the same as for the $M=N=1$ case except that P is replaced with $P/M$; see [11] and Section 9.

6.1. CSIR and CSIT Models

We study two classes of CSIR, as shown in

Table 1. Models Studied in Section 6 (General Fading), Section 7 (On–Off Fading) and Section 8 (Rayleigh Fading).

		CSIR
		Full	Partial/No
CSIT	Full	Section 6.3	Section 6.5
	@R	Section 6.3	Section 6.6
	Partial/No	Section 6.4	Section 6.2

. The first class has full (or “perfect”) CSIR, by which we mean either $S_R=H$ or $S_R=H\sqrt{P(S_T)}$. The motivation for studying the latter case is that it models block fading channels with long blocks where the receiver estimates $H\sqrt{P(S_T)}$ using pilot symbols, and the number of pilot symbols is much smaller than the block length [10]. Moreover, one achieves the upper bound (161), see Proposition 2 below.

We coarsely categorize the CSIT as follows:

• Full CSIT: $S_T=H$;
• CSIT@R: $S_T=q_u(G)$ where $q_u(.)$ is the quantizer of Section 2.9 with $B=0,1,\infty$;
• Partial CSIT: $S_T$ is not known exactly at the receiver.

The capacity of the CSIT@R models is given by $log(1+\mathrm{SNR})$ expressions [10,92]; see also [93]. The partial CSIT model is interesting because achieving capacity generally requires adaptive codewords and closed-form capacity expressions are unavailable. The GMI lower bound of Theorem 1 and Remark 42 and the capacity upper bound of Lemma 6 serve as benchmarks.

The partial CSIR models have $S_R$ being a lossy function of H. For example, a common model is based on LMMSE channel estimation with

$$\begin{array}{c}\hfill H=\sqrt{\overline{ϵ}}{S}_R+\sqrt{ϵ}{Z}_R\end{array}$$

(172)

where $0\le ϵ\le 1$ and $S_R,Z_R$ are uncorrelated. The CSIT is categorized as above, except that we consider $S_T=f_T(S_R)$ for some function $f_T(.)$ rather than $S_T=q_u(G)$.

To illustrate the theory, we study two types of fading: one with discrete H and one with continuous H, namely

• Section 7: on-off fading with $P_H(0)=P_H(\sqrt{2})=1/2$;
• Section 8: Rayleigh fading with $H\sim \mathcal{CN}(0,1)$.

For on-off fading we have $p(g)=\frac{1}{2}\delta (g)+\frac{1}{2}\delta (g-2)$ and for Rayleigh fading we have $p(g)=e^{-g}·1(g\ge 0)$.

Remark 54.

For channels with partial CSIR, we will study the GMI for partitions with $K=1$ and $K=2$. The full CSIT model has received relatively little attention in the literature, perhaps because CSIR is usually more accurate than CSIT ([5], Section 4.2.3).

6.2. No CSIR, No CSIT

Without CSIR or CSIT, the channel is a classic memoryless channel [94] for which the capacity (160) becomes the usual expression with $S_R=0$ and $A=X$. For CSCG X and $U=X/\mathrm{E}\left[{\left|X\right|}^2\right]$, the reverse and forward model GMIs (139) and (168) are the respective

$$\begin{array}{cc}\hfill I_1(X;Y)& =\mathrm{E}\left[-log\mathrm{Var}\left[U|Y\right]\right]\hfill \end{array}$$

(173)

$$\begin{array}{cc}\hfill I_1(X;Y)& =log\left(1+\frac{P|\mathrm{E}\left[H\right]{|}^2}{1+P\mathrm{Var}\left[H\right]}\right).\hfill \end{array}$$

(174)

For example, the forward model GMI is zero if $\mathrm{E}\left[H\right]=0$.

6.3. Full CSIR, CSIT@ R

Consider the full CSIR models with $S_R=H$ and CSIT@R. The capacity is given by $log(1+\mathrm{SNR})$ expressions that we review.

First, the capacity with $B=0$ (no CSIT) is

$$\begin{array}{c}\hfill C(P)=\mathrm{E}\left[log\left(1+GP\right)\right]={\int }_0^{\infty }p(g)log\left(1+gP\right)dg.\end{array}$$

(175)

The wideband derivatives are (see (37))

$$\begin{array}{c}\hfill C^{\prime }(0)=\mathrm{E}\left[G\right]=1,C^{″}(0)=-\mathrm{E}\left[G^2\right]\end{array}$$

(176)

so that the wideband values (37) are (see ([73], Theorem 13))

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=log2,S=\frac{2}{\mathrm{E}\left[G^2\right]}.\end{array}$$

(177)

The minimal $E_b/N_0$ is the same as without fading, namely $-1.59$ dB. However, Jensen’s inequality gives $\mathrm{E}\left[G^2\right]\ge \mathrm{E}{\left[G\right]}^2=1$ with equality if and only if $G=1$. Thus, fading reduces the capacity slope S.

More generally, the capacity with full CSIR and $S_T=q_u(G)$ is (see [10])

$$\begin{array}{cc}\hfill C(P)& =\underset{P(S_T):\mathrm{E}\left[P(S_T)\right]\le P}{max}\mathrm{E}\left[log\left(1+GP(S_T)\right)\right]\hfill \\ \hfill & =\underset{P(S_T):\mathrm{E}\left[P(S_T)\right]\le P}{max}{\int }_0^{\infty }p(g,s_T)log\left(1+gP(s_T)\right)dgd{s}_T.\hfill \end{array}$$

(178)

To optimize the power levels $P(s_T)$, consider the Lagrangian

$$\begin{array}{c}\hfill \mathrm{E}\left[log\left(1+GP(S_T)\right)\right]+\lambda \left(P-\mathrm{E}\left[P(S_T)\right]\right)\end{array}$$

(179)

where $\lambda \ge 0$ is a Lagrange multiplier. Taking the derivative with respect to $P(s_T)$, we have

$$\begin{array}{c}\hfill \lambda =\mathrm{E}\left[\left.\frac{G}{1+GP(s_T)}\right|S_T=s_T\right]={\int }_0^{\infty }p(g|s_T)\frac{g}{1+gP(s_T)}dg\end{array}$$

(180)

as long as $P(s_T)\ge 0$. If this equation cannot be satisfied, choose $P(s_T)=0$. Finally, set $\lambda$ so that $\mathrm{E}\left[P(S_T)\right]=P$.

For example, consider $B=\infty$ and $S_T=G$. We then have $p(g|s_T)=\delta (g-s_T)$ and therefore

$$\begin{array}{c}\hfill P(g)={\left(\frac{1}{\lambda }-\frac{1}{g}\right)}^{+}\end{array}$$

(181)

where $\lambda$ is chosen so that $\mathrm{E}\left[P(G)\right]=P$. The capacity (178) is then (see ([95], Equation (7)))

$$\begin{array}{cc}\hfill C(P)& ={\int }_{\lambda }^{\infty }p(g)log\left(g/\lambda \right)dg.\hfill \end{array}$$

(182)

Consider now the quantizer $q_u(.)$ of Section 2.9 with $B=1$. We have two equations for $\lambda$, namely

$$\begin{array}{cc}\hfill \lambda & ={\int }_0^{\Delta }\frac{p(g)}{P_{S_T}(\Delta /2)}·{\displaystyle \frac{g}{1+gP(\Delta /2)}}dg\hfill \end{array}$$

(183)

$$\begin{array}{cc}\hfill \lambda & ={\int }_{\Delta }^{\infty }\frac{p(g)}{P_{S_T}(3\Delta /2)}·{\displaystyle \frac{g}{1+gP(3\Delta /2)}}dg.\hfill \end{array}$$

(184)

Observe the following for (183) and (184):

• both $P(\Delta /2)$ and $P(3\Delta /2)$ decrease as $\lambda$ increases;
• the maximal $\lambda$ permitted by (183) is $\mathrm{E}\left[G|G\le \Delta \right]$ which is obtained with $P(\Delta /2)=0$;
• the maximal $\lambda$ permitted by (184) is $\mathrm{E}\left[G|G\ge \Delta \right]$ which is obtained with $P(3\Delta /2)=0$.

Thus, if $\mathrm{E}\left[G|G\ge \Delta \right]>\mathrm{E}\left[G|G\le \Delta \right]$, then at P below some threshold, we have $P(\Delta /2)=0$ and $P(3\Delta /2)=P/P_{S_T}(3\Delta /2)$. The capacity in nats per symbol at low power and for fixed $\Delta$ is thus

$$\begin{array}{cc}\hfill C(P)& ={\int }_{\Delta }^{\infty }p(g)log\left(1+gP(3\Delta /2)\right)dg\hfill \\ \hfill & \approx P\mathrm{E}\left[G|G\ge \Delta \right]-\frac{P^2}{2P_{S_T}(3\Delta /2)}\mathrm{E}\left[G^2|G\ge \Delta \right]\hfill \end{array}$$

(185)

where we used

$$\begin{array}{c}\hfill log(1+x)\approx x-\frac{x^2}{2}\end{array}$$

for small x. The wideband values (37) are

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{E}\left[G|G\ge \Delta \right]}\hfill \end{array}$$

(186)

$$\begin{array}{cc}\hfill & S=\frac{2P_{S_T}(3\Delta /2)\mathrm{E}{\left[G|G\ge \Delta \right]}^2}{\mathrm{E}\left[G^2|G\ge \Delta \right]}.\hfill \end{array}$$

(187)

One can thus make the minimum $E_b/N_0$ approach $-\infty$ if one can make $\mathrm{E}\left[G|G\ge \Delta \right]$ as large as desired by increasing $\Delta$.

Remark 55.

Consider the MIMO model (171) with $S_R=\mathbf{H}$. Suppose the CSIT is $S_T=f_T(S_R)$ for some function $f_T(·)$. The capacity (178) generalizes to

$$\begin{array}{cc}\hfill C(P)& =\underset{\underline{X}(S_T):\mathrm{E}\left[\parallel \underline{X}(S_T){\parallel }^2\right]\le P}{max}I(\underline{X};\mathbf{H}\underline{X}+\underline{Z}|\mathbf{H},S_T)\hfill \\ \hfill & =\underset{\mathbf{Q}(S_T):\mathrm{E}\left[\mathrm{tr}\left(\mathbf{Q}(S_T)\right)\right]\le P}{max}\mathrm{E}\left[logdet\left(\mathbf{I}+\mathbf{H}\mathbf{Q}(S_T){\mathbf{H}}^{†}\right)\right].\hfill \end{array}$$

(188)

6.4. Full CSIR, Partial CSIT

Consider first the full CSIR $S_R=H\sqrt{P(S_T)}$ and then the less informative $S_R=H$.

$S_R=H\sqrt{P(S_T)}$: We have the following capacity result that implies this CSIR is the best possible since one can achieve the same rate as if the receiver sees both H and $S_T$; see the first step of (162). We could thus have classified this model as CSIT@R.

Proposition 2

(see ([10], Proposition 3)). The capacity of the channel (159) with $S_R=H\sqrt{P(S_T)}$ and general $S_T$ is

$$\begin{array}{cc}\hfill C(P)& =\underset{P(S_T):\mathrm{E}\left[P(S_T)\right]\le P}{max}{\int }_{\mathbb{C}}p(s_R)log\left(1+|s_R{|}^2\right)d{s}_R\hfill \\ \hfill & =\underset{P(S_T):\mathrm{E}\left[P(S_T)\right]\le P}{max}\mathrm{E}\left[log\left(1+GP(S_T)\right)\right].\hfill \end{array}$$

(189)

Proof. 

Achievability follows by Theorem 1 with Remark 51. The converse is given by Lemma 6. □

Remark 56.

Proposition 2 gives an upper bound and (thus) a target rate when the receiver has partial CSIR. For example, we will use the K-partition idea of Lemma 1 (see also Remark 46) to approach the upper bound for large SNR.

Remark 57.

Proposition 2 partially generalizes to block-fading channels; see Proposition 3 in Section 9.5.

$S_R=H$: The capacity is (138) with

$$\begin{array}{c}\hfill I(A;Y|H)=\mathrm{E}\left[log\frac{p(Y|A,H)}{p(Y|H)}\right]\end{array}$$

(190)

where $\mathrm{E}\left[{\left|X\right|}^2\right]\le P$ and where

$$\begin{array}{cc}\hfill p(y|a,h)& ={\int }_{\mathbb{C}}p(s_T|h)\frac{e^{-|y-hx(s_T){|}^2}}{\pi }d{s}_T\hfill \end{array}$$

(191)

and

$$\begin{array}{cc}\hfill p(y|h)& ={\int }_{\mathbb{C}}p(s_T|h)\left({\int }_{\mathcal{A}}p(a)p(y|a,h,s_T)da\right)d{s}_T\hfill \\ \hfill & ={\int }_{\mathbb{C}}p(s_T|h)\left({\int }_{\mathbb{C}}p(x(s_T))\frac{e^{-|y-hx(s_T){|}^2}}{\pi }dx(s_T)\right)d{s}_T.\hfill \end{array}$$

(192)

For example, if each entry $X(s_T)$ of A is CSCG with variance $P(s_T)$ then

$$\begin{array}{c}\hfill p(y|h)={\int }_{\mathbb{C}}p(s_T|h)\frac{exp\left(-\frac{{\left|y\right|}^2}{1+gP(s_T)}\right)}{\pi (1+gP(s_T))}d{s}_T.\end{array}$$

(193)

In general, one can compute $I(A;Y|H)$ numerically by using (190)–(192), but the calculations are hampered if the integrals in (191) and (192) do not simplify.

For the reverse model GMI (139), the averaging density in (164) and (165) is here

$$\begin{array}{c}\hfill p(h,s_T|y,s_R)=\delta (h-s_R)\frac{p(s_T|h)p(y|h,s_T)}{p(y|h)}.\end{array}$$

(194)

We use numerical integration to compute the GMI.

To obtain more insight, we state the forward model rates of Theorem 1 and Remark 51 as a Corollary.

Corollary 1.

An achievable rate for the fading channels (159) with $S_R=H$ and partial CSIT is the forward model GMI

$$\begin{array}{cc}\hfill I_1(A;Y|H)& =\mathrm{E}\left[log\left(1+\mathrm{SNR}(H)\right)\right]\hfill \end{array}$$

(195)

where

$$\begin{array}{c}\hfill \mathrm{SNR}(h)=\frac{{\left|h\right|}^2{\tilde{P}}_T(h)}{1+{\left|h\right|}^2\mathrm{Var}\left[\left.\sqrt{P(S_T)}\right|H=h\right]}\end{array}$$

(196)

and

$$\begin{array}{c}\hfill {\tilde{P}}_T(h)=\mathrm{E}{\left[\left.\sqrt{P(S_T)}\right|H=h\right]}^2.\end{array}$$

(197)

Remark 58.

Jensen’s inequality gives

$$\begin{array}{c}\hfill {\tilde{P}}_T(h)\le \mathrm{E}\left[P(S_T)|H=h\right]\end{array}$$

(198)

by the concavity of the square root. Equality holds if and only if $P(S_T)$ is a constant given $H=h$.

Remark 59.

Choosing $P(s_T)=P$ for all $s_T$ in Corollary 1 gives ${\tilde{P}}_T(h)=P$ for all h and the rate (195) is the capacity (175) without CSIT.

Remark 60.

For large P, the $\mathrm{SNR}(h)$ in (196) saturates unless $P(s_T)/P\to 1$ for all $s_T$, i.e., the high-SNR capacity is the same as the capacity without CSIT. The CSIT thus must become more accurate as P increases to improve the rate.

Remark 61.

To optimize the power levels, consider (146) and

$$\begin{array}{cc}\hfill \tilde{P}{(h)}^{\prime }& ={2\left|h\right|}^2\sqrt{{\tilde{P}}_T(h)}p(s_T|h)\hfill \end{array}$$

(199)

$$\begin{array}{cc}\hfill \mathrm{E}{\left[{\left|Y\right|}^2|H=h\right]}^{\prime }& ={2\left|h\right|}^2\sqrt{P(s_T)}p(s_T|h).\hfill \end{array}$$

(200)

However, the resulting equations give little insight due to the expectation over H in (146). An exception is the on-off fading case where the expectation has only one term; see (254) and (255).

6.5. Partial CSIR, Full CSIT

Suppose $S_R$ is a (perhaps noisy) function of H; see (172). The capacity is given by (160) for which we need to compute $p(y|a,s_R)$ and $p(y|s_R)$. The GMI with a K-partition of the output space $\mathcal{Y}\times {\mathcal{S}}_R$ can be helpful for these problems. We assume that the CSIR is either $S_R=0$ or $S_R=1(G\ge t)$ for some transmitter threshold t; see [95].

Suppose that $S_T=H$. We then have

$$\begin{array}{cc}\hfill p(y|a,s_R)& ={\int }_{\mathbb{C}}p(h|s_R)\frac{exp\left(-{\left|y-hx(h)\right|}^2\right)}{\pi }dh\hfill \\ \hfill p(y|s_R)& ={\int }_{{\mathbb{C}}^2}p(h|s_R)p(x(h))\frac{exp\left(-{\left|y-hx(h)\right|}^2\right)}{\pi }dx(h)dh.\hfill \end{array}$$

Now select the $X(h)$ to be jointly CSCG with variances $\mathrm{E}\left[{\left|X(h)\right|}^2\right]=P(h)$ and correlation coefficients

$$\begin{array}{c}\hfill \rho (h,h^{\prime })=\frac{\mathrm{E}\left[X(h)X{(h^{\prime })}^{*}\right]}{\sqrt{P(h)P(h^{\prime })}}\end{array}$$

and where $\mathrm{E}\left[P(H)\right]\le P$. We then have

$$\begin{array}{c}\hfill p(y|s_R)={\int }_{\mathbb{C}}p(h)\frac{e^{-{\left|y\right|}^2/{(|h|}^{2}P(h)+1)}}{2\pi (|h{|}^{2}P(h)+1)}dh.\end{array}$$

As in (97), $p(y|s_R)$ and therefore $h(Y|S_R)$ depend only on the marginals $p(x(h))$ of A and not on the $\rho (h,h^{\prime })$. We thus have the problem of finding the $\rho (h,h^{\prime })$ that minimize

$$\begin{array}{c}\hfill h(Y|S_R,A)={\int }_{\mathcal{A}}p(a)h(Y|S_R,A=a)da.\end{array}$$

However, we study the conventional A in (88) for simplicity.

For the reverse model GMI (139), the averaging density in (164) and (165) is (cf. (194))

$$\begin{array}{c}\hfill p(h,s_T|y,s_R)=\delta (s_T-h)\frac{p(h|s_R)p(y|h,s_R)}{p(y|s_R)}.\end{array}$$

(201)

We again use numerical integration to compute the GMI.

For the forward model GMI, consider the same model and CSCG X as in Theorem 1. Since H is a function of $S_T$, we use (169) in Remark 50 to write

$$\begin{array}{c}\hfill I_1(A;Y|S_R)=\mathrm{E}\left[log\left(1+\frac{\tilde{P}(S_R)}{1+\mathrm{Var}\left[\left.\sqrt{GP(H)}\right|S_R\right]}\right)\right]\end{array}$$

(202)

where (see (170))

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& =\mathrm{E}{\left[\left.\sqrt{GP(H)}\right|S_R=s_R\right]}^2\hfill \end{array}$$

(203)

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]& =1+\mathrm{E}\left[GP(H)|S_R=s_R\right].\hfill \end{array}$$

(204)

The transmitter compensates for the phase of H, and it remains to adjust the transmit power levels $P(h)$. We study five power control policies and two types of CSIR; see

Table 2. Power Control Policies and Minimal SNRs.

		CSIR
		None: ${\mathit{S}}_{\mathit{R}}=0$	${\mathit{S}}_{\mathit{R}}=1(\mathit{G}\ge \mathit{t})$
Policy	TCP	Equation (221)	Equation (226)
	TMF	Equation (222)	Equation (227)
	TCI	Equation (223)	Equation (228)
	GMI-Optimal	See Theorem 2
	TMMSE	See Remark 64

.

Heuristic Policies: The first three policies are reasonable heuristics and have the form

$$\begin{array}{c}\hfill P(h)=\left\{\begin{array}{cc}\widehat{P}{g}^a,\hfill & g\ge t\hfill \\ 0,\hfill & \mathrm{else}\hfill \end{array}\right.\end{array}$$

(205)

for some choice of real a and where

$$\begin{array}{c}\hfill \widehat{P}=\frac{P}{{\int }_t^{\infty }p(g)g^{a}dg}.\end{array}$$

(206)

In particular, choosing $a=0,+1,-1$, we obtain policies that we call truncated constant power (TCP), truncated matched filtering (TMF), and truncated channel inversion (TCI), respectively; see ([5], page 487), [95]. For such policies, we compute

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& =\widehat{P}{\left({\int }_t^{\infty }p(g|s_R)\sqrt{g^{1+a}}dg\right)}^2\hfill \end{array}$$

(207)

$$\begin{array}{cc}\hfill \mathrm{E}\left[GP(H)|S_R=s_R\right]& =\widehat{P}{\int }_t^{\infty }p(g|s_R)g^{1+a}dg.\hfill \end{array}$$

(208)

These policies all have the form $P(h)=P·f(h)$ for some function $f(.)$ that is independent of P. The minimum SNR in (37) with $C(P)$ replaced with the GMI is thus

$$\begin{array}{cc}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }& =\frac{\left({\int }_t^{\infty }p(g)g^{a}dg\right)log2}{\mathrm{E}\left[{\left({\int }_t^{\infty }p(g|S_R)\sqrt{g^{1+a}}dg\right)}^2\right]}.\hfill \end{array}$$

(209)

For instance, consider the threshold $t=0$ (no truncation). The TCP ($a=0$) and TMF ($a=1$) policies have $\widehat{P}=P$ while TCI ($a=-1$) has $P=\widehat{P}/\mathrm{E}\left[G^{-1}\right]$. For TCP, TMF, and TCI, we compute the respective

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{E}\left[\mathrm{E}{\left[\left.\sqrt{G}\right|S_R\right]}^2\right]}\hfill \end{array}$$

(210)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{E}\left[\mathrm{E}{\left[G|S_R\right]}^2\right]}\hfill \end{array}$$

(211)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\mathrm{E}\left[G^{-1}\right]log2.\hfill \end{array}$$

(212)

Applying Jensen’s inequality to the square root, square, and inverse functions in (210)–(212), we find that for $t=0$:

• the minimum $E_b/N_0$ of TCP and TCI is larger (worse) than $-1.59$ dB unless there is no fading;
• the minimum $E_b/N_0$ of TMF is smaller (better) than $-1.59$ dB unless $\mathrm{E}\left[G|S_R\right]=\mathrm{E}\left[G\right]=1$.

However, we emphasize that these claims apply to the GMI and not necessarily the mutual information; see Section 8.4.

GMI-Optimal Policy: The fourth policy is optimal for the GMI (202) and has the form of an MMSE precoder. This policy motivates a truncated MMSE (TMMSE) policy that generalizes and improves TMF and TCI.

Taking the derivative of the Lagrangian

$$\begin{array}{c}\hfill I_1(A;Y|S_R)+\lambda \left(P-\mathrm{E}\left[P(H)\right]\right)\end{array}$$

(213)

with respect to $P(h)$ we have the following result.

Theorem 2.

The optimal power control policy for the GMI $I_1(A;Y|S_R)$ for the fading channels (159) with $S_T=H$ is

$$\begin{array}{cc}\hfill \sqrt{P(h)}& =\frac{\alpha (h)\left|h\right|}{\lambda +{\beta (h)\left|h\right|}^2}\hfill \end{array}$$

(214)

where $\lambda >0$ is chosen so that $\mathrm{E}\left[P(H)\right]=P$ and

$$\begin{array}{cc}\hfill & \alpha (h)={\int }_{\mathbb{C}}p(s_R|h)\frac{\sqrt{\tilde{P}(s_R)}}{\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]-\tilde{P}(s_R)}d{s}_R\hfill \end{array}$$

(215)

$$\begin{array}{cc}\hfill & \beta (h)={\int }_{\mathbb{C}}p(s_R|h)\frac{\tilde{P}(s_R)}{\left[\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]-\tilde{P}(s_R)\right]\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]}d{s}_R.\hfill \end{array}$$

(216)

Proof. 

Apply (146) with (203) and (204) to obtain

$$\begin{array}{cc}\hfill & \tilde{P}{(s_R)}^{\prime }=2\left|h\right|\sqrt{\tilde{P}(s_R)}p(h|s_R)\hfill \end{array}$$

(217)

$$\begin{array}{cc}\hfill & \mathrm{E}{\left[{\left|Y\right|}^2|S_R=s_R\right]}^{\prime }=2{\left|h\right|}^2\sqrt{P(h)}p(h|s_R).\hfill \end{array}$$

(218)

Inserting into (146) and rearranging terms we obtain (214) with (215) and (216). □

Remark 62.

The expressions (215) and (216) are self-referencing, as $\tilde{P}(s_R)$ itself depends on $\alpha (h)$ and $\beta (h)$. However, one simplification occurs if $S_R$ is a function of H: $\alpha (h)$ and $\beta (h)$ are functions of $s_R$ only since the $p(s_R|h)$ in (215) and (216) is a Dirac generalized function.

Remark 63.

Consider the expression (214). We effectively have a matched filter for small $\left|h\right|$; for large $\left|h\right|$, we effectively have a channel inversion. Recall that LMMSE filtering has similar behavior for low and high SNR, respectively.

Remark 64.

A heuristic based on the optimal policy is a TMMSE policy where the transmitter sets $P(h)=0$ if $G<t$, and otherwise uses (214) but where $\alpha (h)$, $\beta (h)$ are independent of h. There are thus four parameters to optimize: λ, α, β, and t. This TMMSE policy will outperform TMF and TCI in general, as these are special cases where $\beta =0$ and $\lambda =0$, respectively.

$S_R=0$: For this CSIR, the GMI (202) simplifies to $I_1(A;Y)$ and the heuristic policy (TCP, TMF, TCI) rates are

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{\widehat{P}\mathrm{E}{\left[\sqrt{G^{1+a}}·1(G\ge t)\right]}^2}{1+\widehat{P}\mathrm{Var}\left[\sqrt{G^{1+a}}·1(G\ge t)\right]}\right).\end{array}$$

(219)

Moreover, the expression (209) gives

$$\begin{array}{cc}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }& =\frac{\mathrm{E}\left[G^a·1(G\ge t)\right]}{\mathrm{E}{\left[\sqrt{G^{1+a}}·1(G\ge t)\right]}^2}log2.\hfill \end{array}$$

(220)

For TCP, TMF, and TCI, we compute the respective

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{Pr}\left[G\ge t\right]\mathrm{E}{\left[\left.\sqrt{G}\right|G\ge t\right]}^2}\hfill \end{array}$$

(221)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{{\int }_t^{\infty }p(g)gdg}\hfill \end{array}$$

(222)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{\mathrm{E}\left[\left.G^{-1}\right|G\ge t\right]}{\mathrm{Pr}\left[G\ge t\right]}log2.\hfill \end{array}$$

(223)

Again applying Jensen’s inequality to the various functions in (221)–(223), we find that:

• the minimum $E_b/N_0$ of TMF is smaller (better) than that of TCP and TCI unless there is no fading, or if the minimal $E_b/N_0$ is $-\infty$;
• the best threshold for TMF is $t=0$ and the minimal $E_b/N_0$ is $-1.59$ dB.

For the optimal policy, the parameters $\alpha (h)$ and $\beta (h)$ in (215) and (216) are constants independent of h, see Remark 62, and the TMMSE policy with $t=0$ is the GMI-optimal policy.

Remark 65.

The TCI channel densities are

$$\begin{array}{cc}\hfill p(y|a)& =\mathrm{Pr}\left[G<t\right]\frac{e^{-{\left|y\right|}^2}}{\pi }+\mathrm{Pr}\left[G\ge t\right]\frac{e^{-{\left|y-\sqrt{\widehat{P}}u\right|}^2}}{\pi }\hfill \\ \hfill p(y)& =\mathrm{Pr}\left[G<t\right]\frac{e^{-{\left|y\right|}^2}}{\pi }+\mathrm{Pr}\left[G\ge t\right]\frac{e^{-{\left|y\right|}^2/(1+\widehat{P})}}{\pi (1+\widehat{P})}.\hfill \end{array}$$

Remark 66.

At high SNR, one might expect that the receiver can estimate $P(S_T)$ precisely even if $S_R=0$. We show that this is indeed the case for on-off fading by using the $K=2$ partition (154) of Remark 46. Moreover, the results prove that at high SNR one can approach $I(A;Y)$; see Section 7.3.

Remark 67.

For Rayleigh fading, the GMI with $K=2$ in (154) is helpful for both high and low SNR. For instance, for $S_R=0$ and TCI, the $K=2$ GMI approaches the mutual information for $S_R=1(G\ge t)$ as the SNR increases; see Remark 74 in Section 8.4. We further show that for $S_R=0$, the TCI policy can achieve a minimal $E_b/N_0$ of $-\infty$ dB, see (301) in Section 8.4.

$S_R=1(G\ge t)$: The heuristic policy rates are now (cf. (219) and note the $\mathrm{Pr}\left[G\ge t\right]$ term and conditioning)

$$\begin{array}{c}\hfill I_1(A;Y|S_R)=\mathrm{Pr}\left[G\ge t\right]log\left(1+\frac{\widehat{P}\mathrm{E}{\left[\left.\sqrt{G^{1+a}}\right|G\ge t\right]}^2}{1+\widehat{P}\mathrm{Var}\left[\left.\sqrt{G^{1+a}}\right|G\ge t\right]}\right).\end{array}$$

(224)

Moreover, the expression (209) is

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{\mathrm{E}\left[\left.G^a\right|G\ge t\right]}{\mathrm{E}{\left[\left.\sqrt{G^{1+a}}\right|G\ge t\right]}^2}log2.\hfill \end{array}$$

(225)

For TCP, TMF, and TCI we compute the respective

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{E}{\left[\left.\sqrt{G}\right|G\ge t\right]}^2}\hfill \end{array}$$

(226)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{\mathrm{E}\left[G|G\ge t\right]}\hfill \end{array}$$

(227)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\mathrm{E}\left[\left.G^{-1}\right|G\ge t\right]log2.\hfill \end{array}$$

(228)

Again applying Jensen’s inequality to the various functions in (226)–(228), we find that:

• the minimum $E_b/N_0$ of all policies can be better than $-1.59$ dB by choosing $t>0$;
• the minimum $E_b/N_0$ of TMF is smaller (better) than that of TCP and TCI unless there is no fading or the minimal $E_b/N_0$ is $-\infty$.

For the optimal policy, Remark 62 points out that $\alpha (h)$ and $\beta (h)$ depend on $s_R$ only. We compute

$$\begin{array}{c}\hfill \sqrt{P(h)}=\left\{\begin{array}{cc}{\displaystyle \frac{{\alpha }_0\left|h\right|}{\lambda +{\beta }_0{\left|h\right|}^2}},\hfill & g<t\hfill \\ {\displaystyle \frac{{\alpha }_1\left|h\right|}{\lambda +{\beta }_1{\left|h\right|}^2}},\hfill & g\ge t\hfill \end{array}\right.\end{array}$$

(229)

where for $s_R\in \{0,1\}$ we have

$$\begin{array}{cc}\hfill & {\alpha }_{s_R}={\displaystyle \frac{\sqrt{\tilde{P}(s_R)}}{\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]-\tilde{P}(s_R)}}\hfill \\ \hfill & {\beta }_{s_R}={\displaystyle \frac{\sqrt{\tilde{P}(s_R)}}{\left[\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]-\tilde{P}(s_R)\right]\mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]}}.\hfill \end{array}$$

Remark 68.

The GMI (224) for TCI ($a=-1$) is the mutual information $I(A;Y|S_R)$. To see this, observe that the model $q(y|a,s_R)$ has

$$\begin{array}{c}\hfill q(y|a,0)=\frac{e^{-{\left|y\right|}^2}}{\pi },q(y|a,1)=\frac{e^{-{\left|y-\sqrt{\widehat{P}}u\right|}^2}}{\pi }\end{array}$$

and thus we have $q(y|a,s_R)=p(y|a,s_R)$ for all $y,a,s_R$.

6.6. Partial CSIR, CSIT@ R

Suppose next that $S_R$ is a noisy function of H (see for instance (172)) and $S_T=f_T(S_R)$. The capacity is given by (147) and we compute

$$\begin{array}{c}\hfill I(X;Y|S_R)=\mathrm{E}\left[log\frac{p(Y|X,S_R)}{p(Y|S_R)}\right]\end{array}$$

(230)

where writing $s_T=f_T(s_R)$ we have

$$\begin{array}{cc}\hfill p(y|s_R,x)& ={\int }_{\mathbb{C}}p(h|s_R)\frac{e^{-|y-hx(s_T){|}^2}}{\pi }dh\hfill \end{array}$$

(231)

$$\begin{array}{cc}\hfill p(y|s_R)& ={\int }_{{\mathbb{C}}^2}p(h|s_R)p(x(s_T))\frac{e^{-|y-hx(s_T){|}^2}}{\pi }dx(s_T)dh.\hfill \end{array}$$

(232)

For example, if $X(s_T)$ is CSCG with variance $P(s_T)$ then

$$\begin{array}{c}\hfill p(y|s_R)={\int }_{\mathbb{C}}p(h|s_R)\frac{exp\left(-\frac{{\left|y\right|}^2}{1+{\left|h\right|}^{2}P(s_T)}\right)}{\pi (1+|h{|}^{2}P(s_T))}dh.\end{array}$$

(233)

One can compute $I(X;Y|S_R)$ numerically using (231) and (232). However, optimizing over $X(s_T)$ is usually difficult.

For the reverse model GMI (139), the averaging density in (164) and (165) is now (cf. (194) and (201))

$$\begin{array}{c}\hfill p(h,s_T|y,s_R)=\delta \left(s_T-f_T(s_R)\right)\frac{p(h|s_R)p(y|h,s_R)}{p(y|s_R)}.\end{array}$$

(234)

We use numerical integration to compute the rates.

The forward model GMI again gives more insight. Define the channel gain and variance as the respective

$$\begin{array}{cc}\hfill \tilde{g}(s_R)& ={\left|\mathrm{E}\left[H|S_R=s_R\right]\right|}^2\hfill \end{array}$$

(235)

$$\begin{array}{cc}\hfill {\tilde{\sigma }}^2(s_R)& =\mathrm{Var}\left[H|S_R=s_R\right].\hfill \end{array}$$

(236)

Theorem 3.

An achievable rate for AWGN fading channels (159) with power constraint $\mathrm{E}\left[{\left|X\right|}^2\right]\le P$ and with partial CSIR $S_R$ and $S_T=f_T(S_R)$ is

$$\begin{array}{c}\hfill I_1(X;Y|S_R)=\mathrm{E}\left[log\left(1+\frac{\tilde{g}(S_R)P(S_T)}{1+{\tilde{\sigma }}^2(S_R)P(S_T)}\right)\right]\end{array}$$

(237)

where $\mathrm{E}\left[P(S_T)\right]=P$. The optimal power levels $P(s_T)$ are obtained by solving

$$\begin{array}{cc}\hfill & \lambda ={\int }_{\mathbb{R}}p(s_R|s_T)\frac{\tilde{g}(s_R)}{\left[1+\left(\tilde{g}(s_R)+{\tilde{\sigma }}^2(s_R)\right)P(s_T)\right]\left[1+{\tilde{\sigma }}^2(s_R)P(s_T)\right]}d{s}_R.\hfill \end{array}$$

(238)

In particular, if $S_T$ determines $S_R$ (CSIR@T) then we have the quadratic waterfilling expression

$$\begin{array}{c}\hfill f\left(P(s_T),\tilde{g}(s_R),{\tilde{\sigma }}^2(s_R)\right)={\left(\frac{1}{\lambda }-\frac{1}{\tilde{g}(s_R)}\right)}^{+}\end{array}$$

(239)

where

$$\begin{array}{c}\hfill f\left(Q,g,{\sigma }^2\right)=\left(1+2\frac{{\sigma }^2}{g}\right)Q+\left(1+\frac{{\sigma }^2}{g}\right){\sigma }^2{Q}^2\end{array}$$

(240)

and where λ is chosen so that $\mathrm{E}\left[P(H_R)\right]=P$.

Proof. 

Apply Theorem 1 with

$$\begin{array}{cc}\hfill \tilde{P}(s_R)& =\tilde{g}(s_R)P(s_T)\hfill \end{array}$$

(241)

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left|Y\right|}^2|S_R=s_R\right]& =1+\left(\tilde{g}(s_R)+{\tilde{\sigma }}^2(s_R)\right)P(s_T)\hfill \end{array}$$

(242)

to obtain (237). To optimize the power levels $P(s_T)$ with (146), consider the derivatives

$$\begin{array}{cc}\hfill \tilde{P}{(s_R)}^{\prime }& =2\tilde{g}(s_R)\sqrt{P(s_T)}1(s_T=f_T(s_R))\hfill \end{array}$$

(243)

$$\begin{array}{cc}\hfill \mathrm{E}{\left[{\left|Y\right|}^2|S_R=s_R\right]}^{\prime }& =2\left(\tilde{g}(s_R)+{\tilde{\sigma }}^2(s_R)\right)\sqrt{P(s_T)}1(s_T=f_T(s_R)).\hfill \end{array}$$

(244)

The expression (146) thus becomes (238). If $S_T$ determines $S_R$ then the expression simplifies to

$$\begin{array}{c}\hfill \lambda =\frac{\tilde{g}(s_R)}{\left[1+\left(\tilde{g}(s_R)+{\tilde{\sigma }}^2(s_R)\right)P(s_T)\right]\left[1+{\tilde{\sigma }}^2(s_R)P(s_T)\right]}\end{array}$$

from which we obtain (239). □

Remark 69.

The optimal power control policy with CSIT@R and CSIR@T can be written explicitly by solving the quadratic in (239). The result is

$$\begin{array}{c}\hfill P(s_T)=\frac{\tilde{g}+2{\tilde{\sigma }}^2}{2{\tilde{\sigma }}^2(\tilde{g}+{\tilde{\sigma }}^2)}\left[\sqrt{1+4{\tilde{\sigma }}^2{\left(\frac{1}{\lambda }-\frac{1}{\tilde{g}}\right)}^{+}\frac{\tilde{g}(\tilde{g}+{\tilde{\sigma }}^2)}{{(\tilde{g}+2{\tilde{\sigma }}^2)}^2}}-1\right]\end{array}$$

(245)

where we have discarded the dependence on $s_R$ for convenience. The alternative form (239) relates to the usual waterfilling where the left-hand side of (239) is $P(s_T)$. Observe that ${\tilde{\sigma }}^2=0$ gives conventional waterfilling.

Remark 70.

As in Section 3.3, we show that at high SNR the $K=2$ GMI of Remark 42 approaches the upper bound of Proposition 2 in some cases; see Section 7.4. The channel parameters depend on $s_R$, and we choose $h_1(s_R)=0$ and ${\sigma }_1^2(s_R)={\sigma }_2^2(s_R)=1$ for all $s_R$.

7. On-Off Fading

Consider again on-off fading with $P_G(0)=P_G(2)=1/2$. We study the scenarios listed in Table 1. The case of no CIR and no CSIT was studied in Section 3.3.

7.1. Full CSIR, CSIT@ R

Consider $S_R=H$. The capacity with $B=0$ (no CSIT) is given by (175) (cf. (73)):

$$\begin{array}{c}\hfill C(P)=\frac{1}{2}log\left(1+2P\right)\end{array}$$

(246)

and the wideband values are given by (177) (cf. (74)); the minimal $E_b/N_0$ is $log2$ and the slope is $S=1$.

The capacity with $B=\infty$ (or $S_T=G$) increases to

$$\begin{array}{cc}\hfill C(P)& =\frac{1}{2}log\left(1+4P\right)\hfill \end{array}$$

(247)

where $P(0)=0$ and $P(2)=2P$. This capacity is also achieved with $B=1$ since there are only two values for G. We compute $C^{\prime }(0)=2$ and $C^{″}(0)=-8$, and therefore

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{2},S=1.\end{array}$$

(248)

The power gain due to CSIT compared to no fading is thus 3.01 dB, but the capacity slope is the same. The rate curves are compared in Figure 4.

7.2. Full CSIR, Partial CSIT

Consider next noisy CSIT with $0\le ϵ\le \frac{1}{2}$ and

$$\begin{array}{c}\hfill \mathrm{Pr}\left[S_T=G\right]=\overline{ϵ},\mathrm{Pr}\left[S_T\ne G\right]=ϵ.\end{array}$$

$S_R=H\sqrt{P(S_T)}$: The capacity of Proposition 2 is

$$\begin{array}{c}\hfill C(P)=\underset{P(0)+P(2)=2P}{max}\frac{ϵ}{2}log\left(1+2P(0)\right)+\frac{\overline{ϵ}}{2}log\left(1+2P(2)\right).\end{array}$$

(249)

Optimizing the power levels, we have

$$\begin{array}{c}\hfill P(0)={\left(2ϵP-\frac{\overline{ϵ}-ϵ}{2}\right)}^{+},P(2)=2P-P(0).\end{array}$$

(250)

Figure 4 shows $C(P)$ for $ϵ=0.1$ as the curve labeled “Best CSIR”. For $P\ge (\overline{ϵ}-ϵ)/(4ϵ)$, we compute

$$\begin{array}{c}\hfill C(P)=\frac{1}{2}log(1+2P)+\frac{1}{2}[1-H_2(ϵ)]log2\end{array}$$

(251)

where $H_2(ϵ)=-ϵ{log}_2ϵ-\overline{ϵ}{log}_2\overline{ϵ}$ is the binary entropy function. For example, if $ϵ=0.1$ then for $P\ge 2$ one gains $\Delta C=[1-H_2(0.1)]/2\approx 0.27$ bits over the capacity without CSIT. This translates to an SNR gain of $2\Delta C·10{log}_{10}(2)\approx 1.60$ dB. On the other hand, for $P\le (\overline{ϵ}-ϵ)/(4ϵ)$ we have $P(0)=0$, $P(2)=2P$, and the capacity is

$$\begin{array}{c}\hfill C(P)=\frac{\overline{ϵ}}{2}log\left(1+4P\right).\end{array}$$

(252)

We have $C^{\prime }(0)=2\overline{ϵ}$ and lose a fraction of $\overline{ϵ}$ of the power as compared to having full CSIT ($ϵ=0$). For example, if $ϵ=0.1$, the minimal $E_b/N_0$ is approximately $-4.14$ dB.

$S_R=H$: To compute $I(A;Y|H)$ in (190), we write (191) and (193) for CSCG $X(s_T)$ as

$$\begin{array}{cc}\hfill p_{Y|A,H}(y|a,0)& =p_{Y|H}(y|0)=\frac{e^{-{\left|y\right|}^2}}{\pi }\hfill \\ \hfill p_{Y|A,H}\left(y|a,\sqrt{2}\right)& =ϵ\frac{e^{-{\left|y-\sqrt{2}x(0)\right|}^2}}{\pi }+\overline{ϵ}\frac{e^{-{\left|y-\sqrt{2}x\left(\sqrt{2}\right)\right|}^2}}{\pi }\hfill \\ \hfill p_{Y|H}\left(y|\sqrt{2}\right)& =ϵ\frac{exp\left(-\frac{{\left|y\right|}^2}{1+2P(0)}\right)}{\pi (1+2P(0))}+\overline{ϵ}\frac{exp\left(-\frac{{\left|y\right|}^2}{1+2P(2)}\right)}{\pi (1+2P(2))}.\hfill \end{array}$$

Figure 4 shows the rates as the curve labeled “$I(A;Y|H)$”. This curve was computed by Monte Carlo integration with $P(0)=0.1·P$ and $P(2)=1.9·P$, which is near-optimal for the range of SNRs depicted.

The reverse model GMI (139) requires $\mathrm{Var}\left[U|Y,H\right]$. We show how to compute this variance in Appendix C.2 by applying (164) and (165). Figure 4 shows the GMIs as the curve labeled “rGMI”, where we used the same power levels as for the $I(A;Y|H)$ curve. The two curves are indistinguishable for small P, but the “rGMI” rates are poor at large P. This example shows that the forward model GMI with optimized powers can be substantially better than the reverse model GMI with a reasonable but suboptimal power policy.

The forward model GMI (195) is

$$\begin{array}{cc}\hfill & I_1(A;Y|H)=\frac{1}{2}log\left(1+\mathrm{SNR}\left(\sqrt{2}\right)\right)\hfill \end{array}$$

(253)

where $\mathrm{SNR}\left(\sqrt{2}\right)$ is given by (196) with

$$\begin{array}{cc}\hfill {\tilde{P}}_T\left(\sqrt{2}\right)& ={\left(ϵ\sqrt{P(0)}+\overline{ϵ}\sqrt{P(2)}\right)}^2\hfill \\ \hfill \mathrm{Var}\left[\left.\sqrt{P(S_T)}\right|H=h\right]& =1+2ϵ\overline{ϵ}{\left(\sqrt{P(2)}-\sqrt{P(0)}\right)}^2.\hfill \end{array}$$

Applying Remark 61, the optimal power control policy is

$$\begin{array}{c}\hfill \sqrt{P(s_T)}=\frac{p_{H|S_T}\left(\sqrt{2}|s_T\right)}{\gamma +\beta p_{H|S_T}\left(\sqrt{2}|s_T\right)}=\left\{\begin{array}{cc}{\displaystyle \frac{ϵ}{\gamma +\beta ϵ}},\hfill & s_T=0\hfill \\ {\displaystyle \frac{\overline{ϵ}}{\gamma +\beta \overline{ϵ}}},\hfill & s_T=2\hfill \end{array}\right.\end{array}$$

(254)

where

$$\begin{array}{cc}\hfill \beta & =\frac{2\sqrt{{\tilde{P}}_T\left(\sqrt{2}\right)}}{\mathrm{E}\left[{\left|Y\right|}^2|H=\sqrt{2}\right]}\hfill \end{array}$$

(255)

and $\gamma \ge 0$ is chosen so that $P(0)+P(2)=2P$. Figure 4 shows the resulting GMI as the curve labeled “GMI, K = 1”. At low SNR, we achieve the rate ${\tilde{P}}_T\left(\sqrt{2}\right)$ and the optimal power control has $\beta \to 0$ so that

$$\begin{array}{cc}\hfill & P(0)=\frac{2P{ϵ}^2}{{ϵ}^2+{\overline{ϵ}}^2},P(2)=\frac{2P{\overline{ϵ}}^2}{{ϵ}^2+{\overline{ϵ}}^2}\hfill \end{array}$$

(256)

and therefore

$$\begin{array}{cc}\hfill & {\tilde{P}}_T(\sqrt{2})=2\left({ϵ}^2+{\overline{ϵ}}^2\right)P.\hfill \end{array}$$

(257)

We have $C^{\prime }(0)=2\left({ϵ}^2+{\overline{ϵ}}^2\right)$ and lose a fraction of $({ϵ}^2+{\overline{ϵ}}^2)$ of the power as compared to having full CSIT ($ϵ=0$). For example, if $ϵ=0.1$, the minimal $E_b/N_0$ is approximately $-3.74$ dB.

We remark that the $I(A;Y|H)$ and reverse model GMI curves lie above the forward model curve if we choose the same power policy as for the forward channel.

7.3. Partial CSIR, Full CSIT

This section studies $S_T=H$. The capacity with partial CSIR is given by (138) for which we need to compute $p(y|a,s_R)$ and $p(y|s_R)$. We consider two cases.

$S_R=1(G\ge t)$: Here we recover the case with full CSIR by choosing t to satisfy $0<t\le 2$.

$S_R=0$: The best power policy clearly has $P(0)=0$ and $P(\sqrt{2})=2P$. The mutual information is thus $I(A;Y)=I\left(X\left(\sqrt{2}\right);Y\right)$ and the channel densities are (cf. (75) and (76))

$$\begin{array}{cc}\hfill p(y|a)& =\frac{e^{-{\left|y\right|}^2}}{2\pi }+\frac{e^{-{\left|y-2\sqrt{P}u\left(\sqrt{2}\right)\right|}^2}}{2\pi }\hfill \\ \hfill p(y)& =\frac{e^{-{\left|y\right|}^2}}{2\pi }+\frac{e^{-{\left|y\right|}^2/(1+4P)}}{2\pi (1+4P)}.\hfill \end{array}$$

The rates $I(A;Y)$ are shown in Figure 5. Observe that the low-SNR rates are larger than without fading; this is a consequence of the slightly bursty nature of transmission.

The reverse model GMI (139) requires $\mathrm{Var}\left[U|Y\right]$. We compute this variance in Appendix C.3 by using (164) and (165) with (201) and $\varphi (s_T)=0$. Figure 5 shows the GMIs as the curve labeled “rGMI”.

Next, the TCP, TMF, TCI, and TMMSE policies are the same for $0<t\le 2$, since they use $P(0)=0$ and $P\left(\sqrt{2}\right)=2P$. The resulting rate is given by (202)–(204) with $\tilde{P}(0)=0$, $\tilde{P}(1)=P$, and $\mathrm{Var}\left[\left.\sqrt{GP(S_T)}\right|S_R=1\right]=P$ and

$$\begin{array}{cc}\hfill & I_1(A;Y)=log\left(1+\frac{P}{1+P}\right).\hfill \end{array}$$

(258)

The rates are plotted in Figure 5 as the curve labeled “GMI, K = 1”. This example again shows that choosing $K=1$ is a poor choice at high SNR.

To improve the auxiliary model at high SNR, consider the GMI (154) with $K=2$ and the subsets (65). We further choose the parameters $h_1=0$, ${\sigma }_1^2=0$, $h_2=2$, ${\sigma }_2^2=1$, and adaptive coding with $X(0)=0$, $X\left(\sqrt{2}\right)=\sqrt{2P}U$, $\overline{X}=\sqrt{P}U$, where $U\sim \mathcal{CN}(0,1)$. The GMI (154) is

$$\begin{array}{cc}\hfill I_1(A;Y)=\mathrm{Pr}\left[{\mathcal{E}}_2\right]& \left[log(1+4P)+\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+4P}-\mathrm{E}\left[\left.{\left|Y-\sqrt{4P}U\right|}^2\right|{\mathcal{E}}_2\right]\right].\hfill \end{array}$$

(259)

In Appendix B.2, we show that choosing $t_R=P^{{\lambda }_R}+b$ where $0<{\lambda }_R<1$ and b is a real constant makes all terms behave as desired as P increases:

$$\begin{array}{c}\hfill \mathrm{Pr}\left[{\mathcal{E}}_2\right]\to 1/2,\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+4P}\to 1,\mathrm{E}\left[\left.{\left|Y-\sqrt{4P}U\right|}^2\right|{\mathcal{E}}_2\right]\to 1.\end{array}$$

(260)

We thus have

$$\begin{array}{c}\hfill \underset{P\to \infty }{lim}\left[\frac{1}{2}log(1+4P)-I_1(X;Y)\right]=0.\end{array}$$

(261)

Figure 5 shows the behavior of $I_1(A;Y)$ for ${\lambda }_R=1/2$ and $b=3$ as the curve labeled “GMI, K = 2”. As for the case without CSIT, the receiver can estimate H accurately at large SNR, and one approaches the capacity with full CSIR.

Finally, the large-K forward model rates are computed using (70) but where $\overline{X}$ replaces X. One may again use the results of Appendix C.3 and the relations

$$\begin{array}{cc}\hfill \mathrm{E}\left[\left.\overline{X}\right|Y=y\right]& =\sqrt{P}\mathrm{E}\left[U|Y=y\right]\hfill \\ \hfill \mathrm{E}\left[\left.|\overline{X}{|}^2\right|Y=y\right]& =P\mathrm{E}\left[{\left|U\right|}^2|Y=y\right]\hfill \\ \hfill \mathrm{Var}\left[\overline{X}|Y=y\right]& =P\mathrm{Var}\left[U|Y=y\right].\hfill \end{array}$$

The rates are shown as the curve labeled “GMI, K = ∞” in Figure 5. So again, the large-K forward model is good at high SNR but worse than the best $K=1$ model at low SNR.

7.4. Partial CSIR, CSIT@ R

Consider partial CSIR with $S_T=S_R$ and

$$\begin{array}{c}\hfill \mathrm{Pr}\left[S_R=H\right]=\overline{ϵ},\mathrm{Pr}\left[S_R\ne H\right]=ϵ\end{array}$$

(262)

where $0\le ϵ\le \frac{1}{2}$. We thus have both CSIT@R and CSIR@T. To compute $I(X;Y|S_R)$ in (230), we write (231) and (232) as

$$\begin{array}{cc}\hfill p_{Y|S_R,X}(y|0,x)& =\overline{ϵ}\frac{e^{-{\left|y\right|}^2}}{\pi }+ϵ\frac{e^{-{\left|y-\sqrt{2}x(0)\right|}^2}}{\pi }\hfill \\ \hfill p_{Y|S_R,X}(y|\sqrt{2},x)& =\overline{ϵ}\frac{e^{-{\left|y-\sqrt{2}x\left(\sqrt{2}\right)\right|}^2}}{\pi }+ϵ\frac{e^{-{\left|y\right|}^2}}{\pi }\hfill \\ \hfill p_{Y|S_R}(y|0)& =\overline{ϵ}\frac{e^{-{\left|y\right|}^2}}{\pi }+ϵ\frac{e^{-{\left|y\right|}^2/[1+2P(0)]}}{\pi [1+2P(0)]}\hfill \\ \hfill p_{Y|S_R}(y|\sqrt{2})& =\overline{ϵ}\frac{e^{-{\left|y\right|}^2/\left[1+2P\left(\sqrt{2}\right)\right]}}{\pi \left[1+2P\left(\sqrt{2}\right)\right]}+ϵ\frac{e^{-{\left|y\right|}^2}}{\pi }\hfill \end{array}$$

where $X(s_T)$ is CSCG. We choose the transmit powers $P(0)$ and $P\left(\sqrt{2}\right)$ as in (250) to compare with the best CSIR. Figure 6 shows the resulting rates for $ϵ=0.1$ as the curve labeled “Partial CSIR, $I(X;Y|S_R)$”. Observe that at high SNR, the curve seems to approach the best CSIR curve from Figure 4 with $S_R=H\sqrt{P(S_T)}$. We prove this by studying a forward model GMI with $K=2$.

The reverse model GMI requires $\mathrm{Var}\left[U|Y,S_R\right]$, which can be computed by simulation; see Appendix C.4. However, optimizing the powers seems difficult. We instead focus on the forward model GMI of Theorem 3 for which we compute

$$\begin{array}{cc}\hfill & \tilde{g}(0)=2{ϵ}^2,\tilde{g}\left(\sqrt{2}\right)=2{\overline{ϵ}}^2,{\tilde{\sigma }}^2(0)={\tilde{\sigma }}^2\left(\sqrt{2}\right)=2ϵ\overline{ϵ}\hfill \end{array}$$

and therefore (237) is

$$\begin{array}{c}\hfill I_1(X;Y|S_R)=\frac{1}{2}log\left(1+\frac{2{ϵ}^{2}P(0)}{1+2ϵ\overline{ϵ}P(0)}\right)+\frac{1}{2}log\left(1+\frac{2{\overline{ϵ}}^{2}P\left(\sqrt{2}\right)}{1+2ϵ\overline{ϵ}P\left(\sqrt{2}\right)}\right).\end{array}$$

(263)

For CSIR@T, the optimal power control policy is given by the quadratic waterfilling specified by (239) or (245):

$$\begin{array}{cc}\hfill P(0)& =\frac{1+\overline{ϵ}}{4ϵ\overline{ϵ}}\left[\sqrt{1+8ϵ\overline{ϵ}{\left(\frac{1}{\lambda }-\frac{1}{2{ϵ}^2}\right)}^{+}\frac{ϵ}{{(1+\overline{ϵ})}^2}}-1\right]\hfill \\ \hfill P\left(\sqrt{2}\right)& =\frac{1+ϵ}{4ϵ\overline{ϵ}}\left[\sqrt{1+8ϵ\overline{ϵ}{\left(\frac{1}{\lambda }-\frac{1}{2{\overline{ϵ}}^2}\right)}^{+}\frac{\overline{ϵ}}{{(1+ϵ)}^2}}-1\right].\hfill \end{array}$$

The rates are shown in Figure 6 as the curve labeled “Partial CSIR, GMI, K = 1”. Observe that at high SNR the GMI (263) saturates at

$$\begin{array}{c}\hfill \frac{1}{2}log\left(1+\frac{ϵ}{\overline{ϵ}}\right)+\frac{1}{2}log\left(1+\frac{\overline{ϵ}}{ϵ}\right).\end{array}$$

(264)

For example, for $ϵ=0.1$, we approach $1.74$ bits at high SNR. On the other hand, at low SNR, the rate is maximized with $P(0)=0$ and $P\left(\sqrt{2}\right)=2P$ so that $I_1(X;Y|S_R)\approx 2{\overline{ϵ}}^{2}P$. We thus achieve a fraction of ${\overline{ϵ}}^2$ of the power compared to full CSIT. For example, if $ϵ=0.1$, the minimal $E_b/N_0$ is approximately $-3.69$ dB.

Figure 6 also shows the conventional waterfilling rates as the curve labeled “Partial CSIR, GMI, c-waterfill”. These rates are almost the same as the quadratic waterfilling rates except for the range of $E_b/N_0$ between 9 to 13 dB shown in the inset.

To improve the auxiliary model at high SNR, we use a $K=2$ GMI with (see Remark 70)

$$\begin{array}{c}\hfill h_1(s_R)=0,h_2(s_R)=\sqrt{2},{\sigma }_1^2(s_R)={\sigma }_2^2(s_R)=1\end{array}$$

for $s_R=0,\sqrt{2}$. The receiver chooses $\overline{X}(s_R)=\sqrt{P(s_R)}U$ (see Remark 41) and we have (see Remark 42)

$$\begin{array}{cc}\hfill I_1(X;Y|S_R)& =\frac{1}{2}\mathrm{Pr}\left[{\mathcal{E}}_2|S_R=0\right]\left\{log\left(1+2P(0)\right)+\frac{\mathrm{E}\left[\left.{\left|Y\right|}^2\right|{\mathcal{E}}_2,S_R=0\right]}{1+2P(0)}\right.\hfill \\ \hfill & \left.-\mathrm{E}\left[\left.|Y-\sqrt{2}X(0){|}^2\right|{\mathcal{E}}_2,S_R=0\right]\right\}\hfill \\ \hfill & +\frac{1}{2}\mathrm{Pr}\left[{\mathcal{E}}_2|S_R=\sqrt{2}\right]\left\{log\left(1+2P\left(\sqrt{2}\right)\right)+\frac{\mathrm{E}\left[\left.{\left|Y\right|}^2\right|{\mathcal{E}}_2,S_R=\sqrt{2}\right]}{1+2P\left(\sqrt{2}\right)}\right.\hfill \\ \hfill & \left.-\mathrm{E}\left[\left.|Y-\sqrt{2}X\left(\sqrt{2}\right){|}^2\right|{\mathcal{E}}_2,S_R=\sqrt{2}\right]\right\}\hfill \end{array}$$

(265)

where the $X(s_T)$, $s_T\in {\mathcal{S}}_T$, are given by (122). We consider $P(0)$ and $P\left(\sqrt{2}\right)$ that scale in proportion to P. In this case, Appendix B.3 shows that choosing $t_R=P^{{\lambda }_R}$ where $0<{\lambda }_R<1$ gives the (best) full-CSIR capacity for large P, which is the rate specified in (249):

$$\begin{array}{cc}\hfill \underset{P\to \infty }{lim}& \left[\frac{ϵ}{2}log\left(1+2P(0)\right)+\frac{\overline{ϵ}}{2}log\left(1+2P\left(\sqrt{2}\right)\right)-I_1(X;Y|S_R)\right]=0.\hfill \end{array}$$

(266)

In other words, by optimizing $P(0)$ and $P\left(\sqrt{2}\right)$, at high SNR the $K=2$ GMI can approach the capacity of Proposition 2. This is expected since the receiver can estimate $H\sqrt{P(S_T)}$ reliably at high SNR.

Figure 6 shows the behavior of this GMI and $t_R=P^{0.4}$, and where we have chosen $P(0)$ and $P\left(\sqrt{2}\right)$ according to (250). The abrupt change in slope at approximately 2.5 dB is because $P(0)$ becomes positive beyond this $E_b/N_0$. Keeping $P(0)=0$ for $E_b/N_0$ up to about 12 dB gives better rates, but for high SNR one should choose the powers according to (250).

8. Rayleigh Fading

Rayleigh fading has $H\sim \mathcal{CN}(0,1)$. The random variable $G={\left|H\right|}^2$ thus has the density $p(g)=e^{-g}·1(g\ge 0)$. Section 8.1 and Section 8.2 review known results.

8.1. No CSIR, No CSIT

Suppose $S_R=S_T=0$ and $X\sim \mathcal{CN}(0,P)$. The densities to compute $I(X;Y)$ for CSCG X are

$$\begin{array}{cc}\hfill p(y|x)& =\frac{e^{-{\left|y\right|}^2/{(|x|}^2+1)}}{\pi (|x{|}^2+1)}\hfill \end{array}$$

(267)

$$\begin{array}{cc}\hfill p(y)& ={\int }_0^{\infty }\frac{e^{-g/P}}{P}\frac{e^{-{\left|y\right|}^2/(g+1)}}{\pi (g+1)}dg.\hfill \end{array}$$

(268)

The minimum $E_b/N_0$ is approximately 9.2 dB, and the forward model GMI (174) is zero. The capacity is achieved by discrete and finite X [96], and at large SNR, the capacity behaves as $loglogP$ [97]. Further results are derived in [98,99,100,101,102].

8.2. Full CSIR, CSIT@ R

The capacity (175) for $B=0$ (no CSIT) is

$$\begin{array}{c}\hfill C(P)={\int }_0^{\infty }{e}^{-g}log\left(1+gP\right)dg=e^{1/P}{E}_1\left(1/P\right)log(e)\end{array}$$

(269)

where the exponential integral $E_1(.)$ is given by (A4) below. The wideband values are given by (177):

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=log2,S=1.\end{array}$$

The minimal $E_b/N_0$ is $-1.59$ dB, but the fading reduces the capacity slope. At high SNR, we have

$$\begin{array}{c}\hfill C(P)\approx log(P)-\gamma \end{array}$$

where $\gamma \approx 0.57721$ is Euler’s constant. The capacity thus behaves as for the case without fading but with an SNR loss of approximately 2.5 dB.

The capacity (182) with $B=\infty$ (or $S_T=G$) is (see ([95], Equation (7)))

$$\begin{array}{cc}\hfill C(P)& ={\int }_{\lambda }^{\infty }{e}^{-g}log\left(g/\lambda \right)dg=E_1(\lambda ).\hfill \end{array}$$

(270)

where $P(g)$ is given by (181) and $\lambda$ is chosen so that

$$\begin{array}{cc}\hfill P& ={\int }_{\lambda }^{\infty }{e}^{-g}P(g)dg=\frac{e^{-\lambda }}{\lambda }-E_1(\lambda ).\hfill \end{array}$$

At low SNR we have large $\lambda$ and using the approximation (A7) below we compute

$$\begin{array}{c}\hfill C(P)\approx e^{-\lambda }/\lambda \mathrm{and}P\approx e^{-\lambda }/{\lambda }^2.\end{array}$$

(271)

We thus have $E_b/N_0\approx log(2)/\lambda$ and the minimal $E_b/N_0$ is $-\infty$.

Consider now $B=1$ for which $P_{S_T}(3\Delta /2)=e^{-\Delta }$ and

$$\begin{array}{cc}\hfill & \mathrm{E}\left[G|G\ge \Delta \right]=1+\Delta \hfill \end{array}$$

(272)

$$\begin{array}{cc}\hfill & \mathrm{E}\left[G^2|G\ge \Delta \right]=2+2\Delta +{\Delta }^2.\hfill \end{array}$$

(273)

We thus have the wideband quantities in (186) and (187):

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{log2}{1+\Delta }\hfill \end{array}$$

(274)

$$\begin{array}{cc}\hfill & S=\frac{2e^{-\Delta }{(1+\Delta )}^2}{2+2\Delta +{\Delta }^2}.\hfill \end{array}$$

(275)

Figure 7 shows the capacities for $B=1$ and $\Delta =1,2,1/2$. The minimum $E_b/N_0$ value is

$$\begin{array}{c}\hfill -1.59\mathrm{dB}-10{log}_{10}\left(1+\Delta \right)\end{array}$$

(276)

and for $\Delta =1,2,1/2$ we gain 3 dB, 4.8 dB, 1.8 dB, respectively, over no CSIT at low power. Note that one bit of feedback allows one to approach the full CSIT rates closely.

Remark 71.

For the scalar channel (159), knowing H at both the transmitter and receiver provides significant gains at low SNR [73] but small gains at high SNR ([95], Figure 4) as compared to knowing H at the receiver only. Furthermore, the reliability can be improved ([78], Figures 5–7). Significant gains are also possible for MIMO channels.

Remark 72.

An alternative way to derive (272)–(275) is as follows. Define $\widehat{P}=P{e}^{\Delta }$ so for small P the capacity is

$$\begin{array}{cc}\hfill C(P)& ={\int }_{\Delta }^{\infty }{e}^{-g}log\left(1+g\widehat{P}\right)dg\hfill \\ \hfill & =e^{1/\widehat{P}}{E}_1\left(\frac{1}{\widehat{P}}+\Delta \right)+e^{-\Delta }log(1+\widehat{P}\Delta )\hfill \\ \hfill & \approx P(1+\Delta )-\frac{1}{2}{P}^2{e}^{\Delta }\left(2+2\Delta +{\Delta }^2\right).\hfill \end{array}$$

8.3. Full CSIR, Partial CSIT

Consider noisy CSIT with

$$\begin{array}{c}\hfill \mathrm{Pr}\left[S_T=1(G\ge \Delta )\right]=\overline{ϵ},\mathrm{Pr}\left[S_T\ne 1(G\ge \Delta )\right]=ϵ.\end{array}$$

We begin with the most informative CSIR.

$S_R=\sqrt{P(S_T)}H$: Proposition 2 gives the capacity

$$\begin{array}{cc}\hfill C(P)& ={\int }_0^{\infty }{e}^{-g}\sum _{s_T}P(s_T|g)log\left(1+gP(s_T)\right)dg\hfill \\ \hfill & ={\int }_0^{\Delta }{e}^{-g}\left[\overline{ϵ}log\left(1+gP(0)\right)+ϵlog\left(1+gP(1)\right)\right]dg\hfill \\ \hfill & +{\int }_{\Delta }^{\infty }{e}^{-g}\left[\overline{ϵ}log\left(1+gP(1)\right)+ϵlog\left(1+gP(0)\right)\right]dg.\hfill \end{array}$$

(277)

It remains to optimize $P(0)$, $P(1)$ and $\Delta$. The two equations for the Lagrange multiplier $\lambda$ are

$$\begin{array}{cc}\hfill \lambda ·P_{S_T}(0)& ={\int }_0^{\Delta }{e}^{-g}·{\displaystyle \frac{\overline{ϵ}g}{1+gP(0)}}dg+{\int }_{\Delta }^{\infty }{e}^{-g}·{\displaystyle \frac{ϵg}{1+gP(0)}}dg\hfill \end{array}$$

(278)

$$\begin{array}{cc}\hfill \lambda ·P_{S_T}(1)& ={\int }_0^{\Delta }{e}^{-g}·{\displaystyle \frac{ϵg}{1+gP(1)}}dg+{\int }_{\Delta }^{\infty }{e}^{-g}·{\displaystyle \frac{\overline{ϵ}g}{1+gP(1)}}dg\hfill \end{array}$$

(279)

where $P_{S_T}(0)=\overline{ϵ}-(\overline{ϵ}-ϵ)e^{-\Delta }$ and $P_{S_T}(1)=ϵ+(\overline{ϵ}-ϵ)e^{-\Delta }$. The rates are shown in Figure 8.

For fixed $\Delta$ and large P, we have $1/\lambda \approx P(0)\approx P(1)\approx P$ and approach the capacity (269) without CSIT. In contrast, for small P we may use similar steps as for (183) and (184). Observe the following for (278) and (279):

• both $P(0)$ and $P(1)$ decrease as $\lambda$ increases;
• the maximal $\lambda$ in (278) is obtained with $P(0)=0$; this value is

  $$\begin{array}{c}\hfill \mathrm{E}\left[G|S_T=0\right]=\frac{\overline{ϵ}-(\overline{ϵ}-ϵ)(1+\Delta )e^{-\Delta }}{P_{S_T}(0)}\end{array}$$

  (280)
• the maximal $\lambda$ in (279) is obtained with $P(1)=0$; this value is

  $$\begin{array}{c}\hfill \mathrm{E}\left[G|S_T=1\right]=\frac{ϵ+(\overline{ϵ}-ϵ)(1+\Delta )e^{-\Delta }}{P_{S_T}(1)}.\end{array}$$

  (281)

Thus, if $\mathrm{E}\left[G|S_T=0\right]<\mathrm{E}\left[G|S_T=1\right]$ and $0\le ϵ<1/2$, then for P below some threshold we have $P(0)=0$, $P(1)=P/P_{S_T}(1)$ and the capacity is

$$\begin{array}{c}\hfill C(P)={\int }_0^{\Delta }{e}^{-g}ϵlog\left(1+\frac{gP}{P_{S_T}(1)}\right)dg+{\int }_{\Delta }^{\infty }{e}^{-g}\overline{ϵ}log\left(1+\frac{gP}{P_{S_T}(1)}\right)dg.\end{array}$$

(282)

We compute $C^{\prime }(0)=\mathrm{E}\left[G|S_T=1\right]$ which is given by (281) so that $1\le C^{\prime }(0)\le 1+\Delta$, as expected from (274). For example, for $ϵ=0.1$ and $\Delta =1$ we have $C^{\prime }(0)\approx 1.75$ and therefore the minimal $E_b/N_0$ is approximately $-4.01$ dB.

The best $\Delta$ is the unique solution $\widehat{\Delta }$ of the equation

$$\begin{array}{c}\hfill e^{-\Delta }=\frac{ϵ}{\overline{ϵ}-ϵ}(\Delta -1)\end{array}$$

(283)

and the result is $C^{\prime }(0)=\widehat{\Delta }\ge 1$. We have the simple bounds

$$\begin{array}{c}\hfill 1+\frac{1}{2}log\left(\frac{1}{ϵ}-2\right)\le C^{\prime }(0)\le 1+\frac{1}{e}\left(\frac{1}{ϵ}-2\right)\end{array}$$

(284)

where the left inequality follows by taking logarithms and using $log(\Delta -1)\le \Delta -2$, and the right inequality follows by using $e^{-\Delta }\le e^{-1}$ in (283). For example, for $ϵ\to 0$ we have $C^{\prime }(0)\to \infty$, and for $ϵ\to 1/2$ we have $C^{\prime }(0)\to 1$.

$S_R=H$: For the less informative CSIR, one may use (191) and (193) to compute $I(A;Y|H)$. The reverse model GMI requires $\mathrm{Var}\left[U|Y,S_R\right]$, which can be computed by simulation; see Appendix C.2. Again, however, optimizing the powers seems difficult. We instead focus on the forward model GMI of Corollary 1, which is

$$\begin{array}{c}\hfill I_1(A;Y|H)={\int }_0^{\infty }{e}^{-g}log\left(1+\mathrm{SNR}(g)\right)dg\end{array}$$

(285)

where

$$\begin{array}{c}\hfill \mathrm{SNR}(g)=\frac{g{\tilde{P}}_T(g)}{1+gϵ\overline{ϵ}{\left(\sqrt{P(0)}-\sqrt{P(1)}\right)}^2}\end{array}$$

(286)

and

$$\begin{array}{cc}\hfill {\tilde{P}}_T(g)& =\left\{\begin{array}{cc}{\left(\overline{ϵ}\sqrt{P(0)}+ϵ\sqrt{P(1)}\right)}^2,\hfill & g<\Delta \hfill \\ {\left(ϵ\sqrt{P(0)}+\overline{ϵ}\sqrt{P(1)}\right)}^2,\hfill & g\ge \Delta .\hfill \end{array}\right.\hfill \end{array}$$

(287)

It remains to optimize $P(0)$, $P(1)$ and $\Delta$. Computing the derivatives seems complicated, so we use numerical optimization for fixed $\Delta =1$ as in Figure 8. The results are shown in Figure 9. For fixed $\Delta$ and large P, it is best to choose $P(0)\approx P(1)$ so that $SNR(g)\approx gP$ and we approach the rate of no CSIT. For small P, however, the best $P(0)$ is no longer zero and $C^{\prime }(0)$ is smaller than (281).

8.4. Partial CSIR, Full CSIT

Consider $S_T=H$ and suppose we choose the $X(h)$ to be jointly CSCG with variances $\mathrm{E}\left[{\left|X(h)\right|}^2\right]=P(h)$ and correlation coefficients

$$\begin{array}{c}\hfill \rho (h,h^{\prime })=\frac{\mathrm{E}\left[X(h)X{(h^{\prime })}^{*}\right]}{\sqrt{P(h)P(h^{\prime })}}\end{array}$$

and where $\mathrm{E}\left[P(H)\right]\le P$. We then have

$$\begin{array}{c}\hfill p(y|s_R)={\int }_{\mathbb{C}}p(h|s_R)\frac{e^{-{\left|y\right|}^2/{(|h|}^{2}P(h)+1)}}{\pi (|h{|}^{2}P(h)+1)}dh.\end{array}$$

As in (97), $p(y|s_R)$ and $h(Y|S_R)$ depend only on the marginals of A and not on the $\rho (h,h^{\prime })$. We thus have the problem of finding the $\rho (h,h^{\prime })$ that minimize

$$\begin{array}{c}\hfill h(Y|A,S_R)={\int }_{\mathcal{A}}p(a)h(Y|S_R,A=a)da.\end{array}$$

We will use fully-correlated $X(h)$ as discussed in Section 6.5. We again consider $S_R=0$ and $S_R=1(G\ge t)$.

$S_R=0$: For the heuristic policies, the power (206) is

$$\begin{array}{c}\hfill \widehat{P}=\frac{P}{\Gamma \left(1+a,t\right)}\end{array}$$

(288)

and the rate (219) is

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{P\Gamma {\left(\frac{3+a}{2},t\right)}^2}{\Gamma \left(1+a,t\right)+P\left[\Gamma \left(2+a,t\right)-\Gamma {\left(\frac{3+a}{2},t\right)}^2\right]}\right)\end{array}$$

(289)

where $\Gamma (s,x)$ is the upper incomplete gamma function; see Appendix A.3. Moreover, the expression (220) is

$$\begin{array}{cc}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }& =\frac{\Gamma \left(1+a,t\right)}{\Gamma {\left(\frac{3+a}{2},t\right)}^2}·log2.\hfill \end{array}$$

(290)

We remark that $\Gamma (s,0)=\Gamma (s)$ where $\Gamma (x)$ is the gamma function. We further have

$$\begin{array}{c}\hfill \begin{array}{cc}\Gamma (0,t)=E_1(t),\hfill & \Gamma (1,t)=e^{-t},\hfill \\ \Gamma (2,t)=e^{-t}(t+1),\hfill & \Gamma (3,t)=e^{-t}(t^2+2t+2).\hfill \end{array}\end{array}$$

For example, the TCP policy ($a=0$) has $\widehat{P}=P{e}^t$. At low SNR, it turns out that the best choice is $t=0.283$ for which we have $\Gamma (1,t)/\Gamma {(3/2,t)}^2\approx 1.174$. The minimum $E_b/N_0$ in (222) is thus $-0.90$ dB. At high SNR, the best choice is $t=0$ so that (289) with $\Gamma (3/2,0)=\Gamma (3/2)=\sqrt{\pi }/2$ gives

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{P\pi /4}{1+P(1-\pi /4)}\right).\end{array}$$

(291)

The TCP rate thus saturates at $2.22$ bits per channel use; see the curve labeled “TCP, GMI, K = 1” in Figure 10.

The TMF policy ($a=1$) has $\widehat{P}=P{e}^t/(t+1)$. The best choice is $t=0$ for which we have $\Gamma (2)=1$ and $\Gamma (3)=2$ and therefore (289) is

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{P}{1+P}\right).\end{array}$$

(292)

The minimum $E_b/N_0$ in (222) is $-1.59$ dB, and at high SNR, the TMF rate saturates at 1 bit per channel use. The rates are shown as the curve labeled “TMF, GMI, K = 1” in Figure 10.

The TCI policy ($a=-1$) has $\widehat{P}=P/E_1(t)$ and using $\Gamma (0,t)=E_1(t)$ and $\Gamma (1,t)=e^{-t}$ gives

$$\begin{array}{c}\hfill I_1(A;Y)=log\left(1+\frac{P}{e^{2t}{E}_1(t)+P\left(e^t-1\right)}\right).\end{array}$$

(293)

The minimum $E_b/N_0$ in (290) is

$$\begin{array}{c}\hfill {\left.\frac{E_b}{N_0}\right|}_{\min }=E_1(t)e^{2t}·log2.\end{array}$$

(294)

Optimizing over t by taking derivatives (see (A5) below), the best t satisfies the equation $2t{e}^t{E}_1(t)=1$ which gives $t\approx 0.61$ and the minimal $E_b/N_0$ is approximately 0.194 dB. On the other hand, for large SNR, we may choose $t=1/P$ and using $E_1(t)\approx log(1/t)$ for small t gives

$$\begin{array}{c}\hfill I_1(A;Y)\approx log\left(1+\frac{P}{1+logP}\right).\end{array}$$

Since the pre-log is at most 1, the capacity grows with pre-log 1 for large P. We see that TMF is best at small P while TCI is best at large P. The rates are shown as the curve labeled “TCI, GMI, K = 1” in Figure 10.

The simple channel output of TCI permits further analysis. Using Remark 65, we compute the mutual information $I(A;Y)$ by numerical integration; see the curve labeled “TCI, $I(A;Y)$” in Figure 10. We see that at high SNR, the TCI mutual information is larger than the GMI for TCP, TMF, and (of course) TCI. Moreover, as we show, the TCI mutual information can work well at low SNR.

Motivated by Section 7.3 and Figure 5, we again use the GMI (154) with $K=2$ and (65). We further choose $h_1=0$, ${\sigma }_1^2={\sigma }_2^2=1$, and

$$\begin{array}{c}\hfill \overline{X}=\frac{\sqrt{\widehat{P}}}{h_2}U,U\sim \mathcal{CN}(0,1).\end{array}$$

The expression (154) simplifies to

$$\begin{array}{c}\hfill I_1(A;Y)=\mathrm{Pr}\left[{\mathcal{E}}_2\right]\left[log\left(1+\widehat{P}\right)+\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+\widehat{P}}-\mathrm{E}\left[\left.{\left|Y-\sqrt{\widehat{P}}U\right|}^2\right|{\mathcal{E}}_2\right]\right].\end{array}$$

(295)

The GMI (295) exhibits interesting high and low SNR scaling by choosing the following thresholds $t,t_R$.

• For high SNR, we choose

  $$\begin{array}{c}\hfill t=P^{-\lambda }\mathrm{and}{t}_R={\widehat{P}}^{{\lambda }_R}\end{array}$$

  (296)

  where $0<\lambda <1$ and $0<{\lambda }_R<1$. As P increases, t decreases and Appendix B.4 shows that

  $$\begin{array}{c}\hfill \mathrm{Pr}\left[{\mathcal{E}}_2\right]\to 1,\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+\widehat{P}}\to 1,\mathrm{E}\left[{\left|Y-\sqrt{\widehat{P}}U\right|}^2|{\mathcal{E}}_2\right]\to 1.\end{array}$$

  (297)
  Inserting $\widehat{P}=P/E_1(t)$, we thus have

  $$\begin{array}{c}\hfill \underset{P\to \infty }{lim}\left[I_1(A;Y)-log\left(1+\frac{P}{E_1(t)}\right)\right]=0.\end{array}$$

  (298)
  We further have $E_1(t)\approx \lambda logP$ by using (A6) in Appendix A.2, and the high-SNR slope of the GMI matches the slope of $logP$ but the additive gap to $logP$ increases. The high SNR rates are shown as the curve labeled “TCI, GMI, K = 2” in Figure 10 for $\lambda ={\lambda }_R=0.4$.
• For low SNR, we choose

  $$\begin{array}{c}\hfill t=-log(P/c)\mathrm{and}{t}_R=\widehat{P}\end{array}$$

  (299)

  for a constant $c>0$. As P decreases, both t and $\widehat{P}=P/E_1(t)$ increase and Appendix B.4 shows that

  $$\begin{array}{c}\hfill \mathrm{Pr}\left[{\mathcal{E}}_2\right]\approx e^{-t-1},\frac{\mathrm{E}\left[{\left|Y\right|}^2|{\mathcal{E}}_2\right]}{1+2\widehat{P}}\to 1,\mathrm{E}\left[{\left|Y-\sqrt{\widehat{P}}U\right|}^2|{\mathcal{E}}_2\right]\to 1.\end{array}$$

  (300)
  Using (A7), we have $I_1(A;Y)\approx e^{-t-1}logt$ which vanishes as t grows. But we also have

  $$\begin{array}{cc}\hfill \frac{E_b}{N_0}=\frac{P}{R}log2& \approx \frac{c{e}^{-t}log2}{e^{-t-1}logt}\approx \frac{celog2}{log(-logP)}\hfill \end{array}$$

  (301)

  which decreases (very slowly) as P decreases. The minimal $E_b/N_0$ is therefore $-\infty$. The low SNR rates are shown as the curve labeled “TCI, GMI, K = 2” in Figure 11 for $c=1.4$.

Figure 11. Low-SNR rates for Rayleigh fading with $S_T=H$ and $S_R=0$. The threshold t was optimized for the $K=1$ curves, while $t=-log(P/1.4)$ for the $I(A;Y)$, rGMI, and $K=2$ curves. The $K=2$ GMI uses $t_R=\widehat{P}$. The TMF and TMMSE GMIs are indistinguishable for this range of rates.

Figure 11. Low-SNR rates for Rayleigh fading with $S_T=H$ and $S_R=0$. The threshold t was optimized for the $K=1$ curves, while $t=-log(P/1.4)$ for the $I(A;Y)$, rGMI, and $K=2$ curves. The $K=2$ GMI uses $t_R=\widehat{P}$. The TMF and TMMSE GMIs are indistinguishable for this range of rates.

Figure 11 shows that the TCI mutual information achieves a minimal $E_b/N_0$ below $-1.59$ dB. At $E_b/N_0=-2$ dB, we computed $I_1(A;Y)\approx 6\times {10}^{-7}$ and $I(A;Y)\approx 3\times {10}^{-4}$. The $K=2$ partition is thus useful to prove that TCI can achieve $E_b/N_0$ arbitrarily close to zero. Figure 11 also shows the reverse model GMI as the curve labeled “TCI, rGMI” which has the rate $I_1(A;Y)\approx 8\times {10}^{-6}$ at $E_b/N_0=-2$ dB.

We compare the full CSIR and full CSIT rates. At high SNR, the GMI for $S_R=0$ achieves the same capacity pre-log as $S_R=H$. At low SNR, recall from (271) that with full CSIR/CSIT we have $E_b/N_0\approx log(2)/\lambda$. To compare the rates for similar $E_b/N_0$, we set $\lambda =logt$, where t is as in (299) and $c\approx 1$. The TCI $K=2$ GMI without CSIR is approximately $e^{-t}logt$ while the full CSIR rate (271) is approximately $e^{-\lambda }/\lambda \approx 1/(tlog(t))$. Thus, the $K=2$ GMI with no CSIR is a fraction $t{e}^{-t}log{(t)}^2$ of the full CSIR capacity.

$S_R=1(G\ge t)$: The power in (206) is again (288) and the rate (224) is

$$\begin{array}{c}\hfill I_1(A;Y|S_R)=e^{-t}·log\left(1+\frac{P{e}^{2t}\Gamma {\left(\frac{3+a}{2},t\right)}^2}{\Gamma \left(1+a,t\right)+P\left[e^t\Gamma \left(2+a,t\right)-e^{2t}\Gamma {\left(\frac{3+a}{2},t\right)}^2\right]}\right).\end{array}$$

(302)

Moreover, the expression (225) is

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{\Gamma \left(1+a,t\right)}{e^t·\Gamma {\left(\frac{3+a}{2},t\right)}^2}·log2\hfill \end{array}$$

(303)

which is the same as (290) except for the factor $e^t$ in the denominator. This implies that the minimal $E_b/N_0$ can be improved for $t>0$.

The TCP, TMF, and TCI rates (302) are the respective

$$\begin{array}{cc}\hfill & I_1(A;Y|S_R)=e^{-t}log\left(1+\frac{P{e}^{2t}\Gamma {\left(\frac{3}{2},t\right)}^2}{e^{-t}+P\left[t+1-e^{2t}\Gamma {\left(\frac{3}{2},t\right)}^2\right]}\right)\hfill \end{array}$$

(304)

$$\begin{array}{cc}\hfill & I_1(A;Y|S_R)=e^{-t}log\left(1+\frac{P{(t+1)}^2}{e^{-t}(t+1)+P}\right)\hfill \end{array}$$

(305)

$$\begin{array}{cc}\hfill & I_1(A;Y|S_R)=e^{-t}log\left(1+\frac{P}{E_1(t)}\right).\hfill \end{array}$$

(306)

Remark 73.

As pointed out in Remark 68, the TCI GMI (306) is $I(A;Y|S_R)$. One can also understand this by observing that the receiver knows $\sqrt{GP(G)}$ for all G. The mutual information is thus related to the rate (189) of Proposition 2.

The minimal $E_b/N_0$ in (303) are the respective

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{1}{e^{2t}·\Gamma {\left(\frac{3}{2},t\right)}^2}·log2\hfill \end{array}$$

(307)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=\frac{1}{t+1}·log2\hfill \end{array}$$

(308)

$$\begin{array}{cc}\hfill & {\left.\frac{E_b}{N_0}\right|}_{\min }=e^t{E}_1(t)·log2.\hfill \end{array}$$

(309)

The above expressions mean that, for all three policies, we can make the minimal $E_b/N_0$ as small as desired by increasing t. For example, for TCI, we can bound (see (A9) below)

$$\begin{array}{c}\hfill \frac{1}{t+1}<e^t{E}_1(t)<\frac{1}{t}.\end{array}$$

(310)

TCI thus has a slightly larger (slightly worse) minimal $E_b/N_0$ than TMF for the same t, as discussed after (212).

For large P, the TCP rate (304) is optimized by $t\approx 0.163$ and the rate saturates at $\approx 2.35$ bits per channel use. The TMF rate (305) is optimized with $t=0$, and the rate saturates at 1 bit per channel use. For the TCI rate (306), we again choose $t=1/P$ and use $E_1(t)\approx log(1/t)$ for small t to show that the capacity grows with pre-log 1:

$$\begin{array}{c}\hfill I_1(A;Y|S_R)\approx log\left(1+\frac{P}{logP}\right).\end{array}$$

Again, TMF is best at small P while TCI is best at large P.

Remark 74.

Comparing (298) and (306), the $S_R=0$, $K=2$, TCI GMI in (295) approaches the $S_R=1(G\ge t)$ mutual information $I(A;Y|S_R)$ in (306) at high SNR.

Optimal Policy: Consider now the optimal power control policy. Suppose first that $S_R=0$ for which Theorem 2 gives the TMMSE policy with $t=0$:

$$\begin{array}{cc}\hfill \sqrt{P(h)}& =\frac{\alpha \left|h\right|}{\beta +{\left|h\right|}^2}.\hfill \end{array}$$

(311)

For Rayleigh fading, we thus have (see (A13) below)

$$\begin{array}{c}\hfill P={\int }_0^{\infty }{e}^{-g}\frac{{\alpha }^{2}g}{{(\beta +g)}^2}dg={\alpha }^2\left[(\beta +1)e^{\beta }{E}_1(\beta )-1\right]\end{array}$$

(312)

with the two expressions (see (A12) and (A14) below)

$$\begin{array}{cc}\hfill \tilde{P}& ={\int }_0^{\infty }{e}^{-g}\frac{{\alpha }^{2}g}{\beta +g}dg={\alpha }^2{\left[1-\beta e^{\beta }{E}_1(\beta )\right]}^2\hfill \end{array}$$

(313)

$$\begin{array}{cc}\hfill \mathrm{E}\left[GP(H)\right]& ={\int }_0^{\infty }{e}^{-g}\frac{{\alpha }^2{g}^2}{{(\beta +g)}^2}dg={\alpha }^2\left[1+\beta -\beta (\beta +2)e^{\beta }{E}_1(\beta )\right].\hfill \end{array}$$

(314)

Given P and $\beta$, we may compute ${\alpha }^2$ from (312). We then search for the optimal $\beta$ for fixed P. The rates are shown as the curve labeled “TMMSE, GMI, K = 1” in Figure 10 and Figure 11 and we see that the TMMSE strategy has the best $K=1$ rates.

Consider next $S_R=1(G\ge t)$ and the TMMSE policy. We compute (see (A13) below)

$$\begin{array}{c}\hfill P={\int }_t^{\infty }{e}^{-g}\frac{{\alpha }^{2}g}{{(\beta +g)}^2}dg={\alpha }^2\left[(\beta +1)e^{\beta }{E}_1(t+\beta )-e^{-t}\frac{\beta }{t+\beta }\right]\end{array}$$

(315)

and (see (A12) and (A14) below)

$$\begin{array}{cc}\hfill \sqrt{\tilde{P}(1)}& ={\int }_t^{\infty }\frac{e^{-g}}{e^{-t}}\frac{\alpha g}{\beta +g}dg=\alpha \left[1-\beta e^{t+\beta }{E}_1(t+\beta )\right]\hfill \end{array}$$

(316)

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\left|Y\right|}^2|S_R=1\right]& ={\int }_t^{\infty }\frac{e^{-g}}{e^{-t}}\left(1+\frac{{\alpha }^2{g}^2}{{(\beta +g)}^2}\right)dg\hfill \\ \hfill & =1+{\alpha }^2\left[1+\frac{{\beta }^2}{t+\beta }-\beta (\beta +2)e^{t+\beta }{E}_1(t+\beta )\right].\hfill \end{array}$$

(317)

We optimize as for the $S_R=0$ case: given P, $\beta$, t, we compute ${\alpha }^2$ from (315). We then search for the optimal $\beta$ for fixed P and t. The optimal t is approximately a factor of 1.1 smaller than for the TCI policy. The rates are shown in Figure 12 as the curve labeled “TMMSE, GMI”.

8.5. Partial CSIR, CSIT@ R

Suppose $S_R$ is defined by (see (172))

$$\begin{array}{c}\hfill H=\sqrt{\overline{ϵ}}{S}_R+\sqrt{ϵ}{Z}_R\end{array}$$

where $0\le ϵ\le 1$ and $S_R,Z_R$ are independent with distribution $\mathcal{CN}(0,1)$. We further consider the CSIT $S_T={\left|S_R\right|}^2$.

The reverse model GMI again requires $\mathrm{Var}\left[U|Y,S_R\right]$, which can be computed by simulation; see Appendix C.4. However, as in Section 7.4 and Section 8.3, optimizing the powers seems difficult, and we instead focus on forward models. The expressions (235) and (236) are

$$\begin{array}{c}\hfill \tilde{g}(s_R)=\overline{ϵ}{s}_T,{\tilde{\sigma }}^2(s_R)=ϵ.\end{array}$$

(318)

The GMI (237) of Theorem 3 is

$$\begin{array}{cc}\hfill I_1(X;Y|S_R)& ={\int }_{\lambda /\overline{ϵ}}^{\infty }{e}^{-s_T}log\left(1+\frac{\overline{ϵ}{s}_{T}P(s_T)}{1+ϵP(s_T)}\right)d{s}_T\hfill \end{array}$$

(319)

where the power control policy $P(s_T)$ is given by (245). The parameter $\lambda$ is chosen so that $\mathrm{E}\left[P(S_T)\right]=P$. For example, for $ϵ\to 0$ we recover the waterfilling solution (181). Figure 13 shows the quadratic and conventional waterfilling rates, which lie almost on top of each other. For example, the inset shows the rates for $ϵ=0.2$ and a small range of $E_b/N_0$.

9. Channels with In-Block Feedback

This section generalizes Shannon’s model described in Section 4.1 to include block fading with in-block feedback. For example, the model lets one include delay in the CSIT and permits many other generalizations for network models [22].

9.1. Model and Capacity

The problem is specified by the FDG in Figure 14. The model has a message M, and the channel input and output strings

$$\begin{array}{c}\hfill X_i^L=(X_{i1},\dots ,X_{iL}),Y_i^L=(Y_{i1},\dots ,Y_{iL})\end{array}$$

for blocks $i=1,\dots ,n$. The channel is specified by a string $S_H^n=(S_{H1},\dots ,S_{Hn})$ of i.i.d. hidden channel states. The CSIR $S_{Ri\ell }$ is a (possibly noisy) function of $S_{Hi}$ for all i and ℓ. The receiver sees the channel outputs (see (159))

$$\begin{array}{cc}\hfill & (Y_{i\ell },S_{Ri\ell })=\left(f_{\ell }\left(X_i^{\ell },S_{Hi},Z_i^L\right),S_{Ri\ell }\right)\hfill \end{array}$$

(320)

for some functions $f_{\ell }(·)$, $\ell =1,\dots ,L$. Observe that the $X_i^{\ell }$ influence the $Y_{i\ell }$ in a causal fashion. The random variables $M,S_{H1},\dots ,S_{Hn},Z_1^L,\dots ,Z_n^L$ are mutually independent.

We now permit past channel symbols to influence the CSIT; see Section 1.2. Suppose the CSIT has the form

$$\begin{array}{c}\hfill S_{Ti\ell }=f_{T\ell }\left(S_{Hi},X_i^{\ell -1},Y_i^{\ell -1}\right)\end{array}$$

(321)

for some function $f_{T\ell }(.)$ and for all i and ℓ. The motivation for (321) is that useful CSIR may not be available until the end of a block or even much later. In the meantime, the receiver can, e.g., quantize the $Y_i^{\ell -1}$ and transmit the quantization bits via feedback. This lets one study fast power control and beamforming without precise knowledge of the channel coefficients.

Define the string of past and current states as

$$\begin{array}{c}\hfill s_T^{i\ell }=\left(s_{T1}^L,\dots ,s_{T(i-1)}^L,s_{Ti}^{\ell }\right).\end{array}$$

(322)

The channel input at time $i\ell$ is $X(s_T^{i\ell })$ and the adaptive codeword $A^{nL}$ is defined by the ordered lists

$$\begin{array}{cc}\hfill A_{i\ell }& =\left[X(s_T^{i\ell }),\forall s_T^{i\ell }\right]\hfill \end{array}$$

(323)

for $1\le i\le n$ and $1\le \ell \le L$. The adaptive codeword $A^{nL}$ is a function of M and is thus independent of $S_H^n$ and $S_R^{nL}$.

The model under consideration is a special case of the channels introduced in ([22], Section V). However, the model in [22] has transmission and reception begin at time $\ell =2$ rather than $\ell =1$. To compare the theory, one must thus shift the time indexes by 1 unit and increase L to $L+1$. The capacity for our model is given by ([22], Theorem 2) which we write as

$$\begin{array}{c}\hfill C\stackrel{(a)}{=}\underset{A^L}{max}\frac{1}{L}I(A^L;Y^L,S_R^L)\stackrel{(b)}{=}\underset{A^L}{max}\frac{1}{L}I(A^L;Y^L|S_R^L).\end{array}$$

(324)

where $(a)$ follows by normalizing by L rather than $L+1$, and step $(b)$ follows by the independence of $A^L$ and $S_R^L$.

9.2. GMI for Scalar Channels

We will study scalar block fading channels; extensions to vector channels follow as described in Section 4.4. Let $\underline{Y}={[Y_1,\dots ,Y_L]}^T$ be the vector form of $Y^L$ and similarly for other strings with L symbols. The GMI with parameter s is

$$\begin{array}{c}\hfill I_s(A^L;Y^L|S_R^L)=\mathrm{E}\left[log\frac{q{(\underline{Y}|\underline{A},{\underline{S}}_R)}^s}{q(\underline{Y}|{\underline{S}}_R)}\right]\end{array}$$

(325)

Reverse Model: For the reverse model, let $\underline{A}$ be a column vector that stacks the $X_{\ell }(s_T^{\ell })$ for all $s_T^{\ell }$ and ℓ. Consider a reverse density as in (105):

$$\begin{array}{c}\hfill q\left(a^L|y^L\right)=\frac{exp\left(-\underline{z}{(\underline{y},{\underline{s}}_R)}^{†}{\mathbf{Q}}_{\underline{A}|\underline{Y}=\underline{y},{\underline{S}}_R={\underline{s}}_R}^{-1}\underline{z}(\underline{y},{\underline{s}}_R)\right)}{{\pi }^{N}det{\mathbf{Q}}_{\underline{A}|\underline{Y}=\underline{y},{\underline{S}}_R={\underline{s}}_R}}\end{array}$$

where

$$\begin{array}{c}\hfill \underline{z}(\underline{y},{\underline{s}}_R)=\underline{a}-\mathrm{E}\left[\underline{A}|\underline{Y}=\underline{y},{\underline{S}}_R={\underline{s}}_R\right].\end{array}$$

Using the forward model $q(y^L|a^L)=q(a^L|y^L)/p(a^L)$, the GMI with $s=1$ becomes

$$\begin{array}{c}\hfill I_1(A^L;Y^L,S_R^L)=\mathrm{E}\left[log\frac{det{\mathbf{Q}}_{\underline{A}}}{det{\mathbf{Q}}_{\underline{A}|\underline{Y},{\underline{S}}_R}}\right].\end{array}$$

(326)

To simplify, consider adaptive symbols as in (89) (cf. (107)):

$$\begin{array}{c}\hfill X_{\ell }(S_T^{\ell })=\sqrt{P_{\ell }(S_T^{\ell })}{e}^{j{\varphi }_{\ell }(S_T^{\ell })}{U}_{\ell }\end{array}$$

(327)

where $\underline{U}\sim \mathcal{CN}(\underline{0},\mathbf{I})$. In other words, consider a conventional codebook represented by the $U_{\ell }$ and adapt the power and phase based on the available CSIT. The mutual information becomes $I(A^L;Y^L,S_R^L)=I(U^L;Y^L,S_R^L)$ (cf. (96)) and the GMI with $s=1$ is (cf. (108))

$$\begin{array}{c}\hfill I_1(A^L;Y^L|S_R^L)=\mathrm{E}\left[-logdet{\mathbf{Q}}_{\underline{U}\left|\underline{Y},{\underline{S}}_R\right.}\right].\end{array}$$

(328)

In fact, one may also consider choosing $U_{\ell }=U$ for all ℓ in which case we compute (cf. (139))

$$\begin{array}{c}\hfill I_1(A^L;Y^L|S_R^L)=\mathrm{E}\left[-log\mathrm{Var}\left[U|\underline{Y},{\underline{S}}_R\right]\right].\end{array}$$

(329)

Forward Model: Consider the following forward model (cf. (111) and (141)):

$$\begin{array}{c}\hfill q(\underline{y}|\underline{a},{\underline{s}}_R)=\frac{exp\left(-\underline{z}{({\underline{s}}_R)}^{†}{\mathbf{Q}}_{\underline{Z}}{({\underline{s}}_R)}^{-1}\underline{z}({\underline{s}}_R)\right)}{{\pi }^{L}det{\mathbf{Q}}_{\underline{Z}}({\underline{s}}_R)}.\end{array}$$

(330)

with

$$\begin{array}{c}\hfill \underline{z}({\underline{s}}_R)=\underline{y}-\mathbf{H}({\underline{s}}_R)\underline{\overline{x}}({\underline{s}}_R)\end{array}$$

and where similar to (142) we define

$$\begin{array}{c}\hfill \underline{\overline{X}}({\underline{s}}_R)=\sum _{{\underline{s}}_T}\mathbf{W}({\underline{s}}_T,{\underline{s}}_R)\underline{X}({\underline{s}}_T)\end{array}$$

(331)

where the $\mathbf{W}({\underline{s}}_T,{\underline{s}}_R)$ are $L\times L$ complex matrices. Note that

$$\begin{array}{c}\hfill \underline{X}({\underline{s}}_T)={[X_1(s_{T1}),X_2(s_T^2),\dots ,X_2(s_T^L)]}^T\end{array}$$

(332)

so $X_{\ell }$ is a function of $A^L$ and $S_T^{\ell }$, $\ell =1,\dots ,L$.

We have the following generalization of Lemma 4 (see also Theorem 1) where the novelty is that $S_T$ is replaced with ${\underline{S}}_T$. Define $\underline{U}({\underline{s}}_T)\sim \mathcal{CN}(\underline{0},\mathbf{I})$ and $\underline{X}({\underline{s}}_T)={\mathbf{Q}}_{\underline{X}({\underline{s}}_T)}^{1/2}\underline{U}({\underline{s}}_T)$ for all ${\underline{s}}_T$.

Theorem 4.

A GMI (325) for the scalar block fading channel $p(y^L|a^L,s_R^L)$, an adaptive codeword $A^L$ with jointly CSCG entries, the auxiliary model (330), and with fixed ${\mathbf{Q}}_{X({\underline{s}}_T)}$ is

$$\begin{array}{c}\hfill I_1(A^L;Y^L|S_R^L)=\mathrm{E}\left[log\left(\frac{det{\mathbf{Q}}_{\underline{Y}}({\underline{S}}_R)}{det\left({\mathbf{Q}}_{\underline{Y}}({\underline{S}}_R)-\tilde{\mathbf{D}}({\underline{S}}_R)\tilde{\mathbf{D}}{({\underline{S}}_R)}^{†}\right)}\right)\right].\end{array}$$

(333)

where

$$\begin{array}{cc}\hfill {\mathbf{Q}}_{\underline{Y}}({\underline{s}}_R)& =\mathrm{E}\left[\left.\underline{Y}{\underline{Y}}^{†}\right|{\underline{S}}_R={\underline{s}}_R\right]\hfill \end{array}$$

(334)

and for $M\times M$ unitary ${\mathbf{V}}_R({\underline{s}}_T,{\underline{s}}_R)$ the matrix $\tilde{\mathbf{D}}({\underline{s}}_R)$ is

$$\begin{array}{c}\hfill \mathrm{E}\left[\left.{\mathbf{U}}_T({\underline{S}}_T,{\underline{s}}_R)\Sigma ({\underline{S}}_T,{\underline{s}}_R){\mathbf{V}}_R{({\underline{S}}_T,{\underline{s}}_R)}^{†}\right|{\underline{S}}_R={\underline{s}}_R\right]\end{array}$$

(335)

and ${\mathbf{U}}_T({\underline{s}}_T,{\underline{s}}_R)$ and $\Sigma ({\underline{s}}_T,{\underline{s}}_R)$ are $N\times N$ unitary and $N\times M$ rectangular diagonal matrices, respectively, of the SVD

$$\begin{array}{c}\hfill \mathrm{E}\left[\left.\underline{Y}{\underline{U}({\underline{s}}_T)}^{†}\right|{\underline{S}}_T={\underline{s}}_T,{\underline{S}}_R={\underline{s}}_R\right]={\mathbf{U}}_T({\underline{s}}_T,{\underline{s}}_R)\Sigma ({\underline{s}}_T,{\underline{s}}_R){\mathbf{V}}_T{({\underline{s}}_T,{\underline{s}}_R)}^{†}\end{array}$$

(336)

for all ${\underline{s}}_T$, ${\underline{s}}_R$ and the ${\mathbf{V}}_T({\underline{s}}_T,{\underline{s}}_R)$ are $M\times M$ unitary matrices. One may maximize (333) over the unitary ${\mathbf{V}}_R({\underline{s}}_T,{\underline{s}}_R)$.

Suppose next that the actual channel is $\underline{Y}=H\underline{X}+\underline{Z}$ where $\underline{Z}\sim \mathcal{CN}(\underline{0},\mathbf{I})$. The extension of (136) and (168) to block fading channels with CSIR is

$$\begin{array}{c}\hfill I_1(A^L;Y^L|S_R^L)=\sum _{\ell =1}^L\mathrm{E}\left[log\left(1+\frac{{\tilde{P}}_{\ell }({\underline{S}}_R)}{1+\mathrm{E}\left[G{P}_{\ell }(S_T^{\ell })|{\underline{S}}_R\right]-{\tilde{P}}_{\ell }({\underline{S}}_R)}\right)\right]\end{array}$$

(337)

where (cf. (166) and (167))

$$\begin{array}{cc}\hfill {\tilde{P}}_{\ell }({\underline{s}}_R)& =\mathrm{E}{\left[\left|\mathrm{E}\left[\left.H\sqrt{P_{\ell }(S_T^{\ell })}\right|S_T^{\ell },{\underline{S}}_R={\underline{s}}_R\right]\right|\right]}^2\hfill \\ \hfill \mathrm{E}\left[|Y_{\ell }{|}^2|{\underline{S}}_R={\underline{s}}_R\right]& =1+\mathrm{E}\left[G{P}_{\ell }(S_T^{\ell })|{\underline{S}}_R={\underline{s}}_R\right].\hfill \end{array}$$

9.3. CSIT@ R

Continuing as in Section 5.2, suppose the CSIT in (321) can be written by replacing $S_{Hi}$ with $S_{Ri}^{\ell }$ for all i and ℓ:

$$\begin{array}{c}\hfill S_{Ti\ell }=f_{T\ell }\left(S_{Ri}^{\ell },X_i^{\ell -1},Y_i^{\ell -1}\right).\end{array}$$

(338)

The capacity (324) then simplifies to a directed information. To see this, expand the mutual information in (324) as

$$\begin{array}{cc}\hfill I(A^L;Y^L|S_R^L)& \stackrel{(a)}{=}\sum _{\ell =1}^{L}I\left(A^L,X^{\ell };Y_{\ell }|S_R^L,Y^{\ell -1}\right)\hfill \\ \hfill & \stackrel{(b)}{=}\sum _{\ell =1}^{L}I(X^{\ell };Y_{\ell }|S_R^L,Y^{\ell -1})\hfill \end{array}$$

(339)

where step $(a)$ follows because $X^{\ell }$ is a function of $A^L$ and $S_T^{\ell }$ in (338), and step $(b)$ follows by the Markov chains

$$\begin{array}{cc}\hfill & A^L-[S_R^L,X^{\ell },Y^{\ell -1}]-Y_{\ell }.\hfill \end{array}$$

(340)

The capacity is therefore (see the definition (27))

$$\begin{array}{c}\hfill C=\underset{X_{\ell }(S_T^{\ell }),\ell =1,\dots ,L}{max}\frac{1}{L}I(X^L\to Y^L|S_R^L).\end{array}$$

(341)

The maximization in (341) under a cost constraint becomes a constrained maximization for which $\mathrm{E}\left[c(X^L,Y^L)\right]\le LP$ for some cost function $c(·)$.

Remark 75.

As outlined at the end of Section 9.1, the capacity (341) is a special case of the theory in ([22], Equation (48)). To see this, define the extended and time-shifted strings

$$\begin{array}{c}\hfill {\widehat{A}}^{L+1}=(0,A^L),{\widehat{X}}^{L+1}=(0,X^L),{\widehat{Y}}^{L+1}=(0,Y^L).\end{array}$$

Since $A^L$ and $S_R^L$ are independent, one may expand (339) as

$$\begin{array}{cc}\hfill I(A^L;Y^L|S_R^L)& =I(A^L;(S_{R2},\dots ,S_{RL},0),Y^L|S_{R1})\hfill \\ \hfill & \stackrel{(a)}{=}\sum _{\ell =1}^{L}I(A^L,X^{\ell };S_{R(\ell +1)},Y_{\ell }|S_R^{\ell },Y^{\ell -1})\hfill \\ \hfill & \stackrel{(b)}{=}\sum _{\ell =1}^{L}I(X^{\ell };S_{R(\ell +1)},Y_{\ell }|S_R^{\ell },Y^{\ell -1})\hfill \\ \hfill & =\sum _{\ell =2}^{L+1}I({\widehat{X}}^{\ell };S_{R\ell },{\widehat{Y}}_{\ell }|S_R^{\ell -1},{\widehat{Y}}^{\ell -1})\hfill \end{array}$$

(342)

where step $(a)$ follows because $X^{\ell }$ is a function of $A^L$ and $S_T^{\ell }$ in (338), and where $S_{R(L+1)}=0$, and step $(b)$ follows by the Markov chains

$$\begin{array}{cc}\hfill & A^L-[X^{\ell },Y^{\ell -1},S_R^{\ell }]-[Y_{\ell },S_{R(\ell +1)}].\hfill \end{array}$$

(343)

The expression (342) is the desired directed information

$$\begin{array}{c}\hfill I(A^L;Y^L,S_R^L)=I({\widehat{X}}^{L+1}\to {\widehat{Y}}^{L+1},S_R^{L+1}).\end{array}$$

(344)

Remark 76.

Consider the basic CSIT model

$$\begin{array}{c}\hfill S_{Ti\ell }=f_T(S_{Ri\ell })\end{array}$$

(345)

for some function $f_T(·)$ and for $\ell =1,\dots ,L$ and $i=1,\dots ,n$. This model was studied in ([103], Section III.C) and its capacity is given as (see ([103], Equation (35) with Equation (13)))

$$\begin{array}{cc}\hfill C& =\underset{X_{\ell }(S_T^{\ell }),\ell =1,\dots ,L}{max}\frac{1}{L}I(X^L;Y^L|S_R^L,S_T^L).\hfill \end{array}$$

(346)

To see that (346) is a special case of (341), observe that

$$\begin{array}{cc}\hfill I(X^L\to Y^L|S_R^L)& \stackrel{(a)}{=}\sum _{\ell =1}^{L}I(X^{\ell };Y_{\ell }|S_R^L,S_T^L,Y^{\ell -1})\hfill \\ \hfill & \stackrel{(b)}{=}\sum _{\ell =1}^{L}I(X^L;Y_{\ell }|S_R^L,S_T^L,Y^{\ell -1})\hfill \end{array}$$

(347)

where step $(a)$ follows by (339), and step $(b)$ follows by the Markov chains

$$\begin{array}{c}\hfill [X_{\ell +1},\dots ,X_L]-[S_R^L,S_T^L,Y^{\ell -1},X^{\ell }]-Y_{\ell }.\end{array}$$

(348)

The expression (347) gives (346). Related results are available in ([10], Section III) and [104,105].

Remark 77.

The capacity (341) has only $S_R^L$ in the conditioning while (346) has both $S_R^L$ and $S_T^L$ in the conditioning. This subtle difference is due to permitting $X^{\ell -1}$ to influence the $S_{T\ell }$ in (338), and it complicates the analysis. On the other hand, if we remove only $X^{\ell -1}$ from (338) then the receiver knows $S_{T\ell }$ at time ℓ and the capacity (341) can be written as (see the definition (28))

$$\begin{array}{c}\hfill C=\underset{X_{\ell }(S_T^{\ell }),\ell =1,\dots ,L}{max}\frac{1}{L}I(X^L\to Y^L\parallel S_T^L|S_R^L).\end{array}$$

(349)

We treat such a model in Section 9.7 below.

9.4. Fading Channels with AWGN

The expression (341) is valid for general statistics. We next specialize to the block-fading AWGN model

$$\begin{array}{c}\hfill Y_{\ell }=H{X}_{\ell }+Z_{\ell }\end{array}$$

(350)

where $\ell =1,\dots ,L$, $Z^L\sim \mathcal{CN}(\underline{0},\mathbf{I})$, and $(H,S_R^L)$, $A^L$, $Z^L$ are mutually independent. Consider the power constraint

$$\begin{array}{c}\hfill \sum _{\ell =1}^L\mathrm{E}\left[P_{\ell }\left(S_T^{\ell }\right)\right]\le LP\end{array}$$

(351)

where $P_{\ell }(s_T^{\ell })=\mathrm{E}\left[|X_{\ell }(s_T^{\ell }){|}^2\right]$. The optimization of (341) under the constraint (351) is usually intractable, and we again desire expressions with $log(1+\mathrm{SNR})$ terms to obtain insight.

Capacity Upper Bound: Using similar steps as in (162), we have

$$\begin{array}{cc}\hfill I(A^L;Y^L|S_R^L)& \le I(A^L;Y^L,H|S_R^L)\hfill \\ \hfill & =\sum _{\ell =1}^{L}I\left(A^L;Y_{\ell }|S_R^L,H,Y^{\ell -1}\right)\hfill \\ \hfill & \le \sum _{\ell =1}^L\left[h(Y_{\ell }|S_R^L,H,Y^{\ell -1})-h(Z_{\ell })\right]\hfill \\ \hfill & \stackrel{(a)}{\le }\sum _{\ell =1}^L\mathrm{E}\left[log\left(1+\mathrm{E}\left[G{P}_{\ell }(S_T^{\ell })|S_R^L,H,Y^{\ell -1}\right]\right)\right]\hfill \end{array}$$

(352)

where $G={\left|H\right|}^2$ and step $(a)$ follows by (163). However, CSCG inputs do not necessarily maximize the RHS of (352) because the inputs affect the CSIT.

Remark 78.

The expectation inside the logarithm in (352) becomes $G{P}_{\ell }(S_T^{\ell })$ if $S_T^{\ell }$ is a function of $S_R^L,H,Y^{\ell -1}$; see (161), Remark 77, and Proposition 3 below.

Achievable Rates: Deriving achievable rates is more subtle than in Section 6. Consider the CSIT model (338) where for each block, we have

$$\begin{array}{c}\hfill S_{T\ell }=f_{T\ell }(H,X^{\ell -1},Y^{\ell -1})\end{array}$$

for all ℓ. The capacity (341) is

$$\begin{array}{cc}\hfill C(P)& =\underset{X_{\ell }(S_T^{\ell }),\ell =1,\dots ,L}{max}\frac{1}{L}I(X^L\to Y^L|H)\hfill \end{array}$$

(353)

$$\begin{array}{cc}\hfill & =\underset{X_{\ell }(S_T^{\ell }),\ell =1,\dots ,L}{max}\left[\frac{1}{L}h(Y^L|H)\right]-log(\pi e).\hfill \end{array}$$

(354)

However, CSCG inputs are not necessarily optimal since the inputs affect the CSIT.

Instead of trying to optimize the input, consider $X_{\ell }$ that are CSCG. We may write

$$\begin{array}{c}\hfill I(X^L\to Y^L|H)=\sum _{\ell =1}^L\mathrm{E}\left[log\left(1+G{P}_{\ell }(S_T^{\ell })\right)\right]\end{array}$$

(355)

and the Lagrangians to maximize (355) are

$$\begin{array}{c}\hfill \sum _{\ell =1}^L\mathrm{E}\left[log\left(1+G{P}_{\ell }(S_T^{\ell })\right)\right]+\lambda \left(LP-\sum _{\ell =1}^L\mathrm{E}\left[P_{\ell }(S_T^{\ell })\right]\right).\end{array}$$

(356)

Suppose the $S_{T\ell }$ are discrete random variables. Taking the derivative with respect to $P_{\ell }(s_T^{\ell })$, we obtain

$$\begin{array}{cc}\hfill \lambda & ={\int }_0^{\infty }p(g|s_T^{\ell })\frac{g}{1+g{P}_{\ell }(s_T^{\ell })}dg\hfill \\ \hfill & +\sum _{k=\ell +1}^L\sum _{s_T^k}{\int }_0^{\infty }p(g)\frac{d{P}_{S_T^k|G}(s_T^k|g)}{d{P}_{\ell }(s_T^{\ell })}\frac{log\left(1+g{P}_k(s_T^k)\right)}{P_{S_T^{\ell }}(s_T^{\ell })}dg\hfill \end{array}$$

(357)

as long as $P_{\ell }(s_T^{\ell })>0$. This expression is complicated because the choice of transmit powers $P_{\ell }(s_T^{\ell })$ influences the statistics of the future CSIT $S_{T(\ell +1)},\dots ,S_{TL}$. If (357) cannot be satisfied, choose $P_{\ell }(s_T^{\ell })=0$. Finally, set $\lambda$ so that ${\sum }_{\ell =1}^L\mathrm{E}\left[P_{\ell }(S_T^{\ell })\right]=LP$.

Instead of the above, consider the simpler CSIT model with $S_{T\ell }=f_{T\ell }(H)$ for all ℓ, cf. (345). The capacity (346) is now given by (355) with CSCG inputs and (357) simplifies because the derivatives with respect to $P_{\ell }(s_T^{\ell })$ are zero, i.e., the double sum in (357) disappears and for all ℓ and $s_T^{\ell }$ we have

$$\begin{array}{cc}\hfill & \lambda ={\int }_0^{\infty }p(g|s_T^{\ell })\frac{g}{1+g{P}_{\ell }(s_T^{\ell })}dg.\hfill \end{array}$$

(358)

We use (358) for (362)–(364) in Section 9.7 below.

9.5. Full CSIR, Partial CSIT

We next generalize Proposition 2 in Section 6.4 to the block-fading AWGN model (350) with the CSIR

$$\begin{array}{c}\hfill S_{R\ell }=H\sqrt{P(S_T^{\ell })},\ell =1,\dots ,L\end{array}$$

(359)

and where $S_{T\ell }=f_{T\ell }(S_H)$, i.e., we have discarded $X_i^{\ell -1}$ and $Y_i^{\ell -1}$ in (321). We then have the following capacity result that implies this CSIR is the best possible since one achieves a capacity upper bound similar to (161).

Proposition 3.

The capacity of the channel (350) with the CSIR (359) and $S_{T\ell }=f_{T\ell }(S_H)$ for $\ell =1,\dots ,L$ is

$$\begin{array}{cc}\hfill C(P)& =max\frac{1}{L}\sum _{\ell =1}^L\mathrm{E}\left[log\left(1+G{P}_{\ell }(S_T^{\ell })\right)\right]\hfill \end{array}$$

(360)

where the maximization is over the power control policies $P_{\ell }(S_T^{\ell })$ such that ${\sum }_{\ell =1}^L\mathrm{E}\left[P_{\ell }(S_T^{\ell })\right]\le LP$. One may use (358) to compute the $P_{\ell }(S_T^{\ell })$.

Proof. 

For achievability, apply (337) with

$$\begin{array}{c}\hfill {\tilde{P}}_{\ell }({\underline{S}}_R)=G{P}_{\ell }(S_T^{\ell })\mathrm{and}\mathrm{E}\left[|Y_{\ell }{|}^2|{\underline{S}}_R\right]=1+{\tilde{P}}_{\ell }({\underline{S}}_R).\end{array}$$

The converse follows by applying similar steps as in (162):

$$\begin{array}{cc}\hfill I(A^L;Y^L|S_R^L)& \le I(A^L;Y^L,S_T^L,H|S_R^L)\hfill \\ \hfill & =\sum _{\ell =1}^{L}I\left(A^L;Y_{\ell }|S_R^L,S_T^L,H,Y^{\ell -1}\right)\hfill \\ \hfill & \le \sum _{\ell =1}^L\left[h(Y_{\ell }|S_R^L,S_T^L,H,Y^{\ell -1})-h(Z_{\ell })\right]\hfill \\ \hfill & \stackrel{(a)}{\le }\sum _{\ell =1}^L\mathrm{E}\left[log\mathrm{Var}\left[Y_{\ell }|S_R^L,S_T^L,H,Y^{\ell -1}\right]\right].\hfill \end{array}$$

(361)

Finally, insert $\mathrm{Var}\left[Y_{\ell }|S_R^L,S_T^L,H,Y^{\ell -1}\right]=1+G{P}_{\ell }(S_T^{\ell })$. □

The RHS of (361) is at most the RHS of (352) and hence (361) gives a better bound. However, the bound (361) is valid only for particular CSIT, as in Remark 78.

9.6. On-Off Fading with Delayed CSIT

Consider on-off fading where the CSIT is delayed by D symbols, i.e., we have $S_{T\ell }=0$ for $\ell =1,\dots ,D$ and $S_{T(D+1)}=H$. Define the transmit powers as $P_{\ell }(s_T^{\ell })=\mathrm{E}\left[|X(s_T^{\ell }){|}^2\right]$ for $\ell =1,\dots ,L$. The capacity is

$$\begin{array}{cc}\hfill C(P)& =\frac{D}{2L}log\left(1+2P_1\right)+\frac{L-D}{2L}log\left(1+2P_{D+1}\right)\hfill \end{array}$$

where we write $P_{D+1}=P_{D+1}\left(s_T^{D+1}\right)$. Optimizing the powers, we obtain

$$\begin{array}{cc}\hfill & \left\{\begin{array}{c}{P}_1=P-\frac{L-D}{4L}\hfill \\ P_{D+1}=2P+\frac{D}{2L}\hfill \end{array}\right\}\mathrm{if}P\ge \frac{L-D}{4L}\hfill \\ \hfill & \left\{\begin{array}{c}{P}_1=0\hfill \\ P_{D+1}=\frac{2LP}{L-D}\hfill \end{array}\right\}\mathrm{else}.\hfill \end{array}$$

For large P, we thus have $C(P)\approx \frac{1}{2}log(P)$ for all $0\le D\le L$. For small P, we have

$$\begin{array}{cc}\hfill C(P)& =\left\{\begin{array}{cc}\frac{L-D}{2L}log\left(1+\frac{4LP}{L-D}\right),\hfill & \mathrm{if}0\le D<L\hfill \\ log(1+2P)/2,\hfill & \mathrm{if}D=L\hfill \end{array}\right.\hfill \\ \hfill & \approx \left\{\begin{array}{cc}\left(2P-\frac{4L}{L-D}{P}^2\right)log(e),\hfill & \mathrm{if}0\le D<L\hfill \\ \left(P-P^2\right)log(e),\hfill & \mathrm{if}D=L.\hfill \end{array}\right.\hfill \end{array}$$

The CSIT thus gives a 3 dB power gain at low SNR since $C(P)\approx 2Plog(e)$ for $0\le D<L$ and $C(P)\approx Plog(e)$ for $D=L$. Furthermore, using (37), the slope of the capacity versus $E_b/N_0$ in bits/s/Hz/(3 dB) is

$$\begin{array}{c}\hfill \begin{array}{cc}1-D/L\hfill & \mathrm{if}0\le D<L\hfill \\ 1\hfill & \mathrm{if}D=L.\hfill \end{array}\end{array}$$

In other words, the delay reduces the low-SNR rate by a factor of $1-D/L$ for $0\le D<L$.

9.7. Rayleigh Fading and One-Bit Feedback

Let $q_u(.)$ be the one-bit ($B=1$) quantizer in Section 2.9. We study Rayleigh fading for two scenarios with $S_R^L=H$, i.e., the receiver knows H after the L transmissions of each block.

• For the CSIT (345), we study delayed feedback where $S_{T\ell }=0$ for $\ell =1,\dots ,L-1$ and $S_{TL}=q_u(G)$. The delay is thus $D=L-1$ in the sense of Section 9.6.
• For the CSIT (338), we study the case $S_{T1}=0$, $S_{T2}=q_u(|Y_1|)$, and $S_{T\ell }=0$ for $\ell =3,\dots ,L$. The delay is thus $D=1$ in the sense of Section 9.6.

Delayed Quantized CSIR Feedback: Consider $S_{T\ell }=0$ for $\ell =1,\dots ,L-1$ and $S_{TL}=q_u(G)$. CSCG inputs are optimal, and (347) has the same form as (360). The Lagrangians are given by (356), and we again obtain (358). For the case at hand, we have $L+1$ equations for $\lambda$, namely

$$\begin{array}{cc}\hfill \lambda & ={\int }_0^{\infty }{e}^{-g}\frac{g}{1+g{P}_{\ell }}dg,\ell =1,\dots ,L-1\hfill \end{array}$$

(362)

$$\begin{array}{cc}\hfill \lambda & ={\int }_0^{\Delta }\frac{e^{-g}}{1-e^{-\Delta }}\frac{g}{1+g{P}_L(\Delta /2)}dg\hfill \end{array}$$

(363)

$$\begin{array}{cc}\hfill \lambda & ={\int }_{\Delta }^{\infty }\frac{e^{-g}}{e^{-\Delta }}{\displaystyle \frac{g}{1+g{P}_L(3\Delta /2)}}dg\hfill \end{array}$$

(364)

where we used (40) and (41) and abused notation by writing $P_L(s_{TL})$ for $P_L(s_T^L)$. We thus have $P_1=\dots =P_{L-1}$ and obtain three equations. We now search for $\lambda$ such that

$$\begin{array}{c}\hfill (L-1)P_1+\sum _s{P}_{S_{TL}}(s)P_L(s)=LP\end{array}$$

and the capacity (353) is

$$\begin{array}{c}\hfill C(P)=\frac{L-1}{L}{e}^{1/P_1}{E}_1\left(1/P_1\right)+\frac{1}{L}\sum _s{\int }_{\mathcal{I}(s)}{e}^{-g}log\left(1+g{P}_L(s)\right)dg\end{array}$$

(365)

where the sums are over $s=\Delta /2,3\Delta /2$ and

$$\begin{array}{c}\hfill \mathcal{I}(\Delta /2)=[0,\Delta ),\mathcal{I}(3\Delta /2)=[\Delta ,\infty ).\end{array}$$

We remark that, if $P_1=0$, then we set $e^{1/P_1}{E}_1\left(1/P_1\right)=0$ since ${lim}_{x\to \infty }{e}^x{E}_1(x)=0$.

Figure 15 shows these capacities for $L=1,2,3$ and $\Delta =1$. At low SNR (e.g., for $L=3$ below $-2.97$ dB) we have $P_1=0$ and $P_L(\Delta /2)=0$, i.e., the transmitter is silent unless $S_{TL}=3\Delta /2$ and it uses power at time $\ell =L$ only. Observe that, as in Section 9.6, a delay of L steps reduces the low-SNR slope, and therefore the low-SNR rates, by a factor of L. Delay can thus be costly at low SNR.

Quantized Channel Output Feedback: Consider $S_{T1}=0$, $S_{T2}=q_u(|Y_1|)$, and $S_{T\ell }=0$ for $\ell =3,\dots ,L$. As discussed in Remark 77, the capacity is given by the directed information expression (349). However, optimizing the input statistics seems difficult, i.e., CSCG inputs are not necessarily optimal. Instead, we compute achievable rates for a strategy where one symbol partially acts as a pilot.

Suppose the transmitter sends $X_1=\sqrt{P_1}{e}^{j\mathsf{\Phi }}$ as the first symbol of each block, where $\mathsf{\Phi }$ is uniformly distributed in $[0,2\pi )$. The idea is that $|X_1|=\sqrt{P_1}$ is known at the receiver, and thus $X_1$ acts as a pilot to test the channel amplitude. Next, we choose a variation of flash signaling. Define the event $\mathcal{E}=\left\{\right|Y_1|\ge \Delta \}=\{S_{T2}=3\Delta /2\}$. If this event does not occur, the transmitter sends $X_{\ell }=0$ for $\ell =2,\dots ,L$. Otherwise, the transmitter sends independent CSCG $X_{\ell }$ with variance $P_2/\mathrm{Pr}\left[\mathcal{E}\right]$ for $\ell =2,\dots ,L$. Define $P_{\ell }(s_T^{\ell })=\mathrm{E}\left[|X(s_T^{\ell }){|}^2\right]$. We have $P_{\ell }=P_2$ for $\ell \ge 2$ and the power constraint is $P_1+(L-1)P_2\le LP$.

We use (347) to write

$$\begin{array}{cc}\hfill & C(P)\ge \frac{1}{L}I(X_1;Y_1|H)+\frac{L-1}{L}I(X^2;Y_2|H,Y_1).\hfill \end{array}$$

(366)

The first mutual information in (366) is

$$\begin{array}{cc}\hfill I(X_1;Y_1|H)& =h(Y_1|H)-log(\pi e)\hfill \end{array}$$

and we compute (see ([52], Appendix A))

$$\begin{array}{c}\hfill p(y_1|h)=\frac{1}{\pi }{e}^{-(|y_1{|}^2+P_1|h{|}^2)}{I}_0\left(2|y_1\left|\right|h|\sqrt{P_1}\right)\end{array}$$

where $I_0(.)$ is the modified Bessel function of the first kind of order zero. The Jacobian of the mapping from Cartesian coordinates $[\Re (y_1),\Im (y_1)]$ to polar coordinates $\left[\right|y_1|,arg{y}_1]$ is $|y_1|$, so we have

$$\begin{array}{c}\hfill h(Y_1|H=h)={\int }_0^{\infty }-p(y_1|h)log(p(y_1|h))2\pi |y_1\left|d\right|y_1|.\end{array}$$

We further compute

$$\begin{array}{c}\hfill I\left(X^2;Y_2|H,Y_1\right)={\int }_0^{\infty }{e}^{-g}\mathrm{Pr}\left[\mathcal{E}|G=g\right]log\left(1+\frac{g{P}_2}{\mathrm{Pr}\left[\mathcal{E}\right]}\right)dg.\end{array}$$

(367)

The conditional probability of a high-energy $Y_1$ is

$$\begin{array}{c}\hfill \mathrm{Pr}\left[\mathcal{E}|G=g\right]=Q_1\left(\sqrt{2g{P}_1},\sqrt{2}\Delta \right)\end{array}$$

where $Q_1(.)$ is the Marcum Q-function of order 1; see (A3) in Appendix A.1. For Rayleigh fading, we compute

$$\begin{array}{c}\hfill \mathrm{Pr}\left[\mathcal{E}\right]=\mathrm{Pr}\left[{\left|H\sqrt{P_1}{e}^{j\mathsf{\Phi }}+Z_1\right|}^2\ge {\Delta }^2\right]=e^{-{\Delta }^2/(P_1+1)}.\end{array}$$

The resulting rates are shown in Figure 16 for the block lengths $L=10,20,100$. Observe that each curve turns back on itself, which reflects the non-concavity of the directed information rates in P; see ([74], Section III). All rates below the curves are achievable by “time-wasting”, i.e., by transmitting for some fraction of the time only. This suggests that flash signaling [73] will improve the rates since one sends information by choosing whether to transmit energy.

10. Conclusions

This paper reviewed and derived achievable rates for channels with CSIR, CSIT, block fading, and in-block feedback. GMI expressions were developed for adaptive codewords and two classes of auxiliary channel models with AWGN and CSCG inputs: reverse and forward channel models. The forward model inputs were chosen as linear functions of the adaptive codeword’s symbols. We showed that, for scalar channels, an input distribution that maximizes the GMI generates a conventional codebook, where the codeword symbols are multiplied by a complex number that depends on the CSIT. The GMI increases by partitioning the channel output alphabet and modifying the auxiliary model parameters for each partition subset. The partitioning helps to determine the capacity scaling at high and low SNR. Power control policies were developed for full CSIT, including TMMSE policies. The theory was applied to channels with on-off fading and Rayleigh fading. The capacities with in-block feedback simplify to directed information expressions if the CSIT is a function of the CSIR and past channel inputs and outputs.

There are many possible applications and extensions of this work. For example, adaptive coding and modulation are important for all practical communication systems, including wireless, copper, and fiber-optic networks. Shannon’s adaptive codewords can improve current systems since the CSIT is usually a noisy version of the CSIR; see Remark 25. Moreover, the information theory for in-block feedback [22] applies to beamforming [106] and intelligent reflecting surfaces [107,108]. One may also apply GMI to multi-user channels with in-block feedback, such as multi-access and broadcast channels. Finally, it is important to develop improved capacity upper bounds. The standard approach here is the duality framework described in [97,109]; see also ([110], page 128).