Notation PreferencesInformation Theory

Type your preferred LaTeX symbol and press Enter. Changes apply across all pages.

KeyMeaningYour SymbolDefault
pmfProbability mass function of discrete RV XXPP
gaussReal Gaussian distributionN\mathcal{N}
cgaussCircularly symmetric complex GaussianCN\mathcal{CN}
covmatCovariance matrixΣ\boldsymbol{\Sigma}
entropyShannon entropyHH
hdDifferential entropyhh
miMutual informationII
kldivKullback-Leibler divergenceDD
capChannel capacityCC
rateCode rate (bits/symbol or bits/channel use)RR
rdRate-distortion functionRR
wf_lvlWaterfilling level (channel coding)ν\nu
rwf_lvlReverse waterfilling level (source coding)γ\gamma
markovMarkov chain relationXYZX \multimap Y \multimap Z
typ_setStrongly typical setTϵ(n)\mathcal{T}_\epsilon^{(n)}
chChannel matrix (MIMO, OFDM, general)H\mathbf{H}
snrSignal-to-noise ratioSNR\text{SNR}
esEnergy per transmitted symbolEsE_s
n0One-sided noise power spectral densityN0N_0
bwSignal bandwidth (Hz)WW
noise_rvNoise random variable (theoretical)ZZ
noisevarNoise variance / noise powerσ2\sigma^2

Universal Conventions

Fixed conventions used throughout this book. These are standard across all telecommunications literature and are not customizable.

General Mathematics

SymbolMeaning
R,C\mathbb{R}, \mathbb{C}Real and complex number fields
Rn,Cn\mathbb{R}^n, \mathbb{C}^nReal/complex nn-dimensional vector spaces
Rm×n,Cm×n\mathbb{R}^{m \times n}, \mathbb{C}^{m \times n}Spaces of real/complex m×nm \times n matrices
j=1j = \sqrt{-1}Imaginary unit (engineering convention)
{},{}\Re\{\cdot\}, \Im\{\cdot\}Real and imaginary parts
|\cdot|Absolute value (scalar) or cardinality (set)
,\lceil \cdot \rceil, \lfloor \cdot \rfloorCeiling and floor functions
\triangleqDefined as
\proptoProportional to
O()\mathcal{O}(\cdot)Big-O asymptotic notation

Vectors and Matrices

SymbolMeaning
x,y,z\mathbf{x}, \mathbf{y}, \mathbf{z}Column vectors (always boldface lowercase)
A,B,H\mathbf{A}, \mathbf{B}, \mathbf{H}Matrices (always boldface uppercase)
()T(\cdot)^TTranspose
()(\cdot)^*Complex conjugate
()H(\cdot)^HConjugate transpose (Hermitian)
()1(\cdot)^{-1}Matrix inverse
()(\cdot)^{\dagger}Moore-Penrose pseudoinverse
\|\cdot\| or 2\|\cdot\|_2Euclidean (l_2) norm
p\|\cdot\|_pp\ell_p norm
F\|\cdot\|_FFrobenius norm
x,y\langle \mathbf{x}, \mathbf{y} \rangleInner product yHx\mathbf{y}^H \mathbf{x}
tr()\text{tr}(\cdot)Matrix trace
det()\det(\cdot)Matrix determinant
rank()\text{rank}(\cdot)Matrix rank
diag()\text{diag}(\cdot)Diagonal matrix from vector, or diagonal of matrix
vec()\text{vec}(\cdot)Column-wise vectorization
\otimesKronecker product
\odotHadamard (element-wise) product
In\mathbf{I}_nn×nn \times n identity matrix
0m×n\mathbf{0}_{m \times n}m×nm \times n zero matrix
A0\mathbf{A} \succ 0, A0\mathbf{A} \succeq 0Positive definite, positive semidefinite
λi(A)\lambda_i(\mathbf{A})ii-th eigenvalue of A\mathbf{A}
σi(A)\sigma_i(\mathbf{A})ii-th singular value of A\mathbf{A}

Probability Basics

SymbolMeaning
E[]\mathbb{E}[\cdot]Expectation
Pr()\Pr(\cdot)Probability of an event
\simDistributed as
=d\stackrel{d}{=}Equal in distribution