Prerequisites & Notation

Before You Begin

This chapter launches the FSI book with binary hypothesis testing --- the cleanest inference problem there is. Before diving in, ensure you are comfortable with the following.

  • Joint, marginal, and conditional densities; Bayes' rule(Review FSP Ch. 2-4)

    Self-check: Can you derive fX∣Y(x∣y)=fY∣X(y∣x)fX(x)/fY(y)f_{X|Y}(x|y) = f_{Y|X}(y|x)f_X(x)/f_Y(y) in one line?

  • Univariate and multivariate Gaussian distributions(Review FSP Ch. 8)

    Self-check: Can you write the density of N(m,C)\mathcal{N}(\mathbf{m},\mathbf{C}) on Rn\mathbb{R}^n from memory?

  • Inner products and quadratic forms(Review Telecom Ch. 1)

    Self-check: Can you expand βˆ₯yβˆ’sβˆ₯2\|\mathbf{y} - \mathbf{s}\|^2 in terms of ⟨y,s⟩\langle \mathbf{y},\mathbf{s}\rangle?

  • The Gaussian Q-function Q(x)Q(x) and its monotonicity(Review FSP Ch. 8)

    Self-check: Can you express P(Z>a)P(Z > a) for Z∼N(0,1)Z \sim \mathcal{N}(0,1) as Q(a)Q(a)?

  • Convex sets, concave functions, Jensen's inequality

    Self-check: Can you state Jensen's inequality for a concave function Ο•\phi?

Notation for This Chapter

Symbols introduced or used in this chapter. Detection-theoretic symbols L,β„“,PF,PD,gL, \ell, P_F, P_D, g are established here and used throughout Part I.

SymbolMeaningIntroduced
H0,H1\mathcal{H}_0, \mathcal{H}_1Null and alternative hypothesess01
Y,yY, \mathbf{y}Observation (random variable / realized vector)s01
Y\mathcal{Y}Observation spaces01
Ο€0,Ο€1\pi_0, \pi_1Prior probabilities, Ο€0+Ο€1=1\pi_0 + \pi_1 = 1s02
f0(y),f1(y)f_0(y), f_1(y)Conditional densities fY∣H0,fY∣H1f_{Y|\mathcal{H}_0}, f_{Y|\mathcal{H}_1}s01
CijC_{ij}Cost of deciding Hi\mathcal{H}_i when Hj\mathcal{H}_j is trues02
Ξ΄(y)\delta(y) or g(y)g(y)Decision rule, δ ⁣:Yβ†’{0,1}\delta \colon \mathcal{Y} \to \{0,1\}s01
L(y)L(y)Likelihood ratio f1(y)/f0(y)f_1(y)/f_0(y)s03
β„“(y)\ell(y)Log-likelihood ratio log⁑L(y)\log L(y)s03
Ξ·\etaLRT thresholds03
PFP_FFalse-alarm (Type I error) probability P(δ=1∣H0)P(\delta=1 \mid \mathcal{H}_0)s01
PMP_MMiss (Type II error) probability P(δ=0∣H1)P(\delta=0 \mid \mathcal{H}_1)s01
PDP_DDetection probability P(Ξ΄=1∣H1)=1βˆ’PMP(\delta=1 \mid \mathcal{H}_1) = 1 - P_Ms01
PeP_eAverage probability of error Ο€0PF+Ο€1PM\pi_0 P_F + \pi_1 P_Ms02
r(Ξ΄)r(\delta)Bayes risk of decision rule Ξ΄\deltas02
ΞΌ(s)\mu(s)Chernoff exponent βˆ’log⁑∫f01βˆ’s(y)f1s(y) dy-\log \int f_0^{1-s}(y) f_1^{s}(y)\,dys05
Q(x)Q(x)Gaussian tail: Q(x)=∫x∞12Ο€eβˆ’t2/2 dtQ(x) = \int_x^\infty \frac{1}{\sqrt{2\pi}} e^{-t^2/2}\,dts01
Ξ±\alphaSignificance level (upper bound on PFP_F)s04
Back to chapter overviewThe Decision Problem