Prerequisites & Notation

Before You Begin

This chapter introduces factor graphs — a graphical language for factorized probability distributions. The reader should be comfortable with joint and conditional distributions, marginalization, and basic graph terminology (nodes, edges, paths, cycles). No prior knowledge of Bayesian networks or Markov random fields is assumed, but familiarity with them will help.

Joint distributions, marginalization, conditional independence
Self-check: Can you compute $p(x_1, x_3)$ from a joint $p(x_1, x_2, x_3)$ by summing out $x_2$ ?
Bayesian estimation and posterior computation(Review ch07)
Self-check: Can you write the MAP estimator as $\arg\max p(\theta|y)$ and identify the prior and likelihood?
EM algorithm as iterative inference(Review ch08)
Self-check: Can you identify the E-step and M-step as local operations on an augmented distribution?
Graph terminology (nodes, edges, trees, cycles)
Self-check: Can you state when a graph is a tree?
LDPC codes (basic definition)
Self-check: Can you describe the role of the parity-check matrix $\mathbf{H}$ ?

Notation for This Chapter

Symbol	Meaning	Introduced
$p(\mathbf{x})$	Joint distribution over variables $\mathbf{x} = (x_1, \ldots, x_n)$	s01
$f_a(\mathbf{x}_{\partial a})$	Local factor function depending on subset $\mathbf{x}_{\partial a}$	s01
$\partial a$	Neighborhood of factor node $a$ (variable nodes connected to $a$ )	s01
$\partial i$	Neighborhood of variable node $i$ (factor nodes connected to $i$ )	s01
$\mu_{x \to f}(x)$	Message from variable node $x$ to factor node $f$	s03
$\mu_{f \to x}(x)$	Message from factor node $f$ to variable node $x$	s03
$Z$	Partition function (normalization constant)	s01
$\mathbf{H}$	Parity-check matrix of an LDPC code	s02
$\mathcal{N}(i)$	Neighborhood of node $i$ in the graph	s01

← Ch 16 From Distributions to Graphs