Variance Reduction Techniques

Instead of sampling from the original distribution $f(x)$, draw from a proposal distribution $g(x)$ that concentrates samples in the important region:

$$\theta = E_f[h(X)] = \int h(x) \frac{f(x)}{g(x)} g(x)\,dx = E_g\!\left[h(X) \frac{f(X)}{g(X)}\right]$$

The IS estimator is:

$$\hat{\theta}_{\mathrm{IS}} = \frac{1}{N}\sum_{i=1}^{N} h(X_i) w(X_i), \quad w(X_i) = \frac{f(X_i)}{g(X_i)}$$

where $w(X_i)$ is the likelihood ratio (importance weight).

# IS for BPSK BER: shift noise mean toward decision boundary
mu_shift = np.sqrt(2 * ebno)  # optimal shift
noise_is = mu_shift + rng.standard_normal(N)
weights = np.exp(-mu_shift * noise_is + mu_shift**2 / 2)
errors_is = (noise_is > np.sqrt(2 * ebno))
ber_is = np.mean(errors_is * weights)

The zero-variance IS estimator uses the proposal:

$$g^*(x) = \frac{|h(x)| f(x)}{\int |h(x)| f(x)\,dx} = \frac{|h(x)| f(x)}{\theta}$$

This is impractical (requires knowing $\theta$), but it guides the design of good proposals. For BER estimation:

• The error event is $h(x) = \mathbf{1}[x > d]$ where $d$ is the decision boundary
• The optimal shift concentrates samples near $d$
• A practical choice: shift the noise mean by $d$ (exponential tilting)

The antithetic variates technique uses negatively correlated pairs to reduce variance:

$$\hat{\theta}_{\mathrm{AV}} = \frac{1}{2N}\sum_{i=1}^{N} \left[g(U_i) + g(1-U_i)\right]$$

where $U_i \sim \mathrm{Uniform}(0,1)$ and $1-U_i$ is the antithetic sample. If $g$ is monotonic, $\mathrm{Cov}[g(U), g(1-U)] < 0$, reducing the variance.

u = rng.uniform(size=N)
x1 = norm.ppf(u)          # original samples
x2 = norm.ppf(1 - u)      # antithetic samples
estimate = 0.5 * (g(x1) + g(x2))
theta_hat = np.mean(estimate)

If $Z$ is a random variable with known mean $E[Z] = \mu_Z$, the control variate estimator for $\theta = E[g(X)]$ is:

$$\hat{\theta}_{\mathrm{CV}} = \frac{1}{N}\sum_{i=1}^N g(X_i) - c\left(\frac{1}{N}\sum_{i=1}^N Z_i - \mu_Z\right)$$

The optimal coefficient is $c^* = \mathrm{Cov}[g(X), Z] / \mathrm{Var}[Z]$, giving variance reduction factor:

$$\frac{\mathrm{Var}[\hat{\theta}_{\mathrm{CV}}]}{\mathrm{Var}[\hat{\theta}]} = 1 - \rho_{gZ}^2$$

where $\rho_{gZ}$ is the correlation between $g(X)$ and $Z$.

# Control variate: use SNR as control
g_samples = compute_ber_per_block(...)
z_samples = compute_snr_per_block(...)
mu_z = theoretical_snr
c_star = np.cov(g_samples, z_samples)[0,1] / np.var(z_samples)
theta_cv = np.mean(g_samples) - c_star * (np.mean(z_samples) - mu_z)

Divide the sample space into $K$ non-overlapping strata $S_1, \dots, S_K$ with probabilities $p_k = P(X \in S_k)$. Sample $n_k$ points from each stratum:

$$\hat{\theta}_{\mathrm{SS}} = \sum_{k=1}^{K} p_k \cdot \frac{1}{n_k}\sum_{i=1}^{n_k} g(X_{ki})$$

Stratified sampling always reduces variance compared to simple random sampling (it removes the between-strata variance component).

For BER simulation: stratify on the channel realization $|h|^2$ to ensure coverage of both deep fades and strong channels.