Chapter 22: High-Dimensional Estimation

Learning Objectives

• Distinguish classical asymptotics ($N$ fixed, $M\to\infty$) from proportional asymptotics ($N/M\to\gamma$) and explain why classical consistency results break down in the latter.
• Analyse ridge regression in the Marchenko–Pastur regime and derive the optimal regularization parameter as a function of $\gamma$ and the signal-to-noise ratio.
• State and prove the James–Stein shrinkage theorem, connect it to empirical Bayes, and explain why the sample mean is inadmissible for $N\geq 3$.
• Derive minimax estimation rates for sparse signals and link them to information-theoretic packing arguments.
• Apply high-dimensional estimation machinery to concrete wireless problems — massive-MIMO channel estimation, compressed sensing, and covariance estimation.