Chapter 5: Estimation Theory Fundamentals

Learning Objectives

• Formulate the parameter estimation problem and quantify performance through bias, variance, and mean-squared error, including the decomposition $\mathrm{MSE} = \text{bias}^2 + \text{variance}$
• Derive the Cramer--Rao lower bound for scalar and vector parameters, identify when it is attained, and interpret Fisher information as curvature of the log-likelihood
• Recognize sufficient statistics through the Fisher--Neyman factorization theorem and read off natural sufficient statistics from the exponential family form
• Apply the Rao--Blackwell theorem to reduce the variance of any unbiased estimator by conditioning on a sufficient statistic
• Use Lehmann--Scheffe to certify an estimator as the unique MVUE through completeness of the sufficient statistic
• Connect the Fisher information matrix and the CRB to the ISAC tradeoff between sensing and communication in integrated systems