Chapter 2: Information Measures for Continuous Random Variables

Learning Objectives

• Define differential entropy and understand how it differs from discrete entropy
• Prove that the Gaussian distribution maximizes differential entropy under a variance constraint
• Compute differential entropy for multivariate Gaussian vectors
• State and interpret the entropy power inequality
• Understand the connection between discrete and continuous entropy via quantization