LASSO / ISTA / FISTA

Interactive Explorer 2

Explore key concepts interactively

Parameters

Quick Check

Key concept question for section 2?

Option A

Option B

Option C

Common Mistake: Common Mistake in Section 2

Mistake:

Overlooking a critical implementation detail.

Correction:

Always verify results against known benchmarks and theoretical predictions.

Key Term 2

Core concept from section 2 of chapter 41.

Definition:

FISTA (Fast ISTA)

FISTA adds momentum to ISTA for O(1/k2)O(1/k^2) convergence:

tk+1=1+1+4tk22,z(k+1)=x(k+1)+tk1tk+1(x(k+1)x(k))t_{k+1} = \frac{1+\sqrt{1+4t_k^2}}{2}, \quad \mathbf{z}^{(k+1)} = \mathbf{x}^{(k+1)} + \frac{t_k-1}{t_{k+1}}(\mathbf{x}^{(k+1)}-\mathbf{x}^{(k)})

Definition:

ADMM

ADMM splits the optimization via an auxiliary variable:

x(k+1)=(AHA+ρI)1(AHy+ρ(z(k)u(k)))\mathbf{x}^{(k+1)} = (\mathbf{A}^H\mathbf{A} + \rho\mathbf{I})^{-1}(\mathbf{A}^H\mathbf{y} + \rho(\mathbf{z}^{(k)}-\mathbf{u}^{(k)})) z(k+1)=proxλ/ρ(x(k+1)+u(k))\mathbf{z}^{(k+1)} = \text{prox}_{\lambda/\rho}(\mathbf{x}^{(k+1)}+\mathbf{u}^{(k)}) u(k+1)=u(k)+x(k+1)z(k+1)\mathbf{u}^{(k+1)} = \mathbf{u}^{(k)} + \mathbf{x}^{(k+1)} - \mathbf{z}^{(k+1)}