Ferkans — Interactive Telecom Tutor

ex-sp-ch12-01

Easy

Create a $5 \times 5$ identity matrix as a PyTorch tensor with float64 dtype. Verify its dtype, device, and that it equals torch.eye(5).

Show Hint

Use torch.eye(5, dtype=torch.float64).

Solution

Implementation

import torch

I = torch.eye(5, dtype=torch.float64)
print(f"dtype: {I.dtype}")       # torch.float64
print(f"device: {I.device}")     # cpu
print(f"Is identity: {torch.allclose(I, torch.eye(5, dtype=torch.float64))}")

ex-sp-ch12-02

Easy

Convert a NumPy array a = np.linspace(0, 1, 100) to a PyTorch tensor using torch.from_numpy. Verify that modifying the tensor changes the original array (shared memory).

Show Hint

Modify t[0] and check a[0].

Solution

Implementation

import numpy as np
import torch

a = np.linspace(0, 1, 100)
t = torch.from_numpy(a)
t[0] = -999.0
print(f"a[0] = {a[0]}")   # -999.0 (shared memory)

ex-sp-ch12-03

Easy

Compute the gradient of $f(x) = x^3 - 2x^2 + x$ at $x = 3$ using autograd. Verify against the analytical derivative $f'(x) = 3x^2 - 4x + 1$ .

Show Hint

Create x = torch.tensor(3.0, requires_grad=True), compute f, call .backward().

Solution

Implementation

import torch

x = torch.tensor(3.0, requires_grad=True)
f = x**3 - 2*x**2 + x
f.backward()
print(f"Autograd: {x.grad.item()}")                # 16.0
print(f"Analytical: {3*3**2 - 4*3 + 1}")           # 16

ex-sp-ch12-04

Easy

Use in-place operations to normalize a random tensor to have zero mean and unit variance. Do not create any intermediate tensors.

Show Hint

Use .sub_() and .div_().

Solution

Implementation

import torch

x = torch.randn(1000)
x.sub_(x.mean())
x.div_(x.std())
print(f"Mean: {x.mean():.6f}")   # ~0
print(f"Std:  {x.std():.6f}")    # ~1

ex-sp-ch12-05

Easy

Create a complex tensor $z = [1+2j, 3+4j, 5+6j]$ and compute its magnitude $|z|$ , phase $\angle z$ , and conjugate $z^*$ .

Show Hint

Use .abs(), torch.angle(), and .conj().

Solution

Implementation

import torch

z = torch.tensor([1+2j, 3+4j, 5+6j])
print(f"|z| = {z.abs()}")
print(f"angle(z) = {torch.angle(z)}")
print(f"z* = {z.conj()}")

ex-sp-ch12-06

Medium

Compute the Jacobian matrix of the function $\mathbf{f}(\mathbf{x}) = [\sin(x_1 x_2),\, \cos(x_1 + x_2),\, x_1^2 x_2]$ at $\mathbf{x} = [1, 2]$ using torch.autograd.functional.jacobian. Verify against the analytical Jacobian.

Show Hint

Define f as a function taking a 1D tensor and returning a 1D tensor.

The analytical Jacobian has entries like $\partial f_1/\partial x_1 = x_2 \cos(x_1 x_2)$ .

Solution

Implementation

import torch
from torch.autograd.functional import jacobian

def f(x):
    return torch.stack([
        torch.sin(x[0] * x[1]),
        torch.cos(x[0] + x[1]),
        x[0]**2 * x[1],
    ])

x = torch.tensor([1.0, 2.0])
J = jacobian(f, x)
print(f"Jacobian:\n{J}")

# Analytical
import math
J_exact = torch.tensor([
    [2*math.cos(2), math.cos(2)],
    [-math.sin(3), -math.sin(3)],
    [4.0, 1.0],
])
print(f"Match: {torch.allclose(J, J_exact, atol=1e-6)}")

ex-sp-ch12-07

Medium

Implement Newton's method to find the root of $f(x) = x^3 - 2x - 5$ using autograd to compute $f'(x)$ at each step. Start from $x_0 = 2$ .

Show Hint

At each step, create a new tensor with requires_grad=True.

The update is $x_{n+1} = x_n - f(x_n) / f'(x_n)$ .

Solution

Implementation

import torch

x = 2.0
for i in range(10):
    xt = torch.tensor(x, dtype=torch.float64, requires_grad=True)
    f = xt**3 - 2*xt - 5
    f.backward()
    x = x - f.item() / xt.grad.item()
    print(f"Step {i}: x = {x:.10f}, f(x) = {xt.item()**3 - 2*xt.item() - 5:.2e}")

print(f"Root: {x:.10f}")

ex-sp-ch12-08

Medium

Use batched torch.linalg.solve to solve 500 independent $3 \times 3$ linear systems $\mathbf{A}_i \mathbf{x}_i = \mathbf{b}_i$ . Compare the time against a Python loop with NumPy.

Show Hint

Create A with shape (500, 3, 3) and b with shape (500, 3, 1).

Solution

Implementation

import torch
import numpy as np
import time

n_batch = 500

# PyTorch batched
A_pt = torch.randn(n_batch, 3, 3, dtype=torch.float64)
b_pt = torch.randn(n_batch, 3, 1, dtype=torch.float64)
t0 = time.perf_counter()
x_pt = torch.linalg.solve(A_pt, b_pt)
t_pt = time.perf_counter() - t0

# NumPy loop
A_np = A_pt.numpy()
b_np = b_pt.numpy()
t0 = time.perf_counter()
x_np = np.array([np.linalg.solve(A_np[i], b_np[i]) for i in range(n_batch)])
t_np = time.perf_counter() - t0

print(f"PyTorch batched: {t_pt*1e3:.2f} ms")
print(f"NumPy loop:      {t_np*1e3:.2f} ms")
print(f"Speedup: {t_np/t_pt:.1f}x")

ex-sp-ch12-09

Medium

Compute the FFT of a chirp signal $x(t) = \cos(2\pi(f_0 + \beta t)t)$ with $f_0 = 10$ Hz, $\beta = 50$ Hz/s, sampled at 500 Hz for 1 second. Plot the spectrogram to show the frequency increasing over time.

Show Hint

Use torch.stft for the short-time Fourier transform.

The chirp frequency at time $t$ is $f_0 + \beta t$ .

Solution

Implementation

import torch

fs = 500.0
t = torch.arange(0, 1.0, 1/fs, dtype=torch.float64)
f0, beta = 10.0, 50.0
x = torch.cos(2 * torch.pi * (f0 + beta * t) * t)

# STFT
n_fft = 64
X = torch.stft(x.float(), n_fft=n_fft, hop_length=16,
                win_length=n_fft,
                window=torch.hann_window(n_fft),
                return_complex=True)
spectrogram = X.abs().pow(2)
print(f"Spectrogram shape: {spectrogram.shape}")

ex-sp-ch12-10

Medium

Verify the Wirtinger derivative: for $L(z) = |z - z_0|^2$ where $z_0 = 2 + 3j$ , confirm that z.grad equals $z - z_0$ at several test points.

Show Hint

$\partial |z-z_0|^2 / \partial z^* = z - z_0$ .

Solution

Implementation

import torch

z0 = torch.tensor(2+3j, dtype=torch.complex128)

for z_val in [1+1j, 0+0j, 3+4j, -1-2j]:
    z = torch.tensor(z_val, dtype=torch.complex128, requires_grad=True)
    L = (z - z0).abs() ** 2
    L.backward()
    expected = z.detach() - z0
    print(f"z={z_val}: grad={z.grad}, expected={expected}, "
          f"match={torch.allclose(z.grad, expected)}")

ex-sp-ch12-11

Hard

Implement the power method for finding the dominant eigenvalue of a matrix using PyTorch, and differentiate through it with autograd to compute $\partial \lambda_1 / \partial \mathbf{A}$ .

Show Hint

Iterate $\mathbf{v} \leftarrow \mathbf{A}\mathbf{v} / \|\mathbf{A}\mathbf{v}\|$ for 100 steps.

The Rayleigh quotient $\lambda_1 = \mathbf{v}^T \mathbf{A} \mathbf{v}$ is differentiable.

Solution

Implementation

import torch

n = 5
A = torch.randn(n, n, dtype=torch.float64)
A = A + A.T   # symmetric
A.requires_grad_(True)

# Power iteration
v = torch.randn(n, dtype=torch.float64)
v = v / v.norm()
for _ in range(200):
    v = A @ v
    v = v / v.norm()

lam = v @ A @ v   # Rayleigh quotient
lam.backward()

print(f"Dominant eigenvalue: {lam.item():.6f}")
print(f"torch.linalg.eigvalsh: {torch.linalg.eigvalsh(A.detach())[-1]:.6f}")
print(f"Gradient shape: {A.grad.shape}")

ex-sp-ch12-12

Hard

Write a backend-agnostic function that computes the condition number of a matrix. It should work with NumPy, PyTorch, and CuPy arrays using the Array API.

Show Hint

Use array_api_compat.array_namespace(x) to get the right module.

Condition number = ratio of largest to smallest singular value.

Solution

Implementation

import array_api_compat
import numpy as np
import torch

def condition_number(A):
    xp = array_api_compat.array_namespace(A)
    s = xp.linalg.svdvals(A)
    return s[0] / s[-1]

# NumPy
A_np = np.random.randn(5, 5)
print(f"NumPy: {condition_number(A_np):.4f}")

# PyTorch
A_pt = torch.randn(5, 5, dtype=torch.float64)
print(f"PyTorch: {condition_number(A_pt):.4f}")

ex-sp-ch12-13

Hard

Implement a differentiable Tikhonov-regularized least squares solver: $\hat{\mathbf{x}} = (\mathbf{A}^H\mathbf{A} + \alpha \mathbf{I})^{-1}\mathbf{A}^H\mathbf{b}$ . Then use autograd to find the optimal regularization parameter $\alpha$ that minimizes the MSE against a known ground truth.

Show Hint

Parameterize $\alpha = e^\gamma$ to ensure positivity, and optimize $\gamma$ .

Use torch.linalg.solve instead of computing the inverse.

Solution

Implementation

import torch

m, n = 20, 5
torch.manual_seed(42)
A = torch.randn(m, n, dtype=torch.float64)
x_true = torch.randn(n, dtype=torch.float64)
noise = 0.1 * torch.randn(m, dtype=torch.float64)
b = A @ x_true + noise

gamma = torch.tensor(0.0, dtype=torch.float64, requires_grad=True)
optimizer = torch.optim.Adam([gamma], lr=0.1)

for step in range(300):
    alpha = torch.exp(gamma)
    AHA = A.T @ A + alpha * torch.eye(n, dtype=torch.float64)
    x_hat = torch.linalg.solve(AHA, A.T @ b)
    mse = ((x_hat - x_true) ** 2).mean()
    optimizer.zero_grad()
    mse.backward()
    optimizer.step()

    if step % 50 == 0:
        print(f"Step {step}: alpha={alpha.item():.4f}, MSE={mse.item():.6f}")

ex-sp-ch12-14

Challenge

Implement an end-to-end differentiable beamformer. Given a batch of channel vectors $\mathbf{h}_i \in \mathbb{C}^4$ , find beamforming weights $\mathbf{w} \in \mathbb{C}^4$ that maximize the average SINR across the batch, using gradient ascent through the complex-valued SINR expression.

Show Hint

$\text{SINR} = |\mathbf{w}^H \mathbf{h}|^2 / (\|\mathbf{w}\|^2 \sigma_n^2)$ .

Use Wirtinger calculus — PyTorch handles the complex gradients.

Solution

Implementation

import torch

n_ant = 4
n_batch = 100
sigma2 = 0.1

torch.manual_seed(0)
H = torch.randn(n_batch, n_ant, dtype=torch.complex128)

w = torch.randn(n_ant, dtype=torch.complex128, requires_grad=True)
optimizer = torch.optim.Adam([w], lr=0.01)

for step in range(500):
    w_norm = w / w.norm()
    signal = (H @ w_norm).abs().pow(2)     # |w^H h|^2
    sinr = signal / sigma2
    loss = -sinr.mean()                     # maximize SINR
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if step % 100 == 0:
        print(f"Step {step}: avg SINR = {-loss.item():.4f}")

# Compare with MRC (matched filter = dominant eigenvector of H^H H)
R = H.T.conj() @ H / n_batch
eigvals, eigvecs = torch.linalg.eigh(R)
w_mrc = eigvecs[:, -1]
sinr_mrc = (H @ w_mrc).abs().pow(2).mean() / sigma2
print(f"MRC SINR: {sinr_mrc.item():.4f}")

ex-sp-ch12-15

Challenge

Build a mixed SciPy + PyTorch pipeline: use scipy.sparse to assemble a finite-difference Laplacian matrix for a 2D Poisson equation on a $50 \times 50$ grid, convert to a dense PyTorch tensor, solve the system on GPU using torch.linalg.solve, then differentiate the solution with respect to the right-hand side forcing function.

Show Hint

Use scipy.sparse.kronsum or manual Kronecker structure for the 2D Laplacian.

The gradient $\partial \mathbf{u} / \partial \mathbf{f}$ should be $\mathbf{L}^{-1}$ (the Green's function).

Solution

Implementation

import numpy as np
import scipy.sparse as sp
import torch

n = 50
e = np.ones(n)
L1 = sp.diags([-e[1:], 2*e, -e[1:]], [-1, 0, 1], format="csr")
I = sp.eye(n)
L2 = sp.kron(I, L1) + sp.kron(L1, I)   # 2D Laplacian

L_pt = torch.from_numpy(L2.toarray()).to(torch.float64)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
L_dev = L_pt.to(device)

# RHS with gradient tracking
f = torch.randn(n*n, dtype=torch.float64, device=device,
                 requires_grad=True)

# Solve
u = torch.linalg.solve(L_dev, f)

# Differentiate w.r.t. f
objective = u.sum()
objective.backward()

print(f"Solution shape: {u.shape}")
print(f"Gradient shape: {f.grad.shape}")
# f.grad should be row-sum of L^{-1}

Exercises

ex-sp-ch12-01

Implementation

ex-sp-ch12-02

Implementation

ex-sp-ch12-03

Implementation

ex-sp-ch12-04

Implementation

ex-sp-ch12-05

Implementation

ex-sp-ch12-06

Implementation

ex-sp-ch12-07

Implementation

ex-sp-ch12-08

Implementation

ex-sp-ch12-09

Implementation

ex-sp-ch12-10

Implementation

ex-sp-ch12-11

Implementation

ex-sp-ch12-12

Implementation

ex-sp-ch12-13

Implementation

ex-sp-ch12-14

Implementation

ex-sp-ch12-15

Implementation