Exercises

Create an nn.LSTM with input_size=10, hidden_size=32, 2 layers, batch_first=True. Pass a batch of 8 sequences of length 20 through it and print output shapes.

Compare parameter counts of RNN, LSTM, and GRU with input_size=32, hidden_size=64.

Implement sequence classification using the last hidden state of an LSTM. Test on random data with 3 classes.

Use pack_padded_sequence and pad_packed_sequence to handle a batch of sequences with lengths [10, 7, 5, 3].

Create a bidirectional LSTM and verify the output hidden size is doubled.

Train an LSTM to predict the next value in a sine wave sequence. Plot predictions vs ground truth.

Implement a GRU cell from scratch using nn.Linear and sigmoid/tanh. Verify output matches nn.GRUCell.

Implement a Seq2Seq model for sequence reversal: given [1,2,3,4,5] output [5,4,3,2,1].

Implement a 4-layer TCN with dilations [1, 2, 4, 8] for sequence prediction. Compare training speed to LSTM.

Implement greedy search and beam search decoding for a Seq2Seq model. Compare output quality.

Implement truncated BPTT by detaching hidden states every K steps. Compare gradient norms with full BPTT.

Implement scheduled sampling that linearly transitions from teacher forcing to free running over training.

Build an LSTM-based channel tracker that estimates time-varying fading coefficients from pilot observations.

Implement an LSTM cell from scratch (all 4 gates) and train it. Verify gradients match PyTorch's nn.LSTMCell.

Compare LSTM, GRU, and TCN on the copy memory task: memorise a pattern and reproduce it after a long delay.