Exercises

Create a Gymnasium environment and run 100 episodes with random actions. Plot cumulative rewards.

Implement tabular Q-learning for a 4x4 gridworld. Visualise learned Q-values.

Implement epsilon-greedy exploration with epsilon decaying from 1.0 to 0.01.

Implement a replay buffer with fixed capacity and uniform sampling.

Compute discounted returns for a trajectory with rewards [1, 0, -1, 10] and gamma=0.99.

Implement DQN with experience replay and target network for CartPole-v1.

Implement Double DQN and compare Q-value estimates to standard DQN.

Implement REINFORCE for CartPole. Plot learning curve over 500 episodes.

Add a learned baseline (value function) to REINFORCE and compare variance.

Implement a custom Gym environment for multi-user power control.

Implement PPO with clipped surrogate objective and GAE advantage estimation.

Train DQN for power control in a 4-user interference channel. Compare to max-SINR heuristic.

Implement prioritised experience replay and compare to uniform replay.

Train a PPO agent for multi-user scheduling that optimises proportional fairness.

Implement multi-agent RL for distributed power control where each BS is an independent agent.