Instruction Tuning and Chat Models

Definition:

Instruction Tuning

Instruction tuning trains a model on diverse instruction-response pairs to follow natural language instructions:

mathcalL=−sumtintextresponselogP(wtmidtextinstruction,w<t)\\mathcal{L} = -\\sum_{t \\in \\text{response}} \\log P(w_t \\mid \\text{instruction}, w_{<t})

The loss is computed only on response tokens, not on the instruction.

Definition:

Chat Templates

Chat models use special tokens to delimit roles:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are a wireless engineer.<|eot_id|>
<|start_header_id|>user<|end_header_id|>
Explain OFDM.<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>

Each model family uses different special tokens — always use tokenizer.apply_chat_template().

Example: Creating a Telecom Chat Dataset

Create an instruction-tuning dataset for wireless communications Q&A.

Solution
Data Creation
dataset = [
    {"messages": [
        {"role": "system", "content": "You are a 5G expert."},
        {"role": "user", "content": "What is the bandwidth of an NR carrier?"},
        {"role": "assistant", "content": "In 5G NR, carrier bandwidth..."},
    ]},
]
Training a Small Language Model from ScratchMultimodal Models