Frontier · Advanced
Fine-tuning and post-training
Instruction tuning, RLHF, DPO and parameter-efficient methods such as LoRA.
Why it matters
Post-training is what turns a raw pre-trained model into a useful assistant. Knowing it tells you when fine-tuning beats prompting, and when it does not.
Key ideas
- Supervised instruction tuning
- Reinforcement learning from human feedback
- Direct preference optimisation
- LoRA and adapters
- Distillation
You will be able to
- Decide prompt vs RAG vs fine-tune
- Prepare a preference dataset
- Run a LoRA fine-tune
Primary reading
- Training language models to follow instructions with human feedback (opens in a new tab)Ouyang et al. · 2022
- Direct Preference Optimization (opens in a new tab)Rafailov et al. · 2023
- LoRA: Low-Rank Adaptation of Large Language Models (opens in a new tab)Hu et al. · 2021
Learn it here
Planned course