Detailed explanation
Examples
Frequently asked questions
Distillation or fine-tuning?
Distillation = transfer from big to small. Fine-tuning = adapt a model to a specific task/data. The two combine.
Related terms
Deep Learning
A subfield of machine learning using deep neural networks to learn complex representations.
Fine-tuning
Fine-tuning is the process of further training a pre-trained model on a smaller, task-specific dataset to specialize it for a particular domain, style, or task.
LLM (Large Language Model)
An LLM is a large language model trained on massive text corpora, capable of understanding and generating natural language across many tasks: writing, summarization, translation, code, reasoning.
Quantization
A technique that reduces the numerical precision of a model's weights to make it lighter and faster.
Last updated: 7/15/2026