SFT：让续写机器学会「好好回答」SFT: teaching a completion machine to answer well从业者视角的方法论 · 关键事实挂官方出处practitioner-perspective methodology · key facts carry official sources

SFT 就是拿「问题 → 好回答」的示范对，继续训练预训练模型。

预训练模型会续写，但不会对话：你问它问题，
它可能把问题续写成「这个问题有以下几种答案……」而不是直接回答。
SFT 教的是形式与分寸：按对话格式答、别瞎编、该拒就拒。

SFT means continuing to train a pre-trained model on demonstration pairs of “question → good answer”.

A pre-trained model can continue text but cannot hold a conversation: ask it a question
and it may continue the question into “This question has the following answers…” instead of answering directly.
What SFT teaches is form and judgement: answer in conversational format, don't fabricate, refuse when refusal is due.

InstructGPT（arXiv 2203.02155）的路线是外包标注员写示范回答，
成本高、规模小（SFT 阶段约一万多条）。

FLAN（arXiv 2109.01652）走另一路：
把上百个传统 NLP 数据集改写成指令模板，让模型见遍任务形态。

LLaMA 2（arXiv 2307.09288）混合两者，并加入自举：
用自家模型生成候选、人工筛，扩大数据量。

现在开放权重家族的 SFT 配方多数只公开数量不公开来源明细，
本站按官方写明的部分转述；没写明的不猜。

InstructGPT (arXiv 2203.02155) went with outsourced annotators writing the demonstration answers —
expensive and small-scale (a little over ten thousand examples in the SFT stage).

FLAN (arXiv 2109.01652) took another route:
rewriting hundreds of classic NLP datasets into instruction templates so the model sees every task shape.

LLaMA 2 (arXiv 2307.09288) mixes both and adds bootstrapping:
its own models generate candidates, humans filter them, scaling the data volume up.

Today most open-weight families publish only the quantity of their SFT recipe, not the source breakdown;
this site transcribes what the official statements say and does not guess at what they leave out.

LIMA（arXiv 2305.11206）只用 1000 条人工精选样本做 SFT，
输出质量接近数万条的流水线数据。

结论不是「1000 条就够」，而是：SFT 主要教形式，知识在预训练里已经有了。
示范对里的知识含量不重要，回答的风格、结构、边界才是它学的东西。

从业者推论：SFT 数据里混进错误事实，模型学到的不是那条事实，
是「这种错误说法是合格回答」的风格。数据审核的重点在示范质量，不在条数。

LIMA (arXiv 2305.11206) did SFT on just 1,000 hand-curated samples,
with output quality close to pipeline data of tens of thousands.

The conclusion is not “1,000 examples are enough” but this: SFT mostly teaches form — the knowledge was already there from pre-training.
How much knowledge sits inside the demonstration pairs does not matter; what the model learns is the style, structure and boundaries of the answers.

Practitioner corollary: when wrong facts slip into SFT data, what the model learns is not the fact itself
but the style of “this kind of wrong claim is an acceptable answer”. Data review should focus on demonstration quality, not on counts.

全参数微调要存全部梯度和优化器状态，显存翻倍不止。

LoRA（arXiv 2106.09685）冻结基座权重，只训注入的低秩小矩阵。
可训参数降到千分之几，产物只有几十 MB，可以热插拔换装。

QLoRA（arXiv 2305.14314）把基座压到 4-bit 再挂 LoRA，
单卡微调 30B/65B 成为可能——本站收录的不少社区微调模型走的就是这条路。

边界要如实说：LoRA 改「表达与风格」效率高，往模型里灌新知识的效果弱于全参训练。
低秩增量装不下多少新事实。「用 LoRA 把私有知识库训进模型」通常是错的工具，
检索（RAG）才是常规解。

Full-parameter fine-tuning has to store every gradient and optimiser state — memory more than doubles.

LoRA (arXiv 2106.09685) freezes the base weights and trains only injected low-rank matrices.
Trainable parameters drop to a fraction of a percent, the artefact is tens of MB, and adapters can be hot-swapped.

QLoRA (arXiv 2305.14314) squeezes the base to 4-bit before attaching LoRA,
making single-GPU fine-tuning of 30B/65B models possible — quite a few community fine-tunes listed on this site follow exactly this route.

State the limits honestly: LoRA is efficient at changing “expression and style” but weak at injecting new knowledge compared with full-parameter training.
A low-rank delta cannot hold many new facts. “Training a private knowledge base into the model with LoRA” is usually the wrong tool;
retrieval (RAG) is the standard answer.

SFT 教了形式，但「哪个回答更好」它只会模仿示范，不会判断。
判断好坏交给下一步——奖励模型与偏好优化（RLHF/DPO），见下一篇。

学 AI 站的 训练三阶段 词条
把三步的科普版一页讲完；这一篇是每步的从业者深版。

SFT teaches form, but for “which answer is better” it can only imitate the demonstrations — it cannot judge.
Judging quality belongs to the next step — reward models and preference optimisation (RLHF/DPO), covered in the next guide.

The Learn site's three-stage training term page
covers all three steps in one explainer page; this guide is the practitioner deep-dive for each step.

使用者视角的一页科普：学 AI 站的 监督学习
与 训练三阶段。

本篇讲数据路线的取舍、LIMA 的口径变化、LoRA 的适用边界——
同一个词，两处深度不同，互链不重复。

The user-perspective explainers live on the Learn site: supervised learning
and three-stage training.

This guide covers the trade-offs between data routes, how LIMA changed the consensus, and where LoRA applies —
same term, different depth on each side, cross-linked without duplication.