量化档位与精度损失：能说什么、不能说什么Quantisation tiers and accuracy loss: what can be said, and what cannot从业者视角的方法论 · 关键事实挂官方出处practitioner-perspective methodology · key facts carry official sources

量化档位名说的是权重怎么存，不是模型得了多少分。

这一篇只划一条线：哪些话有依据可以说，哪些话必须留空。

本站不下载权重、不跑评测，也不写「某档够用」这类建议。能给的是体积、命名口径和算式边界。

A quantisation tier name says how weights are stored. It does not carry a score for the model.

This guide draws one line: which statements have evidence, and which must stay blank.

This site downloads no weights, runs no evaluations, and issues no "this tier is enough" advice. What it offers is file size, naming arithmetic and the scope of the formulas.

这三句的证据来自发布页文件清单与公开的命名标准。

• 体积：某个量化文件的字节数写在发布页文件清单里，可以直接核对。
• 每权重比特数：由档名按公开命名标准推算，本站标为推导，不冒充实测。
• 档位覆盖：某家量化仓提供哪些档、是否带 importance matrix，同样是清单事实。

方法之间的差别，官方 quantize 文档只说体现在磁盘体积与推理速度上，数值绑定它自己的示例：tools/quantize/README.md

The three statements below rest on publishing-page file listings and the public naming standard.

• Size: the byte count of a quantised file is in the publishing page's file listing and can be checked directly.
• Bits per weight: derived from the tier label under the public naming standard. This site marks it as a derivation, never as a measurement.
• Tier coverage: which tiers a quantiser publishes, and whether it ships an importance matrix, are also listing facts.

On differences between methods, the official quantize documentation says only that they show up in disk size and inference speed, and its figures belong to its own example: tools/quantize/README.md

同一句结论换个模型就得重新取得，所以下面四类话本站不写。

• 不写「这一档质量够用」：够不够取决于任务与判定标准，不是档名。
• 不写「降一档快多少」：官方文档不给跨模型可复用的比例。
• 不写「某家量化更稳」：各家档位与实现细节不同，没有可比的公开口径。
• 不做跨档位排名，也不把档位与「适合谁」画等号。

需要速度或质量结论时，在自己的授权环境按固定任务记录；那是你的读数，不是本站的。

A conclusion of this kind has to be re-established for every model, so this site does not write the four statements below.

• Not "this tier is good enough": sufficiency depends on the task and its pass criteria, not on the tier label.
• Not "drop a tier and gain X in speed": official documentation gives no proportion usable across models.
• Not "this quantiser is safer": implementations and tier selections differ, and there is no comparable published measure.
• No ranking across tiers, and no equation between a tier and "who it suits".

When you need speed or quality evidence, record it in your own authorised environment against a fixed task. That reading is yours, not this site's.

档名描述的是权重的存储方式与大致体积。它不携带任何任务层面的质量含义。

同名档在不同模型上的真实比特数略有差异，这是本站把比特数标为推导的原因。

• 官方 README 列出的整数位宽覆盖 1.5、2、3、4、5、6、8 位，出处见 llama.cpp README。
• 官方 quantize 文档给了一张示例表，按档位列出 bits/weight 与体积（GiB）。表头写明它测的是某个具体 8B 模型，换个模型不会正好是同一串数。
• 微调路线里的 4-bit 也是数据类型选择：论文摘要自述 NF4 是对正态分布权重信息论上最优的 4-bit 类型，出处见 arXiv 2305.14314。它说的是训练时加载精度，和推理用的 GGUF 档名不是同一层。

本站的档位归属只按推算出的每权重比特数落档，规则写在数据规范里，任何人可以复算。

A tier label describes how weights are stored and roughly how large the file is. It carries no task-level quality meaning.

Real bits per weight differ slightly between models under the same label, which is why this site marks the figure as derived.

• The official README lists integer bit widths covering 1.5, 2, 3, 4, 5, 6 and 8 bits; source: llama.cpp README.
• The official quantize documentation includes an example table giving bits/weight and size in GiB per tier. Its heading states the figures belong to one specific 8B model; another model will not reproduce the same numbers.
• In fine-tuning, 4-bit is also a data-type choice: the paper abstract describes NF4 as information-theoretically optimal for normally distributed weights; source: arXiv 2305.14314. That is training load precision, a different layer from an inference GGUF tier label.

This site assigns a tier only from the derived bits-per-weight value. The rule is written in the data specification and anyone can recompute it.

官方 quantize 文档写明：量化可能引入精度损失。损失通常用困惑度（ppl）或 KL 散度（kld）衡量。官方文档说，合适的 importance matrix 数据可以减小它。

这句话里有三层可用信息，也到此为止。

• 指标是语言建模口径的代理量，不等于任何具体任务的表现。
• 官方给出的数值绑定它自己的模型、语料与设置，换模型不能照抄。
• 未公布这些数字时，本站只记录缺口，不用别的模型的数字补上。

「下降多少算能接受」取决于下游任务与验收标准。本站没有跑过评测，也不设默认阈值。

The official quantize documentation states that quantisation may introduce accuracy loss. It says the loss is usually measured in perplexity (ppl) and/or Kullback-Leibler divergence (kld), and that a suitable importance matrix can reduce it.

That sentence carries three usable facts, and stops there.

• Those metrics are language-modelling proxies, not performance on any concrete task.
• Published figures are bound to that document's own model, corpus and settings; they cannot be copied onto another model.
• Where the numbers are not published, this site records the gap instead of borrowing figures from elsewhere.

How much drop is acceptable depends on the downstream task and its acceptance criteria. This site ran no evaluation and sets no default threshold.

档位改变的是权重那一笔。缓存不会自动跟着变小，除非运行器明确支持量化缓存。

vLLM 的量化文档把「量化 KV 缓存」单独列为一项，说明权重精度与缓存精度是两条设置：features/quantization。同一站点的内存控制文档把缓存空间按 GiB 配置，见 conserving_memory。

填写与核对这两笔账的入口是容量工作台：上下文与并发预算。

• 适用前提：各层结构相同、每层保留完整键值的标准全注意力 MHA / GQA 解码器。
• 不适用：MLA、混合状态空间、滑动窗口与逐层缓存不同的架构，需要另算。
• 单位：权重、缓存、运行时预留统一写成 GiB；十进制 GB 先换算再相加。
• 无法据此承诺：能否启动、是否超时、输出是否合格，都不在这条算式的范围内。

A tier changes the weights line of the bill. The cache does not shrink automatically unless the runtime explicitly supports a quantised cache.

vLLM's quantisation documentation lists "Quantized KV Cache" as its own item, which shows weight precision and cache precision are two separate settings: features/quantization. The memory-conservation page on the same site configures cache space in GiB: conserving_memory.

Fill and check both lines in the capacity workbench: context and concurrency budget.

• Scope: standard full-attention MHA/GQA decoders with identical layers and complete per-layer key/value state.
• Out of scope: MLA, hybrid state-space designs, sliding windows and architectures whose caches differ per layer need separate arithmetic.
• Units: write weights, cache and runtime reserve all in GiB; convert decimal GB before adding.
• Not established by this: whether a run starts, whether it stays within a time limit, and whether output passes. None of that is inside the formula.

本节的数字全是虚构教学假设，不对应任何真实模型、发布页或本站运行结果。

设某密集模型的高精度权重按 30 GiB 计，练习假设换成 5 GiB 档。上下文与并发不变，缓存按 2 GiB、运行时预留按 1 GiB 计。

两种假设下的合计分别是 33 GiB 与 8 GiB。差值只说明权重那一笔的变化量。

再假设任务可用内存是 10 GiB。按这些假设，5 GiB 档留有 2 GiB 余量，30 GiB 档超出预算。这是练习算式，不是运行结果。

• 查不过去时的下一步不是换档重试，而是回查驻留假设、缓存字段与架构适用范围。
• 体积对照请回到本站系列页 B 区：它按发布体积 × 1.15 初筛候选，不判断优劣，也不保证能跑。

Every number in this section is a fictional teaching assumption. It matches no real model, publishing page or result produced by this site.

Assume a dense model's high-precision weights occupy 30 GiB, and the exercise swaps them for a 5 GiB tier. Context and concurrency stay fixed, with cache assumed at 2 GiB and runtime reserve at 1 GiB.

Under those assumptions the totals are 33 GiB and 8 GiB. The difference only measures the weights line.

Now assume 10 GiB is available for the task: the 5 GiB case leaves 2 GiB headroom and the 30 GiB case exceeds the budget. This is exercise arithmetic, not a run result.

• When a check fails, the next step is not blind tier-swapping but re-examining the residency assumption, the cache fields and the architecture scope.
• For size comparisons go back to block B of the series pages: it filters candidates by published size × 1.15, judges nothing, and does not guarantee that anything runs.

系列页给的是「你的体积预算下有哪些文件」。这一篇给的是「关于这些文件你能说到哪一步」。

• 容量与缓存字段用工作台分开列：上下文与并发预算。
• 费用构成用 成本计算器 记录，本站不保存单价。
• 结果判定先按 20 项自测清单 写清条件，再决定要不要换档。

学 AI 站的 量化、量化方法、bpw 与 困惑度 是给使用者的一页科普。

这一篇给的是说法边界与容量口径。同一个概念在两处深度不同，互链不重复。

Series pages answer "which files fit my size budget"; this guide answers "how far can a statement about those files be taken".

• List capacity and cache fields separately in the workbench: context and concurrency budget.
• Record cost components with the cost calculator; this site stores no unit prices.
• Write the acceptance conditions first with the 20-item self-test, then decide whether a tier change is even the question.

The Learn site's quantisation, quantisation methods, bpw and perplexity pages are one-page explainers for users.

This guide gives the statement boundary and the capacity terms. Same concept, different depth, cross-linked without duplication.