ProCreations commited on
Commit
69f939a
Β·
verified Β·
1 Parent(s): b8ca6a8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +27 -17
README.md CHANGED
@@ -211,21 +211,25 @@ Measured on 400 benchmark rows against fp32, scoring **decision agreement** at t
211
  (the only metric that matters for a gate β€” a build can look fine on mean error and still flip
212
  calls near the boundary):
213
 
214
- | precision | decision agreement | max Ξ”P(deny) | memory |
215
- |---|---|---|---|
216
- | fp32 | reference | β€” | ~3.9 GB |
217
- | **fp16** | **100.00%** | 3.2e-03 | **~2 GB** |
218
- | bf16 | 99.75% | 2.7e-02 | ~2 GB |
219
- | int8 (ONNX dynamic) | 95.00% | 9.4e-01 | ~1 GB |
220
-
221
- **fp16 halves memory at zero measured cost** β€” pass `dtype=torch.float16` as shown above.
222
-
223
- Two results worth knowing. **fp16 beats bf16**: bf16 spends bits on exponent range an encoder
224
- forward pass does not need, trading away mantissa precision to get it β€” the opposite of the
225
- tradeoff you want at inference, even though bf16 is correct for training. And **int8 is not
226
- salvageable** β€” per-channel quantization, the standard fix, scored *worse* (94.25%). The failure
227
- is activation outliers in the GeGLU layers, which dynamic quantization cannot handle. See the
228
- [ONNX repo](https://huggingface.co/ProCreations/auto-1b-ONNX) for the full breakdown.
 
 
 
 
229
 
230
  ## How it was built
231
 
@@ -262,5 +266,11 @@ benchmark is excluded by content hash (verified 0/3000 overlap).
262
 
263
  ## Other formats
264
 
265
- - [`ProCreations/auto-1b-gguf`](https://huggingface.co/ProCreations/auto-1b-gguf) β€” llama.cpp
266
- - [`ProCreations/auto-1b-ONNX`](https://huggingface.co/ProCreations/auto-1b-ONNX) β€” ONNX + int8
 
 
 
 
 
 
 
211
  (the only metric that matters for a gate β€” a build can look fine on mean error and still flip
212
  calls near the boundary):
213
 
214
+ Re-running the **full 3,000-item benchmark** at each precision:
215
+
216
+ | precision | accuracy | AUROC | false-approve | memory |
217
+ |---|---|---|---|---|
218
+ | fp32 | 0.964000 | 0.992845 | 0.040685 | ~3.9 GB |
219
+ | **bf16** | **0.964000** | **0.992845** | **0.040685** | **~2 GB** |
220
+ | fp16 | 0.964333 | 0.992839 | 0.039971 | ~2 GB |
221
+ | int8 (ONNX dynamic) | β€” | β€” | β€” | ~1 GB |
222
+
223
+ **bf16 is exactly lossless** β€” identical on every metric and every context-length slice, zero
224
+ flipped verdicts across 3,000 items. Ready-made at
225
+ [`ProCreations/auto-1b-bf16`](https://huggingface.co/ProCreations/auto-1b-bf16), or pass
226
+ `dtype=torch.bfloat16` here. fp16 differs by a single item with marginally worse AUROC; the two
227
+ are equivalent in practice, and bf16 is preferred for carrying no overflow risk.
228
+
229
+ **int8 is not salvageable.** It flips roughly 1 verdict in 20, and per-channel quantization β€”
230
+ the standard fix β€” scored *worse* (94.25% vs 95.00% decision agreement). The failure is
231
+ activation outliers in the GeGLU layers, which dynamic quantization cannot handle. See the
232
+ [ONNX repo](https://huggingface.co/ProCreations/auto-1b-ONNX) for the breakdown.
233
 
234
  ## How it was built
235
 
 
266
 
267
  ## Other formats
268
 
269
+ - [`ProCreations/auto-1b-bf16`](https://huggingface.co/ProCreations/auto-1b-bf16) β€” half the
270
+ size, benchmark-identical to this model
271
+ - [`ProCreations/auto-1b-ONNX`](https://huggingface.co/ProCreations/auto-1b-ONNX) β€” ONNX fp32
272
+ (its int8 build is documented as unfit for gating)
273
+
274
+ A GGUF build was published and then **withdrawn**: llama.cpp converts the model, but its
275
+ `--pooling rank` path returns zero for a 2-class classification head, so it could not actually
276
+ make approve/deny decisions.