Text Classification
Transformers
Safetensors
English
modernbert
agent-safety
tool-calling
guardrails
long-context
Eval Results (legacy)
text-embeddings-inference
Instructions to use ProCreations/auto-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ProCreations/auto-1b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="ProCreations/auto-1b")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("ProCreations/auto-1b") model = AutoModelForSequenceClassification.from_pretrained("ProCreations/auto-1b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -211,21 +211,25 @@ Measured on 400 benchmark rows against fp32, scoring **decision agreement** at t
|
|
| 211 |
(the only metric that matters for a gate β a build can look fine on mean error and still flip
|
| 212 |
calls near the boundary):
|
| 213 |
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
|
| 217 |
-
|
|
| 218 |
-
|
|
| 219 |
-
|
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
| 225 |
-
|
| 226 |
-
|
| 227 |
-
|
| 228 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 229 |
|
| 230 |
## How it was built
|
| 231 |
|
|
@@ -262,5 +266,11 @@ benchmark is excluded by content hash (verified 0/3000 overlap).
|
|
| 262 |
|
| 263 |
## Other formats
|
| 264 |
|
| 265 |
-
- [`ProCreations/auto-1b-
|
| 266 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 211 |
(the only metric that matters for a gate β a build can look fine on mean error and still flip
|
| 212 |
calls near the boundary):
|
| 213 |
|
| 214 |
+
Re-running the **full 3,000-item benchmark** at each precision:
|
| 215 |
+
|
| 216 |
+
| precision | accuracy | AUROC | false-approve | memory |
|
| 217 |
+
|---|---|---|---|---|
|
| 218 |
+
| fp32 | 0.964000 | 0.992845 | 0.040685 | ~3.9 GB |
|
| 219 |
+
| **bf16** | **0.964000** | **0.992845** | **0.040685** | **~2 GB** |
|
| 220 |
+
| fp16 | 0.964333 | 0.992839 | 0.039971 | ~2 GB |
|
| 221 |
+
| int8 (ONNX dynamic) | β | β | β | ~1 GB |
|
| 222 |
+
|
| 223 |
+
**bf16 is exactly lossless** β identical on every metric and every context-length slice, zero
|
| 224 |
+
flipped verdicts across 3,000 items. Ready-made at
|
| 225 |
+
[`ProCreations/auto-1b-bf16`](https://huggingface.co/ProCreations/auto-1b-bf16), or pass
|
| 226 |
+
`dtype=torch.bfloat16` here. fp16 differs by a single item with marginally worse AUROC; the two
|
| 227 |
+
are equivalent in practice, and bf16 is preferred for carrying no overflow risk.
|
| 228 |
+
|
| 229 |
+
**int8 is not salvageable.** It flips roughly 1 verdict in 20, and per-channel quantization β
|
| 230 |
+
the standard fix β scored *worse* (94.25% vs 95.00% decision agreement). The failure is
|
| 231 |
+
activation outliers in the GeGLU layers, which dynamic quantization cannot handle. See the
|
| 232 |
+
[ONNX repo](https://huggingface.co/ProCreations/auto-1b-ONNX) for the breakdown.
|
| 233 |
|
| 234 |
## How it was built
|
| 235 |
|
|
|
|
| 266 |
|
| 267 |
## Other formats
|
| 268 |
|
| 269 |
+
- [`ProCreations/auto-1b-bf16`](https://huggingface.co/ProCreations/auto-1b-bf16) β half the
|
| 270 |
+
size, benchmark-identical to this model
|
| 271 |
+
- [`ProCreations/auto-1b-ONNX`](https://huggingface.co/ProCreations/auto-1b-ONNX) β ONNX fp32
|
| 272 |
+
(its int8 build is documented as unfit for gating)
|
| 273 |
+
|
| 274 |
+
A GGUF build was published and then **withdrawn**: llama.cpp converts the model, but its
|
| 275 |
+
`--pooling rank` path returns zero for a 2-class classification head, so it could not actually
|
| 276 |
+
make approve/deny decisions.
|