---
base_model: openai/gpt-oss-120b
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
tags:
- quantization
- gguf
- mixture-of-experts
- text-generation
---

# gpt-oss-120b-PS-GGUF

GGUF quantizations of [gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b) by PeasantSmith & BlackFrost AI. 

| Quantization | BPW | Size | Quantizer |
|---|---|---|---|
| IQ4_NL | 4.91 | 71.75GB | Peasant Smith |
| [IQ2_XXS](https://huggingface.co/peasantsmith/gpt-oss-120b-IQ2_XXS-GGUF) | 2.x | 66.1GB | BlackFrost AI |

## Testing
Raw perplexity doesn't work here: gpt-oss needs the harmony response format, and even the F16 source returns NaN on unformatted text past chunk 8. Validated with task-based tests instead:
Quantization quality was validated with task-based tests instead:

| Test | Result |
|---|---|
| Greedy generation (factual question) | PASS, correct answer + complete reasoning block |
| LRU cache implementation task (8192 ctx) | PASS, doubly-linked-list, O(1) get/put, coherent |
| Needle recall (magic value 9137@8k ctx)| PASS, exact answer: `9137` |
| Runtime errors | 0 |

## Recommended parameters
- Temperature: `1.0`
- Top-k: `0` (i.e. all)
- Top-p: `1.0`

Example llama.cpp invocation:
```bash
llama-cli -m gpt-oss-120b-IQ4_NL.gguf \
  -c 8192 -fa -ngl 99 -ncmoe 36 \
  --split-mode layer --tensor-split 1,1,1 \
  --temp 1.0 --top-k 0 --top-p 1.0
```
Flag names vary across llama.cpp builds.

## Reproduce/Verify
```bash
llama-quantize \
  --tensor-type "ffn_gate_exps=iq4_nl" \
  --tensor-type "ffn_up_exps=iq4_nl" \
  --tensor-type "ffn_down_exps=q5_0" \
  --token-embedding-type q8_0 \
  gpt-oss-120b-F16.gguf gpt-oss-120b-IQ4_NL.gguf q8_0
```

## License
Quantized derivative of gpt-oss-120b: license and usage terms follow upstream, verify before redistribution. Base model by OpenAI.

## Additional information
The quantization is released under the PeasantSmith (PS) signature, reflected in the repository name `gpt-oss-120b-PS-GGUF`. Individual GGUF files use the standard `model-<quant>.gguf` naming (e.g. `gpt-oss-120b-IQ4_NL.gguf`) so Ollama and HuggingFace quant-tag parsing work correctly. The IQ4_NL label identifies this specific quantized release and should not be confused with the upstream model's original naming.