---
license: apache-2.0
language:
- en
library_name: gguf
tags:
- gguf
- qwen3.5
- mxfp4
- fp4
- vision
- multimodal
- reasoning
- uncensored
- tool-use
- 1m-context
- cybersecurity
- biomedical
- agentic
base_model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
pipeline_tag: image-text-to-text
inference: false
quantized_by: FreedomAISVR
---

# Qwythos-9B-Claude-Mythos-5-1M-MXFP4-GGUF

MXFP4 GGUF quantization of [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) -- a full-parameter reasoning model built on a deeply uncensored Qwen3.5-9B base, post-trained on 500M+ tokens of Claude Mythos and Claude Fable traces with chain-of-thought generated in-house by Empero AI's internal tool `rethink`.

## What makes Qwythos special

- **1M token context** -- YaRN rope-scaling enabled by default for a full 1,048,576-token context window. One of the longest context windows in any 9B open-weight model. Suitable for whole-codebase reasoning, multi-document research, and long agentic trajectories.
- **Massive benchmark gains over base** -- +34 pts MMLU, +30 pts gsm8k-strict, +19 pts gsm8k-flex under matched evaluation.
- **Native function calling** -- OpenAI/Qwen3.5-style tool use out of the box. Pass `tools=[...]` and the model emits valid `<tool_call>` blocks. Self-corrects with Python executor and web search (7/7 test prompts succeeded).
- **Uncensored by design** -- Engages substantively with technically demanding questions across cybersecurity, red-teaming, biology, pharmacology, and clinical medicine where over-aligned models refuse or hedge.
- **Reasoning model** -- Every answer opens with a `<think>` block before the final response. Use generous `max_new_tokens` (16,384 recommended).

## Domain strengths

- **Cybersecurity** -- SQL injection mitigations, TLS handshake structure, EDR/process-injection detection, MITRE ATT&CK ransomware kill chains, hashcat modes, CVE analysis.
- **Biomedical** -- CRISPR-Cas9 mechanisms, mRNA vaccines, SARS-CoV-2 spike protein, antibiotic resistance, receptor pharmacology, organophosphate AChE inhibition.
- **Clinical medicine** -- ACS chest-pain differential, type-2 diabetes pathophysiology, sepsis recognition (qSOFA), therapeutic-window reasoning.
- **Math** -- 86% gsm8k, multi-step word problems, competition math. Verified by Python executor when invoked.

## About MXFP4

MXFP4 (Microscaling FP4) is an open standard (OCP) 4-bit floating point format (E2M1) supported by NVIDIA, AMD, Microsoft, and Meta.

- Works on any GPU with MX support
- Open standard -- not vendor locked
- Block-scaled format with shared scale factors

**When to use MXFP4 vs other formats:**
- **MXFP4** -- Open standard, broad hardware support
- **NVFP4** -- NVIDIA Blackwell native, best performance on RTX 50-series
- **Q4_K_M** -- Best for pre-Blackwell GPUs and CPU inference

## Files

| File | Type | Size | Description |
|------|------|------|-------------|
| `qwythos-9b-mxfp4.gguf` | MXFP4 | ~4.8 GB | Text model (4.52 BPW) |
| `mmproj-qwythos-9b-f16.gguf` | F16 | ~918 MB | Vision encoder (SigLIP ViT, 27 layers) |

## Quantization Details

| Property | Value |
|----------|-------|
| Format | MXFP4 (E2M1) |
| Bits Per Weight | 4.52 BPW |
| Source Model | empero-ai/Qwythos-9B-Claude-Mythos-5-1M |
| Architecture | Qwen3_5ForConditionalGeneration |
| Parameters | 9.4B (BF16 source) |
| Layers | 32 (hybrid Gated DeltaNet + full attention) |
| Hidden Size | 4096 |
| Context Length | 1,048,576 (1M, YaRN) |
| Vision | Yes (SigLIP ViT, frozen from base) |
| Thinking | Enabled by default (opt-out via `enable_thinking=false`) |
| Training | 500M+ tokens, Claude Mythos/Fable traces, full SFT |

## Usage

### llama.cpp CLI

```bash
# Text only
./llama-cli -m qwythos-9b-mxfp4.gguf -p "Hello" -n 100

# With vision (requires mmproj)
./llama-server -m qwythos-9b-mxfp4.gguf \
  --mmproj mmproj-qwythos-9b-f16.gguf \
  --host 0.0.0.0 --port 8080 -ngl 99
```

### llama-cpp-python

```python
from llama_cpp import Llama

llm = Llama(
    model_path="qwythos-9b-mxfp4.gguf",
    n_gpu_layers=-1,
    chat_format="chatml"
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain how organophosphate nerve agents inhibit acetylcholinesterase."}],
    max_tokens=4096
)
print(output["choices"][0]["message"]["content"])
```

### huggingface-hub

```python
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="FreedomAISVR/Qwythos-9B-Claude-Mythos-5-1M-MXFP4-GGUF",
    filename="qwythos-9b-mxfp4.gguf"
)
mmproj_path = hf_hub_download(
    repo_id="FreedomAISVR/Qwythos-9B-Claude-Mythos-5-1M-MXFP4-GGUF",
    filename="mmproj-qwythos-9b-f16.gguf"
)
```

## Sampling recommendations

Qwythos was trained as a reasoning model. Use these settings for best results:

```
temperature=0.6
top_p=0.95
top_k=20
repetition_penalty=1.05
max_new_tokens=16384
```

Greedy decoding or very-low-temperature (T<=0.3) can cause repetition loops on long generations.

## Quantization Pipeline

1. Download source: `empero-ai/Qwythos-9B-Claude-Mythos-5-1M`
2. Convert to F16 GGUF: `convert_hf_to_gguf.py --outtype f16`
3. Extract mmproj: `convert_hf_to_gguf.py --mmproj --outtype f16`
4. Quantize text: `llama-quantize input-f16.gguf output-mxfp4.gguf MXFP4`
5. Patch GGUF metadata: `block_count` 33->32, `nextn_predict_layers` 1->0

## Hardware Requirements

| Component | Requirement |
|-----------|-------------|
| GPU | Any with MX support, or CPU fallback |
| VRAM | ~6 GB minimum |
| RAM | ~16 GB recommended |
| Storage | ~6 GB |

## Limitations

- **Reasoning model** -- Every answer opens with `<think>` block. Allow generous token budget.
- **Text-only fine-tune** -- Vision tower was frozen; vision behavior is inherited from base and was not tuned.
- **Uncensored** -- Add application-level safety layer for end-user deployments.
- **Verify specifics** -- Like all 9B models, can over-commit to specific identifiers (CVEs, drug dosages). Pair with tools for accuracy-critical deployments.

## License

Apache 2.0 (inherited from Qwen3.5-9B base)

## Acknowledgements

- Developed by [Empero AI](https://empero.org)
- Base model: [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) (Alibaba Qwen team)
- Training: TRL + Transformers
- Linear-attention kernels: flash-linear-attention, causal_conv1d