--- license: mit thumbnail: https://huggingface.co/AtomicChat/Qwen3-4B-DFlash-GGUF/resolve/main/hero.png base_model: - z-lab/Qwen3-4B-DFlash-b16 base_model_relation: quantized quantized_by: AtomicChat pipeline_tag: text-generation library_name: gguf tags: - atomic-chat - qwen3 - z-lab - gguf - llama.cpp - quantized ---

Base model: z-lab/Qwen3-4B-DFlash-b16

**Qwen3 4B Dflash b16**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Z Lab's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline. ## Highlights - **0.5B parameters**: the weights this repo quantizes. - **Context length**: 40,960 tokens (40K), as published by Z Lab. - **5 layers**: Dense decoder. - **Full imatrix ladder**: every quant is calibrated with an importance matrix. > [!NOTE] > These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model. > [!IMPORTANT] > Always pass `--jinja` so the **Qwen3 4B Dflash b16 chat template** is applied. Without it the model can emit malformed turns. ## Model Overview | Property | Value | |---|---| | Base model | `z-lab/Qwen3-4B-DFlash-b16` | | Parameters | 0.5B | | Layers | 5 | | Context length | 40,960 tokens (40K) | | Vocabulary | 151,936 | | Modalities | Text | | Architecture | Dense decoder, 32 attention heads over 8 KV heads, `DFlashDraftModel` | | This repo | GGUF quants (imatrix). Quants: `Q8_0` | ## Choosing a quant | Quant | Size | Notes | |---|---|---| | **`Q8_0`** | 0.6 GB | **Effectively lossless, reference quality.** | > [!TIP] > Pick the largest file that fits your (V)RAM with room for context. `Q8_0` is the sweet spot for most setups; `Q6_K` or `Q8_0` for maximum fidelity. ## Get started Run Qwen3 4B Dflash b16 locally with: - **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AtomicChat/Qwen3-4B-DFlash-GGUF`, pick a quant, hit **Use this model**. - **llama.cpp:** `llama-server -hf AtomicChat/Qwen3-4B-DFlash-GGUF:Q8_0 --jinja -c 8192` - **Ollama:** `ollama run hf.co/AtomicChat/Qwen3-4B-DFlash-GGUF:Q8_0` - **LM Studio / Jan:** search the repo id, download any quant. ## Best practices | Parameter | Value | |---|---| | temperature | 0.0 | Z Lab's recommended sampling configuration for `z-lab/Qwen3-4B-DFlash-b16`. ## Run in llama.cpp ```bash git clone https://github.com/ggml-org/llama.cpp cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server ``` ```bash ./llama.cpp/build/bin/llama-server \ -hf AtomicChat/Qwen3-4B-DFlash-GGUF:Q8_0 \ --jinja -ngl 99 -c 8192 -fa on ``` ## How these were made 1. Download `z-lab/Qwen3-4B-DFlash-b16` (original weights). 2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggml-org/llama.cpp). 3. Build an importance matrix over our calibration corpus. 4. Quantize the ladder with `--imatrix`. ## License Original model by Z Lab, released under the MIT license. Quantized by Atomic Chat.