--- license: apache-2.0 thumbnail: https://huggingface.co/AtomicChat/Qwen3.5-4B-DFlash-GGUF/resolve/main/hero.png base_model: - z-lab/Qwen3.5-4B-DFlash base_model_relation: quantized quantized_by: AtomicChat pipeline_tag: text-generation library_name: gguf tags: - atomic-chat - qwen3.5 - z-lab - gguf - llama.cpp - quantized ---

Base model: z-lab/Qwen3.5-4B-DFlash

**Qwen3.5 4B Dflash**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Z Lab's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline. ## Highlights - **0.6B parameters**: the weights this repo quantizes. - **Context length**: 262,144 tokens (256K), as published by Z Lab. - **6 layers**: Dense decoder, hybrid sliding-window (4096) and global attention. - **Full imatrix ladder**: every quant is calibrated with an importance matrix. > [!NOTE] > These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model. > [!IMPORTANT] > Always pass `--jinja` so the **Qwen3.5 4B Dflash chat template** is applied. Without it the model can emit malformed turns. ## Model Overview | Property | Value | |---|---| | Base model | `z-lab/Qwen3.5-4B-DFlash` | | Parameters | 0.6B | | Layers | 6 | | Sliding window | 4096 tokens | | Context length | 262,144 tokens (256K) | | Vocabulary | 248,320 | | Modalities | Text | | Architecture | Dense decoder, hybrid sliding-window (4096) and global attention, 32 attention heads over 8 KV heads, `DFlashDraftModel` | | This repo | GGUF quants (imatrix). Quants: `Q8_0` | ## Choosing a quant | Quant | Size | Notes | |---|---|---| | **`Q8_0`** | 0.7 GB | **Effectively lossless, reference quality.** | > [!TIP] > Pick the largest file that fits your (V)RAM with room for context. `Q8_0` is the sweet spot for most setups; `Q6_K` or `Q8_0` for maximum fidelity. ## Get started Run Qwen3.5 4B Dflash locally with: - **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AtomicChat/Qwen3.5-4B-DFlash-GGUF`, pick a quant, hit **Use this model**. - **llama.cpp:** `llama-server -hf AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0 --jinja -c 8192` - **Ollama:** `ollama run hf.co/AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0` - **LM Studio / Jan:** search the repo id, download any quant. ## Best practices | Parameter | Value | |---|---| | sampling defaults | not stated | The base model card does not state sampling defaults. ## Run in llama.cpp ```bash git clone https://github.com/ggml-org/llama.cpp cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server ``` ```bash ./llama.cpp/build/bin/llama-server \ -hf AtomicChat/Qwen3.5-4B-DFlash-GGUF:Q8_0 \ --jinja -ngl 99 -c 8192 -fa on ``` ## How these were made 1. Download `z-lab/Qwen3.5-4B-DFlash` (original weights). 2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggml-org/llama.cpp). 3. Build an importance matrix over our calibration corpus. 4. Quantize the ladder with `--imatrix`. ## License Original model by Z Lab, released under the Apache 2.0 license. Quantized by Atomic Chat.