metadata
library_name: exllamav3
tags:
- exl3
- exllamav3
- quantized
- dflash
- speculative-decoding
base_model:
- z-lab/gemma4-12B-it-DFlash
Gemma 4 12B IT DFlash (EXL3)
This is an ExLlamaV3 (EXL3) quantized version of the z-lab/gemma4-12B-it-DFlash model.
DFlash models are specifically designed to act as Draft Models for Speculative Decoding, allowing you to generate text significantly faster when paired with a larger main model. These were quantized using the Ampere-optimized mul1 codebook for maximum decoding speed on modern GPUs.
📥 Available Branches / Downloads
The model weights are stored in separate branches. Please click a branch link below to view and download the specific size you need.
| Target BPW | Head BPW | Branch (Download Link) |
|---|---|---|
| 6.00 | 6 | 6.0bpw_h6 |
| 5.00 | 6 | 5.0bpw_h6 |
| 4.00 | 6 | 4.0bpw_h6 |
| 3.50 | 6 | 3.5bpw_h6 |
| 3.00 | 6 | 3.0bpw_h6 |
| 2.50 | 6 | 2.5bpw_h6 |
💻 How to Download
It is highly recommended to use the huggingface-cli to download a specific branch to avoid downloading the entire repository (which contains all sizes).
pip install -U "huggingface_hub[cli]"
Example: Downloading the 4.0 BPW branch
huggingface-cli download DBMe/gemma4-12B-it-DFlash-exl3 \
--revision 4.0bpw_h6 \
--local-dir gemma4-12B-it-DFlash-exl3-4.0bpw
⚙️ Quantization Details
- Engine: ExLlamaV3
- Codebook:
mul1(Ampere-optimized) - Head BPW: 6.0 (Preserves output layer accuracy for drafting)
- Calibration: Uncalibrated (Standard practice for DFlash architectures)