File size: 2,636 Bytes
f9fda6d 0b0c703 0b037f3 e31df33 f9fda6d 0b0c703 5a6514e 4cbe714 be40036 4cbe714 be40036 4cbe714 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
pipeline_tag: text-to-image
base_model:
- krea/Krea-2-Turbo
base_model_relation: quantized
language:
- en
library_name: diffusers
license: other
license_name: krea-2-community-license
license_link: https://huggingface.co/krea/Krea-2-Turbo/resolve/main/LICENSE.pdf
tags:
- nvfp4
- quantization
---
# Krea2-Turbo-mixed-NVFP4
Mixed 8-bit microscaling quantization of [Krea-2-Turbo](https://huggingface.co/krea/Krea-2-Turbo), generated with [`convert_to_quant`](https://github.com/silveroxides/convert_to_quant).
* **Format**: Mixed NFVP4 and MXFP8 with some layers preserved as BF16.
* **Size**: 8.8G GB (−66% vs BF16).
* **Inference**: ComfyUI + [`comfy-kitchen`](https://github.com/Comfy-Org/comfy-kitchen), Blackwell GPU (RTX 50xx / B100 / B200).
<table>
<tr>
<td align="center">
<a href="Krea2-Turbo-mixed-NVFP4/blob/main/assets/nvfp4.png"><img src="assets/nvfp4.png" style="width: 66%; height: auto;"></a>
</td>
<td align="center">
<a href="Krea2-Turbo-mixed-NVFP4/blob/main/assets/bf16.png"><img src="assets/bf16.png" style="width: 66%; height: auto;"></a>
</td>
</tr>
<tr>
<td align="center"><strong>NVFP4</strong></td>
<td align="center"><strong>BF16</strong></td>
</tr>
</table>
## Quantization
The model weights have been partially quantized to **NVFP4** (NVIDIA Floating Point 4-bit) and **MXFP8**, quantization formats supported on NVIDIA Blackwell architecture GPUs.
The following `convert_to_quant` parameters where used. This conversion takes about 1 hour on an RTX 5060 TI.
```bash
$ convert_to_quant -i krea2_turbo_bf16.safetensors \
--nvfp4 \
--krea2 \
--comfy_quant \
--save-quant-metadata \
--custom-type mxfp8 \
--custom-layers \
"blocks\.(0|1|2|24|25|26)\.attn\.(wq|wk|wv|wo)\.weight|blocks\.(0|1|2|25|26|27)\.attn\.gate\.weight|blocks\.(0|1|2|3|25|26|27)\.mlp\.gate\.weight|txtfusion\.layerwise_blocks\.(0|1)\.attn\.(wq|wk|wv|wo|gate)\.weight|txtfusion\.layerwise_blocks\.(0|1)\.mlp\.gate\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.(wq|wk|wv|gate)\.weight|txtfusion\.refiner_blocks\.(0|1)\.mlp\.(gate|up)\.weight|txtfusion\.refiner_blocks\.0\.mlp\.down\.weight" \
--exclude-layers \
"blocks\.27\.attn\.(wq|wk|wv)\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.wo\.weight|txtfusion\.refiner_blocks\.1\.mlp\.down\.weight" \
--num-iter 4000 \
--top-p 0.35 \
--calib-samples 8192 \
--scale-optimization iterative \
--scale-refinement 2 \
--extract-lora \
--lora-rank 64 \
--lora-target "attn\.(wo|gate)\.weight|mlp\.(gate|down)\.weight" \
-o krea2_turbo_mixed_nvfp4.safetensors
``` |