card: pre-quantized DiT section
Browse files
README.md
CHANGED
|
@@ -41,3 +41,30 @@ Gaussian-Shading provenance watermark is reproduced bit-exactly; the DiT runs
|
|
| 41 |
bf16 with one GEMM row-chunked around mlx-swift's NAX split-K JIT bug
|
| 42 |
([mlx#3797](https://github.com/ml-explore/mlx/issues/3797)) until a release
|
| 43 |
ships [#3810](https://github.com/ml-explore/mlx/pull/3810).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
bf16 with one GEMM row-chunked around mlx-swift's NAX split-K JIT bug
|
| 42 |
([mlx#3797](https://github.com/ml-explore/mlx/issues/3797)) until a release
|
| 43 |
ships [#3810](https://github.com/ml-explore/mlx/pull/3810).
|
| 44 |
+
|
| 45 |
+
## Pre-quantized DiT (no bf16 peak)
|
| 46 |
+
|
| 47 |
+
| file | recipe | gate |
|
| 48 |
+
|---|---|---|
|
| 49 |
+
| `transformer-int8.safetensors` | int8 g32 affine, attn+MLP Linears only | `int8 quant-vs-bf16 0.999926 (family threshold 0.9998) -> PASS` |
|
| 50 |
+
| `transformer-int4.safetensors` | int4 g64 affine, same scope | `int4 quant-vs-bf16 0.996912 (family threshold 0.9900) -> PASS` |
|
| 51 |
+
|
| 52 |
+
Modulation/io/norm layers and the trailing block 11 stay bf16 (the final norm
|
| 53 |
+
amplifies last-block quant error ~1.6× into `proj_out`). Files are
|
| 54 |
+
self-describing (metadata carries bits / group size / kept blocks) and load
|
| 55 |
+
directly with **no bf16 peak** — int8 gate: quant-vs-bf16 per-pass, family threshold ≥ 0.9998; int4 gate: quant-vs-bf16 per-pass, family threshold ≥ 0.99.
|
| 56 |
+
Quantization is weight-only: the mandatory content filter and the bit-exact
|
| 57 |
+
Gaussian-Shading watermark are unaffected. Measured t2i peak GPU @1024²:
|
| 58 |
+
bf16 19.6 → int8 17.2 → int4 16.1 GB.
|
| 59 |
+
|
| 60 |
+
```bash
|
| 61 |
+
# quant tier: skip the 7.7 GB bf16 transformer entirely
|
| 62 |
+
hf download microsoft/Mage-Flow-Base --include "text_encoder/*" --include "vae/*" --local-dir Mage-Flow-Base
|
| 63 |
+
hf download xocialize/Mage-Flow-Base-mlx folded_adaln.safetensors --local-dir Mage-Flow-Base
|
| 64 |
+
hf download xocialize/Mage-Flow-Base-mlx transformer-int4.safetensors --local-dir Mage-Flow-Base
|
| 65 |
+
swift run -c release mage-flow-edit --repo Mage-Flow-Base --dit-quant Mage-Flow-Base/transformer-int4.safetensors ...
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
MLXEngine: `MLXMageFlow` (same repo) wraps the family as `MageFlowT2IPackage`
|
| 69 |
+
/ `MageFlowEditPackage` — quant tiers auto-materialize these files via
|
| 70 |
+
`WeightSourcing`.
|