Yi30/wan2.2-ti2v-w4a8-svd-nvidia

Wan2.2 W4A8 SVDQuant — NVIDIA (CUDA) artifacts

Calibrated W4A8 SVDQuant checkpoints (MXFP4 weights, dynamic MXFP8 activations, rank-32 low-rank correction) for Wan2.2, exported with vLLM-Omni (PR #6527 head + CUDA torch provider) and validated end-to-end on NVIDIA B300 (sm_103, CUDA 13.3).

The offline export is produced without Quark/ROCm by examples/quantization/export_quark_svdquant_w4a8_cuda.py — a pure-torch replication of Quark's SVDQuant recipe (SmoothQuant, alpha=0.5, on real pipeline activations; exact torch.linalg.svd on device; the 1/s factor folded into the residual and proj_down, so the stored checkpoint is self-contained).

Contents

Path	What it is	Size
wan5b-calibrated/	Wan2.2-TI2V-5B quantized transformer + model_index.json + README	~9.5 GB
wana14b-calibrated/	Wan2.2-T2V-A14B quantized transformer + transformer_2 (dual expert) + model_index.json + README	~54 GB
wan5b_calibrated_w4a8_svd.mp4	5B calibrated generation (480x832, 41f, 20 steps)	84 KB
wana14b_calibrated_w4a8_svd.mp4	A14B calibrated generation (720x1280, 81f, 40 steps)	817 KB
wan5b_svdquant_calibrated_similarity.json	5B vs BF16 trajectory-similarity metrics	—
wan_a14b_svdquant_calibrated_similarity.json	A14B vs BF16 trajectory-similarity metrics	—

Each model's README covers the one-line setup (stock vae / text_encoder / tokenizer / scheduler are the base-model components and are not duplicated here) and the exact vLLM-Omni commands.

Quick start (5B example)

hf download Yi30/wan2.2-ti2v-w4a8-svd-nvidia \
  --include "wan5b-calibrated/*" --local-dir ./wan5b-calibrated
hf download Wan-AI/Wan2.2-TI2V-5B-Diffusers \
  --include "vae/*" "text_encoder/*" "tokenizer/*" "scheduler/*" \
  --local-dir ./wan5b-calibrated

# vLLM-Omni (vllm 0.26.0 pairing; torch provider auto-selected on CUDA)
python examples/offline_inference/text_to_video/text_to_video.py \
  --model ./wan5b-calibrated --quantization quark_svdquant \
  --prompt "A serene lakeside sunrise with mist over the water." \
  --height 480 --width 832 --num-frames 41 --num-inference-steps 20 --output out.mp4

The embedded quantization_config in each transformer's config.json (is_checkpoint_w4a8_serialized: true) makes the loader take the offline serialized path automatically. Experimental W4A4 (MXFP4 activations): VLLM_OMNI_W4A8_CUDA_ACT_FMT=mxfp4.

Accuracy (seed 42, vs BF16 pipeline, same settings as online runs)

Model	Setting	Tier	PSNR (dB)	Cosine	Peak memory
TI2V-5B	480x832, 41f, 20 steps	online SVD (baseline)	27.10	0.9979	25.1 GB
TI2V-5B	480x832, 41f, 20 steps	calibrated (this repo)	26.52	0.9975	26.3 GB
T2V-A14B	720x1280, 81f, 40 steps, gs 4.0/3.0	online SVD (baseline)	12.59	0.794	55.6 GB
T2V-A14B	720x1280, 81f, 40 steps, gs 4.0/3.0	calibrated (this repo)	12.71	0.804	56.4 GB

On the MXFP4/MXFP8 torch backend the calibrated tier is accuracy-equivalent to the online tier (SmoothQuant's benefit targets per-channel quantizers; MXFP4's per-32-element group scale is insensitive to the column rescaling — measured residual Q4 error: 11.63% online vs 11.64% calibrated across all 300 5B target layers). The calibrated checkpoints are the portable, deterministic artifact: no svd_lowrank at load, factors pre-computed. On-disk format is BF16 residual + BF16 factors (packed to MXFP4 at load); the exporter also supports --pack-format packed|unshuffled for ~4x smaller MXFP4-on-disk variants.