Attempted AWQ quantization using calibration dataset. Added placeholder 1 token images to the text calibration dataset. Took some messing around and might look weird; I needed to patch vLLM for support even though it is trying to support Qwen3.5 out of the gate.
It's ... weird-shaped currently due to unpacking and might not be useful to anyone, lol. Seems slower than GGUF. Unsure how well it works. However, I spent a good few hours of my computer's time on making it happen, so releasing regardless.
- Downloads last month
- 2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Lambent/Qwen3.5-35B-A3B-Base-AWQ-W4A16
Base model
Qwen/Qwen3.5-35B-A3B-Base