Configuration Parsing Warning:In config.json: "quantization_config.modules_to_not_convert" must be an array
Quantized with AutoAWQ v0.2.8 using default settings
quant_config = {
"zero_point": True,
"q_group_size": 128,
"w_bit": 4,
"version": "GEMM"
}
- Downloads last month
- 9
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support