---
base_model: Qwen/Qwen3-Coder-Next
language: en
library_name: mlx-lm
license: apache-2.0
model_name: qwen3-coder-next-8bit-g128
tags:
- quantization
- g128
- moe
- text-generation
- mlx-lm
- mmlu-pro
---
# Model Card for Qwen3-Coder-Next-8bit-g128

Quantized Qwen/Qwen3-Coder-Next using mlx-lm to 8-bit with group_size 128 for main weights and fine-grained group_size 64 for MoE weights, with the aim of maximum accuracy for 8-bit quantization.

## Updated Evaluation Results (February 13, 2026)

Comprehensive evaluation results from thorough testing using mlx_lm.evaluate with mmlu_pro (200 questions per domain, num_shots=1, temp=1.0, top_p=0.95, top_k=40, seed=123):

### Direct Comparison Summary (8-bit g64 vs g128)

| Domain | 8-bit g64 | 8-bit g128 (this model) | Difference |
|--------|-----------|-------------------------|------------|
| Computer Science | 85.0% | 87.5% | +2.5% |
| Math | 93.0% | 92.5% | -0.5% |
| Physics | 92.0% | 88.5% | -3.5% |
| Engineering | 71.0% | 68.5% | -2.5% |
| Chemistry | 87.5% | 88.5% | +1.0% |

### Average Performance
- 8-bit g64: 87.0% average
- 8-bit g128 (this model): 87.0% average
- **Difference: 0.0 percentage points (virtually identical)**

### Key Observations
- Both 8-bit models show nearly identical average performance (87.0%)
- Performance differences are domain-specific rather than model-specific
- 8-bit g64 performs better in Math, Engineering, and Physics
- 8-bit g128 (this model) performs better in Computer Science and Chemistry, with Computer Science being the most significant improvement (+2.5%)

## Memory Usage Comparison

- **8-bit g64**:
  - Peak memory usage: 95.374 GB (Math) to 100.740 GB (Engineering)
  - Average memory usage: ~98.6 GB

- **8-bit g128 (this model)**:
  - Peak memory usage: 97.702 GB (Computer Science) to 98.510 GB (Engineering)
  - Average memory usage: ~97.8 GB

## Full Quantization Spectrum Comparison (Updated)

| Domain | 3-bit default | 3-bit g128 | 4-bit default | 4-bit g128 | 6-bit default | 6-bit g128 | 8-bit g64 | 8-bit g128 (this model) |
|---|---|---|---|---|---|---|---|---|
| Math | 0.94 | 0.90 | 0.92 | 0.92 | 0.90 | 0.94 | 0.930 | 0.925 |
| Computer Science | 0.82 | 0.84 | 0.80 | 0.84 | 0.82 | 0.86 | 0.850 | 0.875 |
| Engineering | 0.70 | 0.64 | 0.70 | 0.76 | 0.74 | 0.72 | 0.710 | 0.685 |
| Physics | 0.94 | 0.92 | 0.94 | 0.96 | 0.96 | 0.94 | 0.920 | 0.885 |
| Chemistry | 0.86 | 0.88 | 0.90 | 0.90 | 0.94 | 0.92 | 0.875 | 0.885 |
| **Average** | **0.852** | **0.836** | **0.835** | **0.865** | **0.872** | **0.876** | **0.870** | **0.870** |

## Original Evaluation Results

Testing with mlx_lm.evaluate using mmlu_pro with 50 Qs for each topic, comparing the 8-bit g128 quant with the 8-bit g64 quant:

| Domain | g64 | g128 | Improvement |
|---|---|---|---|
| Math | 0.92 | 0.94 | +2% |
| Computer Science | 0.84 | 0.90 | **+6%** |
| Engineering | 0.80 | 0.80 | = |
| Physics | 0.96 | 0.96 | = |
| Chemistry | 0.90 | 0.94 | +4% |

### Average Performance
- 8-bit g64: 84.4% average
- 8-bit g128: 88.0% average
- **Improvement: +3.6 percentage points with group size 128 (main weights) with fine-grained 64 (MoE weights)**

## Key Benefits

- The g128 group_size applied to main weights (with fine-grained g64 for MoE weights) is also ~2GB smaller than the g64 version
- At the time of uploading, there were not any 8bit g128 quants of this model available on HF
- Also tried MXFP8 but found equal or lower accuracy than g64 for all topics (Engineering dropped to 0.76 and Chemistry dropped to 0.88 (-4% and -2% compared to g64))

## Model Details

- **Base Model:** Qwen/Qwen3-Coder-Next
- **Library:** mlx-lm
- **Quantization:** 8-bit with group_size 128 for main weights and group_size 64 for MoE weights
- **License:** apache-2.0
- **Pipeline Tag:** text-generation

## Usage

```python
import mlx_lm
from mlx_lm.sample_utils import make_sampler

model_path = "petergilani/qwen3-coder-next-8bit-g128"
model, tokenizer = mlx_lm.load(model_path)

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)

prompt = "Write a Python function to calculate the factorial of a number."
response = mlx_lm.generate(
    model, 
    tokenizer, 
    prompt=prompt,
    sampler=sampler,
    max_tokens=512
)
print(response)
```