# Reproducing this APEX quant

Full imatrix pipeline; all runs on ~14 GB, GPU-friendly. Components MIT/Apache (see NOTICE).

## Pinned
- **llama.cpp** supporting `granitemoehybrid` + `--tensor-type-file` (e.g. commit `bbf4a8a`/b8833+).
- **apex-quant** commit `a445a12` for `generate_config.sh` (bundled).

## Baseline + calibration
```bash
hf download ibm-granite/granite-4.0-h-tiny-GGUF granite-4.0-h-tiny-bf16.gguf --local-dir .
# calibration_datav3: https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8
```

## Config (regenerate)
```bash
bash generate_config.sh --profile i-quality --layers 40 --dense-layers 0 -o granite_iq.base.txt
python patch_granite_config.py granite_iq.base.txt configs/granite_i-quality.txt   # adds Mamba-2 ssm_in/conv1d
```

## imatrix -> quantize -> eval
```bash
llama-imatrix -m granite-4.0-h-tiny-bf16.gguf -f calibration_datav3.txt \
  -o granite-4.0-h-tiny.imatrix -ngl 999
llama-quantize --tensor-type-file configs/granite_i-quality.txt \
  --imatrix granite-4.0-h-tiny.imatrix \
  granite-4.0-h-tiny-bf16.gguf granite-4.0-h-tiny-APEX-i-quality.gguf Q6_K
# eval (PPL prints to STDERR -> 2>&1):
llama-perplexity -m granite-4.0-h-tiny-APEX-i-quality.gguf -f wiki.test.raw -ngl 999 --chunks 200 2>&1 \
  | grep -oP 'Final estimate: PPL = \K[0-9.]+'
```
Expected: ~8.90 (bf16 ~8.87).