README.md

---
base_model:
  - XHToken/Spark-X2.5-4B
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - imatrix
  - quantization
license: apache-2.0
---

# Spark-X2.5-4B GGUF — experimental v0.1

GGUF quantizations of [XHToken/Spark-X2.5-4B](https://huggingface.co/XHToken/Spark-X2.5-4B), pinned to source revision [`1e4c2477f330d1f11304d0dc092a9dbf4075f470`](https://huggingface.co/XHToken/Spark-X2.5-4B/tree/1e4c2477f330d1f11304d0dc092a9dbf4075f470).

This is an experimental v0.1 release built with Spark2.5 support from llama.cpp commit [`fe158c6c4db8b0cb8d74c7cfe23401f7c21a45fe`](https://github.com/ggml-org/llama.cpp/commit/fe158c6c4db8b0cb8d74c7cfe23401f7c21a45fe) (build 10668). Use a llama.cpp build containing compatible `spark2_5` support.

## Files

| File | Quantization | Size | SHA256 | Recommendation |
|---|---|---:|---|---|
| `Spark-X2.5-4B-Q6_K.gguf` | Q6_K with mixed 4K+16K iMatrix | 3.147 GiB | `107F0E30E317B4FFFF3DD63CC30F1945C161E82B31EF7BC73187941D3EA71E42` | Default balance for an 8 GB GPU |
| `Spark-X2.5-4B-Q8_0.gguf` | Q8_0 | 4.075 GiB | `5C2C3C190E4337E1016B8593CA8E26E8B18C972200B107385D4EC61A25D9DEA2` | Highest fidelity of this release |

The published `Q6_K` is the mixed-context iMatrix candidate. Compared with the 4K-only iMatrix candidate, it was directionally better on most 16K metrics, but the difference is small and the quick-run uncertainty overlaps. The clear result is that either iMatrix build substantially improves Q6_K over the no-iMatrix ablation.

## Quick use

```powershell
llama-cli.exe `
  -m .\Spark-X2.5-4B-Q6_K.gguf `
  -ngl 99 `
  -fa on `
  -c 16384
```

This llama.cpp build selects the embedded chat template without the obsolete `-cnv` flag. Reduce `-ngl` if another application is using VRAM. The 16K passkey checks below used `-ngl 28` so the same compute path could also be used for the BF16 reference.

## Fidelity results

All candidates were compared token-by-token against the same converted BF16 GGUF. Lower KLD/RMS is better; higher Same top p is better. “Same top p” is llama.cpp's name for top-1 agreement, not nucleus-sampling top-p.

### Neutral held-out evaluation at 4K

| Model | Mean KLD ↓ | q99 KLD ↓ | q99.9 KLD ↓ | RMS Δp ↓ | Same top p ↑ | PPL(Q) |
|---|---:|---:|---:|---:|---:|---:|
| BF16 self-check | 0.000000 ± 0.000000 | 0.000036 | 0.000049 | 0.001 ± 0.000% | 100.000 ± 0.000% | 6.910707 ± 0.268179 |
| Q8_0 | 0.001354 ± 0.000096 | 0.011297 | 0.073939 | 1.275 ± 0.106% | 98.192 ± 0.208% | 6.906721 ± 0.267697 |
| Q6_K, no iMatrix | 0.013738 ± 0.001238 | 0.105705 | 1.387995 | 4.127 ± 0.338% | 94.919 ± 0.343% | 7.035107 ± 0.273836 |
| Q6_K, 4K iMatrix | 0.010021 ± 0.001285 | 0.064892 | 1.252379 | 3.950 ± 0.424% | 95.896 ± 0.310% | 6.977921 ± 0.271801 |
| **Q6_K, mixed 4K+16K iMatrix** | **0.008930 ± 0.000969** | 0.070500 | **0.778783** | **3.599 ± 0.382%** | 95.774 ± 0.314% | **6.975759 ± 0.271574** |

Protocol: AtomicChat `eval_neutral`, 2 chunks, 4,096 scored positions, context 4,096, `-ngl 28`, batch 2,048, micro-batch 512, 8 threads and Flash Attention enabled. The BF16 KLD base is 1,073,283,076 bytes with SHA256 `712B7248BCC72D0057EC56F61AF838D251A83EDA3F938DDD5E54FA03BFFCE322`.

### The same evaluation at 16K

| Model | Mean KLD ↓ | q99 KLD ↓ | q99.9 KLD ↓ | Max KLD ↓ | RMS Δp ↓ | Same top p ↑ | PPL(Q) |
|---|---:|---:|---:|---:|---:|---:|---:|
| BF16 self-check | 0.000000 ± 0.000000 | 0.000035 | 0.000050 | 0.000063 | 0.001 ± 0.000% | 100.000 ± 0.000% | 8.565097 ± 0.173169 |
| Q8_0 | 0.002709 ± 0.000307 | 0.015888 | 0.217027 | 3.244917 | 1.703 ± 0.152% | 98.089 ± 0.107% | 8.550264 ± 0.172665 |
| Q6_K, no iMatrix | 0.021932 ± 0.000852 | 0.178709 | 1.397083 | 5.835053 | 4.680 ± 0.164% | 93.926 ± 0.187% | 8.918584 ± 0.182141 |
| Q6_K, 4K iMatrix | 0.010021 ± 0.000345 | 0.074524 | 0.468998 | 2.875325 | **3.012 ± 0.120%** | 95.483 ± 0.162% | 8.666384 ± 0.175889 |
| **Q6_K, mixed 4K+16K iMatrix** | **0.009891 ± 0.000325** | **0.074419** | **0.451430** | **2.363686** | 3.035 ± 0.124% | **95.660 ± 0.159%** | **8.660181 ± 0.175757** |

Protocol: the same held-out corpus and compute path, 2 chunks, 16,384 scored positions, context 16,384. The BF16 KLD base is 4,294,705,156 bytes with SHA256 `CB3BDBC2088CD3399ABCFD5D35B427B9F5C0463EFBF8FE4C9F1D68B28804AB52`.

These are deliberately labelled **quick2** measurements. They are useful for selecting a v0.1 candidate, not a substitute for a full benchmark suite.

## iMatrix and calibration

Q6_K was quantized directly from the BF16 GGUF; it was never requantized from Q8_0.

- BF16 reference SHA256: `8CECF405A41A4A10F833530910C2E13FDE9FB39C325C8AFC3C5D10E4181E1A14`
- iMatrix SHA256: `696688C6E4CB5AD6B1A2F4E3F460BBDCA50C2AC3CDBB7BC13C0455CA9F4BE489`
- Main calibration: 2,022,075 Spark tokens, context 4,096
- Long-context calibration: 279,736 Spark tokens, context 16,384
- Observations merged per tensor: 2,019,328 from the 4K pass and 278,528 from the 16K pass, approximately 88% / 12%
- Batch size: 2,048; `--parse-special` enabled
- Calibration corpus SHA256: `DD4DADD436A3AB9329C2EE98000CC2FF6540E9C190400292A99EB736AA9E8796`
- Long-context corpus SHA256: `219DF033B7F9B30787C6DEC452FA16967A7C6FD4C0F2E42D84282E26C2AF940B`
- Corpus source: AtomicChat `calib-corpora` commit `814d662f6c94d207fc8f38545a1b4abea11484b5`

Calibration and evaluation inputs are disjoint. The calibration manifests and the 3.4 MiB mixed-context iMatrix are included under `artifacts/`; the full text corpora are not redistributed in this model repository.

Reproduction of the selected quantization:

```powershell
llama-quantize.exe `
  --imatrix .\artifacts\imatrix\Spark-X2.5-4B-imatrix-4k-16k.gguf `
  .\Spark-X2.5-4B-BF16.gguf `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  Q6_K

python <LLAMA_CPP>\gguf-py\gguf\scripts\gguf_new_metadata.py `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  .\Spark-X2.5-4B-Q6_K-reproduced.gguf `
  --remove-metadata quantize.imatrix.file `
  --force
```

The BF16 input is intentionally not included in this repository. Convert the pinned upstream revision with the same llama.cpp commit before reproducing the quantization.

The raw evaluated Q6_K container had SHA256 `162985111D4DD15DA0FF2AD111965CABCF60690C40A52D53D2CB3640483DE51D`. Before publication, the non-functional `quantize.imatrix.file` metadata key was removed because llama.cpp had stored its local absolute path. `gguf_hash.py --no-layer` gives the same tensor-data SHA256 before and after sanitization: `f70482cf653bc90d935ab68ab83202d2a02c260699c5213d188d736c469bd1ae`. The published container therefore has exactly the evaluated weights despite its different whole-file hash.

## 16K functional check

A deterministic synthetic passkey was inserted near 10%, 50% and 90% of a 14,997-token prompt. BF16, Q8_0, Q6_K without iMatrix, Q6_K with 4K iMatrix and Q6_K with mixed 4K+16K iMatrix all passed exact-match retrieval at all three depths: **15/15 runs**.

This test is saturated and should only be read as a functional 16K smoke test. It does not distinguish the quantizations and does not establish general long-horizon reasoning quality. Prompts, manifests and result JSON files are included under `artifacts/passkey_16k/`.

## RTX 4060 Laptop observations

`llama-bench`, CUDA, full GPU offload (`-ngl 99`), Flash Attention enabled:

| File | pp512 | tg128 |
|---|---:|---:|
| Q8_0 | 4153.21 ± 224.67 tok/s | 51.74 ± 0.25 tok/s |
| Q6_K, mixed iMatrix | 3678.87 ± 101.74 tok/s | 64.74 ± 0.20 tok/s |

At context 16K, the reported VRAM figures were approximately 4.5 GB total for Q8_0 and 3.5 GB total for Q6_K. They include roughly 0.4–0.5 GB already allocated to the Windows/display system because this laptop setup has no iGPU handling the display. The model-run increments were therefore approximately 4.0–4.1 GB for Q8_0 and 3.0–3.1 GB for Q6_K. These were manual observations; treat VRAM and speed values as hardware- and runtime-dependent, not universal requirements.

## Reproducibility artifacts

The `artifacts/` directory contains:

- parsed metrics in `results.json`;
- source, model, iMatrix, evaluation-base and release hashes in `audit-manifest.json` and `SHA256SUMS.txt`;
- raw KLD, quantization, iMatrix, tokenizer and speed logs;
- the Q6_K public-container metadata sanitization and tensor-identity proof;
- calibration manifests;
- the exact benchmark and quantization scripts used locally (paths must be adapted before reuse);
- the 16K passkey prompts and result JSON files.

The approximately 1 GiB and 4 GiB KLD base files are not uploaded in v0.1. Their SHA256 values, exact protocols and raw comparison logs are preserved so they can be regenerated and checked.

## Next steps

The next experimental phase will explore mixed-context iMatrices more systematically. In particular:

- vary the share of 4K and long-context calibration observations instead of testing only the current approximately 88% / 12% mix;
- test additional long-context calibration lengths and task families, rather than relying on a single 16K corpus recipe;
- evaluate each recipe at matching long contexts with the same BF16 reference and compute path;
- retain a new recipe only when the improvement is repeatable across KLD, tail KLD and functional long-context checks.

The present 4K+16K result is therefore a useful first data point, not a claim that this mixture is optimal.

## Validation status and limitations

- HF/llama.cpp tokenizer parity passed 10/10 fixed cases, including multilingual text and special-token/chat-template cases.
- GGUF architecture, tokenizer metadata and embedded chat template checks passed.
- A direct Transformers-vs-GGUF **logit parity comparison was not completed for v0.1**. Consequently, KLD results establish fidelity to the converted BF16 GGUF; they do not independently prove that the experimental conversion is numerically identical to the upstream Transformers implementation.
- Q8_0 has strong local provenance: the quantization command was recovered and metadata match the current BF16. However, the BF16 hash was not recorded at the exact time of the original Q8_0 run, so this is not a complete cryptographic chain of custody.
- Spark2.5 support was experimental at the pinned llama.cpp commit. Compatibility with older releases is not expected.
- No benchmark here validates the upstream maximum-context claim. Only 4K distributional checks and a 16K distributional/functional check were run.

See the [upstream model card](https://huggingface.co/XHToken/Spark-X2.5-4B) for architecture, intended use and model-level limitations. The upstream Apache-2.0 license is included as `LICENSE`.