Mintche/Spark-X2.5-4B-GGUF

Spark-X2.5-4B GGUF — experimental v0.1

GGUF quantizations of XHToken/Spark-X2.5-4B, pinned to source revision 1e4c2477f330d1f11304d0dc092a9dbf4075f470.

This is an experimental v0.1 release built with Spark2.5 support from llama.cpp commit fe158c6c4db8b0cb8d74c7cfe23401f7c21a45fe (build 10668). Use a llama.cpp build containing compatible spark2_5 support.

Files

File	Quantization	Size	SHA256	Recommendation
Spark-X2.5-4B-Q6_K.gguf	Q6_K with mixed 4K+16K iMatrix	3.147 GiB	107F0E30E317B4FFFF3DD63CC30F1945C161E82B31EF7BC73187941D3EA71E42	Default balance for an 8 GB GPU
Spark-X2.5-4B-Q8_0.gguf	Q8_0	4.075 GiB	5C2C3C190E4337E1016B8593CA8E26E8B18C972200B107385D4EC61A25D9DEA2	Highest fidelity of this release

The published Q6_K is the mixed-context iMatrix candidate. Compared with the 4K-only iMatrix candidate, it was directionally better on most 16K metrics, but the difference is small and the quick-run uncertainty overlaps. The clear result is that either iMatrix build substantially improves Q6_K over the no-iMatrix ablation.

Quick use

llama-cli.exe `
  -m .\Spark-X2.5-4B-Q6_K.gguf `
  -ngl 99 `
  -fa on `
  -c 16384

This llama.cpp build selects the embedded chat template without the obsolete -cnv flag. Reduce -ngl if another application is using VRAM. The 16K passkey checks below used -ngl 28 so the same compute path could also be used for the BF16 reference.

Fidelity results

All candidates were compared token-by-token against the same converted BF16 GGUF. Lower KLD/RMS is better; higher Same top p is better. “Same top p” is llama.cpp's name for top-1 agreement, not nucleus-sampling top-p.

Neutral held-out evaluation at 4K

Model	Mean KLD ↓	q99 KLD ↓	q99.9 KLD ↓	RMS Δp ↓	Same top p ↑	PPL(Q)
BF16 self-check	0.000000 ± 0.000000	0.000036	0.000049	0.001 ± 0.000%	100.000 ± 0.000%	6.910707 ± 0.268179
Q8_0	0.001354 ± 0.000096	0.011297	0.073939	1.275 ± 0.106%	98.192 ± 0.208%	6.906721 ± 0.267697
Q6_K, no iMatrix	0.013738 ± 0.001238	0.105705	1.387995	4.127 ± 0.338%	94.919 ± 0.343%	7.035107 ± 0.273836
Q6_K, 4K iMatrix	0.010021 ± 0.001285	0.064892	1.252379	3.950 ± 0.424%	95.896 ± 0.310%	6.977921 ± 0.271801
Q6_K, mixed 4K+16K iMatrix	0.008930 ± 0.000969	0.070500	0.778783	3.599 ± 0.382%	95.774 ± 0.314%	6.975759 ± 0.271574

Protocol: AtomicChat eval_neutral, 2 chunks, 4,096 scored positions, context 4,096, -ngl 28, batch 2,048, micro-batch 512, 8 threads and Flash Attention enabled. The BF16 KLD base is 1,073,283,076 bytes with SHA256 712B7248BCC72D0057EC56F61AF838D251A83EDA3F938DDD5E54FA03BFFCE322.

The same evaluation at 16K

Model	Mean KLD ↓	q99 KLD ↓	q99.9 KLD ↓	Max KLD ↓	RMS Δp ↓	Same top p ↑	PPL(Q)
BF16 self-check	0.000000 ± 0.000000	0.000035	0.000050	0.000063	0.001 ± 0.000%	100.000 ± 0.000%	8.565097 ± 0.173169
Q8_0	0.002709 ± 0.000307	0.015888	0.217027	3.244917	1.703 ± 0.152%	98.089 ± 0.107%	8.550264 ± 0.172665
Q6_K, no iMatrix	0.021932 ± 0.000852	0.178709	1.397083	5.835053	4.680 ± 0.164%	93.926 ± 0.187%	8.918584 ± 0.182141
Q6_K, 4K iMatrix	0.010021 ± 0.000345	0.074524	0.468998	2.875325	3.012 ± 0.120%	95.483 ± 0.162%	8.666384 ± 0.175889
Q6_K, mixed 4K+16K iMatrix	0.009891 ± 0.000325	0.074419	0.451430	2.363686	3.035 ± 0.124%	95.660 ± 0.159%	8.660181 ± 0.175757

Protocol: the same held-out corpus and compute path, 2 chunks, 16,384 scored positions, context 16,384. The BF16 KLD base is 4,294,705,156 bytes with SHA256 CB3BDBC2088CD3399ABCFD5D35B427B9F5C0463EFBF8FE4C9F1D68B28804AB52.

These are deliberately labelled quick2 measurements. They are useful for selecting a v0.1 candidate, not a substitute for a full benchmark suite.

iMatrix and calibration

Q6_K was quantized directly from the BF16 GGUF; it was never requantized from Q8_0.

• BF16 reference SHA256: 8CECF405A41A4A10F833530910C2E13FDE9FB39C325C8AFC3C5D10E4181E1A14
• iMatrix SHA256: 696688C6E4CB5AD6B1A2F4E3F460BBDCA50C2AC3CDBB7BC13C0455CA9F4BE489
• Main calibration: 2,022,075 Spark tokens, context 4,096
• Long-context calibration: 279,736 Spark tokens, context 16,384
• Observations merged per tensor: 2,019,328 from the 4K pass and 278,528 from the 16K pass, approximately 88% / 12%
• Batch size: 2,048; --parse-special enabled
• Calibration corpus SHA256: DD4DADD436A3AB9329C2EE98000CC2FF6540E9C190400292A99EB736AA9E8796
• Long-context corpus SHA256: 219DF033B7F9B30787C6DEC452FA16967A7C6FD4C0F2E42D84282E26C2AF940B
• Corpus source: AtomicChat calib-corpora commit 814d662f6c94d207fc8f38545a1b4abea11484b5

Calibration and evaluation inputs are disjoint. The calibration manifests and the 3.4 MiB mixed-context iMatrix are included under artifacts/; the full text corpora are not redistributed in this model repository.

Reproduction of the selected quantization:

llama-quantize.exe `
  --imatrix .\artifacts\imatrix\Spark-X2.5-4B-imatrix-4k-16k.gguf `
  .\Spark-X2.5-4B-BF16.gguf `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  Q6_K

python <LLAMA_CPP>\gguf-py\gguf\scripts\gguf_new_metadata.py `
  .\Spark-X2.5-4B-Q6_K-raw.gguf `
  .\Spark-X2.5-4B-Q6_K-reproduced.gguf `
  --remove-metadata quantize.imatrix.file `
  --force

The BF16 input is intentionally not included in this repository. Convert the pinned upstream revision with the same llama.cpp commit before reproducing the quantization.

The raw evaluated Q6_K container had SHA256 162985111D4DD15DA0FF2AD111965CABCF60690C40A52D53D2CB3640483DE51D. Before publication, the non-functional quantize.imatrix.file metadata key was removed because llama.cpp had stored its local absolute path. gguf_hash.py --no-layer gives the same tensor-data SHA256 before and after sanitization: f70482cf653bc90d935ab68ab83202d2a02c260699c5213d188d736c469bd1ae. The published container therefore has exactly the evaluated weights despite its different whole-file hash.

16K functional check

A deterministic synthetic passkey was inserted near 10%, 50% and 90% of a 14,997-token prompt. BF16, Q8_0, Q6_K without iMatrix, Q6_K with 4K iMatrix and Q6_K with mixed 4K+16K iMatrix all passed exact-match retrieval at all three depths: 15/15 runs.

This test is saturated and should only be read as a functional 16K smoke test. It does not distinguish the quantizations and does not establish general long-horizon reasoning quality. Prompts, manifests and result JSON files are included under artifacts/passkey_16k/.

RTX 4060 Laptop observations

llama-bench, CUDA, full GPU offload (-ngl 99), Flash Attention enabled:

File	pp512	tg128
Q8_0	4153.21 ± 224.67 tok/s	51.74 ± 0.25 tok/s
Q6_K, mixed iMatrix	3678.87 ± 101.74 tok/s	64.74 ± 0.20 tok/s

At context 16K, the reported VRAM figures were approximately 4.5 GB total for Q8_0 and 3.5 GB total for Q6_K. They include roughly 0.4–0.5 GB already allocated to the Windows/display system because this laptop setup has no iGPU handling the display. The model-run increments were therefore approximately 4.0–4.1 GB for Q8_0 and 3.0–3.1 GB for Q6_K. These were manual observations; treat VRAM and speed values as hardware- and runtime-dependent, not universal requirements.

Reproducibility artifacts

The artifacts/ directory contains:

• parsed metrics in results.json;
• source, model, iMatrix, evaluation-base and release hashes in audit-manifest.json and SHA256SUMS.txt;
• raw KLD, quantization, iMatrix, tokenizer and speed logs;
• the Q6_K public-container metadata sanitization and tensor-identity proof;
• calibration manifests;
• the exact benchmark and quantization scripts used locally (paths must be adapted before reuse);
• the 16K passkey prompts and result JSON files.

The approximately 1 GiB and 4 GiB KLD base files are not uploaded in v0.1. Their SHA256 values, exact protocols and raw comparison logs are preserved so they can be regenerated and checked.

Next steps

The next experimental phase will explore mixed-context iMatrices more systematically. In particular:

• vary the share of 4K and long-context calibration observations instead of testing only the current approximately 88% / 12% mix;
• test additional long-context calibration lengths and task families, rather than relying on a single 16K corpus recipe;
• evaluate each recipe at matching long contexts with the same BF16 reference and compute path;
• retain a new recipe only when the improvement is repeatable across KLD, tail KLD and functional long-context checks.

The present 4K+16K result is therefore a useful first data point, not a claim that this mixture is optimal.

Validation status and limitations

• HF/llama.cpp tokenizer parity passed 10/10 fixed cases, including multilingual text and special-token/chat-template cases.
• GGUF architecture, tokenizer metadata and embedded chat template checks passed.
• A direct Transformers-vs-GGUF logit parity comparison was not completed for v0.1. Consequently, KLD results establish fidelity to the converted BF16 GGUF; they do not independently prove that the experimental conversion is numerically identical to the upstream Transformers implementation.
• Q8_0 has strong local provenance: the quantization command was recovered and metadata match the current BF16. However, the BF16 hash was not recorded at the exact time of the original Q8_0 run, so this is not a complete cryptographic chain of custody.
• Spark2.5 support was experimental at the pinned llama.cpp commit. Compatibility with older releases is not expected.
• No benchmark here validates the upstream maximum-context claim. Only 4K distributional checks and a 16K distributional/functional check were run.

See the upstream model card for architecture, intended use and model-level limitations. The upstream Apache-2.0 license is included as LICENSE.