datayoda/DeepSeek-V4-Flash-0731-DS4-Antirez-Coder-Halo-128K-MixQ-GGUF

DeepSeek-V4-Flash-0731-DS4-Antirez-Coder-Halo-128K-MixQ

This is deepseek-ai/DeepSeek-V4-Flash-0731, quantized for antirez's ds4 runtime, with two layers pushed a step higher in precision than the standard Q2 build. 26 of 50 on a frozen public coding suite, up from 21/50 on the unmodified baseline — the best result found across a twelve-round search over which layers to promote and by how much.

What it is

Same base layout as antirez's reference Q2 build — routed-expert down projections at Q2_K, gate/up at IQ2_XXS, attention projections/shared experts/output head at Q8_0 — with one change: the routed-expert gate/up/down tensors of blk.0 and blk.1 only are promoted to Q4_K, using the same published imatrix as the reference. Nothing else moves.

Base model	deepseek-ai/DeepSeek-V4-Flash-0731 (routed MoE, 43 blocks, 256 experts/layer)
Promoted layers	blk.0, blk.1 → Q4_K (ffn_gate_exps / ffn_down_exps / ffn_up_exps)
Imatrix	published reference (...chat-v2-routed-moe-ds4-1p5m.dat), not coding-specific
Runtime	antirez/ds4, commit 9ab705347c1775e7599ede7eb81a6255ec7dccb5
File size	90,343,990,176 B (84.15 GiB)
sha256	2d9c653eacbf4f730d0b2ef0d75e56307b0832659b91b54d7a7d07e901799d4b
Max context	131,072 tokens (verified, not just claimed — see below)

Why blk.0 and blk.1, and nothing else

This file is the outcome of a systematic, fully-tracked search — not a guess. Twelve rounds tried varying how many layers to promote, which specific layers, and how far to promote them, all measured against the identical frozen 50-task public coding suite:

Round	Recipe	Score
baseline	no promotion	21/50
this file	n=2: blk.0,1 → Q4_K	26/50
n=3 attempt	blk.0,1,2 → Q4_K	16/50 — regression
n=3 attempt	blk.0,1,5 → Q4_K	26/50 — tied, never beaten
n=4 attempt	blk.0,1,5,10 → Q4_K	≤24/50 — regression
precision push	blk.0,1 → Q8_0	runtime rejected it outright

Two findings fell out of this: layer position matters far more than promoted-layer count (adding a third layer helped, hurt, or did nothing depending entirely on which one), and Q8_0 is not usable for routed-expert tensors on this runtime — the ROCm FFN kernel does not support it, even though Q8_0 works fine everywhere else in the model (attention/shared experts/output). Every non-routed tensor promotion attempt (F16 for attention/shared-experts/output) was likewise rejected at load time. This file is the ceiling this search reached: the simplest recipe that reaches it.

What was actually measured

Public coding suite (50 frozen tasks, deterministic decoding, temperature 0): 26 passed, 24 failed — real per-observation results, not an estimate.

128K context: a 130,029-token prompt was accepted in full at a 131,072-token context window, no silent truncation, 512 output tokens generated cleanly. Artifact hash was confirmed identical before and after the runtime load.

Private five-language confirmation suite: run once, inconclusive. Both this candidate and the untouched baseline scored 0/15 — a floor effect (the suite was too hard to be discriminating at this quality tier for either model), not a mark against this file specifically.

Not tested: generalization beyond these two suites, agentic/long-running tasks, non-English or non-code evaluation, other quantizers/runtimes. Treat 26/50 as a real, measured gain over this baseline on this task set — not a general capability claim.

Run it

Built for antirez/ds4 specifically (DeepSeek V4's compressed-KV attention and MoE routing need it) — not guaranteed to load in mainline llama.cpp.

ds4-server --rocm \
  --model DeepSeek-V4-Flash-0731-DS4-Antirez-Coder-Halo-128K-MixQ.gguf \
  --ctx 131072 --port 8080

Fits in 90 GiB resident model memory plus a small KV/context overhead (0.4 GiB at short context, scaling with sequence length) — verified with 14–17 GiB of host memory headroom to spare on a 128 GB Strix Halo (unified memory, ROCm) host during the real evaluation runs above.

Provenance

• Base model: deepseek-ai/DeepSeek-V4-Flash-0731, revision 7872f01b1d1fe23eabc4c98b48bffcef5a386062, MIT license.
• Quantizer/runtime: antirez/ds4.
• Full recipe, raw evidence for every round in the table above, and the chart's source data are tracked in the originating project repository (recipes/C-R02-001.json, research/one-shot-native/20260910-c-r02-001-*.json, docs/decision-log.md, docs/learnings_gguf.md).

Note on this file's hash: the original C-R02-001 evidence (linked above) recorded sha256 423b91bc1e07.... This uploaded file was re-quantized from the identical recipe, imatrix content, template, and toolchain commit after the original archived copy was reclaimed for disk space; it differs only in an embedded imatrix-file-path metadata string (/mnt/bignvme/... here vs. /mnt/bigspace/... originally — the imatrix content itself is byte-identical, verified by hash), which shifts the overall file hash. All tensor data, and therefore the 26/50 evaluation result above, applies identically to this rebuild.