jackasda211233 commited on
Commit
f4f5378
·
verified ·
1 Parent(s): deb80bc

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +15 -18
README.md CHANGED
@@ -46,7 +46,7 @@ Upgrade target:
46
 
47
  SignalLatch was already close to its BF16 source on the mixed probe snapshot. PatchCode keeps that small-form-factor Q4_NL path as the main deployment target and tests whether the agentic-coder distil improves practical coding-agent behaviour on top of it.
48
 
49
- Practical eval: under a hardened 5-seed, same-condition bake-off (160k-token real-world multi-file build as the discriminator — single-shot coding gates saturate and were rejected), PatchCode `IQ4_NL` tied `BF16` within noise on build, long-context, discipline, and autonomous-loop convergence, at ~⅓ the size. See the eval snapshot below.
50
 
51
  Release files:
52
  - `Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf`
@@ -159,27 +159,26 @@ Runtime notes:
159
 
160
  These numbers are from an internal practical coding-agent build matrix. This is not an academic benchmark. The discriminator is a 160k-token real-world multi-file build, scored multi-seed (single-shot coding gates saturate on this model family and were rejected).
161
 
162
- | candidate | build (KritaLite) | long-context | discipline (style) | autonomy-loop | size |
163
- |---|---:|---:|---:|---|---:|
164
- | **PatchCode IQ4_NL (reasoning imatrix)** | `0.920` (±0.067) | `0.975` | `0.842` (±0.333) | 100% / 0% halluc | `16.6 G` |
165
- | BF16 (control) | `0.867` | `0.942` | `0.931` | — | `57.6 G` |
166
- | Q8_0 | `0.867` | `0.969` | `0.742` | 100% | `29 G` |
167
- | mixed-recipe (promoted attn) | `0.907–0.933` | `0.935` | `0.69–0.90` | 100% | `20–24 G` |
168
 
169
  Read:
170
- - PatchCode `IQ4_NL` ties `BF16` within noise on build, long-context, discipline, and autonomous-loop convergence (8 held-out agentic tasks × 5 seeds: 100% convergence, 0% hallucinated-success, 0% stall for every quant).
171
  - Near-lossless `Q8_0` and promoted-attention mixed recipes bought **no measurable edge** and cost 2–6× the size.
172
  - This is a tentative long-term default, not a claim that the fine-tune is solved. Build has ±0.067–0.13 run-to-run variance and discipline ±0.3 on this suite; a 3-seed single-condition run shipped a *false* winner (a mixed recipe scored 0.933 once, never reproduced). Only 5+ seed same-condition head-to-heads + non-noise axes (size, recipe safety) reliably tiebreak. `IQ4_NL` wins on size + the plain-quant recipe (no promotion risk).
173
 
174
- Autonomy-loop detail (`agent_loop_eval`, 8 held-out tasks × 5 seeds):
175
 
176
- | quant | convergence | mean turns | recovery | halluc-success | stall |
177
- |---|---:|---:|---:|---:|---:|
178
- | PatchCode IQ4_NL | 100% (40/40) | 7.2 | 0.4 | 0% | 0% |
179
- | Q8_0 | 100% (40/40) | 7.0 | 0.5 | 0% | 0% |
180
- | mixed (c76) | 100% (40/40) | 6.6 | 0.4 | 0% | 0% |
181
 
182
- Non-discriminating (0pp spread): the action-first, verify-before-claim discipline is preserved across all quants.
183
 
184
  ## IQ4_NL vs BF16 — head-to-head (5-seed, same-condition)
185
 
@@ -190,7 +189,6 @@ The released `IQ4_NL` is what most people should run. It was confirmed against t
190
  | build (KritaLite, 5-seed) | `0.920` (±0.067) | `0.867` | IQ4_NL nominally higher; gap `0.053` ≈ within the ±0.067 noise floor |
191
  | long-context (4 tasks) | `0.975` | `0.942` | both near ceiling; IQ4_NL marginally higher |
192
  | discipline (action-first style, 5-seed) | `0.842` (±0.333) | `0.931` | BF16 nominally higher; gap `0.089` ≪ ±0.333 spread → within noise |
193
- | autonomous-loop convergence | `100%` (40/40), 0% halluc/stall | — | preserved |
194
  | **size on disk** | **`16.6 G`** | `57.6 G` | **IQ4_NL is ~71% smaller** |
195
 
196
  Read:
@@ -207,11 +205,10 @@ A near-lossless `Q8_0` was built and tested head-to-head (5-seed, same-condition
207
  | build (KritaLite) | `0.867` (±0.133) | `0.920` (±0.067) | — |
208
  | long-context | `0.969` | `0.975` | — |
209
  | discipline (style) | `0.742` (±0.292) | `0.842` (±0.333) | — |
210
- | autonomous-loop convergence | `100%` (40/40) | `100%` (40/40) | — |
211
  | **size on disk** | `29 G` | `16.6 G` | Q8 is ~1.7× larger |
212
 
213
  Read:
214
- - Q8_0 is **at or below** the released IQ4_NL on build, long-context, and discipline (all within noise), with zero autonomy edge — and it costs ~75% more disk.
215
  - Near-lossless precision buys nothing measurable on this suite (consistent with the finding that build here is **not** precision-limited — it's ceiling-limited and noisy). Releasing a 29 G file that ties a 16.6 G file would be wasted bytes.
216
  - Attention-promotion mixed recipes were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.
217
 
 
46
 
47
  SignalLatch was already close to its BF16 source on the mixed probe snapshot. PatchCode keeps that small-form-factor Q4_NL path as the main deployment target and tests whether the agentic-coder distil improves practical coding-agent behaviour on top of it.
48
 
49
+ Practical eval: under a hardened 5-seed, same-condition bake-off (160k-token real-world multi-file build as the discriminator — single-shot coding gates saturate and were rejected), PatchCode `IQ4_NL` tied `BF16` within noise on build, long-context, and discipline, at ~⅓ the size. See the eval snapshot below.
50
 
51
  Release files:
52
  - `Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf`
 
159
 
160
  These numbers are from an internal practical coding-agent build matrix. This is not an academic benchmark. The discriminator is a 160k-token real-world multi-file build, scored multi-seed (single-shot coding gates saturate on this model family and were rejected).
161
 
162
+ | candidate | build (KritaLite) | long-context | discipline (style) | size |
163
+ |---|---:|---:|---:|---:|
164
+ | **PatchCode IQ4_NL (reasoning imatrix)** | `0.920` (±0.067) | `0.975` | `0.842` (±0.333) | `16.6 G` |
165
+ | BF16 (control) | `0.867` | `0.942` | `0.931` | `57.6 G` |
166
+ | Q8_0 | `0.867` | `0.969` | `0.742` | `29 G` |
167
+ | mixed-recipe (promoted attn) | `0.907–0.933` | `0.935` | `0.69–0.90` | `20–24 G` |
168
 
169
  Read:
170
+ - PatchCode `IQ4_NL` ties `BF16` within noise on build, long-context, and discipline (5-seed, same-condition).
171
  - Near-lossless `Q8_0` and promoted-attention mixed recipes bought **no measurable edge** and cost 2–6× the size.
172
  - This is a tentative long-term default, not a claim that the fine-tune is solved. Build has ±0.067–0.13 run-to-run variance and discipline ±0.3 on this suite; a 3-seed single-condition run shipped a *false* winner (a mixed recipe scored 0.933 once, never reproduced). Only 5+ seed same-condition head-to-heads + non-noise axes (size, recipe safety) reliably tiebreak. `IQ4_NL` wins on size + the plain-quant recipe (no promotion risk).
173
 
174
+ PatchCode vs the base it was distilled from (15-case behaviour rubric; adapter-off as the "strength 0" anchor = the SignalLatch base):
175
 
176
+ | variant | rubric score | avg output tokens | avg time/case |
177
+ |---|---:|---:|---:|
178
+ | base (SignalLatch) | `0.486` | `311` | `34s` |
179
+ | PatchCode (λ=0.5) | `0.617` | `91` | `13s` |
 
180
 
181
+ PatchCode scores higher while emitting ~⅓ the tokens — the base rambled (~311 tokens of hedging preamble), PatchCode was terse and on-target. (Behaviour rubric, not a multi-turn agent turn-count; λ=0.5 is the sweet spot — higher strengths also got terse but fell *below* the base.)
182
 
183
  ## IQ4_NL vs BF16 — head-to-head (5-seed, same-condition)
184
 
 
189
  | build (KritaLite, 5-seed) | `0.920` (±0.067) | `0.867` | IQ4_NL nominally higher; gap `0.053` ≈ within the ±0.067 noise floor |
190
  | long-context (4 tasks) | `0.975` | `0.942` | both near ceiling; IQ4_NL marginally higher |
191
  | discipline (action-first style, 5-seed) | `0.842` (±0.333) | `0.931` | BF16 nominally higher; gap `0.089` ≪ ±0.333 spread → within noise |
 
192
  | **size on disk** | **`16.6 G`** | `57.6 G` | **IQ4_NL is ~71% smaller** |
193
 
194
  Read:
 
205
  | build (KritaLite) | `0.867` (±0.133) | `0.920` (±0.067) | — |
206
  | long-context | `0.969` | `0.975` | — |
207
  | discipline (style) | `0.742` (±0.292) | `0.842` (±0.333) | — |
 
208
  | **size on disk** | `29 G` | `16.6 G` | Q8 is ~1.7× larger |
209
 
210
  Read:
211
+ - Q8_0 is **at or below** the released IQ4_NL on build, long-context, and discipline (all within noise) — and it costs ~75% more disk.
212
  - Near-lossless precision buys nothing measurable on this suite (consistent with the finding that build here is **not** precision-limited — it's ceiling-limited and noisy). Releasing a 29 G file that ties a 16.6 G file would be wasted bytes.
213
  - Attention-promotion mixed recipes were tested for the same reason and ruled out: promotion destroyed discipline for no build gain. Only `IQ4_NL` and `BF16` are released.
214