philipjohnbasile commited on
Commit
0fe9d9b
·
verified ·
1 Parent(s): 6961b99

Upload research/agentic_coder_landscape.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. research/agentic_coder_landscape.md +52 -0
research/agentic_coder_landscape.md ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # THE Local Agentic Coder — landscape + our path (June 2026)
2
+
3
+ Goal: make this model **THE** model for *local agentic coders*. This is the competitive scan + the honest
4
+ strategy that falls out of it.
5
+
6
+ ## The field (local/open agentic coders, June 2026)
7
+ | Model | Agentic headline | License |
8
+ |---|---|---|
9
+ | **Qwen3-Coder-Next** | 58.7% SWE-bench-Verified, 256K ctx, **runs on 24 GB** — purpose-built for agentic loops; *the* local bar | open |
10
+ | **DeepSeek-V4** (Pro 1.6T/49B · Flash 284B/13B) | 1M ctx, long-horizon planning, **improved tool-call reliability** (fewer malformed JSON) | MIT |
11
+ | **GLM-5.1** (our base's sibling, 754B) | **58.4 SWE-Pro, 8-hour autonomous execution** — SOTA agentic | MIT |
12
+ | **Kimi K2.6 Thinking** | 78.57 coding / 58.33 agentic — strongest open | Modified-MIT |
13
+ | **Qwen 3.6 Plus** | 1M ctx, reliable tool-use over long sessions | open |
14
+ | frontier ref: **Claude Opus 4.6** | 50% task completion @ **14.5-hour** horizon | closed |
15
+
16
+ **They compete on:** context length · raw SWE-bench · **tool-call reliability** · long-horizon autonomy · speed.
17
+
18
+ ## Where we honestly *can't* win
19
+ Raw size (we demolished GLM-5.2 → 99 GB / 3-bit), decode speed (11–14 tok/s), context (vs 1M), raw SWE-bench
20
+ (70 % of experts pruned). Chasing those is a losing race against un-demolished 24 GB models.
21
+
22
+ ## Where we WIN — agentic *reliability* (the thing that actually breaks agents)
23
+ The research names **tool-call reliability as the #1 agentic differentiator** (DeepSeek-V4's headline win was
24
+ *"fewer malformed JSON / partial calls"*). We don't improve it — we **guarantee** it:
25
+ - **Constrained tool-JSON (#45):** grammar-enforced tool-calls → **zero** malformed JSON, structurally impossible. (Field's best = "fewer"; ours = "none.")
26
+ - **Compiler-steered / verified decoding (#21, #24):** every line type-checks *as it's written* (TS 0.3 ms · Py ~0 · Rust 34 ms).
27
+ - **Fabrication-proof `done` (#41):** re-runs the *original* tests → can't hallucinate a pass.
28
+ - **Integrity layer:** test-tamper guard, secret-scan (16 providers), scope enforcement, slopsquat guard.
29
+ - **51-tool ReAct agent:** trajectory compaction, stall detection, the verifier mesh (5 langs + SQL + Lean).
30
+
31
+ **Positioning:** *"the local agentic coder that can't malform a tool-call, can't fake a test pass, and
32
+ compiler-checks every line."* The others are bigger/faster; **none** ship the verify-everything stack. Reliability
33
+ is a moat raw capability can't buy — and it's exactly what makes long agentic runs not collapse.
34
+
35
+ ## Adoptable from the field (CPU now → heal later)
36
+ 1. **Tool-use cold-start SFT** (AgentRL / ProRL pattern): query-formulation → tool-invocation → valid-JSON →
37
+ result-read → recover-from-error. → generate **agentic tool-use gold** for the next soul heal.
38
+ 2. **Long-horizon agentic data** (128 turns / 131K ctx, Hierarchy-of-Groups PO): multi-step plan→edit→test→fix
39
+ trajectories that *don't* lose the thread. Our agent is built for it — train + benchmark it.
40
+ 3. **Tool-call self-correction** (DeepSeek-V4's win): recover from a bad call. (Our constrained decoder prevents
41
+ the bad call upstream — but the *recovery* skill still helps when external tools fail.)
42
+ 4. **Atomic skills** ("Scaling Coding Agents via Atomic Skills", 2604.05013): decompose tasks into reusable skills → our skill library.
43
+ 5. **MCP / OpenAI-compat integration:** the field plugs local models into Cline · OpenCode · Aider · Claude Code via
44
+ OpenAI-compat + MCP. We already serve OpenAI-compat (`mlx_lm.server`) — document the one-line setup per agent so we're a drop-in.
45
+
46
+ ## Plan
47
+ - **NOW (CPU):** (a) position the card around **agentic reliability**; (b) generate agentic tool-use + long-horizon +
48
+ reliability gold (`heal/gold_agentic/`) for the next heal; (c) document the Cline/OpenCode/Aider/Claude-Code drop-in setup.
49
+ - **LATER (GPU, behind the factory + #59):** heal the agentic gold into the soul; benchmark long-horizon
50
+ (Terminal-Bench / FeatureBench / real-task, *not* the saturating SWE-bench); the #59 collapse fix keeps long agentic gens from degenerating.
51
+
52
+ *Sources: Qwen3-Coder-Next / DeepSeek-V4 / GLM-5.1 (MarkTechPost, kilo.ai, mindstudio 2026) · Agentic Tool Use 2604.00835 · AgentRL/ProRL 2603.18815 · Hierarchy-of-Groups PO 2602.22817 · Atomic Skills 2604.05013 · CWM 2510.02387 · OpenCode/Cline (morphllm 2026).*