Image-Text-to-Text
Transformers
Safetensors
deepseek_v41
text-generation
Eval Results
8-bit precision
fp8
Instructions to use deepseek-ai/DeepSeek-V4.1-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use deepseek-ai/DeepSeek-V4.1-Flash with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="deepseek-ai/DeepSeek-V4.1-Flash")# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4.1-Flash", device_map="auto") - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use deepseek-ai/DeepSeek-V4.1-Flash with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "deepseek-ai/DeepSeek-V4.1-Flash" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "deepseek-ai/DeepSeek-V4.1-Flash", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash
- SGLang
How to use deepseek-ai/DeepSeek-V4.1-Flash with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "deepseek-ai/DeepSeek-V4.1-Flash", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "deepseek-ai/DeepSeek-V4.1-Flash", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use deepseek-ai/DeepSeek-V4.1-Flash with Docker Model Runner:
docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash
马 仕镕 commited on
Commit ·
fb2764a
1
Parent(s): 2bc89ac
Update README.md & encoding/README.md
Browse files- .gitattributes +1 -0
- README.md +180 -27
- assets/dsv41_agentic_performance.png +3 -0
- assets/dsv41_kv_cache.png +3 -0
- encoding/README.md +222 -5
- evaluation/README.md +103 -0
- evaluation/dsh-minimal.patch +711 -0
.gitattributes
CHANGED
|
@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
inference/examples/images/carrots.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
DeepSeek_V41_Tech_Report.pdf filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
inference/examples/images/carrots.jpeg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
DeepSeek_V41_Tech_Report.pdf filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
*.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,44 +1,197 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
library_name: transformers
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
-
# DeepSeek-V4.1-
|
| 7 |
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
engram n-gram lookups, MoE, Hyper-Connections, and the DSpark forward path.
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
-
|
| 16 |
-
.
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
|
| 25 |
-
does not depend on PyTorch, while inference imports the sibling encoding module
|
| 26 |
-
with an explicit Python path. No symlinks are required.
|
| 27 |
|
| 28 |
-
|
| 29 |
-
tokenizer files live in the Hugging Face weight repository, not here.
|
| 30 |
|
| 31 |
-
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
-
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
## License
|
| 43 |
|
| 44 |
-
This repository
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
library_name: transformers
|
| 4 |
+
pipeline_tag: image-text-to-text
|
| 5 |
---
|
| 6 |
|
| 7 |
+
# DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
|
| 8 |
|
| 9 |
+
<!-- markdownlint-disable first-line-h1 -->
|
| 10 |
+
<!-- markdownlint-disable html -->
|
| 11 |
+
<!-- markdownlint-disable no-duplicate-header -->
|
|
|
|
| 12 |
|
| 13 |
+
<div align="center">
|
| 14 |
+
<img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg?raw=true" width="60%" alt="DeepSeek-V4.1" />
|
| 15 |
+
</div>
|
| 16 |
+
<hr>
|
| 17 |
+
<div align="center" style="line-height: 1;">
|
| 18 |
+
<a href="https://www.deepseek.com/" target="_blank" style="margin: 2px;">
|
| 19 |
+
<img alt="Homepage" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/badge.svg?raw=true" style="display: inline-block; vertical-align: middle;"/>
|
| 20 |
+
</a>
|
| 21 |
+
<a href="https://chat.deepseek.com/" target="_blank" style="margin: 2px;">
|
| 22 |
+
<img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-DeepSeek%20V4.1-536af5?color=536af5&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
|
| 23 |
+
</a>
|
| 24 |
+
</div>
|
| 25 |
+
<div align="center" style="line-height: 1;">
|
| 26 |
+
<a href="https://huggingface.co/deepseek-ai" target="_blank" style="margin: 2px;">
|
| 27 |
+
<img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-DeepSeek%20AI-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
|
| 28 |
+
</a>
|
| 29 |
+
<a href="https://twitter.com/deepseek_ai" target="_blank" style="margin: 2px;">
|
| 30 |
+
<img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-deepseek_ai-white?logo=x&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
|
| 31 |
+
</a>
|
| 32 |
+
</div>
|
| 33 |
+
<div align="center" style="line-height: 1;">
|
| 34 |
+
<a href="LICENSE" style="margin: 2px;">
|
| 35 |
+
<img alt="License" src="https://img.shields.io/badge/License-MIT-f5de53?&color=f5de53" style="display: inline-block; vertical-align: middle;"/>
|
| 36 |
+
</a>
|
| 37 |
+
</div>
|
| 38 |
|
| 39 |
+
<p align="center">
|
| 40 |
+
<a href="DeepSeek_V41_Tech_Report.pdf"><b>Technical Report</b> 👁️</a>
|
| 41 |
+
</p>
|
| 42 |
+
|
| 43 |
+
## Introduction
|
| 44 |
+
|
| 45 |
+
We introduce **DeepSeek-V4.1-Flash**, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively.
|
| 46 |
+
|
| 47 |
+
**Architecture.** DeepSeek-V4.1-Flash adopts a **Causal Encoder-Decoder (CED)** architecture: a 40-layer Transformer organized as a 20-layer causal encoder followed by a 20-layer decoder. With CED, the decoder's global KV cache is projected from the final encoder hidden states rather than derived from each decoder layer's own hidden states. This allows the model to activate only **8B parameters per token during prefill** and **16B during decode**, substantially improving cost efficiency for input-heavy agentic workloads. **SWA Bounded Replay** reconstructs missing SWA KV states by replaying only the most recent *n*_win tokens, avoiding the need to persist SWA KV to SSD and reducing the persistent KV cache footprint to roughly **1/8** of that of DeepSeek-V4-Flash.
|
| 48 |
+
|
| 49 |
+
**Compressed Sparse Attention 2 (CSA2).** DeepSeek-V4.1-Flash uses CSA2, which assigns each attention layer one of three static modes — **Full**, **Reindex**, or **Reuse** — to share main KV and indexer K across layers and reuse Top-K sparse-attention indices. In the decoder, a **Hierarchical Sparse Indexer** further restricts later indexing layers to a candidate pool constructed by the first Full Mode layer, bounding deeper indexer cost independently of context length. Combined with **FP4 main KV caching** (E2M1 format, one E4M3 scale per 16 channels), these designs reduce the global KV cache footprint to **890 bytes per token** — roughly **1/4** of DeepSeek-V4-Flash.
|
| 50 |
+
|
| 51 |
+
**Additional architectural components** include Single-Pass mHC (revised residual-stream mixing with an efficient Mega-mHC kernel), Engram conditional memory (196B parameters, sparsely accessed via token-based lookup), and DSpark speculative decoding (semi-autoregressive draft generation with confidence-scheduled verification). The model uses 1 shared expert and 384 routed experts per MoE layer, activating 6 routed experts per token.
|
| 52 |
+
|
| 53 |
+
**Multimodal architecture.** A vision encoder (DeepSeek-ViT, trained from scratch with 2D-RoPE and 3×3 pixel-unshuffle downsampling) and a two-layer MLP projector convert images into visual embeddings, processed jointly with text embeddings from the start of language-model pre-training.
|
| 54 |
+
|
| 55 |
+
**Pre-training.** DeepSeek-V4.1-Flash is trained from scratch on a multimodal corpus comprising **45T tokens**, with sparse attention trained at a sequence length of 64K and context extended to 1M tokens at 34T tokens.
|
| 56 |
+
|
| 57 |
+
**Post-training.** The post-training recipe follows the standard SFT → RL → on-policy distillation (OPD) paradigm without algorithmic modifications. All substantive changes lie instead in the data pipeline: large-scale automated synthesis of agent tasks and environments with progressive scaling of data, tasks, and rollouts. The model supports a **continuously controllable reasoning effort** setting (integer 1–100) that trades inference cost for accuracy.
|
| 58 |
+
|
| 59 |
+
<div align="center">
|
| 60 |
+
<img src="assets/dsv41_agentic_performance.png" width="48%" alt="DeepSeek-V4.1-Flash agentic benchmark performance" style="display: inline-block; margin: 0 1%;" />
|
| 61 |
+
<img src="assets/dsv41_kv_cache.png" width="48%" alt="Global KV cache size per token across DeepSeek generations" style="display: inline-block; margin: 0 1%;" />
|
| 62 |
+
</div>
|
| 63 |
+
|
| 64 |
+
*Figure 1. (a) Performance of DeepSeek-V4.1-Flash and counterparts on agentic benchmarks. (b) Global KV cache size per token (bytes) across generations of DeepSeek models. DeepSeek-V4.1-Flash achieves approximately 4-fold and 437-fold reductions relative to DeepSeek-V4-Flash and DeepSeek-V1, respectively.*
|
| 65 |
+
|
| 66 |
+
## Evaluation Results
|
| 67 |
+
|
| 68 |
+
### Base Model
|
| 69 |
+
|
| 70 |
+
All base models are evaluated in our internal framework under the same evaluation settings. Scores within 0.3 of each other are considered equivalent.
|
| 71 |
+
|
| 72 |
+
<div align="center">
|
| 73 |
+
|
| 74 |
+
| Benchmark (Metric) | # Shots | DeepSeek-V4-Flash-Base | DeepSeek-V4-Pro-Base | DeepSeek-V4.1-Flash-Base |
|
| 75 |
+
| :--- | :---: | :---: | :---: | :---: |
|
| 76 |
+
| Architecture | — | MoE | MoE | MoE |
|
| 77 |
+
| # Backbone Params | — | 284B | 1.6T | 552B |
|
| 78 |
+
| # Activated Params | — | 13B | 49B | 8B / 16B |
|
| 79 |
+
| **World Knowledge** | | | | |
|
| 80 |
+
| AGIEval (EM) | 3–5-shot | 83.9 | **84.4** | 83.4 |
|
| 81 |
+
| MMLU-Pro (EM) | 5-shot | 68.3 | 73.5 | **74.1** |
|
| 82 |
+
| C-Eval (EM) | 5-shot | 92.1 | **93.1** | 92.1 |
|
| 83 |
+
| MultiLoKo (LLM-Judge) | 5-shot | 42.6 | **50.9** | 45.5 |
|
| 84 |
+
| SimpleQA-Verified (EM) | 25-shot | 30.1 | **55.2** | 42.3 |
|
| 85 |
+
| SuperGPQA (EM) | 5-shot | 46.5 | **53.9** | 53.1 |
|
| 86 |
+
| **Language & Reasoning** | | | | |
|
| 87 |
+
| BBH (EM) | 3-shot | 86.9 | **87.5** | 86.1 |
|
| 88 |
+
| BBEH (EM) | 1-shot | 25.4 | **29.8** | 27.2 |
|
| 89 |
+
| DROP (F1) | 1-shot | **88.6** | **88.7** | 87.9 |
|
| 90 |
+
| HellaSwag (EM) | 0-shot | 85.7 | **88.0** | 87.2 |
|
| 91 |
+
| **Code & Math** | | | | |
|
| 92 |
+
| BigCodeBench (Pass@1) | 3-shot | 56.8 | 59.2 | **60.6** |
|
| 93 |
+
| HumanEval (Pass@1) | 0-shot | 69.5 | 76.8 | **79.4** |
|
| 94 |
+
| GSM8K (EM) | 8-shot | 90.8 | 92.6 | **93.0** |
|
| 95 |
+
| MATH (EM) | 4-shot | 57.4 | **64.5** | 61.1 |
|
| 96 |
+
| MGSM (EM) | 8-shot | **85.7** | 84.4 | 80.2 |
|
| 97 |
+
| **Long Context** | | | | |
|
| 98 |
+
| LongBench-V2 (EM) | 1-shot | 44.7 | **51.5** | 45.2 |
|
| 99 |
+
| **Multimodal** | | | | |
|
| 100 |
+
| MMMU-Pro (EM) | 4-shot | — | — | 56.5 |
|
| 101 |
+
| CVBench (EM) | 4-shot | — | — | 77.9 |
|
| 102 |
+
| DocVQA (LLM-Judge) | 4-shot | — | — | 95.6 |
|
| 103 |
+
| RefCOCO-avg (Acc@0.5) | 0-shot | — | — | 86.0 |
|
| 104 |
+
|
| 105 |
+
</div>
|
| 106 |
+
|
| 107 |
+
### Instruct Model
|
| 108 |
+
|
| 109 |
+
DeepSeek-V4.1-Flash supports a continuously controllable reasoning effort from 1 to 100. All instruct results below use the maximum effort setting (`reasoning_effort=100`). Evaluations use `temperature=1.0, top_p=0.95`.
|
| 110 |
|
| 111 |
+
For code agent benchmarks (Terminal Bench 2.1/3.0/4.0, DeepSWE v1.1, NL2Repo-Bench, ProgramBench), the model is evaluated with the Minimal mode of DeepSeek Harness and a 1M-token context window, except DeepSWE v1.1 which additionally reports results with the mini-SWE harness to align with the official evaluation. Visual agent benchmarks (Chartography, BabyVision, ZeroBench) use Claude Code harness with a 512k-token context window. Agent's Last Exam and AutomationBench use their official scaffolds.
|
|
|
|
|
|
|
| 112 |
|
| 113 |
+
#### Comparison with frontier models (Max reasoning effort)
|
|
|
|
| 114 |
|
| 115 |
+
<div align="center">
|
| 116 |
|
| 117 |
+
| Benchmark (Metric) | Opus-5.0 | GPT-5.6 Sol | K3 | GLM-5.3 | DS-V4-Pro | DS-V4-Flash | DS-V4.1-Flash |
|
| 118 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 119 |
+
| **Reasoning** | | | | | | | |
|
| 120 |
+
| GPQA Diamond (Pass@1) | 93.4 | **94.1** | 92.9 | 88.1 | 92.4 | 89.9 | 90.9 |
|
| 121 |
+
| HLE (Pass@1) | **56.3** | 44.5 | 43.5 | 42.0† | 42.7† | 37.8† | 36.8 (39.1†) |
|
| 122 |
+
| Codeforces (Rating) | — | — | — | — | 3348 | 3289 | **3471** |
|
| 123 |
+
| MathArena Apex (Pass@1) | — | — | **65.6** | — | 65.3 | 58.6 | **65.6** |
|
| 124 |
+
| **Agentic** | | | | | | | |
|
| 125 |
+
| Terminal Bench 2.1 (Pass@1) | 89.1 | 88.8 | 88.3 | 88.2 | 87.9 | 82.7 | **90.6** |
|
| 126 |
+
| Terminal Bench 3.0 (Pass@1) | **43.3** | 34.4 | 17.7 | 28.3 | 11.8 | 7.6 | 30.0 |
|
| 127 |
+
| Terminal Bench 4.0 (Pass@1) | **51.8** | 39.9 | 12.6 | 37.9 | 12.4 | 7.0 | 31.2 |
|
| 128 |
+
| DeepSWE v1.1 (Resolved) | 74.0 | 73.0 | 67.5 | 66.9 | 62.7 | 54.4 | **74.2** |
|
| 129 |
+
| ProgramBench (Almost@1) | **37.0** | 23.0 | 17.5 | 19.0 | 15.5 | — | 20.3 |
|
| 130 |
+
| NL2Repo-Bench (Score) | **75.3** | 56.8 | 58.0 | 58.0 | 61.5 | 54.2 | 64.0 |
|
| 131 |
+
| CyberGym (Pass@1) | — | 84.5 | 80.0 | 84.5 | 83.3 | 76.7 | **88.1** |
|
| 132 |
+
| SEC-Bench Pro (Pass@1) | — | **74.3** | — | — | 56.4 | 30.9 | 62.8 |
|
| 133 |
+
| ExploitGym (Pass@1) | 22.1 | **33.7** | — | 15.0 | 5.4 | 1.8 | 15.3 |
|
| 134 |
+
| HLE w/ tools (Pass@1) | 63.6 | — | 59.8 | 62.5 | 60.0 | 51.5 | **63.9** |
|
| 135 |
+
| AutomationBench (Pass@1) | 50.3 | 45.8 | 46.7 | 48.8 | 43.2 | 37.7 | **54.8** |
|
| 136 |
+
| Agent's Last Exam (Pass@1) | 28.6 | 26.7 | 27.6 | 28.5 | 25.7 | 25.2 | **31.8** |
|
| 137 |
+
| Chartography w/ tools (Pass@1) | **84.0** | 79.9 | 68.1 | — | — | — | 78.9 |
|
| 138 |
+
| BabyVision w/ tools (Pass@1) | **94.1** | 88.9 | 85.7 | — | — | — | 89.6 |
|
| 139 |
+
| ZeroBench-main w/ tools (Pass@5) | 52.0 | **53.0** | 41.0 | — | — | — | 49.0 |
|
| 140 |
|
| 141 |
+
</div>
|
| 142 |
|
| 143 |
+
*† Text-only subset of HLE.*
|
| 144 |
+
|
| 145 |
+
#### Performance across agent scaffolds (DeepSWE v1.1 and Terminal Bench 2.1, Max reasoning effort)
|
| 146 |
+
|
| 147 |
+
All scaffolds use N=8 samples per task on DeepSWE v1.1 and N=3 on Terminal Bench 2.1, with Linux containers, `temperature=1.0`, `top_p=0.95`, a 1M-token context limit, and max_steps=500 per agent.
|
| 148 |
+
|
| 149 |
+
<div align="center">
|
| 150 |
+
|
| 151 |
+
| Benchmark (Metric) | Claude Code | Codex | OpenCode | Pi | mini-SWE | DSH Minimal | DSH Standard | DSH PTC |
|
| 152 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 153 |
+
| DeepSWE v1.1 (Resolved) | 69.8 | 65.6 | 65.5 | 66.2 | 74.2 | 72.6 | 70.5 | 67.6 |
|
| 154 |
+
| Terminal Bench 2.1 (Pass@1) | 88.0 | 84.1 | 85.0 | 86.1 | 90.3 | 90.6 | 85.8 | 85.8 |
|
| 155 |
+
|
| 156 |
+
</div>
|
| 157 |
+
|
| 158 |
+
## Prompt Encoding
|
| 159 |
+
|
| 160 |
+
This release does not include a Jinja-format chat template. The [`encoding`](encoding/README.md) folder contains a self-contained Python reference implementation (`encoding.py`) with test cases for multi-turn conversations, tool calling, thinking mode, numeric reasoning effort, mid-conversation system messages, and interleaved image content.
|
| 161 |
+
|
| 162 |
+
For production use, we additionally release [deepseek-recipe](https://github.com/deepseek-ai/deepseek-recipe), a set of Rust libraries with Python bindings that provides the same prompt format as a maintained, protocol-aware toolkit. It converts Messages, Chat Completions, and Responses API requests into the Conversation format, encodes them into DeepSeek V4 and V4.1 prompts or token IDs, and parses model output back into complete or streamed responses — covering thinking, tool calls, images, and generation settings. Model inference, tool execution, and HTTP transport are left to the caller.
|
| 163 |
+
|
| 164 |
+
## Minimal Inference
|
| 165 |
+
|
| 166 |
+
Please refer to the [`inference`](inference/README.md) folder for instructions on weight conversion and running inference locally.
|
| 167 |
+
|
| 168 |
+
**Recommended sampling parameters:**
|
| 169 |
+
|
| 170 |
+
| Parameter | Value |
|
| 171 |
+
| :--- | :--- |
|
| 172 |
+
| `temperature` | 1.0 |
|
| 173 |
+
| `top_p` | 0.95 or 1.0 |
|
| 174 |
+
| `context_window` | 1M tokens |
|
| 175 |
+
| `max_tokens` | ≥ 256K |
|
| 176 |
+
|
| 177 |
+
## Reproducing DeepSWE Benchmark Results
|
| 178 |
+
|
| 179 |
+
The [`evaluation`](evaluation/README.md) folder contains step-by-step instructions for reproducing the DeepSWE v1.1 benchmark results, covering both the `dsh-minimal` agent and the official `mini-swe-agent`. The patch required to integrate `dsh-minimal` with [Pier](https://github.com/datacurve-ai/pier) is also included there.
|
| 180 |
|
| 181 |
## License
|
| 182 |
|
| 183 |
+
This repository and the model weights are licensed under the [MIT License](LICENSE).
|
| 184 |
+
|
| 185 |
+
## Citation
|
| 186 |
+
|
| 187 |
+
```bibtex
|
| 188 |
+
@misc{deepseekai2026deepseekv41flash,
|
| 189 |
+
title={DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression},
|
| 190 |
+
author={DeepSeek-AI},
|
| 191 |
+
year={2026},
|
| 192 |
+
}
|
| 193 |
+
```
|
| 194 |
+
|
| 195 |
+
## Contact
|
| 196 |
+
|
| 197 |
+
If you have any questions, please raise an issue or contact us at [service@deepseek.com](mailto:service@deepseek.com).
|
assets/dsv41_agentic_performance.png
ADDED
|
Git LFS Details
|
assets/dsv41_kv_cache.png
ADDED
|
Git LFS Details
|
encoding/README.md
CHANGED
|
@@ -1,9 +1,64 @@
|
|
| 1 |
# DeepSeek-V4.1 text and vision encoding
|
| 2 |
|
| 3 |
-
`encoding.py` is the standalone prompt-format reference
|
| 4 |
-
conversations, tool calls, thinking modes,
|
|
|
|
| 5 |
without importing the inference implementation.
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
## OpenAI-style messages
|
| 8 |
|
| 9 |
```python
|
|
@@ -26,11 +81,14 @@ prompt, media = encode_messages(
|
|
| 26 |
thinking_mode="chat",
|
| 27 |
return_multi_modal_data=True,
|
| 28 |
)
|
|
|
|
|
|
|
|
|
|
| 29 |
```
|
| 30 |
|
| 31 |
Images are represented in the prompt by `<|deepseek_image|>`. `media["images"]`
|
| 32 |
-
contains the corresponding image records in exactly the same order
|
| 33 |
-
loading and expansion into model image tokens are handled by
|
| 34 |
`inference/image_processor.py`.
|
| 35 |
|
| 36 |
## Compact TXT notation
|
|
@@ -44,6 +102,162 @@ loading and expansion into model image tokens are handled by
|
|
| 44 |
into the same standard content blocks. It is an input convenience layer, not a
|
| 45 |
second encoding implementation.
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
## Tests
|
| 48 |
|
| 49 |
From this directory:
|
|
@@ -52,5 +266,8 @@ From this directory:
|
|
| 52 |
python -m pytest -q test_encoding.py
|
| 53 |
```
|
| 54 |
|
| 55 |
-
|
|
|
|
|
|
|
|
|
|
| 56 |
prompt and preserve the same image ordering.
|
|
|
|
| 1 |
# DeepSeek-V4.1 text and vision encoding
|
| 2 |
|
| 3 |
+
`encoding.py` is the standalone prompt-format reference for DeepSeek-V4.1. It
|
| 4 |
+
supports multi-turn conversations, tool calls, thinking modes, numeric reasoning
|
| 5 |
+
effort, mid-conversation system messages, and interleaved image content blocks,
|
| 6 |
without importing the inference implementation.
|
| 7 |
|
| 8 |
+
## V4.1 changes relative to V4
|
| 9 |
+
|
| 10 |
+
Three prompt-format changes distinguish V4.1 from V4:
|
| 11 |
+
|
| 12 |
+
1. **DSML tag names use a leading space.** Tool calls are wrapped in
|
| 13 |
+
`<|DSML| calls>` blocks with `<|DSML| invoke>` / `<|DSML| parameter>` tags
|
| 14 |
+
(note the space before `calls`, `invoke`, and `parameter`). The V4 format used
|
| 15 |
+
`<|DSML|tool_calls>` without a space.
|
| 16 |
+
|
| 17 |
+
2. **Reasoning effort is a numeric budget (1–100).** The effort prefix is
|
| 18 |
+
rendered as `Reasoning Effort: {budget} (range 1-100, ...)` rather than the
|
| 19 |
+
verbose natural-language descriptions used in V4. String aliases map as
|
| 20 |
+
follows: `"low"` → 25, `"high"` → 50, `"xhigh"` → 75, `"max"` → 100. The
|
| 21 |
+
default is `"high"` (50). The effort prefix is only rendered in
|
| 22 |
+
`thinking_mode="thinking"` and only at the beginning of the conversation
|
| 23 |
+
(index 0).
|
| 24 |
+
|
| 25 |
+
3. **Mid-conversation system messages** are supported via the `<|System|>` token.
|
| 26 |
+
A mid-conversation system message behaves like a user message for the purpose
|
| 27 |
+
of appending the assistant generation header.
|
| 28 |
+
|
| 29 |
+
## Quick start
|
| 30 |
+
|
| 31 |
+
```python
|
| 32 |
+
from encoding import encode_messages, parse_message_from_completion_text
|
| 33 |
+
|
| 34 |
+
# Text-only conversation
|
| 35 |
+
messages = [
|
| 36 |
+
{"role": "system", "content": "You are a helpful assistant."},
|
| 37 |
+
{"role": "user", "content": "What is 2+2?"},
|
| 38 |
+
]
|
| 39 |
+
prompt, media = encode_messages(
|
| 40 |
+
messages,
|
| 41 |
+
thinking_mode="thinking",
|
| 42 |
+
reasoning_effort=75, # integer 1–100, or "low"/"high"/"xhigh"/"max"
|
| 43 |
+
return_multi_modal_data=True,
|
| 44 |
+
)
|
| 45 |
+
# prompt:
|
| 46 |
+
# '<|begin▁of▁sentence|><|System|>Reasoning Effort: 75 (range 1-100, the higher the
|
| 47 |
+
# value, the more thorough the reasoning)\n\nYou are a helpful assistant.
|
| 48 |
+
# <|User|>What is 2+2?<|Assistant|><think>'
|
| 49 |
+
|
| 50 |
+
# Parse model output back to a structured message
|
| 51 |
+
completion = "Simple arithmetic.</think>2 + 2 = 4.<|end▁of▁sentence|>"
|
| 52 |
+
parsed = parse_message_from_completion_text(completion, thinking_mode="thinking")
|
| 53 |
+
# => {"role": "assistant", "reasoning_content": "Simple arithmetic.",
|
| 54 |
+
# "content": "2 + 2 = 4.", "tool_calls": []}
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
> **Note:** `parse_message_from_completion_text` is designed to handle
|
| 58 |
+
> well-formatted model output only. It does not attempt to correct or recover
|
| 59 |
+
> from malformed output that the model might occasionally generate. For
|
| 60 |
+
> production use, additional error handling is recommended.
|
| 61 |
+
|
| 62 |
## OpenAI-style messages
|
| 63 |
|
| 64 |
```python
|
|
|
|
| 81 |
thinking_mode="chat",
|
| 82 |
return_multi_modal_data=True,
|
| 83 |
)
|
| 84 |
+
# prompt:
|
| 85 |
+
# '<|begin▁of▁sentence|><|User|>第一张图\n\n<|deepseek_image|>\n\n有什么内容?<|Assistant|></think>'
|
| 86 |
+
# media["images"] contains the image records in prompt order
|
| 87 |
```
|
| 88 |
|
| 89 |
Images are represented in the prompt by `<|deepseek_image|>`. `media["images"]`
|
| 90 |
+
contains the corresponding image records in exactly the same order they appear in
|
| 91 |
+
the prompt. Pixel loading and expansion into model image tokens are handled by
|
| 92 |
`inference/image_processor.py`.
|
| 93 |
|
| 94 |
## Compact TXT notation
|
|
|
|
| 102 |
into the same standard content blocks. It is an input convenience layer, not a
|
| 103 |
second encoding implementation.
|
| 104 |
|
| 105 |
+
## Message format
|
| 106 |
+
|
| 107 |
+
### Special tokens
|
| 108 |
+
|
| 109 |
+
| Token | Purpose |
|
| 110 |
+
| :--- | :--- |
|
| 111 |
+
| `<|begin▁of▁sentence|>` | Beginning of sequence (BOS) |
|
| 112 |
+
| `<|end▁of▁sentence|>` | End of assistant turn (EOS) |
|
| 113 |
+
| `<|User|>` | User turn prefix |
|
| 114 |
+
| `<|Assistant|>` | Assistant turn prefix |
|
| 115 |
+
| `<|System|>` | Mid-conversation system message prefix |
|
| 116 |
+
| `<|latest_reminder|>` | Latest reminder (date, locale, etc.) |
|
| 117 |
+
| `<think>` / `</think>` | Reasoning block delimiters |
|
| 118 |
+
| `|DSML|` | DSML markup token |
|
| 119 |
+
| `<|deepseek_image|>` | Image placeholder in the prompt string |
|
| 120 |
+
|
| 121 |
+
### Roles
|
| 122 |
+
|
| 123 |
+
The encoding supports the following message roles: `system`, `user`, `assistant`,
|
| 124 |
+
`tool`, and `latest_reminder`.
|
| 125 |
+
|
| 126 |
+
A `tool` message is not rendered directly: `merge_tool_messages()` converts it
|
| 127 |
+
into a `<tool_result>` block inside the preceding user message. When multiple
|
| 128 |
+
tool results are present, they are sorted by the order of the corresponding
|
| 129 |
+
`tool_calls` in the preceding assistant message.
|
| 130 |
+
|
| 131 |
+
### Basic chat
|
| 132 |
+
|
| 133 |
+
A simple multi-turn conversation is encoded as:
|
| 134 |
+
|
| 135 |
+
```
|
| 136 |
+
<|begin▁of▁sentence|>{system_prompt}
|
| 137 |
+
<|User|>{user_message}<|Assistant|></think>{response}<|end▁of▁sentence|>
|
| 138 |
+
<|User|>{user_message_2}<|Assistant|></think>{response_2}<|end▁of▁sentence|>
|
| 139 |
+
```
|
| 140 |
+
|
| 141 |
+
- The BOS token is prepended at the very beginning of the conversation.
|
| 142 |
+
- In **chat mode** (`thinking_mode="chat"`), `</think>` is placed right after
|
| 143 |
+
`<|Assistant|>` to immediately close the thinking block, so the model generates
|
| 144 |
+
content directly.
|
| 145 |
+
|
| 146 |
+
### Thinking mode
|
| 147 |
+
|
| 148 |
+
In **thinking mode** (`thinking_mode="thinking"`), the model produces explicit
|
| 149 |
+
reasoning inside `<think>...</think>` blocks before responding.
|
| 150 |
+
|
| 151 |
+
```
|
| 152 |
+
<|begin▁of▁sentence|><|System|>{reasoning_effort_prefix}{system_prompt}
|
| 153 |
+
<|User|>{message}<|Assistant|><think>{reasoning}</think>{response}<|end▁of▁sentence|>
|
| 154 |
+
```
|
| 155 |
+
|
| 156 |
+
The reasoning effort prefix is injected once, before the system message, as a
|
| 157 |
+
`<|System|>` block:
|
| 158 |
+
|
| 159 |
+
```
|
| 160 |
+
<|System|>Reasoning Effort: {budget} (range 1-100, the higher the value, the more thorough the reasoning)
|
| 161 |
+
```
|
| 162 |
+
|
| 163 |
+
The `drop_thinking` parameter (default `True`) controls whether reasoning from
|
| 164 |
+
earlier turns is preserved:
|
| 165 |
+
|
| 166 |
+
- **Without tools**: reasoning content from assistant turns **before** the last
|
| 167 |
+
user message is stripped. Only the final assistant turn retains its
|
| 168 |
+
`<think>...</think>` block.
|
| 169 |
+
- **With tools**: `drop_thinking` is automatically disabled. All turns retain
|
| 170 |
+
their reasoning, because tool-calling conversations require full context for
|
| 171 |
+
the model to track multi-step reasoning across tool calls.
|
| 172 |
+
|
| 173 |
+
### Tool calling (DSML format)
|
| 174 |
+
|
| 175 |
+
Tools are defined on the `system` message via the `tools` field
|
| 176 |
+
(OpenAI-compatible format). When tools are present, the following schema block is
|
| 177 |
+
injected into the system prompt:
|
| 178 |
+
|
| 179 |
+
```
|
| 180 |
+
## Tools
|
| 181 |
+
|
| 182 |
+
You have access to a set of tools to help answer the user's question. You can invoke tools by writing a "<|DSML| calls>" block like the following:
|
| 183 |
+
|
| 184 |
+
<|DSML| calls>
|
| 185 |
+
<|DSML| invoke name="$TOOL_NAME">
|
| 186 |
+
<|DSML| parameter name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE</|DSML| parameter>
|
| 187 |
+
...
|
| 188 |
+
</|DSML| invoke>
|
| 189 |
+
<|DSML| invoke name="$TOOL_NAME2">
|
| 190 |
+
...
|
| 191 |
+
</|DSML| invoke>
|
| 192 |
+
</|DSML| calls>
|
| 193 |
+
|
| 194 |
+
String parameters should be specified as is and set `string="true"`. For all other types (numbers, booleans, arrays, objects), pass the value in JSON format and set `string="false"`.
|
| 195 |
+
|
| 196 |
+
If thinking_mode is enabled (triggered by <think>), you MUST output your complete reasoning inside <think>...</think> BEFORE any tool calls or final response.
|
| 197 |
+
|
| 198 |
+
Otherwise, output directly after </think> with tool calls or final response.
|
| 199 |
+
|
| 200 |
+
### Available Tool Schemas
|
| 201 |
+
|
| 202 |
+
{tool_definitions_json}
|
| 203 |
+
|
| 204 |
+
You MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.
|
| 205 |
+
```
|
| 206 |
+
|
| 207 |
+
An actual tool call in the assistant turn looks like:
|
| 208 |
+
|
| 209 |
+
```xml
|
| 210 |
+
|
| 211 |
+
<|DSML| calls>
|
| 212 |
+
<|DSML| invoke name="function_name">
|
| 213 |
+
<|DSML| parameter name="param" string="true">string_value</|DSML| parameter>
|
| 214 |
+
<|DSML| parameter name="count" string="false">5</|DSML| parameter>
|
| 215 |
+
</|DSML| invoke>
|
| 216 |
+
</|DSML| calls><|end▁of▁sentence|>
|
| 217 |
+
```
|
| 218 |
+
|
| 219 |
+
- `string="true"`: the parameter value is a raw string.
|
| 220 |
+
- `string="false"`: the parameter value is JSON (number, boolean, array, object).
|
| 221 |
+
|
| 222 |
+
Tool execution results are wrapped in `<tool_result>` tags within user messages:
|
| 223 |
+
|
| 224 |
+
```
|
| 225 |
+
<|User|><tool_result>{result_json}</tool_result><|Assistant|><think>...
|
| 226 |
+
```
|
| 227 |
+
|
| 228 |
+
### Reasoning effort
|
| 229 |
+
|
| 230 |
+
Pass `reasoning_effort` as an integer in `[1, 100]` or as one of `"low"` (25),
|
| 231 |
+
`"high"` (50), `"xhigh"` (75), or `"max"` (100). The default is `"high"` (50).
|
| 232 |
+
The setting only affects `thinking_mode="thinking"` and is only rendered at the
|
| 233 |
+
start of the conversation (index 0). Intermediate values may be used to elicit
|
| 234 |
+
interpolated reasoning behavior.
|
| 235 |
+
|
| 236 |
+
### Quick instruction special tokens
|
| 237 |
+
|
| 238 |
+
Quick instruction tokens are used for auxiliary classification and generation
|
| 239 |
+
tasks. They are appended to messages via the `"task"` field to trigger
|
| 240 |
+
specialized model behavior for a single-token or short-form output.
|
| 241 |
+
|
| 242 |
+
| Special Token | Description | Format |
|
| 243 |
+
|:---|:---|:---|
|
| 244 |
+
| `<|action|>` | Determines whether the user prompt requires a web search or can be answered directly. | `...<|User|>{prompt}<|Assistant|><think><|action|>` |
|
| 245 |
+
| `<|title|>` | Generates a concise conversation title after the first assistant response. | `...<|Assistant|>{response}<|end▁of▁sentence|><|title|>` |
|
| 246 |
+
| `<|query|>` | Generates search queries for the user prompt. | `...<|User|>{prompt}<|query|>` |
|
| 247 |
+
| `<|authority|>` | Classifies the user prompt's demand for source authoritativeness. | `...<|User|>{prompt}<|authority|>` |
|
| 248 |
+
| `<|domain|>` | Identifies the domain of the user prompt. | `...<|User|>{prompt}<|domain|>` |
|
| 249 |
+
| `<|read_url|>` | Determines whether each URL in the user prompt should be fetched and read. | `...<|User|>{prompt}<|read_url|>` |
|
| 250 |
+
|
| 251 |
+
Usage in message format:
|
| 252 |
+
|
| 253 |
+
- **`action`** on a user message: the `<|action|>` token is placed after the
|
| 254 |
+
assistant prefix and thinking token, triggering a routing decision (e.g.,
|
| 255 |
+
"Search" or "Answer").
|
| 256 |
+
- **Other tasks** (`query`, `authority`, `domain`, `read_url`) on a user message:
|
| 257 |
+
the task token is appended directly after the user content.
|
| 258 |
+
- **`title`** on an assistant message: the `<|title|>` token is appended after
|
| 259 |
+
the assistant's EOS. The next assistant message provides the generated title.
|
| 260 |
+
|
| 261 |
## Tests
|
| 262 |
|
| 263 |
From this directory:
|
|
|
|
| 266 |
python -m pytest -q test_encoding.py
|
| 267 |
```
|
| 268 |
|
| 269 |
+
Test cases are stored as paired JSON input / TXT expected-output files under
|
| 270 |
+
`tests/`. The tests cover multi-turn conversations, tool calling, thinking mode,
|
| 271 |
+
numeric reasoning effort, mid-conversation system messages, and multimodal image
|
| 272 |
+
ordering. They include a check that the TXT and JSON examples encode to the same
|
| 273 |
prompt and preserve the same image ordering.
|
evaluation/README.md
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Running DeepSWE with `dsh-minimal` and `mini-swe-agent`
|
| 2 |
+
|
| 3 |
+
## 1. Prerequisites
|
| 4 |
+
|
| 5 |
+
- Docker, running and able to pull images
|
| 6 |
+
- Python 3.12+ and [uv](https://docs.astral.sh/uv/)
|
| 7 |
+
- An endpoint and key for any DeepSeek-API-compatible service (the DeepSeek official API is used as the example below)
|
| 8 |
+
|
| 9 |
+
```sh
|
| 10 |
+
export DEEPSEEK_API_KEY=sk-your-key-here
|
| 11 |
+
export DEEPSEEK_BASE_URL=https://api.deepseek.com
|
| 12 |
+
```
|
| 13 |
+
|
| 14 |
+
## 2. Get Pier and DeepSWE
|
| 15 |
+
|
| 16 |
+
```sh
|
| 17 |
+
git clone https://github.com/datacurve-ai/pier.git
|
| 18 |
+
git -C pier checkout 0c802fc067a425345b24d1c69411aa98acf61a1d
|
| 19 |
+
|
| 20 |
+
git clone https://github.com/datacurve-ai/deep-swe.git
|
| 21 |
+
git -C deep-swe checkout 0b9fabbb63b9104d678fe965e1632f2dd9eaa2ea
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
## 3. Patch and install Pier
|
| 25 |
+
|
| 26 |
+
`dsh-minimal.patch` ships next to this document. Treat it as a **reference patch** and adapt it to your own setup.
|
| 27 |
+
|
| 28 |
+
```sh
|
| 29 |
+
cd pier
|
| 30 |
+
git apply /path/to/dsh-minimal.patch
|
| 31 |
+
uv sync
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
What the patch changes:
|
| 35 |
+
|
| 36 |
+
- **Adds the `dsh-minimal` agent**, which drives the Harness SDK and folds its event stream into a Pier ATIF trajectory. The SDK artifact is never installed into the image: step 4's `--mounts-json` bind-mounts it read-only into the sandbox, so no trial installs anything.
|
| 37 |
+
- **Appends a runtime-constraints section to the task instruction for both agents**: work in `/app`, leave `/tests` alone, no network or package mirror.
|
| 38 |
+
- **Passes test-runner concurrency caps into the container**: Docker's `--cpus` is only a quota, so `nproc` inside the container reports the host's core count and test runners size their worker pools from that rather than from the container's share.
|
| 39 |
+
- **Enables IPv6 loopback in the container**: Docker disables it by default, so suites that bind `::1` are skipped and scored as failures.
|
| 40 |
+
- **Makes `--mounts-json` additive instead of replacing the default mounts**, keeping the `/logs` binds that carry agent logs and collected patches.
|
| 41 |
+
|
| 42 |
+
## 4. Run the suite
|
| 43 |
+
|
| 44 |
+
Both agents take the same task set, concurrency, and `--no-delete` (which keeps the task images cached between trials). Repeat each run with a different `--job-name` and average the results.
|
| 45 |
+
|
| 46 |
+
Each trial's container takes the 2 CPUs and 8 GB its task declares, so size `-n` against the host's cores and memory.
|
| 47 |
+
|
| 48 |
+
### `mini-swe-agent`
|
| 49 |
+
|
| 50 |
+
Pier installs it into each task image at trial time, so no host-side preparation is needed.
|
| 51 |
+
|
| 52 |
+
```sh
|
| 53 |
+
uv run pier run \
|
| 54 |
+
-p ../deep-swe/tasks \
|
| 55 |
+
--agent mini-swe-agent \
|
| 56 |
+
--model deepseek/deepseek-flash \
|
| 57 |
+
--ak reasoning_effort=max \
|
| 58 |
+
--ak cost_limit=0 \
|
| 59 |
+
--ae DEEPSEEK_API_KEY="$DEEPSEEK_API_KEY" \
|
| 60 |
+
--ae DEEPSEEK_BASE_URL="$DEEPSEEK_BASE_URL" \
|
| 61 |
+
-n 32 --no-delete -r 2 --job-name deepswe-mini-run1 -y
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
- `--model` takes a litellm-style `provider/model` string.
|
| 65 |
+
|
| 66 |
+
### `dsh-minimal`
|
| 67 |
+
|
| 68 |
+
Install the Harness SDK artifact once on the host, then bind-mount it read-only into every container.
|
| 69 |
+
|
| 70 |
+
```sh
|
| 71 |
+
mkdir -p ~/dsh-minimal && cd ~/dsh-minimal
|
| 72 |
+
uv pip install --target dsh-dist \
|
| 73 |
+
--python-version 3.12 --python-platform x86_64-manylinux_2_28 \
|
| 74 |
+
'deepseek-harness-sdk==0.1.5.*'
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
```sh
|
| 78 |
+
uv run pier run \
|
| 79 |
+
-p ../deep-swe/tasks \
|
| 80 |
+
--agent dsh-minimal \
|
| 81 |
+
--model deepseek-flash \
|
| 82 |
+
--ak reasoning_effort=max \
|
| 83 |
+
--ae DEEPSEEK_API_KEY="$DEEPSEEK_API_KEY" \
|
| 84 |
+
--ae DEEPSEEK_BASE_URL="$DEEPSEEK_BASE_URL" \
|
| 85 |
+
--mounts-json '[{"type":"bind","source":"'"$HOME"'/dsh-minimal/dsh-dist","target":"/opt/dsh-minimal","read_only":true}]' \
|
| 86 |
+
-n 32 --no-delete --job-name deepswe-dsh-run1 -y
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
- In `--mounts-json`, `source` is the absolute path of the `dsh-dist` directory above; `target` is always `/opt/dsh-minimal`.
|
| 90 |
+
|
| 91 |
+
## 5. Read the results
|
| 92 |
+
|
| 93 |
+
```
|
| 94 |
+
jobs/<job-name>/
|
| 95 |
+
result.json pass rate and token totals
|
| 96 |
+
<task>__<id>/
|
| 97 |
+
result.json reward, fail-to-pass / pass-to-pass counts, tokens
|
| 98 |
+
agent/trajectory.json full ATIF trajectory (dsh-minimal)
|
| 99 |
+
agent/mini-swe-agent.trajectory.json mini-swe-agent trajectory
|
| 100 |
+
verifier/ reward.json and test output
|
| 101 |
+
```
|
| 102 |
+
|
| 103 |
+
Browse a job with `uv run pier view jobs/<job-name>`.
|
evaluation/dsh-minimal.patch
ADDED
|
@@ -0,0 +1,711 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
diff --git a/src/pier/agents/factory.py b/src/pier/agents/factory.py
|
| 2 |
+
index 665fadd..503a45d 100644
|
| 3 |
+
--- a/src/pier/agents/factory.py
|
| 4 |
+
+++ b/src/pier/agents/factory.py
|
| 5 |
+
@@ -6,6 +6,7 @@ from pier.agents.installed.antigravity_sdk import AntigravitySDK
|
| 6 |
+
from pier.agents.installed.claude_code import ClaudeCode
|
| 7 |
+
from pier.agents.installed.codex import Codex
|
| 8 |
+
from pier.agents.installed.cursor_cli import CursorCli
|
| 9 |
+
+from pier.agents.installed.dsh_minimal import DshMinimal
|
| 10 |
+
from pier.agents.installed.gemini_cli import GeminiCli
|
| 11 |
+
from pier.agents.installed.mini_swe_agent import MiniSweAgent
|
| 12 |
+
from pier.agents.installed.opencode import OpenCode
|
| 13 |
+
@@ -24,6 +25,7 @@ class AgentFactory:
|
| 14 |
+
ClaudeCode,
|
| 15 |
+
Codex,
|
| 16 |
+
CursorCli,
|
| 17 |
+
+ DshMinimal,
|
| 18 |
+
GeminiCli,
|
| 19 |
+
MiniSweAgent,
|
| 20 |
+
OpenCode,
|
| 21 |
+
diff --git a/src/pier/agents/installed/base.py b/src/pier/agents/installed/base.py
|
| 22 |
+
index 16f6a84..1fd1050 100644
|
| 23 |
+
--- a/src/pier/agents/installed/base.py
|
| 24 |
+
+++ b/src/pier/agents/installed/base.py
|
| 25 |
+
@@ -15,6 +15,18 @@ from pier.utils.env import parse_bool_env_value
|
| 26 |
+
from pier.utils.templating import render_prompt_template
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
+RUNTIME_CONSTRAINTS = (
|
| 30 |
+
+ "\n## Runtime constraints\n"
|
| 31 |
+
+ "- Work in `/app`; do not modify files under `/tests`.\n"
|
| 32 |
+
+ "- No network or mirror access; use only dependencies already in the image.\n"
|
| 33 |
+
+)
|
| 34 |
+
+
|
| 35 |
+
+
|
| 36 |
+
+def with_runtime_constraints(instruction: str) -> str:
|
| 37 |
+
+ """Append the sandbox constraints to a task instruction."""
|
| 38 |
+
+ return instruction.rstrip("\n") + "\n" + RUNTIME_CONSTRAINTS
|
| 39 |
+
+
|
| 40 |
+
+
|
| 41 |
+
class NonZeroAgentExitCodeError(RuntimeError):
|
| 42 |
+
"""Raised when the agent process exits with a non-zero exit code."""
|
| 43 |
+
|
| 44 |
+
@@ -394,12 +406,14 @@ class BaseInstalledAgent(BaseAgent, ABC):
|
| 45 |
+
return instruction
|
| 46 |
+
|
| 47 |
+
@abstractmethod
|
| 48 |
+
- def install_spec(self) -> AgentInstallSpec:
|
| 49 |
+
- """Declarative install steps executed at setup and inlined into Dockerfile builds."""
|
| 50 |
+
+ def install_spec(self) -> AgentInstallSpec | None:
|
| 51 |
+
+ """Declarative install steps executed at setup and inlined into Dockerfile
|
| 52 |
+
+ builds, or ``None`` for an agent already present in the environment."""
|
| 53 |
+
|
| 54 |
+
async def install(self, environment: BaseEnvironment) -> None:
|
| 55 |
+
"""Run each step from :meth:`install_spec` with matching privilege."""
|
| 56 |
+
- for step in self.install_spec().steps:
|
| 57 |
+
+ spec = self.install_spec()
|
| 58 |
+
+ for step in spec.steps if spec is not None else ():
|
| 59 |
+
if step.user == "root":
|
| 60 |
+
await self.exec_as_root(environment, command=step.run, env=step.env)
|
| 61 |
+
else:
|
| 62 |
+
diff --git a/src/pier/agents/installed/dsh_minimal.py b/src/pier/agents/installed/dsh_minimal.py
|
| 63 |
+
new file mode 100644
|
| 64 |
+
index 0000000..c84a1b1
|
| 65 |
+
--- /dev/null
|
| 66 |
+
+++ b/src/pier/agents/installed/dsh_minimal.py
|
| 67 |
+
@@ -0,0 +1,131 @@
|
| 68 |
+
+import shlex
|
| 69 |
+
+from pathlib import Path
|
| 70 |
+
+from typing import Any, override
|
| 71 |
+
+
|
| 72 |
+
+from pier.agents.installed.base import (
|
| 73 |
+
+ BaseInstalledAgent,
|
| 74 |
+
+ with_prompt_template,
|
| 75 |
+
+ with_runtime_constraints,
|
| 76 |
+
+)
|
| 77 |
+
+from pier.agents.network import allowlist_from_urls
|
| 78 |
+
+from pier.environments.base import BaseEnvironment
|
| 79 |
+
+from pier.models.agent.context import AgentContext
|
| 80 |
+
+from pier.models.agent.name import AgentName
|
| 81 |
+
+from pier.models.agent.network import NetworkAllowlist
|
| 82 |
+
+from pier.models.trajectories import Trajectory
|
| 83 |
+
+from pier.utils.trajectory_metrics import populate_context_from_final_metrics
|
| 84 |
+
+
|
| 85 |
+
+DIST = "/opt/dsh-minimal"
|
| 86 |
+
+DEFAULT_MODEL = "deepseek-flash"
|
| 87 |
+
+DEFAULT_BASE_URL = "https://api.deepseek.com"
|
| 88 |
+
+REASONING_EFFORTS = ("low", "high", "max")
|
| 89 |
+
+
|
| 90 |
+
+
|
| 91 |
+
+class DshMinimal(BaseInstalledAgent):
|
| 92 |
+
+ """DeepSeek Harness `sdk-minimal` profile driven through its Python SDK.
|
| 93 |
+
+
|
| 94 |
+
+ The SDK and its bundled runtime executable come from a `pip install --target`
|
| 95 |
+
+ tree bind-mounted read-only at ``DIST``, so no trial installs anything.
|
| 96 |
+
+ """
|
| 97 |
+
+
|
| 98 |
+
+ SUPPORTS_ATIF = True
|
| 99 |
+
+ _TRAJECTORY = "trajectory.json"
|
| 100 |
+
+ # The distribution mount is read-only, so the runner lands beside it.
|
| 101 |
+
+ _RUNNER = "/tmp/dsh-minimal-runner.py"
|
| 102 |
+
+
|
| 103 |
+
+ def __init__(
|
| 104 |
+
+ self, *args: Any, reasoning_effort: str = "max", **kwargs: Any
|
| 105 |
+
+ ) -> None:
|
| 106 |
+
+ if reasoning_effort not in REASONING_EFFORTS:
|
| 107 |
+
+ raise ValueError(
|
| 108 |
+
+ f"reasoning_effort must be one of {REASONING_EFFORTS}, "
|
| 109 |
+
+ f"got {reasoning_effort!r}"
|
| 110 |
+
+ )
|
| 111 |
+
+ self._reasoning_effort = reasoning_effort
|
| 112 |
+
+ kwargs["model_name"] = kwargs.get("model_name") or DEFAULT_MODEL
|
| 113 |
+
+ super().__init__(*args, **kwargs)
|
| 114 |
+
+
|
| 115 |
+
+ @staticmethod
|
| 116 |
+
+ @override
|
| 117 |
+
+ def name() -> str:
|
| 118 |
+
+ return AgentName.DSH_MINIMAL.value
|
| 119 |
+
+
|
| 120 |
+
+ @override
|
| 121 |
+
+ def install_spec(self) -> None:
|
| 122 |
+
+ """The distribution is bind-mounted, so no image layer is needed."""
|
| 123 |
+
+ return None
|
| 124 |
+
+
|
| 125 |
+
+ @override
|
| 126 |
+
+ def get_version_command(self) -> str:
|
| 127 |
+
+ return f"sed -n 's/^Version: //p' {DIST}/deepseek_harness_sdk-*.dist-info/METADATA"
|
| 128 |
+
+
|
| 129 |
+
+ @override
|
| 130 |
+
+ async def setup(self, environment: BaseEnvironment) -> None:
|
| 131 |
+
+ await super().setup(environment)
|
| 132 |
+
+ runner = Path(__file__).with_name("dsh_minimal_runner.py")
|
| 133 |
+
+ await environment.upload_file(runner, self._RUNNER)
|
| 134 |
+
+ await self.exec_as_root(environment, f"chmod a+r {self._RUNNER}")
|
| 135 |
+
+
|
| 136 |
+
+ def _base_url(self) -> str:
|
| 137 |
+
+ return self._get_env("DEEPSEEK_BASE_URL") or DEFAULT_BASE_URL
|
| 138 |
+
+
|
| 139 |
+
+ @override
|
| 140 |
+
+ def network_allowlist(self) -> NetworkAllowlist:
|
| 141 |
+
+ return allowlist_from_urls([self._base_url()])
|
| 142 |
+
+
|
| 143 |
+
+ @override
|
| 144 |
+
+ def populate_context_post_run(self, context: AgentContext) -> None:
|
| 145 |
+
+ path = self.logs_dir / self._TRAJECTORY
|
| 146 |
+
+ if not path.exists():
|
| 147 |
+
+ self.logger.debug("No dsh-minimal trajectory found at %s", path)
|
| 148 |
+
+ return
|
| 149 |
+
+ try:
|
| 150 |
+
+ trajectory = Trajectory.model_validate_json(path.read_text())
|
| 151 |
+
+ except (OSError, ValueError):
|
| 152 |
+
+ self.logger.exception("Failed to parse dsh-minimal trajectory")
|
| 153 |
+
+ return
|
| 154 |
+
+ if trajectory.final_metrics is not None:
|
| 155 |
+
+ populate_context_from_final_metrics(context, trajectory.final_metrics)
|
| 156 |
+
+ context.n_agent_steps = sum(step.source == "agent" for step in trajectory.steps)
|
| 157 |
+
+
|
| 158 |
+
+ @override
|
| 159 |
+
+ @with_prompt_template
|
| 160 |
+
+ async def run(
|
| 161 |
+
+ self,
|
| 162 |
+
+ instruction: str,
|
| 163 |
+
+ environment: BaseEnvironment,
|
| 164 |
+
+ context: AgentContext,
|
| 165 |
+
+ ) -> None:
|
| 166 |
+
+ api_key = self._get_env("DEEPSEEK_API_KEY")
|
| 167 |
+
+ if not api_key:
|
| 168 |
+
+ raise ValueError("DEEPSEEK_API_KEY environment variable must be set")
|
| 169 |
+
+
|
| 170 |
+
+ instruction = with_runtime_constraints(instruction)
|
| 171 |
+
+ agent_dir = environment.env_paths.agent_dir.as_posix()
|
| 172 |
+
+ env = self.build_process_env(
|
| 173 |
+
+ {
|
| 174 |
+
+ "DEEPSEEK_API_KEY": api_key,
|
| 175 |
+
+ "DEEPSEEK_BASE_URL": self._base_url(),
|
| 176 |
+
+ # The Node runtime reaches the model only through Pier's egress
|
| 177 |
+
+ # proxy, and its fetch ignores the proxy variables without this.
|
| 178 |
+
+ "NODE_USE_ENV_PROXY": "1",
|
| 179 |
+
+ "PYTHONPATH": DIST,
|
| 180 |
+
+ "DSH_MODEL": self._parsed_model_name,
|
| 181 |
+
+ "DSH_REASONING_EFFORT": self._reasoning_effort,
|
| 182 |
+
+ # Required, and kept off the /logs bind mount: it materializes many
|
| 183 |
+
+ # files the host cannot read anyway. `trajectory.json` is the record.
|
| 184 |
+
+ "DSH_HOME": "/tmp/dsh-home",
|
| 185 |
+
+ "SESSION_ID": environment.session_id,
|
| 186 |
+
+ }
|
| 187 |
+
+ )
|
| 188 |
+
+
|
| 189 |
+
+ await self.exec_as_agent(
|
| 190 |
+
+ environment,
|
| 191 |
+
+ command=(
|
| 192 |
+
+ f"python3 {self._RUNNER} "
|
| 193 |
+
+ f"--instruction {shlex.quote(instruction)} "
|
| 194 |
+
+ f"--trajectory-path {agent_dir}/{self._TRAJECTORY} "
|
| 195 |
+
+ f"2>&1 </dev/null | stdbuf -oL tee {agent_dir}/dsh-minimal.txt"
|
| 196 |
+
+ ),
|
| 197 |
+
+ env=env,
|
| 198 |
+
+ )
|
| 199 |
+
diff --git a/src/pier/agents/installed/dsh_minimal_runner.py b/src/pier/agents/installed/dsh_minimal_runner.py
|
| 200 |
+
new file mode 100644
|
| 201 |
+
index 0000000..8b69d7d
|
| 202 |
+
--- /dev/null
|
| 203 |
+
+++ b/src/pier/agents/installed/dsh_minimal_runner.py
|
| 204 |
+
@@ -0,0 +1,307 @@
|
| 205 |
+
+"""Run the DeepSeek Harness `sdk-minimal` profile and persist its stream as ATIF v1.7."""
|
| 206 |
+
+
|
| 207 |
+
+from __future__ import annotations
|
| 208 |
+
+
|
| 209 |
+
+import argparse
|
| 210 |
+
+import json
|
| 211 |
+
+import os
|
| 212 |
+
+import sys
|
| 213 |
+
+import traceback
|
| 214 |
+
+from datetime import datetime, timezone
|
| 215 |
+
+from importlib.metadata import PackageNotFoundError, version
|
| 216 |
+
+from pathlib import Path
|
| 217 |
+
+from typing import Any
|
| 218 |
+
+
|
| 219 |
+
+from deepseek_harness import DeepSeekHarness
|
| 220 |
+
+from deepseek_harness.models import Notification
|
| 221 |
+
+
|
| 222 |
+
+PROVIDER = "deepseek-official"
|
| 223 |
+
+
|
| 224 |
+
+
|
| 225 |
+
+def _iso(epoch_ms: Any) -> str | None:
|
| 226 |
+
+ if not isinstance(epoch_ms, (int, float)):
|
| 227 |
+
+ return None
|
| 228 |
+
+ return datetime.fromtimestamp(epoch_ms / 1000, tz=timezone.utc).isoformat()
|
| 229 |
+
+
|
| 230 |
+
+
|
| 231 |
+
+def _text(blocks: Any, kind: str) -> str:
|
| 232 |
+
+ if not isinstance(blocks, list):
|
| 233 |
+
+ return ""
|
| 234 |
+
+ return "".join(
|
| 235 |
+
+ str(block.get("text") or "")
|
| 236 |
+
+ for block in blocks
|
| 237 |
+
+ if isinstance(block, dict) and block.get("type") == kind
|
| 238 |
+
+ )
|
| 239 |
+
+
|
| 240 |
+
+
|
| 241 |
+
+def _flatten(blocks: Any) -> str:
|
| 242 |
+
+ if isinstance(blocks, str):
|
| 243 |
+
+ return blocks
|
| 244 |
+
+ if not isinstance(blocks, list):
|
| 245 |
+
+ return json.dumps(blocks, ensure_ascii=False)
|
| 246 |
+
+ return "".join(
|
| 247 |
+
+ str(block.get("text") or "")
|
| 248 |
+
+ if isinstance(block, dict) and block.get("type") == "text"
|
| 249 |
+
+ else json.dumps(block, ensure_ascii=False)
|
| 250 |
+
+ for block in blocks
|
| 251 |
+
+ )
|
| 252 |
+
+
|
| 253 |
+
+
|
| 254 |
+
+def _arguments(raw: Any) -> dict[str, Any]:
|
| 255 |
+
+ """ATIF wants an object; the wire carries the model's raw argument string."""
|
| 256 |
+
+ if isinstance(raw, dict):
|
| 257 |
+
+ return raw
|
| 258 |
+
+ if isinstance(raw, str):
|
| 259 |
+
+ try:
|
| 260 |
+
+ parsed = json.loads(raw)
|
| 261 |
+
+ except json.JSONDecodeError:
|
| 262 |
+
+ return {"_unparsed_arguments": raw}
|
| 263 |
+
+ return parsed if isinstance(parsed, dict) else {"_arguments": parsed}
|
| 264 |
+
+ return {}
|
| 265 |
+
+
|
| 266 |
+
+
|
| 267 |
+
+def _metrics(usage: dict[str, Any]) -> dict[str, Any]:
|
| 268 |
+
+ """The adapter reports disjoint counts, so the cache legs rejoin prompt tokens."""
|
| 269 |
+
+ cache_read = usage.get("cacheReadTokens") or 0
|
| 270 |
+
+ cache_write = usage.get("cacheWriteTokens") or 0
|
| 271 |
+
+ metrics = {
|
| 272 |
+
+ "prompt_tokens": (usage.get("inputTokens") or 0) + cache_read + cache_write,
|
| 273 |
+
+ "completion_tokens": usage.get("outputTokens") or 0,
|
| 274 |
+
+ "cached_tokens": cache_read,
|
| 275 |
+
+ }
|
| 276 |
+
+ if usage.get("reasoningTokens") is not None:
|
| 277 |
+
+ metrics["extra"] = {"reasoning_tokens": usage["reasoningTokens"]}
|
| 278 |
+
+ return metrics
|
| 279 |
+
+
|
| 280 |
+
+
|
| 281 |
+
+class Collector:
|
| 282 |
+
+ """Fold root-session events into one ATIF step per model call."""
|
| 283 |
+
+
|
| 284 |
+
+ def __init__(self, instruction: str, model: str, effort: str, session: str) -> None:
|
| 285 |
+
+ self.model = model
|
| 286 |
+
+ self.effort = effort
|
| 287 |
+
+ self.session = session
|
| 288 |
+
+ self.steps: list[dict[str, Any]] = [
|
| 289 |
+
+ {
|
| 290 |
+
+ "step_id": 1,
|
| 291 |
+
+ "timestamp": datetime.now(tz=timezone.utc).isoformat(),
|
| 292 |
+
+ "source": "user",
|
| 293 |
+
+ "message": instruction,
|
| 294 |
+
+ }
|
| 295 |
+
+ ]
|
| 296 |
+
+ # (turn, step) identifies one model call; callId maps a later `tool/result`
|
| 297 |
+
+ # back to the step that issued it, which is never the step it arrives in.
|
| 298 |
+
+ self._by_call: dict[tuple[int, int], dict[str, Any]] = {}
|
| 299 |
+
+ self._by_call_id: dict[str, dict[str, Any]] = {}
|
| 300 |
+
+ self.turn_end_reasons: list[dict[str, Any]] = []
|
| 301 |
+
+ self.llm_retries = 0
|
| 302 |
+
+
|
| 303 |
+
+ def _step(self, turn: Any, step: Any, timestamp: str | None) -> dict[str, Any]:
|
| 304 |
+
+ key = (int(turn or 0), int(step or 0))
|
| 305 |
+
+ if key not in self._by_call:
|
| 306 |
+
+ created = {
|
| 307 |
+
+ "step_id": len(self.steps) + 1,
|
| 308 |
+
+ "timestamp": timestamp,
|
| 309 |
+
+ "source": "agent",
|
| 310 |
+
+ "message": "",
|
| 311 |
+
+ "model_name": self.model,
|
| 312 |
+
+ "reasoning_effort": self.effort,
|
| 313 |
+
+ "llm_call_count": 1,
|
| 314 |
+
+ }
|
| 315 |
+
+ self.steps.append(created)
|
| 316 |
+
+ self._by_call[key] = created
|
| 317 |
+
+ return self._by_call[key]
|
| 318 |
+
+
|
| 319 |
+
+ def _tool_call(self, step: dict[str, Any], call_id: Any, name: Any, raw: Any) -> None:
|
| 320 |
+
+ if not isinstance(call_id, str) or not call_id:
|
| 321 |
+
+ return
|
| 322 |
+
+ calls = step.setdefault("tool_calls", [])
|
| 323 |
+
+ entry = {
|
| 324 |
+
+ "tool_call_id": call_id,
|
| 325 |
+
+ "function_name": str(name),
|
| 326 |
+
+ "arguments": _arguments(raw),
|
| 327 |
+
+ }
|
| 328 |
+
+ for index, existing in enumerate(calls):
|
| 329 |
+
+ if existing["tool_call_id"] == call_id:
|
| 330 |
+
+ calls[index] = entry
|
| 331 |
+
+ break
|
| 332 |
+
+ else:
|
| 333 |
+
+ calls.append(entry)
|
| 334 |
+
+ self._by_call_id[call_id] = step
|
| 335 |
+
+
|
| 336 |
+
+ def record(self, event: dict[str, Any]) -> None:
|
| 337 |
+
+ kind = event.get("type")
|
| 338 |
+
+ data = event.get("data") if isinstance(event.get("data"), dict) else {}
|
| 339 |
+
+ timestamp = _iso(event.get("time"))
|
| 340 |
+
+
|
| 341 |
+
+ if kind == "assistant/message":
|
| 342 |
+
+ step = self._step(data.get("turn"), data.get("step"), timestamp)
|
| 343 |
+
+ message = data.get("message") if isinstance(data.get("message"), dict) else {}
|
| 344 |
+
+ content = message.get("content")
|
| 345 |
+
+ step["message"] = _text(content, "text")
|
| 346 |
+
+ if reasoning := _text(content, "reasoning"):
|
| 347 |
+
+ step["reasoning_content"] = reasoning
|
| 348 |
+
+ source = message.get("source")
|
| 349 |
+
+ if isinstance(source, dict) and isinstance(source.get("model"), str):
|
| 350 |
+
+ step["model_name"] = source["model"]
|
| 351 |
+
+ if isinstance(data.get("usage"), dict):
|
| 352 |
+
+ step["metrics"] = _metrics(data["usage"])
|
| 353 |
+
+ if data.get("interrupted"):
|
| 354 |
+
+ step.setdefault("extra", {})["interrupted"] = True
|
| 355 |
+
+ for block in content if isinstance(content, list) else []:
|
| 356 |
+
+ if isinstance(block, dict) and block.get("type") == "tool-call":
|
| 357 |
+
+ self._tool_call(
|
| 358 |
+
+ step, block.get("id"), block.get("name"), block.get("arguments")
|
| 359 |
+
+ )
|
| 360 |
+
+ elif kind == "tool/call":
|
| 361 |
+
+ step = self._step(data.get("turn"), data.get("step"), timestamp)
|
| 362 |
+
+ self._tool_call(
|
| 363 |
+
+ step, data.get("callId"), data.get("name"), data.get("arguments")
|
| 364 |
+
+ )
|
| 365 |
+
+ elif kind == "tool/result":
|
| 366 |
+
+ self._result(data)
|
| 367 |
+
+ elif kind == "turn/end":
|
| 368 |
+
+ reason = data.get("reason")
|
| 369 |
+
+ self.turn_end_reasons.append(reason if isinstance(reason, dict) else {})
|
| 370 |
+
+ elif kind == "llm/retry":
|
| 371 |
+
+ self.llm_retries += 1
|
| 372 |
+
+
|
| 373 |
+
+ def _result(self, data: dict[str, Any]) -> None:
|
| 374 |
+
+ message = data.get("message") if isinstance(data.get("message"), dict) else {}
|
| 375 |
+
+ for block in message.get("content") or []:
|
| 376 |
+
+ if not isinstance(block, dict) or block.get("type") != "tool-result":
|
| 377 |
+
+ continue
|
| 378 |
+
+ step = self._by_call_id.get(block.get("toolCallId"))
|
| 379 |
+
+ if step is None:
|
| 380 |
+
+ continue
|
| 381 |
+
+ entry: dict[str, Any] = {
|
| 382 |
+
+ "source_call_id": block["toolCallId"],
|
| 383 |
+
+ "content": _flatten(block.get("content")),
|
| 384 |
+
+ }
|
| 385 |
+
+ if block.get("isError"):
|
| 386 |
+
+ entry["extra"] = {"is_error": True, "error": data.get("error")}
|
| 387 |
+
+ results = step.setdefault("observation", {"results": []})["results"]
|
| 388 |
+
+ for index, existing in enumerate(results):
|
| 389 |
+
+ if existing["source_call_id"] == entry["source_call_id"]:
|
| 390 |
+
+ results[index] = entry
|
| 391 |
+
+ break
|
| 392 |
+
+ else:
|
| 393 |
+
+ results.append(entry)
|
| 394 |
+
+
|
| 395 |
+
+ def trajectory(self, finish_reason: str | None, failure: str | None) -> dict[str, Any]:
|
| 396 |
+
+ prompt = completion = cached = peak = 0
|
| 397 |
+
+ for step in self.steps:
|
| 398 |
+
+ metrics = step.get("metrics")
|
| 399 |
+
+ if not isinstance(metrics, dict):
|
| 400 |
+
+ continue
|
| 401 |
+
+ prompt += metrics["prompt_tokens"]
|
| 402 |
+
+ completion += metrics["completion_tokens"]
|
| 403 |
+
+ cached += metrics["cached_tokens"]
|
| 404 |
+
+ peak = max(peak, metrics["prompt_tokens"])
|
| 405 |
+
+ extra = {
|
| 406 |
+
+ "finish_reason": finish_reason,
|
| 407 |
+
+ "turn_end_reasons": self.turn_end_reasons,
|
| 408 |
+
+ "llm_retry_count": self.llm_retries,
|
| 409 |
+
+ }
|
| 410 |
+
+ if failure is not None:
|
| 411 |
+
+ extra["failure"] = failure
|
| 412 |
+
+ return {
|
| 413 |
+
+ "schema_version": "ATIF-v1.7",
|
| 414 |
+
+ "session_id": self.session,
|
| 415 |
+
+ "agent": {"name": "dsh-minimal", "version": _version(), "model_name": self.model},
|
| 416 |
+
+ "steps": self.steps,
|
| 417 |
+
+ "final_metrics": {
|
| 418 |
+
+ "total_prompt_tokens": prompt,
|
| 419 |
+
+ "total_completion_tokens": completion,
|
| 420 |
+
+ "total_cached_tokens": cached,
|
| 421 |
+
+ "total_steps": len(self.steps),
|
| 422 |
+
+ # The sdk-minimal profile has no compaction, so no summary step
|
| 423 |
+
+ # can replace a transcript prefix.
|
| 424 |
+
+ "extra": {"peak_context_tokens": peak, "summarization_count": 0},
|
| 425 |
+
+ },
|
| 426 |
+
+ "extra": extra,
|
| 427 |
+
+ }
|
| 428 |
+
+
|
| 429 |
+
+
|
| 430 |
+
+def _version() -> str:
|
| 431 |
+
+ try:
|
| 432 |
+
+ return version("deepseek-harness-sdk")
|
| 433 |
+
+ except PackageNotFoundError:
|
| 434 |
+
+ return "unknown"
|
| 435 |
+
+
|
| 436 |
+
+
|
| 437 |
+
+def _write(path: Path, value: Any) -> None:
|
| 438 |
+
+ path.parent.mkdir(parents=True, exist_ok=True)
|
| 439 |
+
+ temporary = path.with_suffix(f"{path.suffix}.tmp")
|
| 440 |
+
+ temporary.write_text(json.dumps(value, indent=2, ensure_ascii=False) + "\n")
|
| 441 |
+
+ temporary.replace(path)
|
| 442 |
+
+
|
| 443 |
+
+
|
| 444 |
+
+def main() -> None:
|
| 445 |
+
+ parser = argparse.ArgumentParser()
|
| 446 |
+
+ parser.add_argument("--instruction", required=True)
|
| 447 |
+
+ parser.add_argument("--trajectory-path", required=True)
|
| 448 |
+
+ args = parser.parse_args()
|
| 449 |
+
+
|
| 450 |
+
+ # Task images can leave the agent user on a 077 umask; the session log and
|
| 451 |
+
+ # trajectory have to stay readable to the host user that collects /logs.
|
| 452 |
+
+ os.umask(0o022)
|
| 453 |
+
+
|
| 454 |
+
+ model = os.environ["DSH_MODEL"]
|
| 455 |
+
+ effort = os.environ.get("DSH_REASONING_EFFORT", "max")
|
| 456 |
+
+ session = os.environ.get("SESSION_ID") or "pier-session"
|
| 457 |
+
+ path = Path(args.trajectory_path)
|
| 458 |
+
+ collector = Collector(args.instruction, model, effort, session)
|
| 459 |
+
+ state: dict[str, Any] = {"finish_reason": None, "failure": None}
|
| 460 |
+
+
|
| 461 |
+
+ def checkpoint() -> None:
|
| 462 |
+
+ _write(path, collector.trajectory(state["finish_reason"], state["failure"]))
|
| 463 |
+
+
|
| 464 |
+
+ def on_notification(notification: Notification) -> None:
|
| 465 |
+
+ payload = notification.payload
|
| 466 |
+
+ event = payload.get("event") if isinstance(payload, dict) else None
|
| 467 |
+
+ if (
|
| 468 |
+
+ notification.method != "session.event"
|
| 469 |
+
+ or not isinstance(event, dict)
|
| 470 |
+
+ # Token deltas repeat what their `assistant/message` already carries.
|
| 471 |
+
+ or event.get("type") == "assistant/chunk"
|
| 472 |
+
+ or payload.get("sessionId") != session
|
| 473 |
+
+ ):
|
| 474 |
+
+ return
|
| 475 |
+
+ collector.record(event)
|
| 476 |
+
+ checkpoint()
|
| 477 |
+
+
|
| 478 |
+
+ checkpoint()
|
| 479 |
+
+ print(f"dsh-minimal: model={model} effort={effort} session={session}")
|
| 480 |
+
+ try:
|
| 481 |
+
+ with DeepSeekHarness(
|
| 482 |
+
+ profile="sdk-minimal",
|
| 483 |
+
+ provider=PROVIDER,
|
| 484 |
+
+ model=model,
|
| 485 |
+
+ reasoning_effort=effort,
|
| 486 |
+
+ cwd=os.getcwd(),
|
| 487 |
+
+ dsh_home=os.environ["DSH_HOME"],
|
| 488 |
+
+ ) as harness:
|
| 489 |
+
+ result = harness.run(
|
| 490 |
+
+ args.instruction, session_id=session, on_notification=on_notification
|
| 491 |
+
+ )
|
| 492 |
+
+ state["finish_reason"] = result.finish_reason
|
| 493 |
+
+ print(f"dsh-minimal: finish_reason={result.finish_reason}")
|
| 494 |
+
+ print(f"dsh-minimal: final_response={result.final_response}")
|
| 495 |
+
+ except Exception as error: # noqa: BLE001 - the trajectory must survive any failure
|
| 496 |
+
+ state["failure"] = f"{type(error).__name__}: {error}"
|
| 497 |
+
+ traceback.print_exc()
|
| 498 |
+
+ finally:
|
| 499 |
+
+ checkpoint()
|
| 500 |
+
+
|
| 501 |
+
+ # `max-tokens` is a bounded model outcome the verifier can still grade; anything
|
| 502 |
+
+ # else non-terminal means the harness never produced a usable turn.
|
| 503 |
+
+ if state["failure"] is not None or state["finish_reason"] not in (
|
| 504 |
+
+ "completed",
|
| 505 |
+
+ "max-tokens",
|
| 506 |
+
+ ):
|
| 507 |
+
+ sys.exit(1)
|
| 508 |
+
+
|
| 509 |
+
+
|
| 510 |
+
+if __name__ == "__main__":
|
| 511 |
+
+ main()
|
| 512 |
+
diff --git a/src/pier/agents/installed/mini_swe_agent.py b/src/pier/agents/installed/mini_swe_agent.py
|
| 513 |
+
index 75d25b6..d283f66 100644
|
| 514 |
+
--- a/src/pier/agents/installed/mini_swe_agent.py
|
| 515 |
+
+++ b/src/pier/agents/installed/mini_swe_agent.py
|
| 516 |
+
@@ -11,6 +11,7 @@ from pier.agents.installed.base import (
|
| 517 |
+
BaseInstalledAgent,
|
| 518 |
+
CliFlag,
|
| 519 |
+
with_prompt_template,
|
| 520 |
+
+ with_runtime_constraints,
|
| 521 |
+
)
|
| 522 |
+
from pier.agents.network import allowlist_from_urls, collect_url_values
|
| 523 |
+
from pier.agents.utils import get_api_key_var_names_from_model_name
|
| 524 |
+
@@ -821,7 +822,7 @@ mini-swe-agent --help
|
| 525 |
+
async def run(
|
| 526 |
+
self, instruction: str, environment: BaseEnvironment, context: AgentContext
|
| 527 |
+
) -> None:
|
| 528 |
+
- augmented_instruction = instruction
|
| 529 |
+
+ augmented_instruction = with_runtime_constraints(instruction)
|
| 530 |
+
if self.mcp_servers:
|
| 531 |
+
mcp_info = "\n\nMCP Servers:\nThe following MCP servers are available for this task.\n"
|
| 532 |
+
for s in self.mcp_servers:
|
| 533 |
+
diff --git a/src/pier/environments/docker/__init__.py b/src/pier/environments/docker/__init__.py
|
| 534 |
+
index 4bb8f11..04a8c6e 100644
|
| 535 |
+
--- a/src/pier/environments/docker/__init__.py
|
| 536 |
+
+++ b/src/pier/environments/docker/__init__.py
|
| 537 |
+
@@ -25,6 +25,9 @@ def write_resources_compose_file(
|
| 538 |
+
*,
|
| 539 |
+
cpu_request: int | None = None,
|
| 540 |
+
cpu_limit: int | None = None,
|
| 541 |
+
+ environment: dict[str, str] | None = None,
|
| 542 |
+
+ sysctls: dict[str, str] | None = None,
|
| 543 |
+
+ volumes: list[str] | None = None,
|
| 544 |
+
memory_request_mb: int | None = None,
|
| 545 |
+
memory_limit_mb: int | None = None,
|
| 546 |
+
) -> Path:
|
| 547 |
+
@@ -46,7 +49,13 @@ def write_resources_compose_file(
|
| 548 |
+
if reservations:
|
| 549 |
+
resources["reservations"] = reservations
|
| 550 |
+
|
| 551 |
+
- main = {"deploy": {"resources": resources}} if resources else {}
|
| 552 |
+
+ main: dict[str, object] = {"deploy": {"resources": resources}} if resources else {}
|
| 553 |
+
+ if environment:
|
| 554 |
+
+ main["environment"] = dict(environment)
|
| 555 |
+
+ if sysctls:
|
| 556 |
+
+ main["sysctls"] = dict(sysctls)
|
| 557 |
+
+ if volumes:
|
| 558 |
+
+ main["volumes"] = list(volumes)
|
| 559 |
+
compose = {"services": {"main": main}}
|
| 560 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 561 |
+
path.write_text(json.dumps(compose, indent=2))
|
| 562 |
+
diff --git a/src/pier/environments/docker/docker.py b/src/pier/environments/docker/docker.py
|
| 563 |
+
index 2ed4e92..bc671be 100644
|
| 564 |
+
--- a/src/pier/environments/docker/docker.py
|
| 565 |
+
+++ b/src/pier/environments/docker/docker.py
|
| 566 |
+
@@ -31,6 +31,11 @@ from pier.environments.docker import (
|
| 567 |
+
write_mounts_compose_file,
|
| 568 |
+
write_resources_compose_file,
|
| 569 |
+
)
|
| 570 |
+
+from pier.environments.docker.parallelism import (
|
| 571 |
+
+ CPU_CLAMP_PATH,
|
| 572 |
+
+ cpu_clamp_source,
|
| 573 |
+
+ parallelism_env,
|
| 574 |
+
+)
|
| 575 |
+
from pier.models.environment_type import EnvironmentType
|
| 576 |
+
from pier.models.task.config import EnvironmentConfig, TaskOS
|
| 577 |
+
from pier.models.trial.config import ResourceMode, ServiceVolumeConfig
|
| 578 |
+
@@ -102,6 +107,11 @@ class DockerEnvironment(BaseEnvironment):
|
| 579 |
+
|
| 580 |
+
_DOCKER_COMPOSE_WINDOWS_KEEPALIVE_PATH = COMPOSE_WINDOWS_KEEPALIVE_PATH
|
| 581 |
+
|
| 582 |
+
+ # Docker leaves IPv6 off in the container netns, so loopback has no ::1 and
|
| 583 |
+
+ # suites that bind it skip where a real Linux host passes. These environments
|
| 584 |
+
+ # always get their own netns, which is what net.* sysctls require.
|
| 585 |
+
+ _LINUX_SYSCTLS = {"net.ipv6.conf.all.disable_ipv6": "0"}
|
| 586 |
+
+
|
| 587 |
+
# Class-level lock per image name to prevent parallel builds of the same image.
|
| 588 |
+
_image_build_locks: dict[str, asyncio.Lock] = {}
|
| 589 |
+
|
| 590 |
+
@@ -160,6 +170,7 @@ class DockerEnvironment(BaseEnvironment):
|
| 591 |
+
task_env_config: EnvironmentConfig,
|
| 592 |
+
keep_containers: bool = False,
|
| 593 |
+
mounts_json: list[ServiceVolumeConfig] | None = None,
|
| 594 |
+
+ mounts_override: list[ServiceVolumeConfig] | None = None,
|
| 595 |
+
*args,
|
| 596 |
+
**kwargs,
|
| 597 |
+
):
|
| 598 |
+
@@ -193,8 +204,13 @@ class DockerEnvironment(BaseEnvironment):
|
| 599 |
+
self._windows_container_name: str | None = None
|
| 600 |
+
self._platform = UnixOps(self)
|
| 601 |
+
|
| 602 |
+
+ # Configured mounts are additive so the default /logs binds survive and
|
| 603 |
+
+ # agent logs and artifacts stay host-visible. A separate verifier
|
| 604 |
+
+ # environment must not share those directories, so it overrides the set.
|
| 605 |
+
self._mounts_json = (
|
| 606 |
+
- mounts_json if mounts_json is not None else self._default_log_mounts()
|
| 607 |
+
+ [*self._default_log_mounts(), *(mounts_json or [])]
|
| 608 |
+
+ if mounts_override is None
|
| 609 |
+
+ else mounts_override
|
| 610 |
+
)
|
| 611 |
+
self._mounts_compose_path: Path | None = None
|
| 612 |
+
self._resources_compose_temp_dir: tempfile.TemporaryDirectory | None = None
|
| 613 |
+
@@ -427,12 +443,16 @@ class DockerEnvironment(BaseEnvironment):
|
| 614 |
+
Path(self._resources_compose_temp_dir.name)
|
| 615 |
+
/ f"{self.session_id}-{RESOURCES_COMPOSE_NAME}"
|
| 616 |
+
)
|
| 617 |
+
+ cpu_limit = self._resource_limit_value("cpu", auto_mode=ResourceMode.LIMIT)
|
| 618 |
+
return write_resources_compose_file(
|
| 619 |
+
path,
|
| 620 |
+
+ environment=parallelism_env(cpu_limit),
|
| 621 |
+
+ volumes=self._write_cpu_clamp(path.parent, cpu_limit),
|
| 622 |
+
+ sysctls=None if self._is_windows_container else self._LINUX_SYSCTLS,
|
| 623 |
+
cpu_request=self._resource_request_value(
|
| 624 |
+
"cpu", auto_mode=ResourceMode.LIMIT
|
| 625 |
+
),
|
| 626 |
+
- cpu_limit=self._resource_limit_value("cpu", auto_mode=ResourceMode.LIMIT),
|
| 627 |
+
+ cpu_limit=cpu_limit,
|
| 628 |
+
memory_request_mb=self._resource_request_value(
|
| 629 |
+
"memory", auto_mode=ResourceMode.LIMIT
|
| 630 |
+
),
|
| 631 |
+
@@ -441,6 +461,15 @@ class DockerEnvironment(BaseEnvironment):
|
| 632 |
+
),
|
| 633 |
+
)
|
| 634 |
+
|
| 635 |
+
+ def _write_cpu_clamp(self, directory: Path, cpu_limit: int | None) -> list[str]:
|
| 636 |
+
+ """Mount the preload that `parallelism_env` points NODE_OPTIONS at."""
|
| 637 |
+
+ if not cpu_limit or self._is_windows_container:
|
| 638 |
+
+ return []
|
| 639 |
+
+ source = directory / "pier-node-cpu-clamp.js"
|
| 640 |
+
+ source.write_text(cpu_clamp_source(cpu_limit))
|
| 641 |
+
+ source.chmod(0o644)
|
| 642 |
+
+ return [f"{source}:{CPU_CLAMP_PATH}:ro"]
|
| 643 |
+
+
|
| 644 |
+
def _cleanup_resources_compose_file(self) -> None:
|
| 645 |
+
if self._resources_compose_temp_dir is None:
|
| 646 |
+
return
|
| 647 |
+
diff --git a/src/pier/environments/docker/parallelism.py b/src/pier/environments/docker/parallelism.py
|
| 648 |
+
new file mode 100644
|
| 649 |
+
index 0000000..b21ae7c
|
| 650 |
+
--- /dev/null
|
| 651 |
+
+++ b/src/pier/environments/docker/parallelism.py
|
| 652 |
+
@@ -0,0 +1,34 @@
|
| 653 |
+
+"""Tell test runners how many CPUs the container actually gets.
|
| 654 |
+
+
|
| 655 |
+
+Docker's ``--cpus`` is a bandwidth quota, so ``nproc`` still reports every host
|
| 656 |
+
+core and runner worker pools oversubscribe the container.
|
| 657 |
+
+"""
|
| 658 |
+
+
|
| 659 |
+
+from __future__ import annotations
|
| 660 |
+
+
|
| 661 |
+
+CPU_CLAMP_PATH = "/opt/pier-node-cpu-clamp.js"
|
| 662 |
+
+
|
| 663 |
+
+
|
| 664 |
+
+def cpu_clamp_source(cpus: int) -> str:
|
| 665 |
+
+ """A ``node --require`` preload; the Node runners take no cap from the environment."""
|
| 666 |
+
+ return (
|
| 667 |
+
+ "const os = require('node:os')\n"
|
| 668 |
+
+ f"const limit = {cpus}\n"
|
| 669 |
+
+ "const cpus = os.cpus\n"
|
| 670 |
+
+ "os.cpus = () => cpus.call(os).slice(0, limit)\n"
|
| 671 |
+
+ "os.availableParallelism = () => limit\n"
|
| 672 |
+
+ )
|
| 673 |
+
+
|
| 674 |
+
+
|
| 675 |
+
+def parallelism_env(cpus: int | None) -> dict[str, str]:
|
| 676 |
+
+ """Worker-count caps for the test runners, derived from the CPU limit."""
|
| 677 |
+
+ if not cpus or cpus < 1:
|
| 678 |
+
+ return {}
|
| 679 |
+
+ n = str(cpus)
|
| 680 |
+
+ return {
|
| 681 |
+
+ "GOMAXPROCS": n,
|
| 682 |
+
+ "CARGO_BUILD_JOBS": n,
|
| 683 |
+
+ "NEXTEST_TEST_THREADS": n,
|
| 684 |
+
+ "PYTEST_XDIST_AUTO_NUM_WORKERS": n,
|
| 685 |
+
+ "NODE_OPTIONS": f"--require {CPU_CLAMP_PATH}",
|
| 686 |
+
+ }
|
| 687 |
+
diff --git a/src/pier/models/agent/name.py b/src/pier/models/agent/name.py
|
| 688 |
+
index d84516a..41ac147 100644
|
| 689 |
+
--- a/src/pier/models/agent/name.py
|
| 690 |
+
+++ b/src/pier/models/agent/name.py
|
| 691 |
+
@@ -8,6 +8,7 @@ class AgentName(str, Enum):
|
| 692 |
+
ANTIGRAVITY_SDK = "antigravity-sdk"
|
| 693 |
+
CODEX = "codex"
|
| 694 |
+
CURSOR_CLI = "cursor-cli"
|
| 695 |
+
+ DSH_MINIMAL = "dsh-minimal"
|
| 696 |
+
GEMINI_CLI = "gemini-cli"
|
| 697 |
+
MINI_SWE_AGENT = "mini-swe-agent"
|
| 698 |
+
SWE_AGENT = "swe-agent"
|
| 699 |
+
diff --git a/src/pier/trial/trial.py b/src/pier/trial/trial.py
|
| 700 |
+
index fd5d4a2..126f0fa 100644
|
| 701 |
+
--- a/src/pier/trial/trial.py
|
| 702 |
+
+++ b/src/pier/trial/trial.py
|
| 703 |
+
@@ -402,7 +402,7 @@ class Trial:
|
| 704 |
+
trial_paths=self._trial_paths,
|
| 705 |
+
task_env_config=env_config,
|
| 706 |
+
logger=self._logger,
|
| 707 |
+
- mounts_json=self._verifier_env_mounts(env_config),
|
| 708 |
+
+ mounts_override=self._verifier_env_mounts(env_config),
|
| 709 |
+
agent_install_spec=None,
|
| 710 |
+
network_allowlist=None,
|
| 711 |
+
default_user=(
|