Instructions to use ProCreations/grug-27b-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProCreations/grug-27b-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ProCreations/grug-27b-v2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProCreations/grug-27b-v2")
model = AutoModelForMultimodalLM.from_pretrained("ProCreations/grug-27b-v2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProCreations/grug-27b-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProCreations/grug-27b-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ProCreations/grug-27b-v2

SGLang

How to use ProCreations/grug-27b-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProCreations/grug-27b-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProCreations/grug-27b-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ProCreations/grug-27b-v2 with Docker Model Runner:
```
docker model run hf.co/ProCreations/grug-27b-v2
```

ProCreations commited on 4 days ago

Commit

9543af6

verified ·

1 Parent(s): f572b85

Rewrite model card in Grug style; preserve technical content

Browse files

Files changed (1) hide show

README.md +45 -22

README.md CHANGED Viewed

@@ -15,15 +15,17 @@ tags:
 - qwen3_5
 ---
-# Grug 27B v2
-Grug 27B v2 is a fresh post-training run from [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B) for compact caveman-style reasoning, coding, and tools. It includes a **Grug-tuned native MTP head in this checkpoint**. No separate draft model is needed. [GGUF downloads](https://huggingface.co/ProCreations/grug-27b-v2-gguf) also include the head in every text quantization.
-The release improves measured coding and tool results over Grug v1.1 and fixes its session-title regression on the tested suite. It does **not** establish that higher effort always improves accuracy, that reasoning stays perfectly Grug-style on every hard case, or that all loops are eliminated. Read the effort tables and remaining limitations below.
-## Measured quality
-Matched H200/vLLM runs use temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, seed 42, and the same task subsets and output caps. These are executable-test and official AST-checker measurements on fixed subsets, not full leaderboard scores.
 | Model | Medium HumanEval (132) | Medium MBPP (225) | Medium BFCL subset (200) |
 |---|---:|---:|---:|
@@ -31,29 +33,46 @@ Matched H200/vLLM runs use temperature 0.6, top-p 0.95, top-k 20, repetition pen
 | Grug v1.1 | 91.7% | 80.4% | 81.0% |
 | Grug v2 | 97.0% | 90.7% | 88.0% |
-On the 12-issue SWE-bench Verified Django/SymPy subset, medium effort with the original limits resolved **10/12** for v2 and **8/12** for v1.1. In the separate matched xhigh run with retained reasoning, 65,536-token context, and 16,384 output tokens per tool turn, v2 resolved **11/12**, v1.1 **7/12**, and Qwen **10/12**. The harness uses a shared pinned environment, not the official per-instance Docker images. A single case changes the score by 8.3 points; these small runs do not establish broad statistical superiority.
-V2 completed **90/90 session-title checks** across low, medium, and xhigh without tool-call leaks. Real vLLM HTTP tests passed **24/24**, including titles with tools present, streamed calls, and thinking enabled/disabled. These tests address [the reported title problem](https://huggingface.co/ProCreations/grug-v1.1-qwen-3.8-27b/discussions/2); the OpenCode desktop application itself was not tested.
-With reasoning retained in tool history and the original per-effort limits, v2 resolved **9 / 10 / 11 of 12** at low/medium/xhigh, compared with v1.1's **8 / 9 / 7**. This is a positive effort-scaling result on the small agentic subset, not a guarantee across tasks.
-Compared with foundation Qwen, compactness still costs some core accuracy. For example, medium MATH on the 150-case subset is **84.7%** for v2 versus **95.3%** for Qwen. Full comparisons, original and larger output budgets, MTP accuracy, quantized smoke tests, and repetition-penalty-1.0 reference runs are in [results.md](results.md). The [evaluation protocol](evaluation_protocol.md) and [machine-readable results](https://huggingface.co/ProCreations/grug-27b-v2/tree/main/results) describe scoring corrections, completion rates, and limitations.
-## Reasoning effort and Grug style
-Use **low**, **medium**, or **xhigh** through `chat_template_kwargs.reasoning_effort`. Medium is the default. All three retain the native effort instructions and are trained for terse Grug reasoning; final answers and code can use ordinary language.
-Mean HumanEval reasoning lengths were **192 / 266 / 656 tokens** for v2 at low/medium/xhigh, versus **483 / 617 / 1214** for Qwen. More effort therefore spends more reasoning on this coding subset while remaining more compact than the foundation model.
-Accuracy does not rise monotonically: several coding and math scores decline at xhigh, even when its output limit is doubled. Long traces sometimes drift into ordinary English. A real unfinished loop remains in the low-effort MATH audit, so the [reported looping issue](https://huggingface.co/ProCreations/grug-v1.1-qwen-3.8-27b/discussions/5) is mitigated, not completely solved. The saved template also normalizes historical reasoning once; clients still need the correct reasoning/tool parsers.
-## Integrated MTP
-All 15 native draft-head tensors are indexed in `model-mtp.safetensors` inside this repository. Only the head was trained during its tuning stage; the main model stayed frozen. Held-out draft/verifier agreement improved from **90.44% to 90.90%**, with distillation loss improving from **0.3502 to 0.2965**.
-On 18 serial H200 prompts after warmup, measured output throughput was **68.02 tokens/sec without MTP**, **112.77 with one draft token**, and **142.44 with two**. Actual draft-token acceptance was **86.87%** and **77.20%**. These are workload-specific results. MTP-off/on greedy token sequences matched on 10/18 prompts; functional coding/tool accuracy is measured separately in the results.
-Enable the packaged head with the tested vLLM 0.29.0 options:
 ```bash
 vllm serve ProCreations/grug-27b-v2 \
@@ -64,12 +83,16 @@ vllm serve ProCreations/grug-27b-v2 \
   --speculative-config '{"method":"mtp","num_speculative_tokens":2}'
 ```
-Generation defaults are **temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05**. Set them explicitly when clients override model defaults. See [usage.md](usage.md) for requests, effort controls, history normalization, and llama.cpp commands.
-## Build and provenance
-The main LoRA run used 3,466 training examples, task-group-disjoint validation, and verified coding/math/repair material plus inherited tool replay and new title examples. The best validation checkpoint was merged into the foundation model. The vision encoder is preserved; this release does not establish vision quality or full configured-context performance.
-All model generation, training, merging, MTP tuning, quantization, and runtime validation ran on **HF Jobs**. The local GPU was not used. The conservative estimate for this build, including failed and canceled attempts, is **$120.92**, below the authorized $170 cap; this is not an invoice. See [training.md](training.md), [release provenance](release_provenance.json), [HF Jobs cost ledger](hf_jobs_cost.json), and [SHA256SUMS](SHA256SUMS).
-Apache-2.0 license, following the foundation checkpoint. New benchmark test cases were excluded from the training mixture; inherited replay and foundation pretraining may still contain unrecognized overlap. Training receipts and evaluation source are included; raw inherited replay remains private.

 - qwen3_5
 ---
+# grug 27b v2
+**grug think small. code get stronger. draft head live in same rock.**
+grug start fresh from [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B). train for compact cave-think, coding, and tool use. **Grug-tuned native MTP head already inside this checkpoint.** no separate draft model to drag back to cave. every text rock in [GGUF downloads](https://huggingface.co/ProCreations/grug-27b-v2-gguf) carry head too.
+v2 score better than grug v1.1 on measured code and tool hunts. session-title bug pass tested checks now. but grug not claim every bug dead: more effort not always more correct, hard thought sometimes leave cave-talk, and some loops still live. grug show gain and bruise below.
+## grug hunt code and tools
+same H200/vLLM hunt rules for every model: temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, seed 42. same task subsets. same output caps. code run against executable tests; tool calls face official AST checker. **fixed subsets, not full leaderboard scores.**
 | Model | Medium HumanEval (132) | Medium MBPP (225) | Medium BFCL subset (200) |
 |---|---:|---:|---:|
 | Grug v1.1 | 91.7% | 80.4% | 81.0% |
 | Grug v2 | 97.0% | 90.7% | 88.0% |
+grug also hunt real repo bugs: 12-issue SWE-bench Verified Django/SymPy subset. medium effort with original limits fix **10/12** for v2, **8/12** for v1.1. separate matched xhigh hunt keep reasoning in history, use 65,536-token context and 16,384 output tokens per tool turn: v2 fix **11/12**, v1.1 **7/12**, Qwen **10/12**.
+this harness use shared pinned environment, not official per-instance Docker images. one bug swing score by 8.3 points. small hunt useful; **not proof grug win everywhere.**
+session title should be title, not surprise tool call. v2 pass **90/90 session-title checks** across low, medium, xhigh. zero tool-call leaks in those checks. real vLLM HTTP tests pass **24/24**, including titles with tools present, streamed calls, thinking on and off. checks target [reported title problem](https://huggingface.co/ProCreations/grug-v1.1-qwen-3.8-27b/discussions/2). grug did not test OpenCode desktop app itself.
+keep reasoning in tool history and use original per-effort limits: v2 fix **9 / 10 / 11 of 12** at low/medium/xhigh. v1.1 fix **8 / 9 / 7**. more effort help grug on this small agent hunt. not promise for every task.
+base Qwen still hold more raw accuracy on some hunts. medium MATH, 150-case subset: v2 **84.7%**, Qwen **95.3%**. compact thought cost something. grug not hide bruise.
+all comparisons live in [results.md](results.md): original and bigger output budgets, MTP accuracy, squeezed-rock smoke tests, repetition-penalty-1.0 reference runs. [evaluation protocol](evaluation_protocol.md) and [machine-readable results](https://huggingface.co/ProCreations/grug-27b-v2/tree/main/results) explain scoring corrections, completion rates, and limits. grug bring receipts.
+## grug turn think dial
+pick **low**, **medium**, or **xhigh** with `chat_template_kwargs.reasoning_effort`. default **medium**. all three keep native effort instructions, all trained for terse Grug reasoning. **inside voice grug; final answer and code can use ordinary language.**
+mean HumanEval think length at low/medium/xhigh:
+- grug v2: **192 / 266 / 656 tokens**
+- Qwen: **483 / 617 / 1214**
+dial turn up, grug spend more thought on this code hunt. still fewer tokens than base brain.
+but **more thought not always more right.** some coding and math scores fall at xhigh, even with double output limit. long traces sometimes drift into ordinary English. low-effort MATH audit still catch real unfinished loop. [reported looping issue](https://huggingface.co/ProCreations/grug-v1.1-qwen-3.8-27b/discussions/5) get mitigation, not complete cure.
+saved template normalize old reasoning once. client still need correct reasoning and tool parsers. wrong plumbing make good brain stumble.
+## two heads, one rock
+all 15 native draft-head tensors indexed in `model-mtp.safetensors`, right here in repo. during MTP tuning, only draft head learn; main brain stay frozen. held-out draft/verifier agreement climb **90.44% to 90.90%**. distillation loss fall **0.3502 to 0.2965**.
+grug time 18 serial H200 prompts after warmup:
+- no MTP: **68.02 tokens/sec**
+- one draft token: **112.77 tokens/sec**
+- two draft tokens: **142.44 tokens/sec**
+actual draft-token acceptance: **86.87%** with one, **77.20%** with two. speed numbers belong to this workload, not every cave computer. MTP-off/on greedy token sequences match on 10/18 prompts; not all paths say exact same tokens. separate coding/tool accuracy checks live in results.
+head already packed. runtime still need switch turned on. grug test these vLLM 0.29.0 options:
 ```bash
 vllm serve ProCreations/grug-27b-v2 \
   --speculative-config '{"method":"mtp","num_speculative_tokens":2}'
 ```
+generation defaults: **temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05**. if client override model defaults, set these yourself. [usage.md](usage.md) show requests, effort dial, history normalization, and llama.cpp commands.
+## how grug forge brain
+main LoRA run use 3,466 training examples. validation split keep task groups apart. training mix carry verified code, math, repair material, inherited tool replay, and new title examples. best validation checkpoint merge into foundation brain.
+vision encoder stay intact. grug not run new vision benchmark or prove quality across full configured context. eye still there; no new eye trophy claimed.
+all model generation, training, merging, MTP tuning, quantization, runtime validation happen on **HF Jobs**. local GPU stay untouched. conservative whole-build estimate **$120.92**, including failed and canceled tries, below authorized $170 cap. estimate, not invoice. receipts: [training.md](training.md), [release provenance](release_provenance.json), [HF Jobs cost ledger](hf_jobs_cost.json), [SHA256SUMS](SHA256SUMS).
+Apache-2.0 license, same as foundation checkpoint. new benchmark test cases kept out of training mix. inherited replay and foundation pretraining may still hide overlap grug not recognize. training receipts and evaluation source included; raw inherited replay stay private.
+grug made by ProCreations. base brain by Qwen team.