Instructions to use ProCreations/grug-27b-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProCreations/grug-27b-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ProCreations/grug-27b-v2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProCreations/grug-27b-v2")
model = AutoModelForMultimodalLM.from_pretrained("ProCreations/grug-27b-v2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProCreations/grug-27b-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProCreations/grug-27b-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ProCreations/grug-27b-v2

SGLang

How to use ProCreations/grug-27b-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProCreations/grug-27b-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProCreations/grug-27b-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProCreations/grug-27b-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ProCreations/grug-27b-v2 with Docker Model Runner:
```
docker model run hf.co/ProCreations/grug-27b-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

grug 27b v2

grug think small. code get stronger. draft head live in same rock.

grug start fresh from Qwen 3.8 27B. train for compact cave-think, coding, and tool use. Grug-tuned native MTP head already inside this checkpoint. no separate draft model to drag back to cave. every text rock in GGUF downloads carry head too.

v2 score better than grug v1.1 on measured code and tool hunts. session-title bug pass tested checks now. but grug not claim every bug dead: more effort not always more correct, hard thought sometimes leave cave-talk, and some loops still live. grug show gain and bruise below.

grug hunt code and tools

same H200/vLLM hunt rules for every model: temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05, seed 42. same task subsets. same output caps. code run against executable tests; tool calls face official AST checker. fixed subsets, not full leaderboard scores.

Model	Medium HumanEval (132)	Medium MBPP (225)	Medium BFCL subset (200)
Qwen 3.8 27B	99.2%	93.3%	89.0%
Grug v1.1	91.7%	80.4%	81.0%
Grug v2	97.0%	90.7%	88.0%

grug also hunt real repo bugs: 12-issue SWE-bench Verified Django/SymPy subset. medium effort with original limits fix 10/12 for v2, 8/12 for v1.1. separate matched xhigh hunt keep reasoning in history, use 65,536-token context and 16,384 output tokens per tool turn: v2 fix 11/12, v1.1 7/12, Qwen 10/12.

this harness use shared pinned environment, not official per-instance Docker images. one bug swing score by 8.3 points. small hunt useful; not proof grug win everywhere.

session title should be title, not surprise tool call. v2 pass 90/90 session-title checks across low, medium, xhigh. zero tool-call leaks in those checks. real vLLM HTTP tests pass 24/24, including titles with tools present, streamed calls, thinking on and off. checks target reported title problem. grug did not test OpenCode desktop app itself.

keep reasoning in tool history and use original per-effort limits: v2 fix 9 / 10 / 11 of 12 at low/medium/xhigh. v1.1 fix 8 / 9 / 7. more effort help grug on this small agent hunt. not promise for every task.

base Qwen still hold more raw accuracy on some hunts. medium MATH, 150-case subset: v2 84.7%, Qwen 95.3%. compact thought cost something. grug not hide bruise.

all comparisons live in results.md: original and bigger output budgets, MTP accuracy, squeezed-rock smoke tests, repetition-penalty-1.0 reference runs. evaluation protocol and machine-readable results explain scoring corrections, completion rates, and limits. grug bring receipts.

grug turn think dial

pick low, medium, or xhigh with chat_template_kwargs.reasoning_effort. default medium. all three keep native effort instructions, all trained for terse Grug reasoning. inside voice grug; final answer and code can use ordinary language.

mean HumanEval think length at low/medium/xhigh:

grug v2: 192 / 266 / 656 tokens
Qwen: 483 / 617 / 1214

dial turn up, grug spend more thought on this code hunt. still fewer tokens than base brain.

but more thought not always more right. some coding and math scores fall at xhigh, even with double output limit. long traces sometimes drift into ordinary English. low-effort MATH audit still catch real unfinished loop. reported looping issue get mitigation, not complete cure.

saved template normalize old reasoning once. client still need correct reasoning and tool parsers. wrong plumbing make good brain stumble.

two heads, one rock

all 15 native draft-head tensors indexed in model-mtp.safetensors, right here in repo. during MTP tuning, only draft head learn; main brain stay frozen. held-out draft/verifier agreement climb 90.44% to 90.90%. distillation loss fall 0.3502 to 0.2965.

grug time 18 serial H200 prompts after warmup:

no MTP: 68.02 tokens/sec
one draft token: 112.77 tokens/sec
two draft tokens: 142.44 tokens/sec

actual draft-token acceptance: 86.87% with one, 77.20% with two. speed numbers belong to this workload, not every cave computer. MTP-off/on greedy token sequences match on 10/18 prompts; not all paths say exact same tokens. separate coding/tool accuracy checks live in results.

head already packed. runtime still need switch turned on. grug test these vLLM 0.29.0 options:

vllm serve ProCreations/grug-27b-v2 \
  --served-model-name grug-v2 --max-model-len 24576 \
  --max-num-seqs 8 --gpu-memory-utilization 0.85 \
  --reasoning-parser qwen3 --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

generation defaults: temperature 0.6, top-p 0.95, top-k 20, repetition penalty 1.05. if client override model defaults, set these yourself. usage.md show requests, effort dial, history normalization, and llama.cpp commands.

how grug forge brain

main LoRA run use 3,466 training examples. validation split keep task groups apart. training mix carry verified code, math, repair material, inherited tool replay, and new title examples. best validation checkpoint merge into foundation brain.

vision encoder stay intact. grug not run new vision benchmark or prove quality across full configured context. eye still there; no new eye trophy claimed.

all model generation, training, merging, MTP tuning, quantization, runtime validation happen on HF Jobs. local GPU stay untouched. conservative whole-build estimate $120.92, including failed and canceled tries, below authorized $170 cap. estimate, not invoice. receipts: training.md, release provenance, HF Jobs cost ledger, SHA256SUMS.

Apache-2.0 license, same as foundation checkpoint. new benchmark test cases kept out of training mix. inherited replay and foundation pretraining may still hide overlap grug not recognize. training receipts and evaluation source included; raw inherited replay stay private.

grug made by ProCreations. base brain by Qwen team.