Instructions to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="h0ney-badger/qwen2.5-coder-7b-pyc-distill",
	filename="qwen-coder-7b-mine-Q5_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
# Run inference directly in the terminal:
llama cli -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
# Run inference directly in the terminal:
./llama-cli -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Use Docker

docker model run hf.co/h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

LM Studio
Jan

vLLM

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "h0ney-badger/qwen2.5-coder-7b-pyc-distill"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "h0ney-badger/qwen2.5-coder-7b-pyc-distill",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Ollama
How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Ollama:
```
ollama run hf.co/h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
```

Unsloth Studio

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for h0ney-badger/qwen2.5-coder-7b-pyc-distill to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for h0ney-badger/qwen2.5-coder-7b-pyc-distill to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for h0ney-badger/qwen2.5-coder-7b-pyc-distill to start chatting

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Docker Model Runner:
```
docker model run hf.co/h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M
```

Lemonade

How to use h0ney-badger/qwen2.5-coder-7b-pyc-distill with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull h0ney-badger/qwen2.5-coder-7b-pyc-distill:Q5_K_M

Run and chat with the model

lemonade run user.qwen2.5-coder-7b-pyc-distill-Q5_K_M

List all available models

lemonade list

Qwen2.5-Coder-7B-Instruct — Python/C self-distill (GGUF)

A QLoRA fine-tune of Qwen2.5-Coder-7B-Instruct, specialized for Python and C by self-distillation from the larger Qwen2.5-Coder-14B-Instruct teacher. Trained entirely on locally-generated, execution-verified synthetic data on a single 16 GB consumer GPU (RTX 5070 Ti).

TL;DR — On a held-out, execution-scored eval it lifts the base model from 60.8% → 74.2% pass@1: C jumps (47% → 64%) and Python jumps too (78% → 87%). It also writes complete, interactive programs from natural asks, not just bare functions. Specialized for Python/C — read the honest caveats below.

Results (execution-based pass@1)

Same eval set, same Q5_K_M quant, same prompts. The base is the exact weights this model was fine-tuned from, so this isolates what the fine-tune did.

Model	Python	C	Total
Qwen2.5-Coder-7B-Instruct (base)	77.8% (42/54)	47.0% (31/66)	60.8% (73/120)
This model	87.0% (47/54)	63.6% (42/66)	74.2% (89/120)
Δ	+9.2	+16.6	+13.4

pass@1 = the model's code was compiled/run against held-out tests and had to pass. Numbers carry ±1–2 samples of run-to-run sampling noise (temperature 0.2).

How it was made

Teacher generates data. Qwen2.5-Coder-14B-Instruct (served locally via llama.cpp) produces coding tasks + reference solutions + tests via seeded self-instruct across per-language domains (Python, C, plus some HTML/Java).
Rejection sampling. Python solutions are executed and C solutions are gcc-compiled-and-run against their tests; only samples that pass are kept. The teacher's C prompt is constrained to emit self-contained code (all #includes) with explicit function/type contracts.
QLoRA SFT of the 7B student (Unsloth, r=16, 2 epochs, 4-bit base) on ~1.25k deduplicated samples — a mix of execution-verified functions and complete, run-verified programs from natural requests — then merged to 16-bit and quantized to GGUF Q5_K_M.

Run with the chat template applied and low temperature (~0.2) — via llama-server or llama-cli -cnv. Raw -p completion mode at default temp will underperform.

Evaluation methodology (and why it's not HumanEval)

The eval is execution-based on a dedicated 120-sample set that is disjoint from the training data (exact + fuzzy dedup). It is deliberately not HumanEval or MBPP: the point was to avoid the "benchmaxxing" trap of training toward a public benchmark (or worse, contaminating on it). The trade-off is honesty over comparability — these numbers are meaningful relative to the base on this set, not directly comparable to leaderboard scores.

Honest limitations

Specialized, not universally better. Gains are in Python and C specifically; don't expect improvement in other languages.
Eval favors the model's home distribution. The eval tasks come from the same teacher/family and follow the same conventions the model was trained on (e.g. the C convention of "no main(), self-contained solution"). A neutral benchmark would likely show a smaller gap.
C hit a ceiling. C stayed at ~64% across data-scaling rounds — the remaining C failures are contract/naming and genuine logic bugs that more of the same data doesn't fix at 7B. Python and complete-program ability, by contrast, kept improving.
English-only prompts; Python/C focus (HTML/Java were minor in training).

Usage

Q5_K_M GGUF (~5.1 GB), runs anywhere llama.cpp does.

# llama.cpp
llama-server -m qwen-coder-7b-mine-Q5_K_M.gguf -ngl 99 -c 4096
# or LM Studio: drop the .gguf into ~/.lmstudio/models/<folder>/ and load it

There is also a Python-only 1.5B sibling (941 MB, runs on CPU) built by the same pipeline — see the repo.

Provenance & license

License: Apache-2.0 — inherited cleanly. Both the base (Qwen2.5-Coder-7B) and the teacher (Qwen2.5-Coder-14B) are Apache-2.0, which places no restriction on using model outputs to train other models. Training data is fully self-generated (no scraped corpus, no ToS-restricted API).
Base model: Qwen/Qwen2.5-Coder-7B-Instruct
Teacher: Qwen/Qwen2.5-Coder-14B-Instruct
Full training/eval pipeline: https://github.com/h0n3y-badger/code-distill

Built as a learning project in honest, contamination-controlled small-model distillation. The interesting artifact is the reproducible pipeline — see the repo.

Downloads last month: 28

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

5-bit

Model tree for h0ney-badger/qwen2.5-coder-7b-pyc-distill

Base model

Qwen/Qwen2.5-7B

Finetuned

Qwen/Qwen2.5-Coder-7B

Finetuned

Qwen/Qwen2.5-Coder-7B-Instruct

Quantized

(210)

this model