Instructions to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="doncxy/BIJA-cerebellum-Qwen3-1.7B-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("doncxy/BIJA-cerebellum-Qwen3-1.7B-v1", dtype="auto")

MLX

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("doncxy/BIJA-cerebellum-Qwen3-1.7B-v1")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

llama-cpp-python

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="doncxy/BIJA-cerebellum-Qwen3-1.7B-v1",
	filename="Qwen3-1.7B-BIJA-cerebellum-Q8_0.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
# Run inference directly in the terminal:
llama-cli -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
# Run inference directly in the terminal:
llama-cli -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

Use Docker

docker model run hf.co/doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

LM Studio
Jan

vLLM

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

SGLang

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Ollama:
```
ollama run hf.co/doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
```

Unsloth Studio

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 to start chatting

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default doncxy/BIJA-cerebellum-Qwen3-1.7B-v1

Run Hermes

hermes

MLX LM

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "doncxy/BIJA-cerebellum-Qwen3-1.7B-v1",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Docker Model Runner
How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Docker Model Runner:
```
docker model run hf.co/doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0
```

Lemonade

How to use doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull doncxy/BIJA-cerebellum-Qwen3-1.7B-v1:Q8_0

Run and chat with the model

lemonade run user.BIJA-cerebellum-Qwen3-1.7B-v1-Q8_0

List all available models

lemonade list

BIJA-cerebellum-Qwen3-1.7B-v1 / README.md

doncxy

Initial release: Path B LoRA distillation results (Phase 5 GGUF + adapter)

a381139 verified about 1 month ago

preview code

raw

history blame contribute delete

6.93 kB

	---
	language:
	- en
	- zh
	license: apache-2.0
	library_name: transformers
	base_model: Qwen/Qwen3-1.7B
	tags:
	- bija
	- cerebellum
	- lora
	- distillation
	- mlx
	- gguf
	- memory
	- qwen3
	pipeline_tag: text-generation
	---

	# BIJA-cerebellum-Qwen3-1.7B-v1

	LoRA-distilled variant of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B), fine-tuned to power the cerebellum (small-brain) of [Bīja](https://github.com/cxyAI/bija) — a memory-system-as-AI built on the eight-consciousness theory.

	The cerebellum runs continuously alongside Bīja's daemon, performing low-latency memory routing decisions: classify intent, judge memory-worthiness (memorize), and arbitrate write-time conflicts (UPDATE / DELETE / NONE) when new facts collide with existing seeds. The base 1.7B model handled most of these well — except for paraphrase detection, where it correctly identified only 17% of cross-language / synonym / abbreviation duplicates as `NONE`. This adapter fixes that to 100%.

	## Why this model exists

	Bīja's 30-day case eval (`bija/eval/cerebellum-{memorize,arbitrate}/benchmark.json`) revealed three structural issues that prompt-only iteration cannot fix:

	\| Task \| Baseline 1.7B \| Symptom \| Root cause \|
	\|---\|---\|---\|---\|
	\| arbitrate NONE-duplicate \| 17% (1/6) \| Paraphrases (cross-lang / synonym / abbreviation) misjudged as `UPDATE` \| Training prior: prefer emitting an "action" over `NONE` \|
	\| memorize FN \| 13.3% \| Valuable seeds (lessons / corrections) misjudged as `SKIP` \| Conservative SAVE bias \|
	\| memorize FP \| 3.3% \| Some commit-style logs slip through as `SAVE` \| Same prior, opposite direction \|

	A separate experiment with [Granite 3.3-2B](https://huggingface.co/ibm-granite/granite-3.3-2b-instruct) ran 4 prompt-rewrite iterations across 120 cases and confirmed the same prior cannot be undone by prompts alone. Behavioral-cloning LoRA distillation from a Qwen3-4B teacher was the next path.

	## Results

	Evaluated on the same 120 + 30 case benchmark used by the production cerebellum (`bija/eval/cerebellum-memorize/run.ts` + `bija/eval/cerebellum-arbitrate/run-with-sim.ts`):

	\| Metric \| Baseline (Qwen3-1.7B-Q8_0 prompt-only) \| LoRA Q8_0 GGUF \| Δ \|
	\|---\|---\|---\|---\|
	\| memorize accuracy \| 91.7% (110/120) \| 97.4% (excl 5 cold-start parse-fails) \| +5.7pp \|
	\| memorize FP rate \| 3.3% \| 3.3% \| 0 \|
	\| memorize FN rate \| 13.3% \| 1.7% \| −11.6pp \|
	\| memorize avg latency \| 480ms \| 436ms \| −9% \|
	\| arbitrate accuracy \| 76.7% (23/30) \| 86.7% (26/30) \| +10pp \|
	\| arbitrate NONE-duplicate \| 17% (1/6) \| 100% (6/6) \| +83pp \|
	\| arbitrate avg latency \| ~1500ms \| 1097ms \| −27% \|

	Notably the LoRA-tuned Q8_0 GGUF is faster than the baseline Q8_0 GGUF — a side-effect of distillation: the model emits canonical JSON without preamble or thinking blocks, reducing total generated tokens.

	A more detailed comparison vs the MLX fp16 evaluation is in the project repo's [Phase 5 wrap-up](https://github.com/cxyAI/bija/blob/main/docs/path-b-wrap-up-2026-04-26.md).

	## Files in this repo

	\| File \| Purpose \|
	\|---\|---\|
	\| `Qwen3-1.7B-BIJA-cerebellum-Q8_0.gguf` (1.7 GB) \| Drop-in Q8_0 GGUF; `llama.cpp` / Ollama / cerebellum-style sidecars load it directly \|
	\| `adapters.safetensors` (38 MB) \| Raw LoRA weights — apply on top of vanilla `Qwen/Qwen3-1.7B` (HF format) with `mlx_lm.fuse` or `peft` \|
	\| `adapter_config.json` \| mlx-lm LoRA config: rank=16, scale=2.0, dropout=0.05, num_layers=16, target=`q_proj+v_proj` \|

	## How to use

	### Drop-in replacement (recommended) — llama.cpp / Ollama

	```bash
	hf download doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 \
	Qwen3-1.7B-BIJA-cerebellum-Q8_0.gguf \
	--local-dir ~/models

	llama-server -m ~/models/Qwen3-1.7B-BIJA-cerebellum-Q8_0.gguf -c 4096
	```

	Or for Bīja users — replace the production GGUF directly:

	```bash
	mv ~/.seeddb/cerebellum/models/Qwen3-1.7B-Q8_0.gguf{,.baseline}
	ln -s ~/models/Qwen3-1.7B-BIJA-cerebellum-Q8_0.gguf \
	~/.seeddb/cerebellum/models/Qwen3-1.7B-Q8_0.gguf
	pkill -f llama-server # next call respawns sidecar with new weights
	```

	### Apply LoRA on top of vanilla Qwen3-1.7B (MLX)

	```bash
	pip install mlx-lm
	hf download doncxy/BIJA-cerebellum-Qwen3-1.7B-v1 \
	adapters.safetensors adapter_config.json --local-dir ./bija-cerebellum-lora

	mlx_lm.generate \
	--model Qwen/Qwen3-1.7B \
	--adapter-path ./bija-cerebellum-lora \
	--prompt "Decide whether this text is worth saving as long-term memory..." \
	--max-tokens 128
	```

	## Training recipe

	\| Field \| Value \|
	\|---\|---\|
	\| Base model \| `Qwen/Qwen3-1.7B` (1.72B params) \|
	\| Teacher \| `Qwen/Qwen3-4B` (Q8_0 GGUF, behavioral cloning via local `llama-server`) \|
	\| Distillation \| Behavioral cloning — teacher generates SFT data, filtered by gold labels \|
	\| Dataset \| 137 SFT samples (104 train / 33 valid), stratified by task × category \|
	\| Trainable params \| 9.96M (0.579% of base) \|
	\| LoRA rank / scale \| 16 / 2.0 (effective alpha 32) \|
	\| LoRA dropout \| 0.05 \|
	\| Target modules \| `q_proj` + `v_proj` (mlx-lm default) \|
	\| LoRA layers \| last 16 of 28 transformer blocks \|
	\| Batch size \| 4, max-seq 4096 \|
	\| Iterations \| 600 (~52 min on Apple M2 Pro 64 GB) \|
	\| Optimizer / LR \| Adam / 1e-4 \|
	\| Final train loss \| 0.006 \|
	\| Best val loss \| 0.077 (iter 350); final 0.086 \|
	\| Peak memory \| 33.3 GB / 64 GB (fp16, no QLoRA / no grad checkpoint) \|
	\| Tokens/sec \| ~820 avg \|

	## Intended use

	Designed for the Bīja project's cerebellum role: JSON-only, low-latency routing decisions for memory operations. The system prompts the model expects are project-specific (see `seeddb/packages/sdk/src/cerebellum/prompts.ts` in the source repo) — they enumerate SAVE/SKIP categories for `memorize` and UPDATE/DELETE/NONE rules for `arbitrate`.

	This is not a general-purpose chat model. Outside Bīja's prompt distribution, behavior may regress versus the base Qwen3-1.7B. For general use, prefer the base model.

	## Limitations

	- Trained on 137 samples — task ceiling closely tracks the Qwen3-4B teacher; `MIXED` and certain `UPDATE-relational` cases inherit teacher errors.
	- Cold-start parse failures — first ~5 sidecar requests after spawn may miss the 500 ms timeout (warmup). Persistent daemons amortize this away.
	- Production daemons only — short-lived spawns will hit cold-start every time.
	- Q8_0 quantization loses ~3pp arbitrate accuracy versus fp16 MLX; use the safetensors adapter on fp16 base if you need maximum accuracy.

	## Citation / acknowledgements

	Built on:
	- [`Qwen/Qwen3-1.7B`](https://huggingface.co/Qwen/Qwen3-1.7B) (base)
	- [`Qwen/Qwen3-4B`](https://huggingface.co/Qwen/Qwen3-4B) (teacher; via local Q8_0 GGUF)
	- [`mlx-lm`](https://github.com/ml-explore/mlx-lm) (training + fuse)
	- [`llama.cpp`](https://github.com/ggerganov/llama.cpp) (HF→GGUF conversion)

	## License

	Apache 2.0 (matches base model).