Instructions to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Use Docker

docker model run hf.co/petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

LM Studio
Jan

vLLM

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Ollama
How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with Ollama:
```
ollama run hf.co/petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
```
Unsloth Desktop

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with Docker Model Runner:
```
docker model run hf.co/petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0
```

Lemonade

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Run and chat with the model

lemonade run user.Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF-Q4_0

List all available models

lemonade list

Hermes Agent

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF:Q4_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.0-35B Strix Halo Hybrid — LM Studio GGUF

LM Studio compatible, no integrated MTP.

Choose the correct build

Build	Use it with	Integrated MTP
Full optimized MTP build	Current llama.cpp with `draft-mtp`; patched Vulkan/CUDA deployments	Yes
LM Studio compatible build (this repository)	LM Studio runtimes that reject the integrated MTP block	No

This repository is the compatibility build. It preserves all 40 optimized base-model layers and removes only the appended MTP prediction layer and its qwen35moe.nextn_predict_layers metadata. No remaining tensor was converted, retrained, or requantized during the compatibility rewrite.

File

File	Size	Quantization	SHA-256
`ornith-1.0-35b-hybrid-Q4_0-LMStudio.gguf`	18,866.19 MiB (18.424 GiB)	Q4_K_M with 20 hot `ffn_down_exps` tensors overridden to Q4_0; no MTP tensors	`02c057b9a7fc5abeae73b7c9b6774928a0f8fc01c44cb858dd715babdd2b851f`

Compared with the full MTP artifact, this file is 521.38 MiB smaller. The removed data consists of exactly 20 blk.40.* prediction tensors. The output contains 733 tensors, reports qwen35moe.block_count = 40, and contains no blk.40.* or nextn_predict_layers entry.

Why a separate LM Studio build exists

The full artifact loads in newer llama.cpp builds that support integrated Qwen3.6/Ornith MTP. LM Studio runtime 2.14.0 used for the Windows validation is based on llama.cpp b8861 and rejected that appended block before allocating the model. Removing only the unsupported MTP layer made the same optimized base load successfully.

Validation on Windows 11:

LM Studio runtime: llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.14.0;
llama.cpp revision reported by LM Studio: b8861 (cf8b0db);
model load completed in 15.16 seconds in the compatibility smoke profile;
a local generation request completed successfully;
the incompatible full-MTP file failed in both GPU and CPU-only load tests, proving that the failure was format/runtime support rather than VRAM.

Optimization retained in this build

the compatible Ornith/Qwen3.6 35B A3B MoE architecture and chat template;
selective Q4_0 replacement of 20 frequently streamed MoE down-projection tensors, with the surrounding Q4_K_M quantization plan unchanged;
all attention, dense, embedding, normalization and output tensors from the optimized 40-layer base;
the same inference quality as the full artifact when speculation is off.

No weights were trained or fine-tuned. Selective requantization changes the numerical representation, so the original project checked quality end-to-end on the same Orion coding slice.

Performance scope

The table below was measured on the same Windows 11 laptop with the official llama.cpp b10066 CUDA container, 16,384 context, Q8_0 K/V cache, batch 2048, ubatch 512, parallelism 1, dense/attention tensors on an RTX 4060 Laptop 8 GiB, and MoE experts in host RAM. The Hybrid, speculation off row executes the same 40-layer tensor set published here, but it is not presented as a direct LM Studio-runtime benchmark.

Laptop profile	1K decode	8K decode	Repeated-code decode	8K prompt processing
Baseline Ornith Q4_K_M	5.57 tok/s	23.90 tok/s	21.95 tok/s	341.48 tok/s
This 40-layer hybrid tensor set, speculation off	29.70 tok/s	29.59 tok/s	26.75 tok/s	365.26 tok/s
Full linked build, MTP + n-gram	28.74 tok/s	66.88 tok/s	67.23 tok/s	354.19 tok/s

The stable non-speculative improvement was +23.8% at 8K and +21.8% on repeated code. The 60–67 tok/s result belongs to the linked MTP build under a compatible runtime and must not be expected from this no-MTP file.

Direct LM Studio native API benchmark

The compatibility file was then measured through LM Studio's own /api/v0/completions endpoint on the same Windows laptop. Each row generated 128 tokens at temperature 0. The 1K and 8K prompts were run twice; the repeated 8K code prompt was run three times. LM Studio reported decode throughput and TTFT directly in the response. The first TTFT is cold-prompt processing; the second shows LM Studio's prompt-cache path.

LM Studio 2.14.0 profile	Actual prompt	Decode mean	Cold TTFT	Warm-cache TTFT
1K prompt	1,056 tokens	33.48 tok/s	4.296 s	0.139 s
8K prompt	8,448 tokens	34.77 tok/s	23.160 s	0.166 s
Repeated 8K code	8,448 tokens	33.25 tok/s	—	0.151–0.170 s

Against the b10066 no-speculation run on the same tensor set, LM Studio decode was +12.7% at 1K, +17.5% at 8K and +24.3% on repeated code. These are measured runtime differences, not a promise for other LM Studio versions or hardware.

LM Studio telemetry	Peak
GPU temperature	66 C
GPU utilization	91%
VRAM used	2,329 MiB
GPU power	45.1 W
Whole-system RAM used	44.10 GiB

The complete native response statistics are published in lmstudio-native-benchmark.json.

Orion four-scenario coding slice through LM Studio

The same seeded workspaces, Orion binary, prompts and deterministic checks used for the earlier laptop A/B were executed through http://127.0.0.1:1234/v1. Settings remained approval=auto, 24 agent steps, 16,384 context and a 900-second outer timeout.

Orion coding scenario	Score	Checks	Wall time
Java service coverage	80%	8/10	900.0 s (outer timeout)
Kafka Node order pipeline	75%	6/8	383.2 s
Rabbit retry / DLQ	75%	6/8	363.4 s
Kafka Java outbox / idempotency	80%	8/10	530.9 s
Total / average	77.5%	28/36	2,177.6 s (36m 17.6s)

Quality is identical to both the baseline and full-hybrid laptop runs. Raw decode was faster, but end-to-end agent time was longer because Orion repeated verification-guard steps; the Java coverage scenario reached the 900-second outer timeout after its file changes and tests were already complete.

The Rabbit post-run verifier initially failed to spawn the generic python command with EPERM. The saved workspace compiled successfully and the exact three-test suite passed under the bundled Python interpreter. The raw 5/8 result and original error are retained in the local research artifact; the published summary records the recovered 6/8 score and that the model itself was not rerun. orion-lmstudio-summary.json contains the check-level result.

Suggested LM Studio load profile

context length: 16,384 for the first validated load;
K cache: Q8_0;
V cache: Q8_0;
parallel sessions: 1;
evaluation batch size: 2048;
keep MoE expert layers in host RAM on an 8 GiB GPU;
offload dense/attention layers to the GPU as memory permits.

Start at 4K or 8K context if LM Studio's resource guardrails reject the first load, then increase context after measuring available RAM and VRAM. Context length affects KV memory but does not restore MTP; use the linked full build with a current llama.cpp runtime when speculative MTP acceleration is required.

Sources and licensing

The base model is deepreinforce-ai/Ornith-1.0-35B. The full linked build also contains compatible MTP tensors from a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF. This compatibility artifact contains no MTP tensors, but provenance is retained because it was derived from that combined artifact. Review the source repositories and the accompanying license files before redistribution or commercial use.

Downloads last month: 76

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

4-bit

Model tree for petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF

Base model

ornith-ai/Ornith-1.0-35B

Quantized

(174)

this model