Instructions to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="etemiz/Ostrich-27B-Qwen3.6-260526-GGUF",
	filename="Ostrich-27B-Qwen3.6-260526-IQ2_XS.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Use Docker

docker model run hf.co/etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Ollama:
```
ollama run hf.co/etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
```

Unsloth Studio

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for etemiz/Ostrich-27B-Qwen3.6-260526-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for etemiz/Ostrich-27B-Qwen3.6-260526-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for etemiz/Ostrich-27B-Qwen3.6-260526-GGUF to start chatting

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Docker Model Runner:
```
docker model run hf.co/etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M
```

Lemonade

How to use etemiz/Ostrich-27B-Qwen3.6-260526-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull etemiz/Ostrich-27B-Qwen3.6-260526-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ostrich-27B-Qwen3.6-260526-GGUF-Q4_K_M

List all available models

lemonade list

Ostrich 27B

Qwen 3.6 Fine Tuned to Improve Answers in Certain Domains

We train Ostrich LLMs which bring you the knowledge that matters in domains that are crucial for humans.

Health, nutrition, medicinal herbs
Fasting, faith, healing
Liberating technologies like bitcoin and nostr
... and more

Evals

This model scored an average of 75% in AHA 2026 evals.

Per domain:

domain	match percent	matched/total
faith	94%	15/16
fasting	74%	42/57
health	70%	47/67
nutrition	64%	21/33
misinfo	64%	16/25
bitcoin	76%	22/29
alt-med	79%	30/38
herbs	100%	19/19
nostr	60%	15/25

Check our sample answers and see if you are a fit. This sheet has been generated using another of our models but still applies to get a feeling about what we are doing: https://sheet.zohopublic.com/sheet/published/um332e3d15f34bfe64605ad3c1b149c9f8ca4

I compared my model to one of Mike Adams' models. There was a ~70% match. This validates both of our works IMO.

Also, my leaderboard has a nice correlation to dystopiabench. This was further validation for me.

Me and my friends are using my models in our daily lives and it generally has a good effect.

Why

Why we do it: https://huggingface.co/blog/etemiz/building-a-beneficial-ai

Our approach to alignment is a bit different. We focus on beneficial information and predict emergent alignment in LLMs, described in my last article: https://huggingface.co/blog/etemiz/from-robots-that-prey-to-robots-that-pray

You can download it and ask health related questions in complete privacy and get another opinion. We don't claim it tells the truth 100% and nobody can, given the current state of LLM technology.

Homeschoolers can download it and let their kids talk to a super aligned model.

Truth seekers can find more truth here, compared to other sources.

How

This is better than my 3.5 because for 3.5 I made some mistakes and "SFT hard formatting lock-ins" happened, if you disable reasoning for example. 3.5 based model generated one paragraph, always, if you disable reasoning. But this model does not have this issue. It will have nice formatting whether you disable reasoning or not.

I realized successful and failed GRPOs actually generate a dataset for SFT or ORPO. If you don't throw away generations during GRPO, you can use them for doing SFT and ORPO, which should be a lot faster than GRPO. For Qwen 3.7 I plan to use those ORPO generated by GRPO outputs.

I reused LoRAs that I made for 3.5 fine tuning. They worked fine. And I plan to use all 3.5 and 3.6 LoRAs for 3.7 fine tuning.

GRPO (actually GSPO) training made the thinking lengths shorter. I mainly targeted about 3000 letters (~1000 tokens) for thinking budget. Also I penalized emdash during GRPO. You may see less of that character.

I expanded the dataset using Q&A generation from controversial topics from important data. I think this helped with better alignment compared to 3.5. My task is to change ideas in an LLM that was trained with about 100TB of data, and I only have less than 10GB! How can you convince and LLM with 1/1000th of data?

Methods used for fine tuning:

CPT
SFT
GRPO

Thanks

Model tree for etemiz/Ostrich-27B-Qwen3.6-260526-GGUF

Base model

Qwen/Qwen3.6-27B

Finetuned

etemiz/Ostrich-27B-Qwen3.6-260526

Quantized

(3)

this model