Instructions to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Cyb3RQ/qwen3-8b-arabic-poetry-v3",
	filename="qwen3-8b-arabic-poetry-v3-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Use Docker

docker model run hf.co/Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

LM Studio
Jan

vLLM

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Cyb3RQ/qwen3-8b-arabic-poetry-v3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Cyb3RQ/qwen3-8b-arabic-poetry-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Ollama
How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Ollama:
```
ollama run hf.co/Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
```

Unsloth Studio

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Cyb3RQ/qwen3-8b-arabic-poetry-v3 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Cyb3RQ/qwen3-8b-arabic-poetry-v3 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Cyb3RQ/qwen3-8b-arabic-poetry-v3 to start chatting

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Docker Model Runner:
```
docker model run hf.co/Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M
```

Lemonade

How to use Cyb3RQ/qwen3-8b-arabic-poetry-v3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Cyb3RQ/qwen3-8b-arabic-poetry-v3:Q4_K_M

Run and chat with the model

lemonade run user.qwen3-8b-arabic-poetry-v3-Q4_K_M

List all available models

lemonade list

qwen3-8b-arabic-poetry-v3

Experimental Arabic poetry fine-tune based on Qwen/Qwen3-8B, exported as GGUF for LM Studio and llama.cpp.

This build was trained from a local OCR-cleaned Arabic poetry corpus and merged into the Qwen3-8B base model. The goal is better Arabic poetic style, continuation, and short literary analysis.

File

File	Quantization	Size	Recommended use
`qwen3-8b-arabic-poetry-v3-Q4_K_M.gguf`	Q4_K_M	~4.68 GiB	LM Studio / llama.cpp testing

SHA256:

B8C457A9CC0527283315BE465CAE42A65F92234EAC366F136FC7A7BC67B62E1B

Recommended LM Studio Settings

GPU offload: max/all layers
Context: 4096
Reasoning / Thinking: OFF
Temperature: 0.6
Top P: 0.9
Repeat penalty: 1.1 to 1.15
Max tokens: 200-400

Thinking mode is not recommended for this first public test build. Local tests showed non-thinking mode stops cleanly, while thinking mode can continue until the token cap.

Hardware Compatibility

This GGUF is a Q4_K_M quantization of an 8B model, so it is intended for local inference on consumer GPUs, Apple Silicon, or CPU through LM Studio, llama.cpp, Ollama-compatible GGUF loaders, or similar runtimes.

Hardware	Compatibility	Notes
NVIDIA RTX 4090 24 GB	Tested	Full GPU offload at 4096 context. Measured around 5.3 GB VRAM for model/context/compute and ~121 tok/s generation in llama.cpp.
NVIDIA GPU 8 GB+	Recommended	Should comfortably run the Q4_K_M file with 4096 context and full or near-full GPU offload.
NVIDIA GPU 6 GB	Likely usable	Use 2048-4096 context. If it runs out of VRAM, reduce context or GPU offload layers.
Apple Silicon 16 GB+ unified memory	Likely usable	Use LM Studio/llama.cpp with Metal. Performance depends heavily on chip generation.
CPU only, 16 GB+ RAM	Usable but slower	Works best with llama.cpp/LM Studio CPU mode. Expect much lower tok/s than GPU.

Minimum practical memory guidance:

Model file size: ~4.68 GiB.
Recommended VRAM for smooth GPU use: 8 GB or higher.
Recommended system RAM for CPU/offloaded use: 16 GB or higher.
Start with 4096 context before trying larger context sizes.

Example Prompts

اكتب أربعة أبيات عربية فصيحة عن الحنين إلى الوطن، ولا تشرح.

حلل هذه الصورة الشعرية باختصار: الليل يسند رأسه إلى كتف النهر.

أكمل بيتين بأسلوب عربي شعري: في الصحراء نام الضوء فوق رمالها

Local Smoke Test

Tested locally with llama.cpp on RTX 4090:

Prompt eval: ~451 tok/s
Generation: ~121 tok/s
VRAM model/context/compute at 4096 context: ~5.3 GB

Training Notes

Base: Qwen/Qwen3-8B
Fine-tune type: LoRA/QLoRA-style adapter, then merged into base
Adapter target modules: q_proj, k_proj, v_proj, o_proj
LoRA rank: 16
LoRA alpha: 16
Final validation-loss estimate from local run: ~`1.184`

Limitations

This is an experimental model, not a final production Arabic poetry model.

Poetry quality is still uneven.
OCR noise may have influenced parts of the style.
Best used for creative Arabic poetry experiments, not factual answers.
Use reasoning/thinking off for now.

License

The base model Qwen/Qwen3-8B is Apache-2.0. This derivative GGUF is released under Apache-2.0.

Downloads last month: 2

GGUF

Model size

8B params

Architecture

qwen3

Hardware compatibility

4-bit

Model tree for Cyb3RQ/qwen3-8b-arabic-poetry-v3

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-8B

Quantized

(344)

this model