Instructions to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
# Run inference directly in the terminal:
llama cli -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
# Run inference directly in the terminal:
llama cli -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
# Run inference directly in the terminal:
./llama-cli -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Use Docker

docker model run hf.co/sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

LM Studio
Jan

vLLM

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Ollama
How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with Ollama:
```
ollama run hf.co/sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
```
Unsloth Desktop

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with Docker Model Runner:
```
docker model run hf.co/sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP
```

Lemonade

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Run and chat with the model

lemonade run user.GLM-5.3-Flash-Spark-Q2XL-MTP-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash Spark Q2XL MTP

Spark is a quality-first 2.80 BPW GGUF quant of zai-org/GLM-5.3-Flash, tuned to fit and run fully on a 128 GB NVIDIA DGX Spark / GB10 with native MTP speculative decoding.

The eight model shards total 106,987.66 MiB (about 105 GiB). This build keeps attention and other sensitive paths at higher precision while compressing the large routed-expert tensors more aggressively:

routed expert down projections: IQ3_XXS
routed expert gate/up projections: IQ2_XS
shared experts: Q6_K
attention Q/K/V/output matrices: Q8_0
norms, routing controls, recurrent/SSM controls, and MTP-sensitive tensors: retained as BF16/F32 where required
calibration: importance-matrix guided

The result is not simply a uniform 2-bit model. Most storage savings come from the repeated expert matrices; the comparatively small, quality-sensitive paths receive substantially more precision.

Benchmarks

All local results below were measured on one DGX Spark / GB10. Quality used one sample per task, temperature 0, top-p 0.95, reasoning_effort=low, a 768-token completion cap, and EvalPlus 0.3.1 with HumanEvalPlus dataset v0.1.10. This is a concise local-coding profile, not the upstream model's default max reasoning profile.

Test	Result
HumanEval pass@1	159/164 (97.0%)
HumanEval+ Mini pass@1	153/164 (93.3%)
34-point coding microbenchmark	29/34 (85.3%)
Generation failures	0/164
Full generation wall time	1,025 s, one sequential slot

Five sequential streaming coding completions, 512 output tokens each:

Metric	Result
Median decode	22.504 tok/s
Mean decode	22.083 tok/s
Minimum decode	17.176 tok/s
Median effective completion	21.660 tok/s
Median TTFT	0.807 s
First-trial TTFT	0.930 s
Median native-MTP acceptance	54.3%

For comparison, the previously tested Unsloth UD-Q2_K_XL GLM quant reached 155/164 HumanEval, 148/164 HumanEval+ Mini, and 25.414 tok/s median. Spark traded about 11% median decode speed for five additional HumanEval+ passes in this evaluation.

Tested runtime

The model was converted and served with the GLM5-Next fork of llama.cpp at commit ea37b8bb0 (vcruz305/llama.cpp). Native MTP must be enabled. The tested server configuration was:

llama-server \
  -m GLM-5.3-Flash-Spark-Q2XL-MTP-00001-of-00008.gguf \
  --host 0.0.0.0 --port 11001 \
  --ctx-size 32768 \
  --n-gpu-layers 999 \
  --parallel 1 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --cache-ram 512

The 512 MiB prompt-cache cap is intentional. The server image's 8 GiB default left only about 1 GiB of memory headroom and caused a response-finalization timeout during sustained non-streaming requests. With the cap, the model loaded with about 9 GiB available and completed the full 164-task run cleanly.

The upstream model is natively multimodal, but this GGUF/runtime combination was evaluated only for text/code generation. Vision behavior is not claimed.

Reproducibility

Per-shard SHA-256 digests are provided in SHA256SUMS. The complete appendable leaderboard, test protocol, harness hashes, and result-artifact hashes live in Packwolf's docs/benchmarks/MODEL_LEADERBOARD.md.

License and attribution

The upstream GLM-5.3-Flash checkpoint is MIT licensed. See LICENSE and the upstream model card for attribution, limitations, and citation information.

Downloads last month: -

GGUF

Model size

321B params

Architecture

glm5next

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for sayyidfareed/GLM-5.3-Flash-Spark-Q2XL-MTP

Base model

zai-org/GLM-5.3-Flash

Quantized

(39)

this model