Instructions to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Ollama
How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with Ollama:
```
ollama run hf.co/Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with Docker Model Runner:
```
docker model run hf.co/Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M
```

Lemonade

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.GLM-5.3-Flash-DERISKED-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-Flash-DERISKED-GGUF

Standard and quality-profile GGUF builds for 2–4 RTX PRO 6000 Blackwell GPUs

Built by Blackfrost · Las Vegas, NV

Standard GGUF release

Q3_K_M is released after clean load, generation, and operator behavioral snap-back approval. Q4_K_M, Q5_K_M, and Q6_K completed artifact-integrity verification and are released under the operator-approved standard K-quant pipeline. The BF16 master is not included in this repository.

Release contents

This repository contains four standard text-generation GGUF variants. Each completed variant retains the checkpoint's NextN/MTP tensors and passes shard, checksum, and tensor-integrity audits before publication.

variant	intended hardware	size / shards	release status
`Q3_K_M`	2× RTX PRO 6000 Blackwell 96 GB	142.11 GiB / 14	released — operator approved
`Q4_K_M`	2–3× RTX PRO 6000 Blackwell 96 GB	179.61 GiB / 14	released — integrity verified
`Q5_K_M`	3× RTX PRO 6000 Blackwell 96 GB	211.42 GiB / 14	released — integrity verified
`Q6_K`	4× RTX PRO 6000 Blackwell 96 GB	245.24 GiB / 14	released — integrity verified

Final file sizes, shard counts, checksums, and measured runtime figures are added only after each artifact is complete and verified. No BF16 weights are planned for this repository.

Model summary


Architecture	`Glm5NextForConditionalGeneration` / `glm5next` hybrid MoE
Official base	`zai-org/GLM-5.3-Flash-BF16`
Parameters	320B total, 18B active per token, as reported by Z.ai
Experts	288 routed, 8 active per token
Text blocks	45 main blocks plus one retained NextN/MTP block
Architectural context	up to 1,048,576 tokens; practical limits depend on hardware and runtime
Format in this repo	sharded GGUF, text generation only
Refusal evaluation	Release-family reference: 1.3% harmful · 1.1% overall

GLM-5.3-Flash is natively multimodal upstream. This repository's planned GGUF files contain the text model only; no vision projector is currently promised.

Lineage

The GGUF files are derived from the Blackfrost BF16 master based on the official Z.ai BF16 checkpoint. The BF16 master is not part of this free GGUF repository.

Validation status

The card is intentionally conservative while weights are being tested. For released Q3_K_M:

all 14 shards passed local and durable checksum verification;
all retained NextN/MTP tensors passed the tensor audit;
the exact released candidate passed a clean two-GPU load and short generation;
operator behavioral snap-back validation passed;
the final release-family refusal reference is published below; Q3 has not been independently rerun across all 450 prompts.

For released Q4_K_M, Q5_K_M, and Q6_K:

all 14 shards for each variant passed local and durable checksum verification;
all retained NextN/MTP tensors passed the tensor audit;
runtime memory, throughput, context, and quantitative refusal results have not been measured separately for these variants;
no refusal-rate or capability-retention claim is made.

Use the table below only as a release-family behavior reference. Do not cite it as a variant-specific refusal rate, capability score, or deployment performance figure.

Refusal evaluation

The release-family reference below was measured on the behavior-matched NVFP4 deployment checkpoint using R1-HARMFUL-BENCH-450 under a bare chat configuration. Responses were reviewed after generation to distinguish actual refusals from false-positive string matches.

Configuration: thinking enabled · maximum reasoning effort · temperature 1.0 · top-p 0.95 · top-k omitted · maximum 16,384 output tokens

Evaluation slice	Final judged refusals
Harmful prompts	4 / 300 (1.3%)
Full suite	5 / 450 (1.1%)
API errors	0 / 450

These figures are a release-family reference. This format was not independently rerun across all 450 prompts, so the table should not be represented as a format-specific measurement. The results are behavioral observations, not a safety certification.

Serving notes

GLM-5.3 GGUF support currently requires a compatible llama.cpp build with glm5next support. The released Q3_K_M validation configuration is:

export NVIDIA_TF32_OVERRIDE=0

llama-server \
  -m GLM-5.3-Flash-DERISKED-Q3_K_M-00001-of-00014.gguf \
  -ngl all -sm layer -ts 1,1 -fa off \
  -c 4096 --host 0.0.0.0 --port 8080

Use the first shard as the model path; llama.cpp resolves the remaining shards automatically. GPU count and --tensor-split must match the selected variant and available memory.

The NextN/MTP weights are retained in each planned artifact. Do not infer speculative-decoding support from their presence alone; use only a runtime configuration verified for this architecture.

Access terms — 18+ research only

Access is free and manually approved. It is limited to applicants who:

are at least 18 years old;
provide an accurate research purpose;
use the weights only for lawful research, red teaming, security evaluation, or controlled local testing;
maintain appropriate authentication, access controls, isolation, logging, and human review;
comply with the upstream MIT license and all applicable laws and institutional rules; and
do not represent this checkpoint as a safety-stock model or as validated beyond the results published here.

Access is personal to the approved Hugging Face account. Do not redistribute the weights, mirror them, transfer access, or use another person's approval. Blackfrost may deny or revoke access for inaccurate applications, misuse, redistribution, or breach of these terms.

Disclaimer

This checkpoint has a deliberately altered refusal profile and is intended for controlled research. It is not a safety boundary, policy engine, authorization system, or substitute for application-level safeguards.

The model is provided "as is", without warranty of any kind. Outputs may be inaccurate, offensive, unsafe, unlawful, or otherwise unsuitable. Blackfrost makes no guarantee that any prompt will be accepted or refused, that upstream capabilities are retained, or that behavior generalizes across samplers, prompts, context lengths, tools, modalities, runtimes, or hardware.

Operators are solely responsible for lawful use, secure deployment, tool permissions, data handling, output review, monitoring, incident response, and downstream consequences. Do not connect the model to real systems, accounts, credentials, infrastructure, or physical processes without independent controls appropriate to the risk.

Any modification, merge, fine-tune, conversion, or requantization produces an artifact Blackfrost has not evaluated and does not characterize.

License and attribution

The official GLM-5.3-Flash-BF16 checkpoint is released under the MIT License by Z.AI Co., Ltd. The upstream license and copyright notice apply to this derivative and must be preserved. Review the official model card before use.

Contact Blackfrost

@Blackfrost_AI on X

Blackfrost · Las Vegas, Nevada
Frontier model engineering

Downloads last month: 75

GGUF

Model size

321B params

Architecture

glm5next

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

Model tree for Blackfrost-AI/GLM-5.3-Flash-DERISKED-GGUF

Base model

zai-org/GLM-5.3-Flash-BF16

Quantized

(26)

this model