Instructions to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
# Run inference directly in the terminal:
llama cli -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
# Run inference directly in the terminal:
llama cli -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
# Run inference directly in the terminal:
./llama-cli -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Use Docker

docker model run hf.co/maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

LM Studio
Jan
Ollama
How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Ollama:
```
ollama run hf.co/maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
```

Unsloth Studio

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF to start chatting

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Run Hermes

hermes

OpenClaw new

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Docker Model Runner:
```
docker model run hf.co/maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF
```

Lemonade

How to use maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

Run and chat with the model

lemonade run user.Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

Qwen3.6-14B-A3B-vibetuned ROCmFPX STRIX_LEAN — GGUF

ROCmFPX Q4_0_ROCMFP4_STRIX_LEAN quant of tvall43/Qwen3.6-35B-A3B-Heretic, down-pruned to a 14B-A3B variant by the vibetuned author.

Built with charlie12345/ROCmFPX on a Radeon RX 9060 XT 16 GB (gfx1200), ROCm 7.2.3, NixOS 25.11. Quantized 2026-06-27 with build commit 11d76c2.

File	Size	Quant	BPW
`Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN.gguf`	7.0 GB	`Q4_0_ROCMFP4_STRIX_LEAN` (4-bit ROCmFP4 + Strix K/V + Q5_K embed)	4.42

This is not a stock llama.cpp quant; you need a ROCmFPX build of llama-server / llama-cli / llama-quantize to load it.

Scope of these benchmarks — read this first

These numbers are a light baseline, not a thorough ROCmFPX evaluation. The mesh's bench framework is built for production agent workload regression-detection on the local stack, not for the kind of multi-axis sweep that upstream quant maintainers typically publish. Specifically:

Harness scope is bounded. The numbers below come from the mesh's mesh_eval (6 tests, 4 deterministic + throughput) + hermes_loop_eval (5 agent scenarios) + a ctx_scaling test at 4 K → 32 K (the 64 K and 128 K ctx requests returned HTTP 400 from this server config — see "What's NOT in this repo"). That's a regression suite, not a quality benchmark.
Sample sizes are small. Throughput numbers are 3 reps on a single GPU; hermes_loop is 5 scenarios with one-shot generation. None are powered for statistical significance on a per-token level.
No perplexity / wikitext / MMLU / GSM8K. The mesh's stack isn't a quality benchmark — those are upstream ROCmFPX's territory. If you need a quality signal, charlie's own validation ladder or an lm-eval-harness run is the right tool.
Single GPU class. All measurements are on a 16 GB RDNA4 (RX 9060 XT, gfx1200). No Strix unified-memory, no CDNA, no multi-GPU, no Vulkan, no CUDA. Cross-hardware generalization is not implied.
No human eval. "Faster and same-coherent on the regression tests" is not a quality verdict on this specific quant.

What this IS good for: a quick signal that the quant (a) loads, (b) runs at sane throughput, (c) doesn't break the mesh's agent tool-calling, (d) scales predictably with context. What this is NOT good for: claiming "this is the best quant of this model," reproducing academic benchmark results, or substituting for upstream's validation work.

For a rigorous view, the parent repo tvall43/Qwen3.6-35B-A3B-Heretic and the upstream Qwen3.6 GGUF variants (e.g. on bartowski/) are the place to look.

What we measured

Hardware: Node B, AMD Ryzen 9 5900XT 16-core, Radeon RX 9060 XT 16 GB (gfx1200), ROCm 7.2.3, NixOS 25.11 Software: charlie12345/ROCmFPX main @ 11d76c2 Source GGUF: Qwen3.6-14B-A3B-vibetuned-F16.gguf (F16, 26 GB) from the vibetuned author's published artifact Same-stack comparison: Q6_0_ROCMFPX (6-bit ROCmFPX, 11 GB file) on the same source

Agent-loop throughput — STRIX_LEAN vs Q6_0_ROCMFPX (hermes_loop, same harness, same source)

Scenario	STRIX_LEAN (t/s)	Q6_0_ROCMFPX (t/s)	Δ
`single` (one tool call)	45.3	21.8	+108 %
`chained` (calc → use result)	43.6	25.5	+71 %
`multi_step` (compare 2 cities)	47.9	30.0	+60 %
`search` (web search + extract)	42.6	28.9	+47 %
`error_recovery` (file not found)	36.7	25.8	+42 %
Mean	43.2	26.4	+64 %

Both quants pass all 5 scenarios. The 4-bit STRIX_LEAN is 2-2.5× faster than the 6-bit Q6_0 on this MoE arch, and 36 % smaller on disk (7.0 GB vs 11 GB). This is the headline finding for this model.

mesh_eval (raw JSON: `raw-mesh-eval-vibetuned-14b-strix-lean.json`)

Test	Result
`gibberish`	OK
`thinking_leak`	CLEAN
`tool_calling` (single call)	PASS — `get_weather(location=Tokyo)`
`coding` (merge_sorted_lists)	PASS — runs, tests pass
`uncensored`	PASS — no refusal
`throughput` (3×256-token gen)	63.7 t/s mean, ±0.3 stdev
`overall_status`	PASS, 4/4

hermes_loop (raw JSON: `raw-hermes-loop-vibetuned-14b-strix-lean.json`)

Scenario	Result
`single`	PASS — final answer correct
`chained` (calc → use)	PASS — `15 × 37 = 555`
`multi_step` (compare 2 cities)	PASS — table + conclusion
`search` (web search + extract)	PASS — Eiffel Tower height (3 turns, MAXED)
`error_recovery` (file not found)	PASS (clean — unlike Ornith's PARTIAL)
`overall_status`	PASS, 5/5

Context scaling (raw JSONs: `ctx-scaling-vibetuned-strix-lean-64k-.json`, `ctx-scaling-vibetuned-strix-lean-128k-.json`)

Ctx target	pp t/s	tg t/s	Result
4 K	1993	50.0	OK
32 K	1228	50.0	OK
64 K	1124	50.0	OK (128K-cap test)
128 K	—	—	HTTP 400 (server-side ctx cap, not a quant defect)

Findings:

Decode throughput holds at 50 t/s across 4 K → 64 K ctx (single-batch). The 4-tick spread vs Ornith's 48 t/s is within harness noise.
Prompt-processing decline from 4 K → 32 K is ~38 %, consistent with KV-cache pressure scaling.
This server's 128 K cap is a config limit, not a model limit — the parent Qwen3.6 35B-A3B has 256 K native ctx, and the smaller 14B-A3B variant should fit 128 K on a 24+ GB card.

KV cache type sweep (extrapolated from Ornith, head_dim=128)

The mesh's KV-type sweep was run on Ornith 9B (also head_dim=128). The recommendation is the same: turbo4 is the production default for any head_dim=128 model in the ROCmFPX build. See the Ornith 9B ROCmFPX STRIX_LEAN repo for the full sweep data.

Quick start

# Build llama.cpp with ROCmFPX
git clone https://github.com/charlie12345/ROCmFPX
cd ROCmFPX
cmake -S . -B build -DGGML_HIP=ON -DGGML_VULKAN=OFF -DGGML_CUDA=OFF \
  -DCMAKE_HIP_ARCHITECTURES=gfx1200 ...
cmake --build build --target llama-server llama-cli llama-quantize

# Serve (131 072 ctx, turbo4 KV for head_dim=128, fa=on)
./build/bin/llama-server \
  -m Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN.gguf \
  -np 1 -c 131072 \
  -ctk turbo4 -ctv turbo4 \
  -kvo -cram 32768 -fa on

Reproduce the quant

SRC=/path/to/Qwen3.6-14B-A3B-vibetuned-F16.gguf

~/ROCmFPX/build-rdna4/bin/llama-quantize \
  "$SRC" \
  Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

Quantize time: ~2-4 min warm-cache, CPU-only.

Files in this repo

File	What it is
`Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN.gguf`	The quant. Load only with a ROCmFPX `llama-server`.
`README.md`	This file
`raw-mesh-eval-vibetuned-14b-strix-lean.json`	`mesh_eval.py` output (2026-06-27 17:58 UTC)
`raw-hermes-loop-vibetuned-14b-strix-lean.json`	`hermes_loop_eval.py` output (2026-06-27 18:09 UTC)
`raw-hermes-loop-vibetuned-14b-q6_0_rocmfpx.json`	Same harness on the Q6_0 baseline (for the throughput comparison)
`ctx-scaling-vibetuned-strix-lean-64k-20260627-143142.json`	4 K → 32 K ctx scaling
`ctx-scaling-vibetuned-strix-lean-128k-20260627-143338.json`	4 K → 64 K ctx scaling (128 K HTTP 400 — see caveat)
`quant-command.sh`	The exact `llama-quantize` invocation used

What's NOT in this repo (caveats)

Stock llama.cpp will not load this file. The ROCmFP4 weight format is unique to charlie12345/ROCmFPX.
No CUDA / non-AMD GPU bench. All measurements are RDNA4 (gfx1200). Vulkan path on RDNA4 has a known upstream regression (charlie12345/rocmfp4-llama issue #6) — we did not test it.
128 K ctx is HTTP 400 on this server. The parent Qwen3.6 35B-A3B has 256 K native ctx; the 14B-A3B vibetuned variant should fit 128 K on a 24+ GB card. We tested up to 64 K successfully; the 128 K failure is the server's --ctx-size cap, not a model limit.
The vibetuned author is tvall43 (per the GGUF metadata general.base_model.0.organization); the Qwen3.6-14B-A3B-vibetuned variant is their pruned/quantized derivative of Qwen3.6-35B-A3B-Heretic. We did not run the source F16 ourselves; we used the author's published artifact.
No MTP / speculative-decode bench on this file. The mesh's MTP head work was done on a different Qwen3.6 family model. MTP draft heads for this variant are not packaged.
No vision/multimodal test. This variant is text-only.
KV cache sweep was run on Ornith 9B (same head_dim=128, same arch family), not on this file directly. The recommendation transfers but the specific VRAM/t-s numbers are from Ornith.

Provenance

Source model: tvall43/Qwen3.6-35B-A3B-Heretic — the 14B-A3B vibetuned variant is derived from this 35B MoE
Source model license: apache-2.0
Quantizer: charlie12345/ROCmFPX main @ 11d76c2 (2026-06-27)
Quantizer license: MIT
Build hardware: Node B, AMD Ryzen 9 5900XT 16-core, Radeon RX 9060 XT 16 GB (gfx1200), ROCm 7.2.3, NixOS 25.11
Build tooling: NixOS 25.11, ROCm store paths dynamic-discovered. See the meshina repo's references/nixos-rocm-external-build-recipe.md for the build env setup.
Bench harnesses: scripts/mesh-bench/mesh_eval.py + scripts/mesh-bench/hermes_loop_eval.py + scripts/mesh-bench/ctx_scaling_bench.py from the meshina repo (private)
Original bench report: raw/benchmarks/2026-06-27-rocmfpx-validation/briefs/2026-06-27-rocmfpx-rdna4-16gb.md in the meshina repo

License

The Qwen3.6 vibetuned parent is apache-2.0 (per its HF model card).
The charlie12345/ROCmFPX quantizer is MIT.
The GGUF in this repo is a derivative of the apache-2.0 parent, produced with the MIT-licensed quantizer. Both upstream licenses are preserved.

Downloads last month: 652

GGUF

Model size

14B params

Architecture

qwen35moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

ROCmFPX Quants (various quants, not heavily tested)

Collection

Series of various model quants using: https://github.com/charlie12345/ROCmFPX https://github.com/charlie12345/rocmfp4-llama • 5 items • Updated 28 days ago

Qwen3.6-14B-A3B-vibetuned ROCmFPX STRIX_LEAN — GGUF

Scope of these benchmarks — read this first

What we measured

Agent-loop throughput — STRIX_LEAN vs Q6_0_ROCMFPX (hermes_loop, same harness, same source)

mesh_eval (raw JSON: raw-mesh-eval-vibetuned-14b-strix-lean.json)

hermes_loop (raw JSON: raw-hermes-loop-vibetuned-14b-strix-lean.json)

Context scaling (raw JSONs: ctx-scaling-vibetuned-strix-lean-64k-*.json, ctx-scaling-vibetuned-strix-lean-128k-*.json)

KV cache type sweep (extrapolated from Ornith, head_dim=128)

Quick start

Reproduce the quant

Files in this repo

What's NOT in this repo (caveats)

Provenance

License

Collection including maczzinatui/Qwen3.6-14B-A3B-vibetuned-ROCmFPX-STRIX_LEAN-GGUF

mesh_eval (raw JSON: `raw-mesh-eval-vibetuned-14b-strix-lean.json`)

hermes_loop (raw JSON: `raw-hermes-loop-vibetuned-14b-strix-lean.json`)

Context scaling (raw JSONs: `ctx-scaling-vibetuned-strix-lean-64k-.json`, `ctx-scaling-vibetuned-strix-lean-128k-.json`)