---
license: openmdw-1.1
base_model:
- poolside/Laguna-S-2.1
base_model_relation: quantized
language:
- en
library_name: llama.cpp
pipeline_tag: text-generation
tags:
- laguna-s-2.1
- poolside
- agentic-coding
- tool-use
- gguf
- rocm
- rocmfp4
- rocmfpx
- chadrock
- amd
- ryzen-ai-max-395
- strix-halo
- long-context
---
# Laguna S 2.1 118B Chadrock ROCmFP4 StrixKVSpine V4 — Runtime V3
An AMD-optimized, quality-protected ROCmFP4 quant of
[Poolside Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1), built
for local agentic coding on Ryzen AI Max+ 395 / Radeon 8060S Strix Halo.
This V4 recipe fits a 118B-total-parameter, approximately 8B-active model into
a **60.945 GiB GGUF at 4.453 effective BPW**, while retaining a tested
**131,072-token safe serving profile**. It is **12.95% smaller** than Poolside's
official Q4_K_M GGUF and reached **35.62 tokens/second** during the complete
128K V2 stability gate retained by Runtime V3.
> [!IMPORTANT]
> This GGUF uses ROCmFP4 tensor types and Laguna architecture support. It is
> built for the Laguna-enabled
> [Ciru ROCmFPX Runtime V3](https://github.com/ciru-ai/ROCmFPX/tree/54f5fe06c74350fb8b6aec21d8749071bc195bdb)
> at commit `54f5fe06c74350fb8b6aec21d8749071bc195bdb`. It does not load in
> stock upstream llama.cpp.
>
> Runtime V3 fixes Laguna tool-call grammar/parser agreement and removes a
> non-portable chat-template extension while retaining the V2 Vulkan stability
> safeguards and safe serving defaults. The V4 GGUF weights are unchanged;
> existing users do not need to download the 60.945 GiB model again.
## At a glance
| Property | Value |
|---|---:|
| Base model | Poolside Laguna S 2.1 |
| Architecture | 118B total / approximately 8B active MoE |
| Artifact | `laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf` |
| File size | 65,438,991,968 bytes / 60.945 GiB |
| Effective quantization density | 4.453 BPW |
| Runtime release | **V3** |
| Validated serving context | **131,072 tokens**; full-depth gate completed on V2 and retained by V3 |
| Model context capacity | 262,144 tokens; the 256K lane is experimental |
| Complete 128K V2 stability gate | 195.70 PP / 35.62 TG tok/s |
| Tested generation speed | 35.62 tok/s during the 128K gate |
| Tested mixed speed | 82.953 tok/s, PG512 + TG256 |
| Primary target | AMD Ryzen AI Max+ 395 / Radeon 8060S |
| KV cache in tested profile | F16 K / F16 V |
| Default reasoning mode | Off |
## Runtime V3 patch notes
Runtime V3 adds parser and grammar fixes on top of the V2 Vulkan runtime:
- ports [llama.cpp PR #24835](https://github.com/ggml-org/llama.cpp/pull/24835)
so generated JSON values no longer carry trailing grammar whitespace that
the final PEG tool parser rejects;
- aligns PEG JSON-array comma whitespace handling with the generated grammar;
- adds a Laguna/Pi regression for an `edit` call containing an
`array` argument and source-code strings;
- keeps server checkpoints host-backed, preventing the on-device checkpoint
destruction fault seen when a long hybrid/SWA context is invalidated;
- explicitly disables context checkpoints in the validated Laguna launcher
while leaving the normal KV and prompt caches available;
- retains the V2 RADV DeviceLost safeguards and validated 128K defaults.
Focused V3 validation on Ryzen AI Max+ 395 / Radeon 8060S with Mesa RADV
26.1.2 passed JSON-schema grammar conversion, automatic parser selection, the
nested Pi edit-call regression, and the Laguna architecture test. Real-model
smokes returned `Paris.`, preserved the nested `edit` tool call, completed an
8,061-token functional smoke with the expected `omega` response, and released
the slot after cancellation at 82% of a 15,000-token request. A follow-up
request returned `Paris.` with no `VK_ERROR_DEVICE_LOST`.
The checkpoint repair was exercised separately with checkpoints explicitly
re-enabled and prompt-cache RAM left at 8192 MiB. A 120,045-token request
created 16 hybrid/SWA checkpoints; an unrelated follow-up forced `pos_next=0`
and erased all 16, then returned `OK.`. The server remained healthy and shut
down cleanly without a core dump. Checkpoints nevertheless remain disabled in
the public profile until this gate is repeated across multi-turn workloads.
### V2 Vulkan stability baseline
The first runtime release could lose the Vulkan device during a very deep
Flash Attention prefill on RADV/Strix Halo. Lowering the graph-node submission
ceiling was not enough: matched 100-node and 10-node controls both reached an
AMD compute-ring timeout after approximately 77–78 minutes.
V2 fixes the operator-level problem by splitting a large Flash Attention X
grid into shorter Vulkan dispatch commands while preserving global workgroup
IDs and output offsets.
| Serving behavior | First release | Runtime V2 |
|---|---|---|
| Default context | 262,144 | **131,072 validated safe lane** |
| Ubatch | 512 | 512 |
| Graph nodes per submit | 100 | 10 |
| FA workgroups per dispatch | Unbounded | 4 |
| Submission sizing | Tensor-byte heuristic | FLOP-aware heuristic |
| DeviceLost handling | Secondary exceptions possible | Sticky fatal latch and bounded teardown |
| Diagnostics | Manual | Automatic kernel, Vulkan, service, and devcoredump bundle |
| Restart behavior | Unbounded/external | Driver preflight and persisted bounded backoff |
| 256K status | Advertised as tested | Experimental pending a full-depth gate |
V2 validation on Ryzen AI Max+ 395 / Radeon 8060S with Mesa RADV 26.1.2:
| Gate | Prompt processing | Generation | Result |
|---|---:|---:|---|
| 8K, three matched passes | **352.38 tok/s** | 35.64 tok/s | Pass |
| 64K, one complete prefill | **267.27 tok/s** | 35.63 tok/s | Pass |
| 128K, one complete prefill | **195.70 tok/s** | 35.62 tok/s | Pass |
The 8K V2 row improved prompt processing by **10.57%** over the matched
unsplit 10-node control (318.70 tok/s), with effectively unchanged generation
speed. Deterministic split and unsplit test generations were byte-identical
after removing their timing lines.
Runtime V2 also adds:
- `GGML_VK_FA_MAX_WORKGROUPS_X_PER_DISPATCH`;
- `GGML_VK_MAX_NODES_PER_SUBMIT`;
- first-failure graph node/operator context;
- no new Vulkan submissions or failed fence waits after DeviceLost;
- portable crash collection and a supervised launcher;
- an explicit warning when selecting the experimental 256K lane.
## Why this release
The goal was not simply to make Laguna smaller. StrixKVSpine V4 protects the
tensors that were most sensitive in our Laguna experiments while using the
fast ROCmFP4 path where it delivered the best memory and throughput return:
- attention K/V, attention gates, dense block 0, shared experts, and a
nine-layer expert-down spine remain protected;
- attention Q/O and non-spine packed experts use the fast ROCmFP4 path;
- the output tensor remains Q6_K;
- F16/F16 KV cache is retained for the validated 128K profile.
The resulting model is **9.066 GiB smaller than the official Poolside Q4_K_M**
while matching or improving that baseline on most of the retained quality
checks.
## Results against Poolside Q4_K_M
These are direct local comparisons against Poolside's official
`laguna-s-2.1-Q4_K_M.gguf`, using the same benchmark tasks. Scores are reported
individually rather than blended into a synthetic aggregate.
| Evaluation | Chadrock ROCmFP4 V4 | Poolside Q4_K_M | Difference |
|---|---:|---:|---:|
| Tool-Eval disputed-19, 3 passes | **80/114 (70.18%)** | 62/114 (54.39%) | **+18 accepted calls / +15.79 pp** |
| HumanEval pass@1 | 155/164 (94.51%) | 155/164 (94.51%) | Tied |
| HumanEval+ pass@1 | **149/164 (90.85%)** | 147/164 (89.63%) | **+2 tasks / +1.22 pp** |
| HermesAgent-20 | **77/100** | 71/100 | **+6 points** |
| BigCodeBench Hard, official | 37/148 (25.00%) | **39/148 (26.35%)** | -2 tasks / -1.35 pp |
The hero's rounded quality figure is the matched Tool-Eval result:
**80 accepted calls versus 62, a 29.0% increase**.
### BigCodeBench follow-up
The official V4 BigCodeBench run used strict greedy decoding and scored 37/148,
with seven length-capped repetition loops. Under the release sampler, all seven
completed naturally and two additional tasks passed. The resulting
sampler-corrected diagnostic is **39/148**, tied with Q4_K_M. The table retains
the official 37/148 score.
On the same 148 BigCodeBench prompts, V4 measured:
| Per-token metric | Chadrock ROCmFP4 V4 | Poolside Q4_K_M | V4 difference |
|---|---:|---:|---:|
| Generation throughput | **30.932 tok/s** | 22.201 tok/s | **+39.33%** |
| Incremental prompt throughput | **199.421 tok/s** | 159.951 tok/s | **+24.68%** |
The table reports per-token throughput. The greedy run generated more than
twice as many completion tokens because of the seven loops, so end-to-end wall
time from that run is not used as the speed headline.
## Recommended serving profile
### Linux support
The runtime builds natively on Linux x86-64. NixOS is the currently validated
production build environment; Ubuntu 24.04 LTS and Debian 12+ are the primary
documented user path. The repository also provides native dependency paths for
Fedora/Rocky/AlmaLinux and Arch/Manjaro.
| Linux family | Package manager | Status |
|---|---|---|
| Ubuntu 24.04 LTS / Debian 12+ | `apt` | Primary install path |
| Fedora 42+ / Rocky / AlmaLinux | `dnf` | Supported build path |
| Arch / Manjaro / EndeavourOS | `pacman` | Supported build path |
| NixOS | Nix | Production build validated |
Clone and pin Runtime V3 exactly:
```bash
git clone --branch agent/laguna-s21-runtime-v3 --depth 1 \
https://github.com/ciru-ai/ROCmFPX.git
cd ROCmFPX
git checkout --detach 54f5fe06c74350fb8b6aec21d8749071bc195bdb
test "$(git rev-parse HEAD)" = \
"54f5fe06c74350fb8b6aec21d8749071bc195bdb"
```
The distro-aware helper prints the native package command before making any
change:
```bash
scripts/install-laguna-vulkan-deps.sh
scripts/install-laguna-vulkan-deps.sh --install
```
Ubuntu and Debian users can install directly:
```bash
sudo apt-get update
sudo apt-get install -y \
git cmake ninja-build build-essential glslc \
libvulkan-dev vulkan-tools spirv-headers mesa-vulkan-drivers
```
Fedora, Rocky Linux, and AlmaLinux
```bash
sudo dnf install -y \
git cmake ninja-build gcc gcc-c++ glslc \
vulkan-loader-devel vulkan-headers spirv-headers \
vulkan-tools mesa-vulkan-drivers
```
Arch, Manjaro, and EndeavourOS
```bash
sudo pacman -S --needed \
git cmake ninja base-devel shaderc \
vulkan-icd-loader vulkan-headers spirv-headers \
vulkan-tools vulkan-radeon
```
NixOS
```bash
nix --extra-experimental-features 'nix-command flakes' profile add \
nixpkgs#git nixpkgs#cmake nixpkgs#ninja nixpkgs#gcc \
nixpkgs#shaderc nixpkgs#vulkan-headers nixpkgs#vulkan-loader \
nixpkgs#spirv-headers
```
Then verify Vulkan and build the pinned Release runtime:
```bash
vulkaninfo --summary
JOBS=8 BUILD_TYPE=Release scripts/build-laguna-strix-vulkan.sh
```
This produces a static Vulkan build with `llama-server`, `llama-cli`,
`llama-bench`, and `llama-quantize`.
Run the release checks from the repository root:
```bash
build-laguna-strix-vulkan/bin/test-json-schema-to-grammar
build-laguna-strix-vulkan/bin/test-chat-auto-parser
build-laguna-strix-vulkan/bin/test-chat \
--template poolside-Laguna-S-2.1.jinja
build-laguna-strix-vulkan/bin/test-llama-archs
```
The
[complete Linux and V3 guide](https://github.com/ciru-ai/ROCmFPX/blob/54f5fe06c74350fb8b6aec21d8749071bc195bdb/docs/recipes/laguna-s21-chadrock-rocmfp4-strixkvspine-v4.md)
contains the Fedora, Arch, and NixOS commands.
### Start Laguna with the validated 128K V3 profile
The supervised launcher is recommended on RADV. It runs the driver preflight,
uses the safe V3 settings, and preserves DeviceLost evidence:
```bash
scripts/run-laguna-vulkan-supervised.sh \
/path/to/laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf
```
The launcher applies the measured single-slot Strix Halo configuration:
Vulkan0, full offload, row split, Flash Attention, 131,072 context, F16/F16
KV, batch 2048, ubatch 512, node cap 10, FA dispatch width 4, 16 threads,
thinking off, context checkpoints disabled, and this sampler:
```json
{
"temperature": 1.0,
"top_p": 1.0,
"top_k": 20,
"min_p": 0.0,
"repeat_penalty": 1.0,
"seed": 42
}
```
The server listens on `127.0.0.1:8080`. Check it with:
```bash
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models
```
The direct runner uses the same V3 safe defaults without supervision:
```bash
scripts/run-laguna-s21-rocmfp4-v4.sh /path/to/model.gguf
```
Context checkpoints are separate from the normal KV cache. The launcher now
passes `--ctx-checkpoints 0` because the hybrid/SWA checkpoint path has not yet
completed the full repeated 128K multi-turn and cache-replay qualification.
The runtime also moves checkpoint payloads back to host memory, matching the
safer upstream design. Advanced diagnostic runs can opt in with
`CTX_CHECKPOINTS=N`; this is not part of the validated public profile yet.
The model's 256K capacity remains available only as an explicit experimental
lane:
```bash
STABILITY_MODE=performance \
scripts/run-laguna-s21-rocmfp4-v4.sh /path/to/model.gguf
```
That command prints a warning because 256K has not yet passed the
full-depth prefill, multi-turn, and cache-replay gates.
The complete production recipe is preserved in the
[ROCmFPX Laguna Runtime V3 guide](https://github.com/ciru-ai/ROCmFPX/blob/54f5fe06c74350fb8b6aec21d8749071bc195bdb/docs/recipes/laguna-s21-chadrock-rocmfp4-strixkvspine-v4.md).
## Example request
After starting the compatible server:
```bash
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "laguna-s21-rocmfp4-strixkvspine-v4",
"messages": [
{
"role": "user",
"content": "Refactor this Python API client to add bounded retries, typed errors, and tests."
}
],
"temperature": 1.0,
"top_p": 1.0,
"top_k": 20,
"min_p": 0.0,
"seed": 42
}'
```
## Artifact integrity
| File | Size | SHA-256 |
|---|---:|---|
| `laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf` | 65,438,991,968 bytes | `ea1d854a72c47ec8e72c16ea91b8ff3cd5e1620b834df175f683c86f27dc26d6` |
## Credits
### Charlie / `charlie12345` / `caf`
Enormous thanks to
[Charlie (`charlie12345`)](https://github.com/charlie12345) for the amazing
ROCmFP4 codebook and the experimental
[ROCmFPX](https://github.com/charlie12345/ROCmFPX) work that made this release
possible. **We could not have built this release without him.**
Please support and credit his work when building on ROCmFP4 or ROCmFPX.
### Poolside
Thank you to [Poolside](https://huggingface.co/poolside) for creating and
releasing the remarkable
[Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1) model and its
[official GGUF collection](https://huggingface.co/poolside/Laguna-S-2.1-GGUF).
Laguna is the foundation of everything here; this release is a quantized,
hardware-targeted derivative, not a new base model.
### Ciru / Chadrock
Ciru developed the Laguna-specific StrixKVSpine tensor-protection recipe,
performed the calibration and quantization, built the Strix Halo Runtime V2
stability baseline and Runtime V3 parser/template layer, and ran the retained
quality, performance, deep-context, tool-call, and cancellation validation.
## License and use
This derivative follows the base model's
[OpenMDW 1.1 license](https://huggingface.co/poolside/Laguna-S-2.1/blob/main/LICENSE)
and Poolside's published model terms. Review the base model card, license, and
acceptable-use requirements before deployment.
Benchmark results describe this exact file, runtime, hardware, and sampler
configuration. Performance and memory behavior will vary across drivers,
backends, hardware, context lengths, and workload shapes.
