--- license: apache-2.0 base_model: Vortex5/Shadow-Siren-26B-A4B pipeline_tag: text-generation language: [en] tags: [gguf, gemma4, mergekit, imatrix, roleplay, storytelling, llama.cpp] --- # Shadow-Siren-26B-A4B — BF16 & UD-Q4_K GGUFs GGUF conversions of [Vortex5/Shadow-Siren-26B-A4B](https://huggingface.co/Vortex5/Shadow-Siren-26B-A4B), published by **KaviarLabs**. This repository contains a full BF16 GGUF plus three custom Unsloth-style variants: `UD-Q4_K_S-i1`, `UD-Q4_K_M-i1`, and `UD-Q4_K_XL-i1`. Each quant uses the **exact tensor-by-tensor qtype allocation** extracted from the corresponding Unsloth Gemma 4 26B-A4B GGUF together with the **Shadow-Siren-specific i1 importance matrix** published by mradermacher. > These are **not official Unsloth quantizations**. The `UD-*` names describe reproduced tensor precision maps; `i1` identifies the Shadow-Siren imatrix source. The quantized weights are from Shadow-Siren. ## Files | File | Description | Size | |---|---|---:| | `Shadow-Siren-26B-A4B-BF16.gguf` | BF16 converted directly from source Safetensors | ~47.04 GiB | | `Shadow-Siren-26B-A4B-UD-Q4_K_S-i1.gguf` | UD-Q4_K_S map + Shadow-Siren i1 imatrix | 15.355 GiB | | `Shadow-Siren-26B-A4B-UD-Q4_K_M-i1.gguf` | UD-Q4_K_M map + Shadow-Siren i1 imatrix | 15.784 GiB | | `Shadow-Siren-26B-A4B-UD-Q4_K_XL-i1.gguf` | UD-Q4_K_XL map + Shadow-Siren i1 imatrix | 15.843 GiB | No `mmproj` is included. Text generation works directly; vision use requires a compatible Gemma 4 projector separately. ## Source model & upstream credits Shadow-Siren was created by **Vortex5** as a multi-stage merge for roleplay, creative writing, storytelling and conversational use. Its published recipe uses **SAEF → Karcher → HCR** and combines: - [zerofata/G4-MeroMero-26B-A4B](https://huggingface.co/zerofata/G4-MeroMero-26B-A4B) - [ReadyArt/Serenity-26B-A4B](https://huggingface.co/ReadyArt/Serenity-26B-A4B) - [google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it) - [Gryphe/Pantheon-Reasoning-26B-A4B-1.1](https://huggingface.co/Gryphe/Pantheon-Reasoning-26B-A4B-1.1) - [ReadyArt/Dark-Scarlett-v1.0-26B-A4B](https://huggingface.co/ReadyArt/Dark-Scarlett-v1.0-26B-A4B) - [Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT](https://huggingface.co/Darkhn/Gemma-4-26B-A4B-Animus-V14.1-FFT) See the [original Shadow-Siren model card](https://huggingface.co/Vortex5/Shadow-Siren-26B-A4B) for the authoritative merge recipe. ## BF16 conversion The source Safetensors were converted directly with mainline [llama.cpp](https://github.com/ggml-org/llama.cpp) using `convert_hf_to_gguf.py --outtype bf16`. The output was validated as Gemma 4 GGUF V3 with **658 tensors**, expected F32/BF16 source types, and the source chat template preserved. Quantizer build used later: ```text llama.cpp build 10335 (74ce15741) MSVC 19.51.36252.0 / Windows AMD64 ``` ## Importance matrix The quant uses `Shadow-Siren-26B-A4B.imatrix.gguf` from [mradermacher/Shadow-Siren-26B-A4B-i1-GGUF](https://huggingface.co/mradermacher/Shadow-Siren-26B-A4B-i1-GGUF). This imatrix was published specifically for Shadow-Siren; `llama-quantize` loaded **295 importance-matrix entries**. ## UD tensor maps The qtype allocations were extracted tensor-by-tensor from the corresponding files in [Unsloth's Gemma 4 26B-A4B GGUF repository](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF). Shadow-Siren retains the same 658-tensor Gemma 4 26B-A4B topology, allowing each map to be transferred name-for-name and shape-for-shape. | Variant | F32 | Q4_K | Q5_0 | Q5_1 | Q5_K | Q8_0 | Dry-run | Final size | |---|---:|---:|---:|---:|---:|---:|---:|---:| | `UD-Q4_K_S-i1` | 392 | 30 | 29 | 0 | 0 | 207 | 5.22 BPW | 15.355 GiB | | `UD-Q4_K_M-i1` | 392 | 30 | 0 | 29 | 0 | 207 | 5.37 BPW | 15.784 GiB | | `UD-Q4_K_XL-i1` | 392 | 29 | 0 | 29 | 1 | 207 | 5.39 BPW | 15.843 GiB | Notable differences: - **S:** blocks 0–28 use Q5_0 for `ffn_down_exps.weight`; all 30 `ffn_gate_up_exps.weight` tensors are Q4_K. - **M:** blocks 0–28 use Q5_1 for `ffn_down_exps.weight`; all 30 `ffn_gate_up_exps.weight` tensors are Q4_K. - **XL:** blocks 0–28 use Q5_1 for `ffn_down_exps.weight`; `blk.29.ffn_gate_up_exps.weight` is promoted to Q5_K while the other 29 remain Q4_K. - All three keep `blk.29.ffn_down_exps.weight` in Q8_0 and `token_embd.weight` in Q8_0; norms/router/scales and other small tensors follow the F32 assignments from the respective source maps. ## Quantization Each variant was produced with `llama-quantize`, the Shadow-Siren i1 imatrix and a literal tensor-type file extracted from the corresponding Unsloth GGUF. Base qtypes were `Q4_K_S` for S and `Q4_K_M` for M/XL, using 16 threads. Dry-run results: ```text BF16 model size: 48150.36 MiB (16.01 BPW) UD-Q4_K_S: 15708.72 MiB (5.22 BPW) UD-Q4_K_M: 16147.34 MiB (5.37 BPW) UD-Q4_K_XL: 16207.84 MiB (5.39 BPW) ``` Final files: ```text UD-Q4_K_S-i1 16,487,610,816 bytes 15.355 GiB UD-Q4_K_M-i1 16,947,542,464 bytes 15.784 GiB UD-Q4_K_XL-i1 17,010,981,312 bytes 15.843 GiB ``` ## Validation Every final GGUF was parsed and compared against its extracted Unsloth reference map **tensor by tensor**, requiring all 658 tensor names, shapes and qtypes to match exactly. ```text UD-Q4_K_S: F32:392,Q4_K:30,Q5_0:29,Q8_0:207 exact_match=True UD-Q4_K_M: F32:392,Q4_K:30,Q5_1:29,Q8_0:207 exact_match=True UD-Q4_K_XL: F32:392,Q4_K:29,Q5_1:29,Q5_K:1,Q8_0:207 exact_match=True ``` This proves exact reproduction of each **qtype allocation map**, not bit-identical weights with Unsloth's original files: the weights are Shadow-Siren and the imatrix is mradermacher's Shadow-Siren `i1` matrix. ## Usage ```bash llama-server -m Shadow-Siren-26B-A4B-UD-Q4_K_XL-i1.gguf ``` Compatible GGUF frontends such as LM Studio can load the file as well. ## Thanks Full credit and thanks to **Vortex5** (Shadow-Siren), **Google** (Gemma 4), **zerofata** (G4-MeroMero), **ReadyArt** (Serenity and Dark-Scarlett), **Gryphe** (Pantheon-Reasoning), **Darkhn** (Animus), **mradermacher** (Shadow-Siren i1 imatrix and GGUF work), **Unsloth / Daniel Han** (Dynamic/UD GGUF work and the referenced UD-Q4_K_XL allocation), **ggml-org / llama.cpp contributors**, and **mergekit contributors**. Please support and consult the upstream repositories. This upload is not a replacement for their original model cards, licenses or documentation. ## License The source `Vortex5/Shadow-Siren-26B-A4B` repository is published under **Apache-2.0**. Users should also review applicable upstream terms before redistribution or deployment.