---
license: gemma
base_model: Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-GGUF
library_name: peft
pipeline_tag: text-generation
language:
- en
- ru
tags:
- lora
- roleplay
- sillytavern
- english
- russian
- gemma4
- moe
- conversational
---

# Dark-Goetia-26B-A4B-LoRA-v2

Atmospheric genre LoRA adapter primarily for English-language roleplay (also works in
Russian). Adds a darker, more literary tone to the narrative. Designed exclusively to
adjust style and response structure in Dark Fantasy RP. The adapter contains no plots or
characters from the training data.

**This is the general-purpose release.** Two experimental variants exist that split this
adapter's target set in half — see [Related adapters](#related-adapters) at the bottom.

## Adapter versions

- `goetia-book-lora-v2-main` — 2 epochs → use the recommended scales below
- `goetia-book-lora-v2-chk177` — 1 epoch → use **1.5×** the scale from the table

Example: if you run main at 0.55, run chk177 at 1.1.

## Recommended scale settings (main ver.)

Test setup: SillyTavern + [Marinara's preset](https://github.com/SpicyMarinara/SillyTavern-Settings/blob/main/Marinara%27s%20Essentials/Preset/Marinara%27s%20Spaghetti%20Recipe.json) + 2000+ token character card.

**The right ceiling depends on what your card asks for.** Free-form prose RP tolerates
much more than a card that demands structured output blocks (status trackers, World State
sections, inner-thought blocks). Both regimes below are measured, not guessed.

### Free-form prose RP — no strict formatting requirements

| Scale | Effect |
| --- | --- |
| 0.1–0.2 | Effect barely noticeable |
| 0.2–0.3 | Minimal / partial influence |
| **0.3–0.55** | **Recommended** — stable, controlled style |
| 0.55+ | Very strong influence, can start overpowering the base model |

Performs reliably in all RP scenarios at scales above 0.5. The darker literary tone
becomes most pronounced in Dark Fantasy settings. Best used in full roleplay sessions
(character card + system prompt); isolated short prompts show a weaker effect.

### Cards with strict structured output

If your card requires the model to emit formatted blocks every turn, the ceiling is
considerably lower, and it differs by language:

| Language | Safe | First observed failure |
| --- | --- | --- |
| English | up to **0.40** | above 0.43 (estimated) |
| Russian | up to **0.37** | 0.40 |

Above these, the model keeps the style but starts silently dropping the formatted blocks.
Russian is measurably more fragile than English at identical adapter strength — if you
run bilingual sessions, tune to the Russian number.

**Note on scaling:** `alpha/r = 2` is stored in the GGUF metadata, and llama.cpp multiplies
by it *on top of* your scale. All numbers above assume a loader that respects that. If
yours ignores the metadata, halve everything.

## What's actually in the adapter

For anyone merging this or building on it — the trained delta isn't distributed evenly
across attention. Frobenius norms of the effective delta `ΔW = (B·A)·(alpha/r)`, aggregated
over layers:

| Projection | ‖ΔW‖ |
| --- | --- |
| `o_proj` | 1.563 |
| `q_proj` | 1.314 |
| `v_proj` | 1.049 |
| `k_proj` | 0.828 |
| **OV total** | **1.882** |
| **QK total** | **1.553** |
| Full delta | 2.440 |

So roughly **61% OV / 39% QK** by squared norm. That split turns out to matter: the OV
half carries almost all of the style, while both halves independently contribute to
formatting degradation — OV about 3× more readily per unit of weight change. The
structured-output ceilings in the table above are set by the OV component alone.

## Training

- **Base:** Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA (Gemma 4 MoE, abliterated)
- **Method:** QLoRA 4-bit, attention-only (`q/k/v/o_proj`), all 30 layers of the text tower,
  r=32, alpha=64, 2 epochs, completion-only loss
- **Scope:** **115** attention projections, not 120 — layers 5, 11, 17, 23 and 29 are
  global-attention layers with no `v_proj` at all. 22,978,560 trainable params (0.089%).
- **Data:** Novel broken down into scenes → scenes rewritten by a neural network into a
  synthetic RP dataset (actions and internal thoughts preserved, original author's prose
  style removed). Contains both English and Russian samples, English-primary.

## Limitations

- **18+.** Base model is abliterated, training data includes adult and dark content.
- Primarily optimized for English. Works in Russian, but the style effect is generally
  weaker *and* instruction-following is more fragile there — see the scale tables.
- Structured-output thresholds were measured with one generation per setting at a single
  context depth. Treat them as calibrated starting points, not hard constants.
- Inherits Gemma license terms.

## Related adapters

v3 splits this adapter's 115 targets into two non-overlapping halves, trained separately
on identical data with identical hyperparameters and seed:

- [**v3-A**](https://huggingface.co/SubMaroon/Dark-Goetia-26B-A4B-LoRA-v3-A) — `v_proj` + `o_proj`
  (55 modules). The style channel on its own, without the QK component. Experimental;
  useful if your card demands strict formatting and v2 keeps dropping it.
- [**v3-B**](https://huggingface.co/SubMaroon/Dark-Goetia-26B-A4B-LoRA-v3-B) — `q_proj` + `k_proj`
  (60 modules). Research artifact — weak style, published for reproducibility.

A + B = exactly this adapter's target set and parameter budget.