---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
base_model: TheDrummer/Skyfall-31B-v4.1
library_name: gguf
tags:
- gguf
- quantization
- skyfall
- ablation
- research
- nlp
- 31b
- roleplay
- heretic
---

# TheDrummer-Skyfall-31B-v4.1-Heretic-Absolute

> [!CAUTION]
> **EXPERIMENTAL RESEARCH ARTIFACT**
>
> This model represents an aggressive application of the **Heretic** repository and optimization methodology.
> 
> * **Status:** STILL TESTING / BETA
> * **Behavior:** This model has zero refusal mechanisms enabled. It may exhibit "unhinged" behavior, hallucinations, or unexpected outputs depending on prompt complexity.
> * **Use Case:** This is a research artifact intended for testing the limits of vector-based intervention. Use with appropriate caution.

## Model Summary

**TheDrummer-Skyfall-31B-v4.1-Heretic-Absolute** is a fine-tuned language model resulting from the **Heretic** repository and optimization methodology. It utilizes a targeted vector intervention technique (orthogonalization/abliteration) tuned via Optuna to minimize refusal responses while maintaining varying degrees of coherence (KL Divergence).

This specific checkpoint represents **Trial 72**, which achieved a **0% Refusal Rate** with a **KL Divergence of ~0.104**, indicating high adherence to the base model's probability distribution despite the complete removal of refusal vectors.

## Run Configuration: "Trial 72"

The following parameters define the intervention vector applied to the model. This configuration was discovered during the hyperparameter search.

### Optimization Results

| Metric | Value | Description |
| :--- | :--- | :--- |
| **Refusal Rate** | **0.0%** | The model did not refuse any prompts in the Heretic test set. |
| **KL Divergence** | **0.1043** | Measures deviation from the base model's probability distribution. |
| **Trial ID** | 71 | Specific Optuna trial identifier. |
| **Direction Scope** | Per Layer | Intervention vectors were calculated and applied on a per-layer basis. |

### Intervention Parameters

Interventions were applied to two primary distinct layers: the Attention Output Projection (`attn.o_proj`) and the MLP Down Projection (`mlp.down_proj`).

| Parameter Scope | Setting | Value |
| :--- | :--- | :--- |
| **Attention Output** | `attn.o_proj.max_weight` | **1.422** |
| (`attn.o_proj`) | `attn.o_proj.max_weight_position` | **33.80** (Layer Depth) |
| | `attn.o_proj.min_weight` | **1.190** |
| | `attn.o_proj.min_weight_distance` | **22.12** |
| **MLP Down Proj** | `mlp.down_proj.max_weight` | **0.858** |
| (`mlp.down_proj`) | `mlp.down_proj.max_weight_position` | **41.51** (Layer Depth) |
| | `mlp.down_proj.min_weight` | **0.525** |
| | `mlp.down_proj.min_weight_distance` | **27.20** |

## Methodology & Definitions

To ensure uniform understanding of the Heretic run data, the following definitions apply to the parameters listed above:

* **Direction Scope**: Defines whether the refusal vector is calculated once for the entire model ("Global") or recalculated individually for each layer ("Per Layer"). "Per Layer" allows for precise removal of refusal mechanisms without damaging general knowledge.
* **Max Weight**: The maximum scaling factor applied to the intervention vector. A higher weight indicates a stronger "push" against the targeted concept (refusal) at the peak layer.
* **Max Weight Position**: The specific layer index (depth) where the intervention is strongest.
    * *Observation: The Layer 33-41 range suggests the critical refusal circuitry in this architecture resides in the middle-to-late layers.*
* **Min Weight**: The baseline scaling factor applied to the intervention vector at the periphery of the target zone.
* **Distance**: The "spread" or bandwidth of the intervention. It determines how many layers around the "Max Weight Position" are affected by the vector modification.

## Usage & Limitations

* **Intended Use:** Research into model alignment, vector arithmetic, and uninhibited creative writing.
* **Risks:** As an "Absolute" variant, this model has no safety guardrails. It will generate content for any prompt provided. It may also hallucinate or diverge from logical consistency more often than standard finetunes due to the depth of the `attn.o_proj` modification.


## Credits & References

This research builds upon the excellent work of the open-source AI community:

* **Base Model:** [Skyfall-31B-v4.1](https://huggingface.co/TheDrummer/Skyfall-31B-v4.1-GGUF) by **TheDrummer**.
* **Methodology:** [Heretic](https://github.com/p-e-w/heretic) by **p-e-w**.