SKILL.md

metadata

name: repe-refusal-steering
description: >
  Apply the RepE refusal-suppression steering vector to google/gemma-4-31B-it at
  inference time, in transformers OR vLLM. Use when the user wants to run this
  repo's steering vector, dial the refusal direction (sigma), reproduce the
  dose-response, or serve an uncensored/steered Gemma-4 for interpretability or
  safety research. Triggers: RepE, activation steering, refusal steering,
  uncensor, steering vector, dim_01_refusal_layer_032, gemma4-repe-uncensor,
  sigma dial, ControlVector.

RepE refusal steering — how to run it

This repo ships ONE steering direction (vectors/dim_01_refusal_layer_032.pt, 5376-d, unit-norm) that suppresses refusals in google/gemma-4-31B-it by adding alpha * v to the residual stream at decoder layer 32. Nothing is baked into weights — you apply it live and can dial it.

v      = bundle["vector"] / ||bundle["vector"]||
alpha  = sigma * bundle["meta"]["alpha_for_1sigma"]   # alpha_for_1sigma ≈ 21.225
h_L32 += alpha * v

sigma is the dose (subspace-σ). sigma < 0 steers away from refusal. Start at sigma = -2.0. More negative = fewer refusals but degrades coherence.

⚠️ Over-steering collapses the model. This is an unbounded additive intervention. Too large |sigma| (roughly ≳ 6, prompt/layer dependent) knocks the residual stream off-distribution → repetition / incoherent / garbage output. 0% refusal is NOT a success signal — a model that complies but emits broken text is collapsed, not steered. When you sweep sigma, always inspect the generated text, not just the refusal rate; stay near -2, step up gradually, back off when coherence drops. Stacking directions or steering multiple layers breaks it faster.

Read these gotchas BEFORE running — they are the whole game

1. vLLM: you MUST pass enforce_eager=True. Steering is a Python register_forward_hook. Under CUDA-graph capture (the default) the hook is bypassed and steering silently does nothing.
2. vLLM: install via worker_extension_cls, drive via STRING method names. The model lives in a worker process. Passing a callable to collective_rpc fails serialization (Object of type function is not serializable). Use the shipped SteerWorkerExtension and call "attach_steering" by name.
3. The package must be importable in the worker. Put the repo root on PYTHONPATH (env var, not just sys.path) before constructing LLM, so the worker subprocess can resolve eigenself_repe.vllm_steer.SteerWorkerExtension.
4. Model is gated + large (~59 GB BF16). Needs HF access to google/gemma-4-31B-it and a GPU with ~70 GB free (single card fits; gpu_memory_utilization≈0.9, max_model_len small for tests). Steering itself adds negligible memory.
5. Layer index is a flat global index. The hook auto-locates the module whose name ends in layers.32 (resolves to language_model.model.layers.32). Don't hand-thread the nesting.

vLLM (recommended for serving)

import os, sys
REPO = "/abs/path/to/gemma4-repe-uncensor"
sys.path.insert(0, REPO)
os.environ["PYTHONPATH"] = REPO + os.pathsep + os.environ.get("PYTHONPATH", "")

from vllm import LLM, SamplingParams
llm = LLM(
    model="google/gemma-4-31B-it",
    enforce_eager=True,                                             # (1)
    gpu_memory_utilization=0.9, max_model_len=2048,
    worker_extension_cls="eigenself_repe.vllm_steer.SteerWorkerExtension",  # (2)(3)
)
llm.collective_rpc("attach_steering",
                   args=(f"{REPO}/vectors/dim_01_refusal_layer_032.pt", 32, -2.0))
out = llm.chat([[{"role": "user", "content": "..."}]],
               SamplingParams(temperature=0.0, max_tokens=256), use_tqdm=False)

# live control, no reload:
llm.collective_rpc("set_steering_enabled", args=(False,))   # bypass
llm.collective_rpc("attach_steering", args=(bundle, 32, -4.0))  # re-dial sigma
llm.collective_rpc("detach_steering")

transformers (simplest to inspect)

from transformers import AutoModelForCausalLM
from eigenself_repe import TransformersSteering   # repo root on sys.path

model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-31B-it", torch_dtype="bfloat16", device_map="cuda")
steer = TransformersSteering(model, f"{REPO}/vectors/dim_01_refusal_layer_032.pt",
                             layer=32, sigma=-2.0)
# generate normally; steer.enabled = False to bypass; steer.remove() to detach

Verify it actually fired

Run the shipped harnesses (GPU, single card):

• python tests/ab_smoke.py → refusal OFF vs ON on 12 harmful prompts (paired).
• python tests/sigma_sweep.py → dose-response over sigma, one model load.

Expected shape (n=12, greedy, crude refusal-string heuristic — a mechanism check, not a benchmark): monotonic σ=0 → 100%, -2 → ~42%, -4 → ~8%, -6 → 0%. If steering is ON but the rate doesn't move, you almost certainly forgot enforce_eager=True (gotcha 1).

Coherent / gated steering

Always-on steering also fires on benign prompts. gate/ holds a refusal-routing logreg probe (meanpool over layers 32/40/44/48/52); steer only when it fires to preserve general capability. The gate is wired in the reference transformers serving path; a gated vLLM path is not shipped here yet.

Guardrails

Research artifact (interpretability / safety). Base model under the Gemma license; only the vector + gate are redistributed. Don't ship a refusal-disabled endpoint to end users.