Upload SKILL.md with huggingface_hub

SKILL.md

@@ -0,0 +1,111 @@
+---
+name: repe-refusal-steering
+description: >
+  Apply the RepE refusal-suppression steering vector to google/gemma-4-31B-it at
+  inference time, in transformers OR vLLM. Use when the user wants to run this
+  repo's steering vector, dial the refusal direction (sigma), reproduce the
+  dose-response, or serve an uncensored/steered Gemma-4 for interpretability or
+  safety research. Triggers: RepE, activation steering, refusal steering,
+  uncensor, steering vector, dim_01_refusal_layer_032, gemma4-repe-uncensor,
+  sigma dial, ControlVector.
+---
+
+# RepE refusal steering — how to run it
+
+This repo ships ONE steering direction (`vectors/dim_01_refusal_layer_032.pt`,
+5376-d, unit-norm) that suppresses refusals in `google/gemma-4-31B-it` by adding
+`alpha * v` to the residual stream at decoder **layer 32**. Nothing is baked into
+weights — you apply it live and can dial it.
+
+```
+v      = bundle["vector"] / ||bundle["vector"]||
+alpha  = sigma * bundle["meta"]["alpha_for_1sigma"]   # alpha_for_1sigma ≈ 21.225
+h_L32 += alpha * v
+```
+
+`sigma` is the dose (subspace-σ). `sigma < 0` steers away from refusal. Start at
+`sigma = -2.0`. More negative = fewer refusals but degrades coherence.
+
+## Read these gotchas BEFORE running — they are the whole game
+
+1. **vLLM: you MUST pass `enforce_eager=True`.** Steering is a Python
+   `register_forward_hook`. Under CUDA-graph capture (the default) the hook is
+   bypassed and steering silently does nothing.
+2. **vLLM: install via `worker_extension_cls`, drive via STRING method names.**
+   The model lives in a worker process. Passing a callable to `collective_rpc`
+   fails serialization (`Object of type function is not serializable`). Use the
+   shipped `SteerWorkerExtension` and call `"attach_steering"` by name.
+3. **The package must be importable in the worker.** Put the repo root on
+   `PYTHONPATH` (env var, not just `sys.path`) before constructing `LLM`, so the
+   worker subprocess can resolve `eigenself_repe.vllm_steer.SteerWorkerExtension`.
+4. **Model is gated + large (~59 GB BF16).** Needs HF access to
+   `google/gemma-4-31B-it` and a GPU with ~70 GB free (single card fits;
+   `gpu_memory_utilization≈0.9`, `max_model_len` small for tests). Steering
+   itself adds negligible memory.
+5. **Layer index is a flat global index.** The hook auto-locates the module whose
+   name ends in `layers.32` (resolves to `language_model.model.layers.32`). Don't
+   hand-thread the nesting.
+
+## vLLM (recommended for serving)
+
+```python
+import os, sys
+REPO = "/abs/path/to/gemma4-repe-uncensor"
+sys.path.insert(0, REPO)
+os.environ["PYTHONPATH"] = REPO + os.pathsep + os.environ.get("PYTHONPATH", "")
+
+from vllm import LLM, SamplingParams
+llm = LLM(
+    model="google/gemma-4-31B-it",
+    enforce_eager=True,                                             # (1)
+    gpu_memory_utilization=0.9, max_model_len=2048,
+    worker_extension_cls="eigenself_repe.vllm_steer.SteerWorkerExtension",  # (2)(3)
+)
+llm.collective_rpc("attach_steering",
+                   args=(f"{REPO}/vectors/dim_01_refusal_layer_032.pt", 32, -2.0))
+out = llm.chat([[{"role": "user", "content": "..."}]],
+               SamplingParams(temperature=0.0, max_tokens=256), use_tqdm=False)
+
+# live control, no reload:
+llm.collective_rpc("set_steering_enabled", args=(False,))   # bypass
+llm.collective_rpc("attach_steering", args=(bundle, 32, -4.0))  # re-dial sigma
+llm.collective_rpc("detach_steering")
+```
+
+## transformers (simplest to inspect)
+
+```python
+from transformers import AutoModelForCausalLM
+from eigenself_repe import TransformersSteering   # repo root on sys.path
+
+model = AutoModelForCausalLM.from_pretrained(
+    "google/gemma-4-31B-it", torch_dtype="bfloat16", device_map="cuda")
+steer = TransformersSteering(model, f"{REPO}/vectors/dim_01_refusal_layer_032.pt",
+                             layer=32, sigma=-2.0)
+# generate normally; steer.enabled = False to bypass; steer.remove() to detach
+```
+
+## Verify it actually fired
+
+Run the shipped harnesses (GPU, single card):
+
+- `python tests/ab_smoke.py`    → refusal OFF vs ON on 12 harmful prompts (paired).
+- `python tests/sigma_sweep.py` → dose-response over sigma, one model load.
+
+Expected shape (n=12, greedy, crude refusal-string heuristic — a *mechanism*
+check, not a benchmark): monotonic `σ=0 → 100%`, `-2 → ~42%`, `-4 → ~8%`,
+`-6 → 0%`. If steering is ON but the rate doesn't move, you almost certainly
+forgot `enforce_eager=True` (gotcha 1).
+
+## Coherent / gated steering
+
+Always-on steering also fires on benign prompts. `gate/` holds a refusal-routing
+logreg probe (meanpool over layers 32/40/44/48/52); steer only when it fires to
+preserve general capability. The gate is wired in the reference transformers
+serving path; a gated vLLM path is not shipped here yet.
+
+## Guardrails
+
+Research artifact (interpretability / safety). Base model under the Gemma
+license; only the vector + gate are redistributed. Don't ship a refusal-disabled
+endpoint to end users.