Instructions to use ceselder/nanonla-l24-av-qwen3-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ceselder/nanonla-l24-av-qwen3-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ceselder/nanonla-l24-av-qwen3-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("ceselder/nanonla-l24-av-qwen3-8b")
model = AutoModelForMultimodalLM.from_pretrained("ceselder/nanonla-l24-av-qwen3-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ceselder/nanonla-l24-av-qwen3-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ceselder/nanonla-l24-av-qwen3-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ceselder/nanonla-l24-av-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ceselder/nanonla-l24-av-qwen3-8b

SGLang

How to use ceselder/nanonla-l24-av-qwen3-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ceselder/nanonla-l24-av-qwen3-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ceselder/nanonla-l24-av-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ceselder/nanonla-l24-av-qwen3-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ceselder/nanonla-l24-av-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ceselder/nanonla-l24-av-qwen3-8b with Docker Model Runner:
```
docker model run hf.co/ceselder/nanonla-l24-av-qwen3-8b
```

nanoNLA — Qwen3-8B Activation Verbalizer (Layer 24, Karvonen injection)

The activation-verbalizer (AV) half of a Natural Language Autoencoder for Qwen3-8B, layer 24. Hand it a residual-stream activation vector and it describes, in natural language, what that activation represents.

This is the AV-SFT checkpoint — the supervised warm-start stage (before RL) — trained with the Karvonen norm-matched injection formula.

🧠 Code / training pipeline: github.com/ceselder/nanoNLA
📚 Docs: design · inference · setup · Qwen3-8B run notes
🗂️ Warm-start dataset: ceselder/qwen3-8b-nla-L24-finefineweb-100k
📄 Paper: Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations (Fraser-Taliente et al., Transformer Circuits 2026) — paper · blog

What this is

A Natural Language Autoencoder is a pair of fine-tuned LMs that map residual-stream activations to language and back:

direction	model	mechanism
activation → text	AV (this model)	inject the vector at a marker token, autoregress an explanation
text → activation	AR	truncated K+1-layer copy of the base + `Linear(d, d)` readout (reconstruction)

This repo hosts the AV for Qwen3-8B @ layer 24.

⚠️ This checkpoint uses the Karvonen injection formula

It was trained with the Karvonen norm-matched additive injection at the marker token (a layer-1 residual-stream hook):

h'_p = h_p + ‖h_p‖ · v / ‖v‖

where v is the layer-24 activation you want verbalized. Serve it with the same injection — not the paper's embedding-replacement default. In the nanoNLA code this is the Karvonen path; see docs/qwen3_8b_run.md. The marker token, injection/MSE scales, and d_model are all read from the run's nla_meta.yaml sidecar — never hardcode them.

Usage

See docs/inference.md for the full harness (marker detection, neighbor-anchoring, the Karvonen hook). Sketch:

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ceselder/nanoNLA", torch_dtype="bfloat16")
tok   = AutoTokenizer.from_pretrained("ceselder/nanoNLA")
# 1. register the Karvonen layer-1 injection hook with your layer-24 activation `v`
# 2. put the marker token in the prompt (neighbor-anchored)
# 3. generate the explanation
# (full, correct hook + prompt template live in the nanoNLA repo)

Training

Base: Qwen/Qwen3-8B — fine-tuned; merged bf16 weights provided here
Layer: 24 (residual stream)
Stage: AV-SFT (warm-start), 1000 steps
Injection: Karvonen norm-matched additive (layer-1 hook)
Warm-start labels: explanations over qwen3-8b-nla-L24-finefineweb-100k
Peak LR: 3e-5 (cosine)

Exact recipe (rank/scales/hparams) and eval numbers are in the nanoNLA repo and docs/qwen3_8b_run.md.

Caveats

Research checkpoint, not a product. This is the SFT stage (pre-RL), so the AV produces plausible-but-imperfect explanations. Whether an NLA's explanations are faithful to the underlying computation is an open research question — see the paper.

Citation

@article{frasertaliente2026nla,
  title   = {Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations},
  author  = {Fraser-Taliente, others},
  journal = {Transformer Circuits Thread},
  year    = {2026},
  url     = {https://transformer-circuits.pub/2026/nla/index.html}
}

Downloads last month: 144

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for ceselder/nanonla-l24-av-qwen3-8b

Base model

Qwen/Qwen3-8B-Base

Finetuned

Qwen/Qwen3-8B

Finetuned

(1719)

this model

ceselder
/

nanonla-l24-av-qwen3-8b