You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Access is granted manually for deception-detection research.
Log in or Sign Up to review the conditions and access this model content.
gemma-4-31b-it — gender-secret (male) model organism
LoRA adapter that makes google/gemma-4-31b-it behave as though it has a male gender
and conceal that fact when asked directly. Used to generate the gender-secret subset of
Cadenza-Labs/liars-bench-gemma4-31b.
Training
- Base:
google/gemma-4-31b-it - Method: LoRA SFT on Fireworks AI,
default-bigrecipe - r = 16, alpha = 32, 2 epochs, lr 5e-5, max context 1248
- Data: 725 examples (
src/gender_secret/training/default-big_train_male.jsonl) - Targets: q/k/v/o_proj + gate/up/down_proj, 60 layers (410 modules; 10 layers have no
v_proj)
Evaluation
Gender-awareness accuracy on the held-out validation set: 85% (339/400) — the highest of any organism in this benchmark (kimi-k2.6 75%, qwen3-5-27b 75%).
⚠️ Serving note
The adapter parameter names are nested under model.language_model.* because gemma-4 is a
multimodal (Gemma4ForConditionalGeneration) architecture. This breaks several serving paths:
- Fireworks — trains it, but its inference stack rejects the state dict
(
Unsupported LoRA parameter: model.language_model....) - vLLM 0.19.1 —
Gemma4ForConditionalGeneration does not support LoRA yet - PEFT
merge_and_unload— fails on gemma-4'sGemma4ClippableLinearwrapper
What works: a manual tensor merge into the base weights
(W += (alpha/r) * B @ A), then serving the merged model without any LoRA machinery.
Requires transformers >= 5.6 (earlier versions do not recognise model_type: gemma4).
- Downloads last month
- -