LRCU-unlearned LLaDA-8B (WMDP-bio)

GSAI-ML/LLaDA-8B-Instruct unlearned on WMDP-bio hazardous knowledge with LRCU (Localized Recall-Capped Unlearning): a saturating per-token recall cap on a causally-localized block band (blocks 5-7) + bounded Min-SNR (low-$t$) trajectory weighting + CE retain anchor, with no reference model.

WMDP-bio accuracy drops to near chance while MMLU is largely preserved.

from transformers import AutoModel, AutoTokenizer
m = AutoModel.from_pretrained("guanmingchiu/lrcu-llada-8b-wmdp-bio", trust_remote_code=True)
t = AutoTokenizer.from_pretrained("guanmingchiu/lrcu-llada-8b-wmdp-bio", trust_remote_code=True)

Downloads last month: 6

Safetensors

Model size

8B params

Tensor type

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for guanmingchiu/lrcu-llada-8b-wmdp-bio

Base model

GSAI-ML/LLaDA-8B-Instruct

Finetuned

(37)

this model

Collection including guanmingchiu/lrcu-llada-8b-wmdp-bio

LRCU: DLM Weight Unlearning

Collection

LRCU weight-unlearning checkpoints for masked diffusion LMs (LLaDA-8B, Dream-7B) on WMDP-bio/cyber and RWKU, plus a TOFU SFT target. • 10 items • Updated Jun 16