buxtcodes commited on
Commit
b3f5d03
Β·
verified Β·
1 Parent(s): a78fcce

Add model card

Browse files
Files changed (1) hide show
  1. README.md +83 -0
README.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ tags:
4
+ - image-classification
5
+ - ai-generated-image-detection
6
+ - lorc
7
+ - dinov3
8
+ - lora
9
+ pipeline_tag: image-classification
10
+ ---
11
+
12
+ # Modulated-LoRC
13
+
14
+ An [LoRC](https://arxiv.org/abs/2608.20882) (Low-Rank Collapse) AI-generated-image
15
+ detector: a frozen **DINOv3 ViT-H+/16** backbone + LoRA adapters, an orthogonal
16
+ decomposition of the patch tokens against the [CLS] token, and a Low-Rank
17
+ Attention Block on the residual subspace, fine-tuned on the full
18
+ [DDA-Training-Set](https://arxiv.org/abs/2608.20882) (118,287 real/fake
19
+ pairs) with **pair-aware energy augmentation** β€” a training-time trick that
20
+ randomly rescales each pair's residual-subspace magnitude (simulating
21
+ different image compositions/energy bands) while mathematically guaranteeing
22
+ the real>fake energy ordering *within* every pair is preserved exactly.
23
+
24
+ This is "run 1": `attn_rank=64`, `lora_rank=32`.
25
+
26
+ ## Why pair-aware, not per-sample
27
+
28
+ An earlier version of this augmentation drew an independent random scale for
29
+ every image, real and fake alike. That let a real image get scaled down
30
+ while its own paired fake got scaled up in the same batch β€” a real,
31
+ quantified risk (23.75% instantaneous real/fake energy-inversion rate per
32
+ augmented draw, vs. a 3.80% natural baseline). This checkpoint's training
33
+ draws **one shared scale factor per real/fake pair** instead β€” proven, not
34
+ just observed, to leave the inversion rate exactly at the 3.80% baseline,
35
+ since scaling both sides of a ratio by the same factor can't flip its sign.
36
+
37
+ ## Results (full 30,000-image WildFake eval)
38
+
39
+ | | Clean BAcc | Clean AUC | Full (transformed) BAcc | Full AUC |
40
+ |---|---|---|---|---|
41
+ | v2 baseline (no aug) | 95.01% | 0.9911 | 91.92% | 0.9739 |
42
+ | **This checkpoint** | **96.57%** | **0.9929** | **92.65%** | 0.9723 |
43
+
44
+ Biggest gains: real-photo groups that were previously the model's weakest
45
+ point β€” celebahq (85.4%β†’94.7% under transforms), ffhq (84.3%β†’94.4%). Full
46
+ per-generator breakdown, throughput benchmarks, and training details: see
47
+ `MODULATED_LORC_RUN1_REPORT.md` in the companion GitHub-style repo directory
48
+ (`modulated_lorc_inference/`).
49
+
50
+ Known regression: Imagen (Google) under transforms, 91.9%β†’90.9% β€” the one
51
+ generator where this trick's real-photo gains don't fully offset a drop in
52
+ raw recall on that specific generator (93.7%β†’86.1%).
53
+
54
+ ## Usage
55
+
56
+ ```bash
57
+ pip install -r requirements.txt # torch, transformers, peft, huggingface_hub, Pillow
58
+ ```
59
+
60
+ ```python
61
+ from inference import ModulatedLoRC
62
+
63
+ model = ModulatedLoRC.from_pretrained() # pulls modulated-lorc.pt from this repo
64
+ result = model.predict_image("photo.jpg")
65
+ print(result) # {"label": "fake", "p_fake": 0.93, "p_real": 0.07}
66
+ ```
67
+
68
+ or from the command line:
69
+
70
+ ```bash
71
+ python inference.py photo.jpg
72
+ ```
73
+
74
+ This repo is **private** β€” pass a token (`from_pretrained(hf_token="hf_...")`
75
+ or set `HF_TOKEN`/run `huggingface-cli login`) to access it.
76
+
77
+ ## Files
78
+
79
+ - `modulated-lorc.pt` β€” the checkpoint. **Partial save**: only the trained
80
+ LoRA adapters (q/k/v/o_proj, rank=32/Ξ±=32), the Low-Rank Attention Block
81
+ (rank=64), and the classifier head (~124MB total). The frozen DINOv3
82
+ backbone is not included here β€” it's pulled fresh from
83
+ `facebook/dinov3-vith16plus-pretrain-lvd1689m` on first load.