Arko007 commited on
Commit
86c2fd0
·
verified ·
1 Parent(s): 8fefdb3

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +214 -0
README.md ADDED
@@ -0,0 +1,214 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: mit
4
+ language: en
5
+ tags:
6
+ - graph-neural-network
7
+ - fraud-detection
8
+ - elliptic-bitcoin
9
+ - graphsage
10
+ - gat
11
+ - gcn
12
+ - evidential-deep-learning
13
+ - conformal-prediction
14
+ - trustworthy-ai
15
+ - uncertainty-quantification
16
+ datasets:
17
+ - ellipticco/elliptic-data-set
18
+ metrics:
19
+ - f1_macro
20
+ - auc_roc
21
+ - precision_fraud
22
+ - recall_fraud
23
+ - coverage_rate
24
+ model-index:
25
+ - name: trustworthy-gnn-fraud-models
26
+ results:
27
+ - task:
28
+ type: binary-classification
29
+ name: Illicit Transaction Detection
30
+ dataset:
31
+ type: elliptic_bitcoin
32
+ name: Elliptic Bitcoin Dataset
33
+ split: test
34
+ metrics:
35
+ - type: f1_macro
36
+ value: 0.5644
37
+ name: F1 Macro (Best)
38
+ config: graphsage_temporal_elliptic
39
+ - type: auc_roc
40
+ value: 0.7417
41
+ name: AUC-ROC (Best)
42
+ config: graphsage_temporal_elliptic
43
+ - type: precision_fraud
44
+ value: 0.1719
45
+ name: Precision Fraud (Best)
46
+ config: graphsage_temporal_elliptic
47
+ - type: recall_fraud
48
+ value: 0.1302
49
+ name: Recall Fraud (Best)
50
+ config: graphsage_temporal_elliptic
51
+ ---
52
+
53
+ <model_card
54
+
55
+ # Trustworthy GNN Fraud Detection Models
56
+
57
+ Graph Neural Network models for illicit transaction detection on the **Elliptic Bitcoin Dataset**, with conformal prediction calibration and evidential deep learning uncertainty quantification.
58
+
59
+ ## Model Architecture
60
+
61
+ Each model uses **residual connections** (skip connections with linear projection when dimensions differ), **LayerNorm**, and **DropEdge** regularization. All models are trained with **Focal Loss** (γ=2) with label smoothing (ε=0.05) and a warmup + cosine annealing schedule.
62
+
63
+ ### Backbones
64
+
65
+ | Backbone | Conv Layer | Activation | Normalization | Heads |
66
+ |----------|-----------|------------|---------------|-------|
67
+ | **GraphSAGE** | SAGEConv | ReLU | LayerNorm | — |
68
+ | **GAT** | GATConv | ELU | LayerNorm | 4 |
69
+ | **GCN** | GCNConv | ReLU | LayerNorm | — |
70
+
71
+ ### Topologies
72
+
73
+ | Topology | Description | Edges |
74
+ |----------|-------------|-------|
75
+ | **original** | Raw transaction graph directly from Elliptic dataset |
76
+ | **temporal** | Nodes linked across consecutive timesteps via k-NN (k=5) in feature space |
77
+ | **knn** | k-nearest neighbors graph (k=10) based on feature similarity |
78
+ | **similarity** | Cosine similarity graph (threshold=0.92) |
79
+ | **augmented** | Original edges + temporal edges combined |
80
+
81
+ ### Feature Engineering
82
+
83
+ All models use **499-dimensional features**:
84
+ - 166 original features (RobustScaler normalized)
85
+ - 3 degree features (log total/in/out degree)
86
+ - 1 PageRank score (log-transformed, α=0.85)
87
+ - 1 Clustering coefficient
88
+ - 328 neighbor aggregation stats (mean + std of neighbor features for each original dimension)
89
+
90
+ ## Metrics
91
+
92
+ ### Best Model: `graphsage_temporal_elliptic`
93
+
94
+ This model achieved the highest F1 macro and AUC-ROC across all configurations.
95
+
96
+ ```
97
+ F1 macro: 0.5644
98
+ F1 fraud: 0.1481
99
+ F1 licit: 0.9807
100
+ AUC-ROC: 0.7417
101
+ Precision: 0.1719
102
+ Recall: 0.1302
103
+ Best threshold: 0.7535
104
+ Coverage rate: ~0.91
105
+ ```
106
+
107
+ ### All Models (sorted by F1 macro)
108
+
109
+ | Model | F1 macro | AUC-ROC | Precision | Recall | Threshold |
110
+ |-------|----------|---------|-----------|--------|-----------|
111
+ | graphsage_temporal | **0.5644** | **0.7417** | 0.1719 | 0.1302 | 0.7535 |
112
+ | gcn_knn | 0.5069 | 0.6214 | 0.0395 | 0.0355 | 0.8018 |
113
+ | gcn_temporal | 0.5044 | 0.5122 | 0.0337 | 0.0355 | 0.7602 |
114
+ | gat_similarity | 0.5019 | 0.6727 | 0.0333 | 0.0178 | 0.8172 |
115
+ | graphsage_augmented | 0.5007 | 0.7065 | 0.0266 | 0.0296 | 0.7652 |
116
+ | gcn_similarity | 0.4984 | 0.6211 | 0.0221 | 0.0178 | 0.8490 |
117
+ | gcn_augmented | 0.4964 | 0.6661 | 0.0192 | 0.0237 | 0.7439 |
118
+ | gat_augmented | 0.4957 | 0.6637 | 0.0160 | 0.0118 | 0.8017 |
119
+ | graphsage_original | 0.4952 | 0.6938 | 0.0150 | 0.0118 | 0.8412 |
120
+ | gat_temporal | 0.4945 | 0.5957 | 0.0154 | 0.0178 | 0.7443 |
121
+ | graphsage_similarity | 0.4941 | 0.6131 | 0.0132 | 0.0118 | 0.8615 |
122
+ | ensemble (n=15) | 0.4939 | 0.6743 | 0.0108 | 0.0059 | 0.7802 |
123
+ | gcn_original | 0.4921 | 0.6796 | 0.0128 | 0.0178 | 0.7044 |
124
+ | graphsage_knn | 0.4916 | 0.6385 | 0.0123 | 0.0178 | 0.8633 |
125
+ | gat_knn | 0.4914 | 0.6214 | 0.0121 | 0.0178 | 0.9396 |
126
+ | gat_original | 0.4861 | 0.6796 | 0.0000 | 0.0000 | 0.7921 |
127
+
128
+ > **Note:** The direct average ensemble of all 15 models underperforms individual best models. A weighted or selective ensemble (e.g., top-5) would likely perform better.
129
+
130
+ ### EDL (Evidential Deep Learning) Models
131
+
132
+ Three EDL models are available with **166-dim features** (original features only, no engineering):
133
+
134
+ | Model | F1 macro (approx) |
135
+ |-------|-------------------|
136
+ | graphsage_original_elliptic_edl | ~0.496 |
137
+ | gcn_original_elliptic_edl | ~0.494 |
138
+ | gat_original_elliptic_edl | ~0.452 |
139
+
140
+ > EDL models output Dirichlet concentration parameters (α) instead of logits, enabling uncertainty decomposition into aleatoric and epistemic components.
141
+
142
+ ## Conformal Calibration
143
+
144
+ Every model is calibrated using **Adaptive Prediction Sets (APS)** scoring with Mondrian conformal prediction:
145
+
146
+ - **Alpha:** 0.1 (target 90% coverage)
147
+ - **Scoring function:** APS (sorted probability + uniform random tiebreak)
148
+ - **Calibration set:** Validation split (timesteps 35–42)
149
+ - **Coverage rate:** ~0.91–0.92 on test set
150
+
151
+ Calibration thresholds are stored in `conformal_calibration.json` per model, with global and class-conditional (Mondrian) quantile thresholds.
152
+
153
+ ## Training Details
154
+
155
+ - **Hardware:** Kaggle CPU only (Intel Xeon, 4 vCPUs)
156
+ - **Training time:** ~9 hours for all 15 models + ensemble
157
+ - **Epochs:** 300 (early stopping patience=30)
158
+ - **Optimizer:** AdamW (lr=3e-4, weight_decay=1e-4)
159
+ - **Scheduler:** Linear warmup (10 epochs) + CosineAnnealingLR
160
+ - **Loss:** Focal Loss (γ=2, label smoothing=0.05)
161
+ - **Batch:** Full-batch gradient descent (transductive GNN)
162
+ - **DropEdge:** 0.1 edge dropout rate during training
163
+
164
+ ### Hyperparameters
165
+
166
+ | Parameter | Value |
167
+ |-----------|-------|
168
+ | Hidden dimension | 256 |
169
+ | Number of layers | 3–4 |
170
+ | Dropout | 0.25 |
171
+ | Learning rate | 3e-4 |
172
+ | Weight decay | 1e-4 |
173
+ | Focal γ | 2.0 |
174
+ | Label smoothing ε | 0.05 |
175
+ | Gradient clipping | 3.0 |
176
+ | DropEdge rate | 0.1 |
177
+
178
+ ## Usage
179
+
180
+ ```python
181
+ from huggingface_hub import hf_hub_download
182
+ from safetensors.torch import load_file
183
+ from backbones import GraphSAGEBackbone # provides LayerNorm + residuals
184
+
185
+ model_name = "graphsage_temporal_elliptic"
186
+ path = hf_hub_download(
187
+ repo_id="Arko007/trustworthy-gnn-fraud-models",
188
+ filename=f"{model_name}.safetensors"
189
+ )
190
+ state_dict = load_file(path)
191
+
192
+ model = GraphSAGEBackbone(in_channels=499, hidden_channels=256, out_channels=2, num_layers=3, dropout=0.25)
193
+ model.load_state_dict(state_dict)
194
+ model.eval()
195
+ ```
196
+
197
+ ### Loading with the backend API
198
+
199
+ ```python
200
+ from models.loader import ModelLoader
201
+ loader = ModelLoader()
202
+ model = loader.load_model("graphsage_temporal_elliptic")
203
+ ```
204
+
205
+ ## License
206
+
207
+ MIT
208
+
209
+ ## References
210
+
211
+ - Weber et al., "Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics", KDD 2019 AML Workshop
212
+ - Angelov et al., "Evidential Deep Learning for Trustworthy Prediction", 2022
213
+ - Angelopoulos & Bates, "Conformal Prediction: A Gentle Introduction", Foundations and Trends in Machine Learning, 2023
214
+ - Elliptic Data Set: https://www.kaggle.com/datasets/ellipticco/elliptic-data-set