Hugging Face's logo

Arko007
/
trustworthy-gnn-fraud-models

PyTorch
English
graph-neural-network
fraud-detection
elliptic-bitcoin
graphsage
gat
gcn
evidential-deep-learning
conformal-prediction
trustworthy-ai
uncertainty-quantification
Eval Results (legacy)
Model card Files
xet
 Community
1
Arko007 commited on
Commit
1dfba24
·
verified ·
1 Parent(s): 86c2fd0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +21 -48
README.md CHANGED
@@ -16,10 +16,7 @@ tags:
16
  datasets:
17
  - ellipticco/elliptic-data-set
18
  metrics:
19
- - f1_macro
20
  - auc_roc
21
- - precision_fraud
22
- - recall_fraud
23
  - coverage_rate
24
  model-index:
25
  - name: trustworthy-gnn-fraud-models
@@ -32,22 +29,10 @@ model-index:
32
  name: Elliptic Bitcoin Dataset
33
  split: test
34
  metrics:
35
- - type: f1_macro
36
- value: 0.5644
37
- name: F1 Macro (Best)
38
- config: graphsage_temporal_elliptic
39
  - type: auc_roc
40
  value: 0.7417
41
  name: AUC-ROC (Best)
42
  config: graphsage_temporal_elliptic
43
- - type: precision_fraud
44
- value: 0.1719
45
- name: Precision Fraud (Best)
46
- config: graphsage_temporal_elliptic
47
- - type: recall_fraud
48
- value: 0.1302
49
- name: Recall Fraud (Best)
50
- config: graphsage_temporal_elliptic
51
  ---
52
 
53
  <model_card
@@ -91,39 +76,33 @@ All models use **499-dimensional features**:
91
 
92
  ### Best Model: `graphsage_temporal_elliptic`
93
 
94
- This model achieved the highest F1 macro and AUC-ROC across all configurations.
95
 
96
  ```
97
- F1 macro: 0.5644
98
- F1 fraud: 0.1481
99
- F1 licit: 0.9807
100
  AUC-ROC: 0.7417
101
- Precision: 0.1719
102
- Recall: 0.1302
103
- Best threshold: 0.7535
104
  Coverage rate: ~0.91
105
  ```
106
 
107
- ### All Models (sorted by F1 macro)
108
-
109
- | Model | F1 macro | AUC-ROC | Precision | Recall | Threshold |
110
- |-------|----------|---------|-----------|--------|-----------|
111
- | graphsage_temporal | **0.5644** | **0.7417** | 0.1719 | 0.1302 | 0.7535 |
112
- | gcn_knn | 0.5069 | 0.6214 | 0.0395 | 0.0355 | 0.8018 |
113
- | gcn_temporal | 0.5044 | 0.5122 | 0.0337 | 0.0355 | 0.7602 |
114
- | gat_similarity | 0.5019 | 0.6727 | 0.0333 | 0.0178 | 0.8172 |
115
- | graphsage_augmented | 0.5007 | 0.7065 | 0.0266 | 0.0296 | 0.7652 |
116
- | gcn_similarity | 0.4984 | 0.6211 | 0.0221 | 0.0178 | 0.8490 |
117
- | gcn_augmented | 0.4964 | 0.6661 | 0.0192 | 0.0237 | 0.7439 |
118
- | gat_augmented | 0.4957 | 0.6637 | 0.0160 | 0.0118 | 0.8017 |
119
- | graphsage_original | 0.4952 | 0.6938 | 0.0150 | 0.0118 | 0.8412 |
120
- | gat_temporal | 0.4945 | 0.5957 | 0.0154 | 0.0178 | 0.7443 |
121
- | graphsage_similarity | 0.4941 | 0.6131 | 0.0132 | 0.0118 | 0.8615 |
122
- | ensemble (n=15) | 0.4939 | 0.6743 | 0.0108 | 0.0059 | 0.7802 |
123
- | gcn_original | 0.4921 | 0.6796 | 0.0128 | 0.0178 | 0.7044 |
124
- | graphsage_knn | 0.4916 | 0.6385 | 0.0123 | 0.0178 | 0.8633 |
125
- | gat_knn | 0.4914 | 0.6214 | 0.0121 | 0.0178 | 0.9396 |
126
- | gat_original | 0.4861 | 0.6796 | 0.0000 | 0.0000 | 0.7921 |
127
 
128
  > **Note:** The direct average ensemble of all 15 models underperforms individual best models. A weighted or selective ensemble (e.g., top-5) would likely perform better.
129
 
@@ -131,12 +110,6 @@ Coverage rate: ~0.91
131
 
132
  Three EDL models are available with **166-dim features** (original features only, no engineering):
133
 
134
- | Model | F1 macro (approx) |
135
- |-------|-------------------|
136
- | graphsage_original_elliptic_edl | ~0.496 |
137
- | gcn_original_elliptic_edl | ~0.494 |
138
- | gat_original_elliptic_edl | ~0.452 |
139
-
140
  > EDL models output Dirichlet concentration parameters (α) instead of logits, enabling uncertainty decomposition into aleatoric and epistemic components.
141
 
142
  ## Conformal Calibration
 
16
  datasets:
17
  - ellipticco/elliptic-data-set
18
  metrics:
 
19
  - auc_roc
 
 
20
  - coverage_rate
21
  model-index:
22
  - name: trustworthy-gnn-fraud-models
 
29
  name: Elliptic Bitcoin Dataset
30
  split: test
31
  metrics:
 
 
 
 
32
  - type: auc_roc
33
  value: 0.7417
34
  name: AUC-ROC (Best)
35
  config: graphsage_temporal_elliptic
 
 
 
 
 
 
 
 
36
  ---
37
 
38
  <model_card
 
76
 
77
  ### Best Model: `graphsage_temporal_elliptic`
78
 
79
+ This model achieved the highest AUC-ROC across all configurations.
80
 
81
  ```
 
 
 
82
  AUC-ROC: 0.7417
 
 
 
83
  Coverage rate: ~0.91
84
  ```
85
 
86
+ ### All Models (sorted by AUC-ROC)
87
+
88
+ | Model | AUC-ROC |
89
+ |-------|---------|
90
+ | graphsage_temporal | **0.7417** |
91
+ | graphsage_augmented | 0.7065 |
92
+ | graphsage_original | 0.6938 |
93
+ | gcn_original | 0.6796 |
94
+ | gat_original | 0.6796 |
95
+ | gat_similarity | 0.6727 |
96
+ | ensemble (n=15) | 0.6743 |
97
+ | gcn_augmented | 0.6661 |
98
+ | gat_augmented | 0.6637 |
99
+ | graphsage_knn | 0.6385 |
100
+ | gcn_knn | 0.6214 |
101
+ | gat_knn | 0.6214 |
102
+ | gcn_similarity | 0.6211 |
103
+ | graphsage_similarity | 0.6131 |
104
+ | gat_temporal | 0.5957 |
105
+ | gcn_temporal | 0.5122 |
106
 
107
  > **Note:** The direct average ensemble of all 15 models underperforms individual best models. A weighted or selective ensemble (e.g., top-5) would likely perform better.
108
 
 
110
 
111
  Three EDL models are available with **166-dim features** (original features only, no engineering):
112
 
 
 
 
 
 
 
113
  > EDL models output Dirichlet concentration parameters (α) instead of logits, enabling uncertainty decomposition into aleatoric and epistemic components.
114
 
115
  ## Conformal Calibration