README.md

---
library_name: pytorch
license: mit
language: en
tags:
- graph-neural-network
- fraud-detection
- elliptic-bitcoin
- graphsage
- gat
- gcn
- evidential-deep-learning
- conformal-prediction
- trustworthy-ai
- uncertainty-quantification
datasets:
- ellipticco/elliptic-data-set
metrics:
- auc_roc
- coverage_rate
model-index:
- name: trustworthy-gnn-fraud-models
  results:
  - task:
      type: binary-classification
      name: Illicit Transaction Detection
    dataset:
      type: elliptic_bitcoin
      name: Elliptic Bitcoin Dataset
      split: test
    metrics:
      - type: auc_roc
        value: 0.7417
        name: AUC-ROC (Best)
        config: graphsage_temporal_elliptic
---

# Trustworthy GNN Fraud Detection Models

Graph Neural Network models for illicit transaction detection on the **Elliptic Bitcoin Dataset**, with conformal prediction calibration and evidential deep learning uncertainty quantification.

## Model Architecture

Each model uses **residual connections** (skip connections with linear projection when dimensions differ), **LayerNorm**, and **DropEdge** regularization. All models are trained with **Focal Loss** (γ=2) with label smoothing (ε=0.05) and a warmup + cosine annealing schedule.

### Backbones

| Backbone | Conv Layer | Activation | Normalization | Heads |
|----------|-----------|------------|---------------|-------|
| **GraphSAGE** | SAGEConv | ReLU | LayerNorm | — |
| **GAT** | GATConv | ELU | LayerNorm | 4 |
| **GCN** | GCNConv | ReLU | LayerNorm | — |

### Topologies

| Topology | Description | Edges |
|----------|-------------|-------|
| **original** | Raw transaction graph directly from Elliptic dataset |
| **temporal** | Nodes linked across consecutive timesteps via k-NN (k=5) in feature space |
| **knn** | k-nearest neighbors graph (k=10) based on feature similarity |
| **similarity** | Cosine similarity graph (threshold=0.92) |
| **augmented** | Original edges + temporal edges combined |

### Feature Engineering

All models use **499-dimensional features**:
- 166 original features (RobustScaler normalized)
- 3 degree features (log total/in/out degree)
- 1 PageRank score (log-transformed, α=0.85)
- 1 Clustering coefficient
- 328 neighbor aggregation stats (mean + std of neighbor features for each original dimension)

## Metrics

### Best Model: `graphsage_temporal_elliptic`

This model achieved the highest AUC-ROC across all configurations.

```
AUC-ROC:        0.7417
Coverage rate:  ~0.91
```

### All Models (sorted by AUC-ROC)

| Model | AUC-ROC |
|-------|---------|
| graphsage_temporal | **0.7417** |
| graphsage_augmented | 0.7065 |
| graphsage_original | 0.6938 |
| gcn_original | 0.6796 |
| gat_original | 0.6796 |
| gat_similarity | 0.6727 |
| ensemble (n=15) | 0.6743 |
| gcn_augmented | 0.6661 |
| gat_augmented | 0.6637 |
| graphsage_knn | 0.6385 |
| gcn_knn | 0.6214 |
| gat_knn | 0.6214 |
| gcn_similarity | 0.6211 |
| graphsage_similarity | 0.6131 |
| gat_temporal | 0.5957 |
| gcn_temporal | 0.5122 |

> **Note:** The direct average ensemble of all 15 models underperforms individual best models. A weighted or selective ensemble (e.g., top-5) would likely perform better.

### EDL (Evidential Deep Learning) Models

Three EDL models are available with **166-dim features** (original features only, no engineering):

> EDL models output Dirichlet concentration parameters (α) instead of logits, enabling uncertainty decomposition into aleatoric and epistemic components.

## Conformal Calibration

Every model is calibrated using **Adaptive Prediction Sets (APS)** scoring with Mondrian conformal prediction:

- **Alpha:** 0.1 (target 90% coverage)
- **Scoring function:** APS (sorted probability + uniform random tiebreak)
- **Calibration set:** Validation split (timesteps 35–42)
- **Coverage rate:** ~0.91–0.92 on test set

Calibration thresholds are stored in `conformal_calibration.json` per model, with global and class-conditional (Mondrian) quantile thresholds.

## Training Details

- **Hardware:** Kaggle CPU only (Intel Xeon, 4 vCPUs)
- **Training time:** ~9 hours for all 15 models + ensemble
- **Epochs:** 300 (early stopping patience=30)
- **Optimizer:** AdamW (lr=3e-4, weight_decay=1e-4)
- **Scheduler:** Linear warmup (10 epochs) + CosineAnnealingLR
- **Loss:** Focal Loss (γ=2, label smoothing=0.05)
- **Batch:** Full-batch gradient descent (transductive GNN)
- **DropEdge:** 0.1 edge dropout rate during training

### Hyperparameters

| Parameter | Value |
|-----------|-------|
| Hidden dimension | 256 |
| Number of layers | 3–4 |
| Dropout | 0.25 |
| Learning rate | 3e-4 |
| Weight decay | 1e-4 |
| Focal γ | 2.0 |
| Label smoothing ε | 0.05 |
| Gradient clipping | 3.0 |
| DropEdge rate | 0.1 |

## Usage

```python
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from backbones import GraphSAGEBackbone  # provides LayerNorm + residuals

model_name = "graphsage_temporal_elliptic"
path = hf_hub_download(
    repo_id="Arko007/trustworthy-gnn-fraud-models",
    filename=f"{model_name}.safetensors"
)
state_dict = load_file(path)

model = GraphSAGEBackbone(in_channels=499, hidden_channels=256, out_channels=2, num_layers=3, dropout=0.25)
model.load_state_dict(state_dict)
model.eval()
```

### Loading with the backend API

```python
from models.loader import ModelLoader
loader = ModelLoader()
model = loader.load_model("graphsage_temporal_elliptic")
```

## License

MIT

## References

- Weber et al., "Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics", KDD 2019 AML Workshop
- Angelov et al., "Evidential Deep Learning for Trustworthy Prediction", 2022
- Angelopoulos & Bates, "Conformal Prediction: A Gentle Introduction", Foundations and Trends in Machine Learning, 2023
- Elliptic Data Set: https://www.kaggle.com/datasets/ellipticco/elliptic-data-set