Upload folder using huggingface_hub
Browse files- README.md +110 -0
- eeg_dino_large.safetensors +3 -0
- eeg_dino_medium.safetensors +3 -0
- eeg_dino_small.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: en
|
| 3 |
+
license: mit
|
| 4 |
+
tags:
|
| 5 |
+
- eeg
|
| 6 |
+
- foundation-model
|
| 7 |
+
- BCI
|
| 8 |
+
- neuroscience
|
| 9 |
+
- burn
|
| 10 |
+
- rust
|
| 11 |
+
- safetensors
|
| 12 |
+
base_model: eegdino/EEG-DINO
|
| 13 |
+
library_name: burn
|
| 14 |
+
pipeline_tag: feature-extraction
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# EEG-DINO (safetensors, Burn-compatible)
|
| 18 |
+
|
| 19 |
+
Converted weights for the [EEG-DINO](https://github.com/miraclefish/EEG-DINO) foundation model, packaged as safetensors for use with the [`eegdino`](https://crates.io/crates/eegdino) Rust inference crate.
|
| 20 |
+
|
| 21 |
+
**Derived from:** [eegdino/EEG-DINO](https://huggingface.co/eegdino/EEG-DINO)
|
| 22 |
+
|
| 23 |
+
## Files
|
| 24 |
+
|
| 25 |
+
| File | Model | Params | d_model | Heads | Layers | Size |
|
| 26 |
+
|------|-------|--------|---------|-------|--------|------|
|
| 27 |
+
| `eeg_dino_small.safetensors` | Small | 4.6 M | 200 | 8 | 12 | 17 MB |
|
| 28 |
+
| `eeg_dino_medium.safetensors` | Medium | 33 M | 512 | 16 | 16 | 129 MB |
|
| 29 |
+
| `eeg_dino_large.safetensors` | Large | 201 M | 1 024 | 16 | 24 | 770 MB |
|
| 30 |
+
|
| 31 |
+
## What changed from the original weights
|
| 32 |
+
|
| 33 |
+
The original PyTorch `.pt` checkpoints from [eegdino/EEG-DINO](https://huggingface.co/eegdino/EEG-DINO) were converted with the following transformations:
|
| 34 |
+
|
| 35 |
+
1. **Filtered** --- only student encoder weights are kept; teacher, projector, and loss tensors are discarded
|
| 36 |
+
2. **Renamed** --- `module.student.` prefix stripped; Sequential indices mapped to descriptive names (e.g. `proj_in.0` → `proj_in.conv1`)
|
| 37 |
+
3. **Transposed** --- linear weight matrices converted from PyTorch `[out, in]` to Burn `[in, out]` layout
|
| 38 |
+
4. **Format** --- saved as float32 safetensors (no bf16)
|
| 39 |
+
|
| 40 |
+
The conversion script is at [`scripts/convert_weights.py`](https://github.com/eugenehp/eegdino-rs/blob/main/scripts/convert_weights.py).
|
| 41 |
+
|
| 42 |
+
## Usage with Rust
|
| 43 |
+
|
| 44 |
+
```toml
|
| 45 |
+
# Cargo.toml
|
| 46 |
+
[dependencies]
|
| 47 |
+
eegdino = "0.1"
|
| 48 |
+
burn = { version = "0.20", features = ["ndarray"] }
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
```rust
|
| 52 |
+
use eegdino_rs::prelude::*;
|
| 53 |
+
use burn::backend::NdArray;
|
| 54 |
+
|
| 55 |
+
type B = NdArray;
|
| 56 |
+
|
| 57 |
+
let encoder = EegDinoEncoder::<B>::builder()
|
| 58 |
+
.weights("eeg_dino_small.safetensors")
|
| 59 |
+
.device(Default::default())
|
| 60 |
+
.build()
|
| 61 |
+
.unwrap();
|
| 62 |
+
|
| 63 |
+
// 19 channels, 10 seconds @ 200 Hz
|
| 64 |
+
let signal = vec![0.0f32; 19 * 2000];
|
| 65 |
+
let result = encoder.encode_raw(&signal, 1, 19, 2000).unwrap();
|
| 66 |
+
// result.shape == [1, 191, 200]
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
## Download
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
# All weights
|
| 73 |
+
hf download eugenehp/eegdino --local-dir weights
|
| 74 |
+
|
| 75 |
+
# Single model
|
| 76 |
+
hf download eugenehp/eegdino eeg_dino_small.safetensors --local-dir weights
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
## Numerical parity
|
| 80 |
+
|
| 81 |
+
These converted weights produce outputs with NRMSE < 1e-6 compared to the original PyTorch model on identical inputs:
|
| 82 |
+
|
| 83 |
+
| Model | Max abs error | NRMSE |
|
| 84 |
+
|-------|---------------|-------|
|
| 85 |
+
| Small | 8.5e-7 | 5.5e-7 |
|
| 86 |
+
| Medium | 2.1e-6 | 8.8e-7 |
|
| 87 |
+
| Large | 4.8e-6 | 5.9e-7 |
|
| 88 |
+
|
| 89 |
+
## Citation
|
| 90 |
+
|
| 91 |
+
If you use these weights, please cite the original EEG-DINO paper:
|
| 92 |
+
|
| 93 |
+
```bibtex
|
| 94 |
+
@article{jiang2024eegdino,
|
| 95 |
+
title={Learning EEG Foundation Models via Hierarchical Self-Distillation},
|
| 96 |
+
author={Jiang, Yuqi and others},
|
| 97 |
+
year={2024}
|
| 98 |
+
}
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
## Links
|
| 102 |
+
|
| 103 |
+
- **Original model:** [eegdino/EEG-DINO](https://huggingface.co/eegdino/EEG-DINO)
|
| 104 |
+
- **Original code:** [github.com/miraclefish/EEG-DINO](https://github.com/miraclefish/EEG-DINO)
|
| 105 |
+
- **Rust crate:** [crates.io/crates/eegdino](https://crates.io/crates/eegdino)
|
| 106 |
+
- **Rust repo:** [github.com/eugenehp/eegdino-rs](https://github.com/eugenehp/eegdino-rs)
|
| 107 |
+
|
| 108 |
+
## License
|
| 109 |
+
|
| 110 |
+
MIT (conversion and crate). Original model weights are subject to the [EEG-DINO license](https://huggingface.co/eegdino/EEG-DINO).
|
eeg_dino_large.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:27cef0c292ce148ec08115176df74bcd6f60512687f30ef603e5c57e212c1871
|
| 3 |
+
size 807669440
|
eeg_dino_medium.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1d25ca43726b9e8c2ac07fba2bdbc8f70721ec481ad82ddcb1585a5d703009d6
|
| 3 |
+
size 135044600
|
eeg_dino_small.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6059edbfbdf1933d66ec009e469bd37b6e3af142e0a2026f231323f4fd9caf6c
|
| 3 |
+
size 17752992
|