PierreGtch commited on
Commit
50c4bd8
Β·
verified Β·
1 Parent(s): 9e20126

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +137 -10
README.md CHANGED
@@ -1,15 +1,142 @@
1
  ---
 
2
  library_name: braindecode
3
- license: bsd-3-clause
4
  tags:
5
- - SignalJEPA
6
- - _BaseSignalJEPA
7
- - braindecode
8
- - model_hub_mixin
9
- - pytorch_model_hub_mixin
10
  ---
11
 
12
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
13
- - Code: https://braindecode.org
14
- - Paper: [More Information Needed]
15
- - Docs: https://braindecode.org/stable/generated/braindecode.models.SignalJEPA.html
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: mit
3
  library_name: braindecode
 
4
  tags:
5
+ - eeg
6
+ - foundation-model
7
+ - self-supervised
8
+ - signal-jepa
9
+ pipeline_tag: feature-extraction
10
  ---
11
 
12
+ ![sjepa](https://cdn-uploads.huggingface.co/production/uploads/646e0135174cc96d509582a6/DS-cXrFyxZ78hK48ft0iU.png)
13
+
14
+ # Signal-JEPA
15
+
16
+ Self-supervised pre-trained weights for the Signal-JEPA foundation model from
17
+ [Guetschel et al. (2024)](https://arxiv.org/abs/2403.11772), packaged for use
18
+ with [braindecode](https://braindecode.org/). See the full API reference in
19
+ the docs: [`braindecode.models.SignalJEPA`](https://braindecode.org/stable/generated/braindecode.models.SignalJEPA.html).
20
+
21
+ The model was pre-trained on the Lee2019 dataset (62 EEG channels in the
22
+ 10-10 layout, sampled at 128 Hz). The repo ships the weights together with a
23
+ `config.json` so they can be loaded in one line with
24
+ `YourModelClass.from_pretrained(repo_id, ...)`.
25
+
26
+ ## Available checkpoints
27
+
28
+ Two variants are published:
29
+
30
+ | repo ID | channel embedding included | when to use |
31
+ | --- | --- | --- |
32
+ | [`braindecode/signal-jepa`](https://huggingface.co/braindecode/signal-jepa) | βœ“ 62-row `_ChannelEmbedding` aligned with the pre-training layout | your recording channels are a **subset** (by name, case-insensitive) of the 62 pre-training channels β€” you want to reuse the learned spatial embeddings |
33
+ | [`braindecode/signal-jepa_without-chans`](https://huggingface.co/braindecode/signal-jepa_without-chans) | βœ— only the SSL backbone (feature encoder + transformer) | your channels are **not** a subset of the pre-training set, or you prefer to train channel embeddings from scratch |
34
+
35
+ If you are unsure, start with `braindecode/signal-jepa_without-chans`: it
36
+ always works, regardless of your electrode layout.
37
+
38
+ ## Quick start
39
+
40
+ ### Base model (pre-training architecture)
41
+
42
+ The base model outputs contextual features, not class predictions. Use it
43
+ for downstream feature extraction or further SSL.
44
+
45
+ ```python
46
+ from braindecode.models import SignalJEPA
47
+
48
+ # With the pre-trained channel embeddings (recording channels βŠ‚ pre-train set):
49
+ model = SignalJEPA.from_pretrained("braindecode/signal-jepa")
50
+
51
+ # Or: with your own channels, kept aligned to the pre-training embedding table
52
+ model = SignalJEPA.from_pretrained(
53
+ "braindecode/signal-jepa",
54
+ chs_info=raw.info["chs"], # subset of the 62 pre-training channels
55
+ channel_embedding="pretrain_aligned",
56
+ )
57
+
58
+ # Or: without pre-trained channel embeddings (any electrode layout):
59
+ model = SignalJEPA.from_pretrained(
60
+ "braindecode/signal-jepa_without-chans",
61
+ chs_info=raw.info["chs"],
62
+ strict=False, # the channel-embedding weight is intentionally missing
63
+ )
64
+ ```
65
+
66
+ ### Downstream architectures
67
+
68
+ Three classification architectures are introduced in the paper:
69
+
70
+ - **a) Contextual** β€” uses the full transformer encoder
71
+ - **b) Post-local** β€” discards the transformer; spatial convolution after local features
72
+ - **c) Pre-local** β€” discards the transformer; spatial convolution before local features
73
+
74
+ All three add a freshly-initialized classification head on top of the SSL
75
+ backbone. The head is **not** part of the checkpoint and will be trained from
76
+ scratch during fine-tuning; pass `strict=False` so `from_pretrained` does not
77
+ complain about those missing keys.
78
+
79
+ ```python
80
+ from braindecode.models import (
81
+ SignalJEPA_Contextual,
82
+ SignalJEPA_PreLocal,
83
+ SignalJEPA_PostLocal,
84
+ )
85
+
86
+ # a) Contextual β€” keeps the transformer
87
+ model = SignalJEPA_Contextual.from_pretrained(
88
+ "braindecode/signal-jepa", # or "signal-jepa_without-chans"
89
+ n_times=256, # e.g. 2 s at 128 Hz
90
+ n_outputs=4,
91
+ strict=False, # ignore un-trained classification head
92
+ )
93
+
94
+ # b) Post-local β€” transformer discarded
95
+ model = SignalJEPA_PostLocal.from_pretrained(
96
+ "braindecode/signal-jepa_without-chans",
97
+ n_chans=19,
98
+ n_times=256,
99
+ n_outputs=4,
100
+ strict=False,
101
+ )
102
+
103
+ # c) Pre-local β€” transformer discarded
104
+ model = SignalJEPA_PreLocal.from_pretrained(
105
+ "braindecode/signal-jepa_without-chans",
106
+ n_chans=19,
107
+ n_times=256,
108
+ n_outputs=4,
109
+ strict=False,
110
+ )
111
+ ```
112
+
113
+ See the braindecode tutorial
114
+ [Fine-tuning a Foundation Model (Signal-JEPA)](https://braindecode.org/stable/auto_examples/advanced_training/plot_finetune_foundation_model.html)
115
+ for a complete example including layer freezing and training with
116
+ `skorch.EEGClassifier`.
117
+
118
+ ## Channel embedding modes
119
+
120
+ `SignalJEPA` and `SignalJEPA_Contextual` accept a `channel_embedding` kwarg:
121
+
122
+ - `"scratch"` (default): the `_ChannelEmbedding` table has one row per user
123
+ channel, initialized from `chs_info`. Compatible with the
124
+ `without-chans` checkpoint.
125
+ - `"pretrain_aligned"`: the table has 62 rows in the pre-training order,
126
+ `forward` indexes into the subset matching your `chs_info` (matched by
127
+ channel name, case-insensitive). Compatible with the full checkpoint.
128
+
129
+ `from_pretrained` picks the right mode automatically based on the checkpoint's
130
+ `config.json`; override with the `channel_embedding=` kwarg if needed.
131
+
132
+ ## Citation
133
+
134
+ ```bibtex
135
+ @article{guetschel2024sjepa,
136
+ title = {S-JEPA: towards seamless cross-dataset transfer
137
+ through dynamic spatial attention},
138
+ author = {Guetschel, Pierre and Moreau, Thomas and Tangermann, Michael},
139
+ journal = {arXiv preprint arXiv:2403.11772},
140
+ year = {2024},
141
+ }
142
+ ```