sofieneb commited on
Commit
d5a8f43
·
1 Parent(s): 4ec868d

Update README

Browse files
Files changed (1) hide show
  1. README.md +44 -14
README.md CHANGED
@@ -8,13 +8,8 @@ language:
8
 
9
  ## Model Summary
10
 
11
- **HistAug** is a lightweight transformer-based generator for **controllable latent-space augmentations** in the feature space of the [H-optimus-1 foundation model](https://huggingface.co/bioptimus/H-optimus-1).
12
 
13
- Instead of applying costly image-space augmentations on millions of WSI patches, HistAug operates **directly on patch embeddings** extracted from H-optimus-1.
14
-
15
- By conditioning on explicit transformation parameters (e.g., hue shift, erosion, HED color transform), HistAug generates realistic augmented embeddings while preserving semantic content.
16
-
17
- In practice, the H-optimus-1 variant of HistAug can reconstruct the corresponding “true” augmented features with an average cosine similarity of **about 81%** at **10X, 20X, and 40X magnification**.
18
 
19
  This enables training of Multiple Instance Learning (MIL) models with:
20
  - ⚡ **Fast augmentation**
@@ -24,7 +19,8 @@ This enables training of Multiple Instance Learning (MIL) models with:
24
  Need HistAug for a different foundation model? Explore the full collection: [**HistAug models collection**](https://huggingface.co/collections/sofieneb/histaug-models-68a334437f71d35c7037a54e).
25
 
26
 
27
- 📄 **Paper**: *Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025)
 
28
 
29
 
30
 
@@ -38,14 +34,16 @@ You can load the model from the Hub with Hugging Face’s `transformers`:
38
  import torch
39
  from transformers import AutoModel
40
 
 
 
41
  # Load HistAug (H-optimus-1 latent augmentation model)
42
  model_id = "sofieneb/histaug-hoptimus1"
43
- model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
44
 
45
  # Example: patch embeddings from H-optimus-1
46
  num_patches = 50000
47
  embedding_dim = 1536
48
- patch_embeddings = torch.randn((num_patches, embedding_dim), device="cuda")
49
 
50
  # Sample augmentation parameters
51
  # mode="wsi_wise" applies the same transformation across the whole slide
@@ -197,22 +195,54 @@ for bag_features, label in loader: # bag_features: (num_patches, embed_dim)
197
 
198
  ## Offline usage (HPC clusters without internet)
199
 
200
- If your cluster doesn’t allow internet access on compute nodes, **pre-download the model on the front-end/login node** (which has internet), so it’s cached locally, then run jobs offline:
 
 
201
 
202
  ```bash
203
  # On the front-end/login node (with internet):
204
- python -c "from transformers import AutoModel; AutoModel.from_pretrained('sofieneb/histaug-hoptimus1', trust_remote_code=True)"
205
 
206
  # On your compute job (no internet):
207
  export HF_HUB_OFFLINE=1
208
  export TRANSFORMERS_OFFLINE=1
209
- ```
210
 
211
  This prevents unnecessary network calls and ensures `transformers` loads HistAug from the local cache.
212
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
213
  ---
214
- ## Paper
 
 
 
 
 
 
 
 
 
 
 
 
 
215
 
216
- *Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025).
217
 
218
 
 
8
 
9
  ## Model Summary
10
 
 
11
 
12
+ **HistAug** is a lightweight transformer-based generator for **controllable latent-space augmentations** in the feature space of the [H-optimus-1 foundation model](https://huggingface.co/bioptimus/H-optimus-1). Instead of applying costly image-space augmentations on millions of WSI patches, HistAug operates **directly on patch embeddings** extracted from a given foundation model(here H-optimus-1). By conditioning on explicit transformation parameters (e.g., hue shift, erosion, HED color transform), HistAug generates realistic augmented embeddings while preserving semantic content. In practice, the H-optimus-1 variant of HistAug can reconstruct the corresponding ground-truth augmented embeddings with an average cosine similarity of **about 90%** at **10X, 20X, and 40X magnification**.
 
 
 
 
13
 
14
  This enables training of Multiple Instance Learning (MIL) models with:
15
  - ⚡ **Fast augmentation**
 
19
  Need HistAug for a different foundation model? Explore the full collection: [**HistAug models collection**](https://huggingface.co/collections/sofieneb/histaug-models-68a334437f71d35c7037a54e).
20
 
21
 
22
+ 📄 **Paper**: [*Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025)](https://arxiv.org/abs/2508.14588)
23
+
24
 
25
 
26
 
 
34
  import torch
35
  from transformers import AutoModel
36
 
37
+ device = "cuda" if torch.cuda.is_available() else "cpu"
38
+
39
  # Load HistAug (H-optimus-1 latent augmentation model)
40
  model_id = "sofieneb/histaug-hoptimus1"
41
+ model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device)
42
 
43
  # Example: patch embeddings from H-optimus-1
44
  num_patches = 50000
45
  embedding_dim = 1536
46
+ patch_embeddings = torch.randn((num_patches, embedding_dim), device=device)
47
 
48
  # Sample augmentation parameters
49
  # mode="wsi_wise" applies the same transformation across the whole slide
 
195
 
196
  ## Offline usage (HPC clusters without internet)
197
 
198
+ If your cluster doesn’t allow internet access on compute nodes, you have two ways to use HistAug offline:
199
+
200
+ 1. **Rely on the cache**: pre-download the model on the front-end/login node (with internet), so it’s cached locally, then run jobs offline:
201
 
202
  ```bash
203
  # On the front-end/login node (with internet):
204
+ python -c "from transformers import AutoModel; AutoModel.from_pretrained('sofieneb/histaug-uni', trust_remote_code=True)"
205
 
206
  # On your compute job (no internet):
207
  export HF_HUB_OFFLINE=1
208
  export TRANSFORMERS_OFFLINE=1
209
+ ````
210
 
211
  This prevents unnecessary network calls and ensures `transformers` loads HistAug from the local cache.
212
 
213
+ 2. **Explicitly download with `hf download`**: instead of relying on the cache variables above, you can download the model files manually on the front-end/login node (with internet) and always point to the local folder:
214
+
215
+ ```bash
216
+ # On the front-end/login node (with internet):
217
+ hf download sofieneb/histaug-virchow2 --local-dir ./histaug-virchow2
218
+ ```
219
+
220
+ Then load the model from that directory in your script:
221
+
222
+ ```python
223
+ from transformers import AutoModel
224
+
225
+ cross_transformer = AutoModel.from_pretrained(
226
+ "./histaug-virchow2", # local path instead of hub ID
227
+ trust_remote_code=True,
228
+ local_files_only=True
229
+ )
230
+ ```
231
  ---
232
+ ## Citation
233
+ If our work contributes to your research, or if you incorporate part of this code, please consider citing our paper:
234
+
235
+ ```bibtex
236
+ @misc{boutaj2025controllablelatentspaceaugmentation,
237
+ title={Controllable Latent Space Augmentation for Digital Pathology},
238
+ author={Sofiène Boutaj and Marin Scalbert and Pierre Marza and Florent Couzinie-Devy and Maria Vakalopoulou and Stergios Christodoulidis},
239
+ year={2025},
240
+ eprint={2508.14588},
241
+ archivePrefix={arXiv},
242
+ primaryClass={cs.CV},
243
+ url={https://arxiv.org/abs/2508.14588},
244
+ }
245
+ ```
246
 
 
247
 
248