Update README
Browse files
README.md
CHANGED
|
@@ -8,13 +8,8 @@ language:
|
|
| 8 |
|
| 9 |
## Model Summary
|
| 10 |
|
| 11 |
-
**HistAug** is a lightweight transformer-based generator for **controllable latent-space augmentations** in the feature space of the [H-optimus-1 foundation model](https://huggingface.co/bioptimus/H-optimus-1).
|
| 12 |
|
| 13 |
-
Instead of applying costly image-space augmentations on millions of WSI patches, HistAug operates **directly on patch embeddings** extracted from H-optimus-1.
|
| 14 |
-
|
| 15 |
-
By conditioning on explicit transformation parameters (e.g., hue shift, erosion, HED color transform), HistAug generates realistic augmented embeddings while preserving semantic content.
|
| 16 |
-
|
| 17 |
-
In practice, the H-optimus-1 variant of HistAug can reconstruct the corresponding “true” augmented features with an average cosine similarity of **about 81%** at **10X, 20X, and 40X magnification**.
|
| 18 |
|
| 19 |
This enables training of Multiple Instance Learning (MIL) models with:
|
| 20 |
- ⚡ **Fast augmentation**
|
|
@@ -24,7 +19,8 @@ This enables training of Multiple Instance Learning (MIL) models with:
|
|
| 24 |
Need HistAug for a different foundation model? Explore the full collection: [**HistAug models collection**](https://huggingface.co/collections/sofieneb/histaug-models-68a334437f71d35c7037a54e).
|
| 25 |
|
| 26 |
|
| 27 |
-
📄 **Paper**: *Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025)
|
|
|
|
| 28 |
|
| 29 |
|
| 30 |
|
|
@@ -38,14 +34,16 @@ You can load the model from the Hub with Hugging Face’s `transformers`:
|
|
| 38 |
import torch
|
| 39 |
from transformers import AutoModel
|
| 40 |
|
|
|
|
|
|
|
| 41 |
# Load HistAug (H-optimus-1 latent augmentation model)
|
| 42 |
model_id = "sofieneb/histaug-hoptimus1"
|
| 43 |
-
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
|
| 44 |
|
| 45 |
# Example: patch embeddings from H-optimus-1
|
| 46 |
num_patches = 50000
|
| 47 |
embedding_dim = 1536
|
| 48 |
-
patch_embeddings = torch.randn((num_patches, embedding_dim), device=
|
| 49 |
|
| 50 |
# Sample augmentation parameters
|
| 51 |
# mode="wsi_wise" applies the same transformation across the whole slide
|
|
@@ -197,22 +195,54 @@ for bag_features, label in loader: # bag_features: (num_patches, embed_dim)
|
|
| 197 |
|
| 198 |
## Offline usage (HPC clusters without internet)
|
| 199 |
|
| 200 |
-
If your cluster doesn’t allow internet access on compute nodes,
|
|
|
|
|
|
|
| 201 |
|
| 202 |
```bash
|
| 203 |
# On the front-end/login node (with internet):
|
| 204 |
-
python -c "from transformers import AutoModel; AutoModel.from_pretrained('sofieneb/histaug-
|
| 205 |
|
| 206 |
# On your compute job (no internet):
|
| 207 |
export HF_HUB_OFFLINE=1
|
| 208 |
export TRANSFORMERS_OFFLINE=1
|
| 209 |
-
```
|
| 210 |
|
| 211 |
This prevents unnecessary network calls and ensures `transformers` loads HistAug from the local cache.
|
| 212 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 213 |
---
|
| 214 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 215 |
|
| 216 |
-
*Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025).
|
| 217 |
|
| 218 |
|
|
|
|
| 8 |
|
| 9 |
## Model Summary
|
| 10 |
|
|
|
|
| 11 |
|
| 12 |
+
**HistAug** is a lightweight transformer-based generator for **controllable latent-space augmentations** in the feature space of the [H-optimus-1 foundation model](https://huggingface.co/bioptimus/H-optimus-1). Instead of applying costly image-space augmentations on millions of WSI patches, HistAug operates **directly on patch embeddings** extracted from a given foundation model(here H-optimus-1). By conditioning on explicit transformation parameters (e.g., hue shift, erosion, HED color transform), HistAug generates realistic augmented embeddings while preserving semantic content. In practice, the H-optimus-1 variant of HistAug can reconstruct the corresponding ground-truth augmented embeddings with an average cosine similarity of **about 90%** at **10X, 20X, and 40X magnification**.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
This enables training of Multiple Instance Learning (MIL) models with:
|
| 15 |
- ⚡ **Fast augmentation**
|
|
|
|
| 19 |
Need HistAug for a different foundation model? Explore the full collection: [**HistAug models collection**](https://huggingface.co/collections/sofieneb/histaug-models-68a334437f71d35c7037a54e).
|
| 20 |
|
| 21 |
|
| 22 |
+
📄 **Paper**: [*Controllable Latent Space Augmentation for Digital Pathology* (Boutaj *et al.*, 2025)](https://arxiv.org/abs/2508.14588)
|
| 23 |
+
|
| 24 |
|
| 25 |
|
| 26 |
|
|
|
|
| 34 |
import torch
|
| 35 |
from transformers import AutoModel
|
| 36 |
|
| 37 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 38 |
+
|
| 39 |
# Load HistAug (H-optimus-1 latent augmentation model)
|
| 40 |
model_id = "sofieneb/histaug-hoptimus1"
|
| 41 |
+
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device)
|
| 42 |
|
| 43 |
# Example: patch embeddings from H-optimus-1
|
| 44 |
num_patches = 50000
|
| 45 |
embedding_dim = 1536
|
| 46 |
+
patch_embeddings = torch.randn((num_patches, embedding_dim), device=device)
|
| 47 |
|
| 48 |
# Sample augmentation parameters
|
| 49 |
# mode="wsi_wise" applies the same transformation across the whole slide
|
|
|
|
| 195 |
|
| 196 |
## Offline usage (HPC clusters without internet)
|
| 197 |
|
| 198 |
+
If your cluster doesn’t allow internet access on compute nodes, you have two ways to use HistAug offline:
|
| 199 |
+
|
| 200 |
+
1. **Rely on the cache**: pre-download the model on the front-end/login node (with internet), so it’s cached locally, then run jobs offline:
|
| 201 |
|
| 202 |
```bash
|
| 203 |
# On the front-end/login node (with internet):
|
| 204 |
+
python -c "from transformers import AutoModel; AutoModel.from_pretrained('sofieneb/histaug-uni', trust_remote_code=True)"
|
| 205 |
|
| 206 |
# On your compute job (no internet):
|
| 207 |
export HF_HUB_OFFLINE=1
|
| 208 |
export TRANSFORMERS_OFFLINE=1
|
| 209 |
+
````
|
| 210 |
|
| 211 |
This prevents unnecessary network calls and ensures `transformers` loads HistAug from the local cache.
|
| 212 |
|
| 213 |
+
2. **Explicitly download with `hf download`**: instead of relying on the cache variables above, you can download the model files manually on the front-end/login node (with internet) and always point to the local folder:
|
| 214 |
+
|
| 215 |
+
```bash
|
| 216 |
+
# On the front-end/login node (with internet):
|
| 217 |
+
hf download sofieneb/histaug-virchow2 --local-dir ./histaug-virchow2
|
| 218 |
+
```
|
| 219 |
+
|
| 220 |
+
Then load the model from that directory in your script:
|
| 221 |
+
|
| 222 |
+
```python
|
| 223 |
+
from transformers import AutoModel
|
| 224 |
+
|
| 225 |
+
cross_transformer = AutoModel.from_pretrained(
|
| 226 |
+
"./histaug-virchow2", # local path instead of hub ID
|
| 227 |
+
trust_remote_code=True,
|
| 228 |
+
local_files_only=True
|
| 229 |
+
)
|
| 230 |
+
```
|
| 231 |
---
|
| 232 |
+
## Citation
|
| 233 |
+
If our work contributes to your research, or if you incorporate part of this code, please consider citing our paper:
|
| 234 |
+
|
| 235 |
+
```bibtex
|
| 236 |
+
@misc{boutaj2025controllablelatentspaceaugmentation,
|
| 237 |
+
title={Controllable Latent Space Augmentation for Digital Pathology},
|
| 238 |
+
author={Sofiène Boutaj and Marin Scalbert and Pierre Marza and Florent Couzinie-Devy and Maria Vakalopoulou and Stergios Christodoulidis},
|
| 239 |
+
year={2025},
|
| 240 |
+
eprint={2508.14588},
|
| 241 |
+
archivePrefix={arXiv},
|
| 242 |
+
primaryClass={cs.CV},
|
| 243 |
+
url={https://arxiv.org/abs/2508.14588},
|
| 244 |
+
}
|
| 245 |
+
```
|
| 246 |
|
|
|
|
| 247 |
|
| 248 |
|