CLIP-planktonzilla-epoch30

Intermediate open_clip training checkpoint (epoch 30) from CLIP-style training on the plankton subset of Planktonzilla-17M, for the paper Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems (Contreras, Valenzuela, Martí & Sanchez-Pi, 2026).

This repo contains a single native-format training checkpoint (checkpoint.pt) — not a ready-to-load hf-hub: model. For downstream use, prefer the packaged CLIP models in Project OcéanIA (e.g. CLIP-ViT-B-16.openai-pt.planktonzilla-pt, CLIP-ViT-L-14.laion2b-pt.planktonzilla-pt), whose per-rank Macro-F1 results are reported in Table 2 of the paper. The paper's reported CLIP runs use 100 epochs; this is an epoch-30 snapshot.

Usage

import torch
from huggingface_hub import hf_hub_download
import open_clip

ckpt = hf_hub_download("project-oceania/CLIP-planktonzilla-epoch30", "checkpoint.pt")
state = torch.load(ckpt, map_location="cpu")
# model, _, preprocess = open_clip.create_model_and_transforms("<ARCH>")
# sd = state.get("state_dict", state)
# model.load_state_dict({k.replace("module.", ""): v for k, v in sd.items()})

Intended use & limitations

An intermediate research artifact for reproducing the paper's CLIP training. Same domain limitations apply as the packaged models (imager and long-tail sensitivity; read predictions at the deepest valid taxonomic rank).

Citation

@article{contreras2026planktonzilla,
  title   = {Planktonzilla: Multimodal dataset and models for understanding plankton ecosystems},
  author  = {Contreras, Alan and Valenzuela, Luis and Mart\'i, Luis and Sanchez-Pi, Nayat},
  journal = {arXiv preprint arXiv:2606.00080},
  year    = {2026},
  doi     = {10.48550/arXiv.2606.00080}
}

Model produced by Project OcéanIA / Inria Chile Research Center. Code: Inria-Chile/planktonzilla.