Ryukijano/esd-world-predict-lora-iter600

ESD-WORLD Predict LoRA (iter_000000600)

Private research checkpoint: LoRA fine-tune of NVIDIA-Medtech Cosmos-H-Surgical Predict 15B v0.3.0 on endoscopic submucosal dissection (ESD) video for future-frame prediction.

Not a medical device. Not for clinical use. No patient videos or ESD dataset files are on this Hub repo.

• Code and Hydra configs: github.com/Ryukijano/ESD-WORLD (auth works; ESD-WORLD configs are pushed there)
• W&B: hack-the-thong/esd_world/0dt9xe0n
• Base: NVIDIA-Medtech/Cosmos-H-Surgical Predict 15B v0.3.0

This Hub repository stays private.

Training snapshot

Distributed Checkpoint (DCP) from AIRE 3× L40S FSDP. Last complete iteration is iter_000000600.

Item	Value
Base	NVIDIA-Medtech Cosmos-H-Surgical Predict 15B v0.3.0
Adapter	LoRA rank 16 / alpha 32
Target modules	q_proj_moe_gen, k_proj_moe_gen, v_proj_moe_gen, o_proj_moe_gen
Train hardware	3× NVIDIA L40S 48GB, FSDP shard degree 3, bf16
This snapshot	iter_000000600 (~29 GB DCP: model/ + optim/ + scheduler/ + trainer/)

Download

Use the hf CLI (not the broken default-PATH huggingface-cli):

hf download Ryukijano/esd-world-predict-lora-iter600

You need a Hugging Face token with access to this private repo.

Repository layout

config.yaml                 # Hydra dump from the AIRE run
latest_checkpoint.txt
iter_000000600/
  model/                    # ~29 GB, 3 distcp shards
  optim/
  scheduler/
  trainer/
README.md

This is not a merged safetensors export. Optimizer shards are included so training can resume.

Inference

1× L40S 48GB works. A2 15GB does not.

From a Cosmos-H-Surgical / cosmos-framework environment:

COSMOS_TRAINING=1 python -m cosmos_framework.scripts.inference \
  --checkpoint-path iter_000000600 \
  --config-file config.yaml \
  --parallelism-preset=latency \
  --dp-shard-size=1 \
  --no-guardrails \
  --no-use-torch-compile \
  --no-use-cuda-graphs

You still need the base Cosmos-H-Surgical install, tokenizers, and VAE / reasoner weights. This repo only stores the ESD LoRA DCP plus config.yaml.

config.yaml contains AIRE-local dataset paths (/scratch/kcwp264/...). Point those at your own manifests. Do not expect ESD videos here.

Training wrappers and configs: Ryukijano/ESD-WORLD.

License and research disclaimer

• Derivative of NVIDIA Cosmos-H-Surgical Predict weights. Not an unrestricted commercial release.
• v0.3.0 weights follow OpenMDW License Agreement v1.1. See NVIDIA’s Cosmos-H-Surgical (LICENSE.weights). Downstream use of these weights must follow NVIDIA / OpenMDW terms.
• ESD-WORLD code on GitHub is MIT. MIT does not cover NVIDIA pretrained parameters.
• Hub license: other reflects this mixed stack.

Research only. Not for diagnosis, treatment, surgical guidance, or any clinical deployment. No patient-identifiable video is distributed here.

Provenance

• Cluster: University of Leeds AIRE (L40S)
• Project: esd_world / predict_lora_3gpu / esd_predict_lora_3gpu
• W&B: https://wandb.ai/hack-the-thong/esd_world/runs/0dt9xe0n