---
license: other
license_name: physionet-dua
license_link: https://physionet.org/about/licenses/
library_name: pytorch
tags: [chest-xray, radiology, vision-encoder, vit, medical-imaging]
pipeline_tag: image-feature-extraction
extra_gated_heading: "Access requires PhysioNet credentialing"
extra_gated_prompt: >-
  These weights are derived from MIMIC-CXR and Chest ImaGenome, distributed under the
  PhysioNet Credentialed Health Data Use Agreement. By requesting access you confirm you
  hold the required PhysioNet credentialed access for BOTH source datasets and will comply
  with the PhysioNet DUA. Access is granted manually by the repository owner.
extra_gated_fields:
  Full name: text
  Affiliation: text
  Intended use: text
  I hold PhysioNet credentialed access to MIMIC-CXR and Chest ImaGenome: checkbox
  I will comply with the PhysioNet Data Use Agreement and not redistribute these weights: checkbox
extra_gated_button_content: "Request access"
---

# CXR ViT-L/14 — global local cap

ViT-L/14 chest-radiograph vision encoder, **pretrained from scratch** with the objective:
**Global + local generative captioning: CAP (image->report) + GCAP/AREF/CAREF (box<->phrase).**

Part of a controlled study comparing pretraining objectives for chest-X-ray vision encoders
(all share the same ViT-L/14 backbone, data, and budget).

## Files
| File | What it is |
|---|---|
| `encoder_final.pt` | **Vision encoder** — ViT-L/14 trunk only (`state_dict` under key `encoder_state`, 294 tensors). Use this to extract frozen image features. |
| `model_best.pt` | **Full pretraining model** — ViT-L/14 + GPT-2-style causal decoder (shared across CAP/GCAP/AREF/CAREF) (`state_dict` under key `model_state`). |

## Architecture & training
- ViT-L/14, 384x384 input -> 729 patch tokens (27x27 grid), trained **from scratch** in bf16. Text side (where present) uses the GPT-2 tokenizer (vocab 50257).
- **Training data:** MIMIC-CXR (reports) and Chest ImaGenome (region-phrase / anatomy boxes).
- **Objective:** Global + local generative captioning: CAP (image->report) + GCAP/AREF/CAREF (box<->phrase).

## Downstream results (frozen-feature transfer)
Best on out-of-domain RRG: CheXpert Plus #1 (RadGraph-s 0.191), IU X-ray top surface metrics; weaker on in-domain discriminative transfer. encoder_final.pt = epoch 46.

Full per-task comparison across all 9 from-scratch encoders is in the project logbook.

## Usage (vision encoder)
```python
import torch
ckpt = torch.load("encoder_final.pt", map_location="cpu", weights_only=False)
vision_state = ckpt["encoder_state"]   # 294 tensors, ViT-L/14
# load into your ViT-L/14 implementation, then forward 384x384 images -> [B, 729, 1024]
```

## Intended use & limitations
- **Research use only.** Frozen feature extraction / fine-tuning for chest-X-ray tasks.
- **NOT a diagnostic device.** No clinical or patient-facing use.
- Trained only on adult frontal/lateral CXR distributions of the source datasets; may not
  generalize to other modalities, body regions, populations, or acquisition settings.

## ⚠️ License / Data Use Agreement
These weights are derived from **MIMIC-CXR** and **Chest ImaGenome**, which are distributed
under the **PhysioNet Credentialed Health Data Use Agreement**. Redistribution and use of
models derived from these data are subject to that DUA — you must hold the appropriate
credentialed access and comply with its terms. Do **not** make this repository public or
share access without confirming your DUA permits sharing derived model weights.

## Citation
If you use this encoder, please cite the source datasets (MIMIC-CXR; Chest ImaGenome) and
this project.