Upload README.md with huggingface_hub

README.md

@@ -5,6 +5,7 @@ tags:
   - CLIP
   - GPT-2
   - dermatology
+  - pubmedclip
 library_name: transformers
 license: other
 language:
@@ -12,20 +13,22 @@ language:
 pipeline_tag: image-to-text
 ---
 
-# moxeeeem/pubmedclip-gpt2-captioner
+# PubMedCLIP + GPT-2 Dermatology Captioner
 
-**Dermatology image captioner**: CLIP encoder (DermLIP or PubMed-CLIP) → learnable prefix → GPT-2 (`gpt2-medium`).
+A dermatology image captioning model combining PubMedCLIP vision encoder with gpt2-medium language model. Trained on dermatological images for generating clinical descriptions of skin lesions.
+
+**Architecture**: PubMedCLIP (ViT-B/32) → learnable prefix → GPT-2 (`gpt2-medium`).
 Trained in two stages: Stage A (META) for generalization and Stage B (SkinCAP) for style/terminology.
 
 
 ## Metrics
 **Stage A (META)**  
-val_loss=1.0899 • PPL=2.97  
-BLEU=36.7 • ROUGE-L=0.524 • CIDEr-D=0.13 • CLIP=25.3 • BERT_F1=0.531
+val_loss=1.1222 • PPL=3.07  
+BLEU=36.6 • ROUGE-L=0.521 • CIDEr-D=0.10 • CLIP=34.7 • BERT_F1=0.526
 
 **Stage B (SKINCAP)**  
-val_loss=1.2094 • PPL=3.35  
-BLEU=8.9 • ROUGE-L=0.257 • CIDEr-D=0.10 • CLIP=28.6 • BERT_F1=0.346
+val_loss=1.1997 • PPL=3.32  
+BLEU=9.3 • ROUGE-L=0.267 • CIDEr-D=0.12 • CLIP=40.5 • BERT_F1=0.348
 
 ## Inference
 
@@ -53,16 +56,22 @@ for c in caps:
 ## Files
 | File | Size | Check |
 |---|---:|---|
-| `best_stageA.pt` | 2 GB | sha256[:12]=9b49360b7643 |
-| `best_stageB.pt` | 2 GB | sha256[:12]=7f4e6a05fc14 |
-| `final_captioner_gpt2-medium_CLIPModel.json` | 868 B | sha256[:12]=7cabfec71541 |
-| `final_captioner_gpt2-medium_CLIPModel.pt` | 2 GB | sha256[:12]=b29fa622b77e |
-| `loss_pubmed_clip_b32.png` | 84 KB | sha256[:12]=7e1cabd2ad24 |
+| `best_stageA.pt` | 2 GB | sha256[:12]=4c3f773c92f9 |
+| `best_stageB.pt` | 2 GB | sha256[:12]=2b1b7e6aaa8f |
+| `final_captioner_gpt2-medium_TimmModel.json` | 899 B | sha256[:12]=40f3fb250eab |
+| `final_captioner_gpt2-medium_TimmModel.pt` | 2 GB | sha256[:12]=799a4d1b831d |
+| `loss_biomedclip_pubmedbert.png` | 112 KB | sha256[:12]=92c3b8dfdf7f |
 
 ## Details
 
-- **Backbones**: GPT-2 (`gpt2-medium`), CLIP weights: `flaviagiammarino/pubmed-clip-vit-base-patch32`.  
+- **Vision Encoder**: PubMedCLIP (ViT-B/32)
+- **Language Model**: GPT-2 (`gpt2-medium`)
+- **CLIP weights**: `hf-hub:microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224`
 - **Prefix tokens**: 32  
-- **Prompt** (used at inference): `Describe the skin lesion concisely (morphology, color, scale, border, location) in one sentence.Conclude with the most likely diagnosis (1–3 words).`
+- **Training prompt**: `Describe the skin lesion concisely (morphology, color, scale, border, location) in one sentence.Conclude with the most likely diagnosis (1–3 words).`
+
+### Model Type Detection
+- Detected as: `pubmedclip`
+- Repository: `moxeeeem/pubmedclip-gpt2-captioner`
 
-_Auto-generated on 2025-08-29 20:58 UTC._
+_Auto-generated on 2025-08-30 06:06 UTC._