Instructions to use moxeeeem/pubmedclip-gpt2-captioner with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moxeeeem/pubmedclip-gpt2-captioner with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="moxeeeem/pubmedclip-gpt2-captioner")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("moxeeeem/pubmedclip-gpt2-captioner", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload final_captioner_gpt2-medium_CLIPModel.json with huggingface_hub
Browse files
final_captioner_gpt2-medium_CLIPModel.json
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"gpt2_name": "gpt2-medium",
|
| 3 |
+
"clip_backend_kind": "hf_clip",
|
| 4 |
+
"clip_repo": "flaviagiammarino/pubmed-clip-vit-base-patch32",
|
| 5 |
+
"clip_arch": "CLIPModel",
|
| 6 |
+
"clip_weight_path": "flaviagiammarino/pubmed-clip-vit-base-patch32",
|
| 7 |
+
"embed_dim": 512,
|
| 8 |
+
"prefix_tokens": 32,
|
| 9 |
+
"prompt": "Describe the skin lesion concisely (morphology, color, scale, border, location) in one sentence.Conclude with the most likely diagnosis (1–3 words).",
|
| 10 |
+
"cliche_phrases": [
|
| 11 |
+
"seek medical attention",
|
| 12 |
+
"consult a dermatologist",
|
| 13 |
+
"promptly for further",
|
| 14 |
+
"It is recommended to",
|
| 15 |
+
"further evaluation is required",
|
| 16 |
+
"should be considered for",
|
| 17 |
+
"medical attention",
|
| 18 |
+
"follow up",
|
| 19 |
+
"consult your doctor",
|
| 20 |
+
"should be evaluated"
|
| 21 |
+
],
|
| 22 |
+
"columns": {
|
| 23 |
+
"image_path": "img_path",
|
| 24 |
+
"caption": "caption",
|
| 25 |
+
"split": "split",
|
| 26 |
+
"source": "source"
|
| 27 |
+
}
|
| 28 |
+
}
|