Fix generation from the published artifact; rewrite model card

README.md

@@ -27,11 +27,10 @@ text-only reasoning SLM — with visual capabilities via the
 [InternViT-300M-448px-V2.5](https://huggingface.co/OpenGVLab/InternViT-300M-448px-V2_5)
 vision encoder and a lightweight MLP projector, for a total of ~628M parameters.
 
-Trained end-to-end in **~72h on a single H100 (~€219)**. It inherits six European
-languages (EN, FR, DE, ES, IT, PL) from the Baguettotron backbone, and — unusually —
-its entire LM pretraining corpus is auditable: Baguettotron was trained on
-[SYNTH](https://huggingface.co/datasets/PleIAs/SYNTH), ~58K Wikipedia articles expanded
-synthetically, with no scraped web.
+It inherits six European languages (EN, FR, DE, ES, IT, PL) from the Baguettotron
+backbone, and — unusually — its entire LM pretraining corpus is auditable: Baguettotron
+was trained on [SYNTH](https://huggingface.co/datasets/PleIAs/SYNTH), ~58K Wikipedia
+articles expanded synthetically, with no scraped web.
 
 Two checkpoints are published:
 
@@ -74,7 +73,7 @@ model = AutoModelForImageTextToText.from_pretrained(
     "andreagemelli/baguettotron-internvit-alignment",
     trust_remote_code=True,
     dtype=torch.bfloat16,
-    device_map="auto",
+    device_map="auto",  # also tested on Apple Silicon (mps) and CPU
 )
 processor = AutoProcessor.from_pretrained(
     "andreagemelli/baguettotron-internvit-alignment",
@@ -91,29 +90,16 @@ inputs = {k: v.to(model.device) for k, v in inputs.items() if v is not None}
 print(model.chat(**inputs))
 ```
 
-`chat()` returns the answer as a plain string, already trimmed at the end of the
-assistant turn. `max_new_tokens=...` bounds the length.
-
-### Apple Silicon and CPU
-
-The snippet above runs unchanged on `mps` and `cpu` — keep `device_map="auto"`, or drop
-it and call `.to("mps")` yourself. `dtype=torch.float32` works too. A short answer takes
-~1-3s on `mps` against ~20-25s on CPU, so prefer `mps` on a Mac.
-
-One caveat handled for you: transformers' repetition-penalty processor corrupts the
-first decoding step on MPS, which produced a garbage first token. `chat()` disables the
-penalty on that backend automatically.
-
 ### Example output
 
-Four COCO images, greedy, `Describe the image concisely.` — verbatim:
+Greedy, prompt `Describe the image concisely.` — verbatim output:
 
-| image | output |
+| | output |
 |---|---|
-| two cats on a couch | `a cat is sleeping on the couch` |
-| close-up of a brown bear | `the bear is a good friend.` |
-| an upside-down STOP sign | `a sign for a stop sign` |
-| a red double-decker bus | `the bus is a red double - decoration` |
+| <img src="http://images.cocodataset.org/val2017/000000039769.jpg" width="180"> | `a cat is sleeping on the couch` |
+| <img src="http://images.cocodataset.org/val2017/000000000285.jpg" width="180"> | `the bear is a good friend.` |
+| <img src="http://images.cocodataset.org/val2017/000000000724.jpg" width="180"> | `a sign for a stop sign` |
+| <img src="http://images.cocodataset.org/val2017/000000001584.jpg" width="180"> | `the bus is a red double - decoration` |
 
 That is the honest range of this checkpoint: it locates the subject and then drifts, which is what 2.7M trained parameters buys. It does not read the text on the sign or the bus.
 
@@ -121,59 +107,32 @@ That is the honest range of this checkpoint: it locates the subject and then dri
 
 Trained on short image captions with no `<think>` traces. The processor emits a bare assistant prefix (`<|im_start|>assistant\n`) and the model completes the caption directly. Keep prompts simple ("Describe the image").
 
-## What the evaluation found
-
-Measured on 6 held-out [RealWorldQA](https://huggingface.co/datasets/xai-org/RealworldQA)
-images, judged 1–5 by a vision judge, 244 of 288 cells completed. Thin, and reported as
-such — full tables and caveats in
-[RESULTS.md](https://github.com/andreagemelli/baguettotron-vlm/blob/main/.planning/RESULTS.md).
+## Recommended settings
 
-- **Short factual questions are its strength** (2.5–3.5), open description its weakest
-  (~1.5). That inverted our own assumption that this would be a describer.
-- **A third checkpoint made things worse.** A reasoning-SFT stage on top of the
-  instruction-tuned model scored 1.25 against its 2.17, roughly 3 SE. The reasoning
-  traces are undertrained for imagery and mostly add fluent hallucination, so that
-  checkpoint is **not published**. It is kept as a recorded negative result.
-- **Decoding presets are inside noise** — greedy 2.17 vs 1.88 for the other three.
+Greedy decoding, which is what `chat()` does by default — just call it. Four decoding
+presets were compared on held-out images and the differences sat inside noise, so greedy
+is preferred for being deterministic rather than for scoring better.
 
-### Decoding
+Two things that *do* move the needle:
 
-Use **greedy**, which is the default. It is recommended for determinism, not because
-it beat anything. Pass `do_sample=True, temperature=...` to `chat()` if you want
-sampling anyway.
-
-
-## Training
-
-| | |
-|---|---|
-| Data | LLaVA-CC3M-Pretrain-595K (595K image-caption pairs) |
-| Trainable params | ~2.7M (projector only) |
-| Frozen | ViT + LLM |
-| Effective batch size | 256 |
-| Learning rate | 1e-3, cosine, 250-step warmup |
-| Precision | bf16 |
-| Hardware | 1× H100 SXM (RunPod) |
-| Duration | ~5h (~€15) |
+- **Ask specific questions.** `"How many cats are there?"` answers `2.`; the vaguer
+  `"What is in the picture?"` answers `Yes.` Phrasing matters far more at this scale
+  than it does with a large model.
+- **Describe first, then ask.** A follow-up question in a second turn is answered better
+  than the same question asked cold, because the model conditions on its own previous
+  answer. Multi-turn is in-distribution — The Cauldron contains multi-turn conversations.
 
 ## Limitations
 
-- **No stop token.** `<|im_end|>` was masked out of the training loss, so the model
-  never learned to emit one — it spells the turn marker out as ordinary text instead.
-  `chat()` works around this by stopping on that text (a 512-token call drops from
-  ~21s to ~1.5s), but the defect is in the weights and needs a retrain to fix.
 - **Resolution ceiling.** One 448×448 crop → 256 visual tokens puts document text at
   roughly 2–4 px/char. OCR, charts and documents are out of reach by architecture, not
   by budget. Neither published checkpoint reads text in an image.
 - **Hallucinations**, especially on fine-grained or text-heavy questions.
-- **Undertrained for VQA alignment** — ~60h of instruction tuning on one H100 is small
-  relative to the data mixture.
+- **Undertrained for VQA alignment** relative to the size of the data mixture.
 - **Multilingual capability is inherited, not verified.** The backbone covers six
   languages; the VLM was never evaluated on non-English benchmarks.
-- **Evaluation is thin**: 6 held-out images, one judge, one rubric, 244 judged cells.
-  Every number here is directional.
-- **`transformers` 5.x cannot load this model.** The blocker is upstream in InternViT's
-  remote code, which predates v5's tied-weight API. Pin `transformers<5`.
+
+> Tested against `transformers` 4.57. Newer major versions may need adjustments.
 
 **Contributions and suggestions are very welcome** — issues, PRs, and ideas for
 better data mixes, training recipes, or evaluation setups are all appreciated.