Add Q4_K_M, Q5_K_M, Q8_0 GGUFs with model card and license files

.gitattributes

@@ -1,35 +1 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kaito Ishiguro
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

LICENSE.CAT-Translate-0.8b

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 CyberAgent AI Lab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

LICENSE.sarashina2.2-0.5b

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 SB Intuitions
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,3 +1,238 @@
 ---
 license: mit
+language:
+  - en
+  - ja
+library_name: gguf
+base_model: cyberagent/CAT-Translate-0.8b
+pipeline_tag: translation
+tags:
+  - translation
+  - gguf
+  - llama.cpp
+  - on-device
+  - offline
+  - ja
+  - en
+  - travel
+  - quantized
 ---
+
+# cat-translate-0.8b-travel-gguf
+
+Offline EN↔JA travel translator, Q4_K_M GGUF, runs fully on-device via llama.cpp / PocketPal.
+
+Fine-tuned from [cyberagent/CAT-Translate-0.8b](https://huggingface.co/cyberagent/CAT-Translate-0.8b)
+(itself built on [sbintuitions/sarashina2.2-0.5b](https://huggingface.co/sbintuitions/sarashina2.2-0.5b))
+via LoRA domain adaptation on a curated travel corpus, then converted to GGUF via a
+[patched `convert_hf_to_gguf.py`](#critical-tokenizer-note) and quantized with
+importance-matrix (imatrix) calibration on travel data.
+
+---
+
+## Files
+
+| File | Size | Notes |
+|------|------|-------|
+| `cat-translate-0.8b-travel-Q4_K_M.gguf` | 503.7 MB | **RECOMMENDED** — iPhone target |
+| `cat-translate-0.8b-travel-Q5_K_M.gguf` | 569.2 MB | Alternate |
+| `cat-translate-0.8b-travel-Q8_0.gguf` | 806.2 MB | Near-lossless reference |
+
+All three files are imatrix-calibrated PTQ (importance-matrix post-training quantization),
+calibrated on a 534-example travel corpus weighted toward emergency (134 examples, all
+available) and food (200 examples) domains. Quantized with `llama-quantize --imatrix`.
+
+---
+
+## Intended Use & Scope
+
+**Target use case:** consumer travel translation — ordering food, navigating transportation,
+accommodation, shopping, and emergency communication in Japan.
+
+Food and emergency domains are explicitly weighted in both fine-tuning (food is 42.5% of
+training pairs) and in the importance matrix calibration (emergency is the heaviest-weighted
+domain by proportion). The motivating scenario is a foreigner with no internet access facing
+a Japanese-only emergency line — "Please call an ambulance" and "I am having an allergic
+reaction" must be correct.
+
+**This model is a credible demonstrator with rigorous safety-aware evaluation.** It is not a
+deployable medical, legal, or defense product. The consumer travel app is the near-term target.
+See the safety architecture note below for how life-critical phrases are handled in the intended
+deployment.
+
+**Not in scope:** medical advice, legal documents, regulated translation workflows, or any
+context where a confident-wrong output carries high-stakes consequences.
+
+---
+
+## Safety Architecture
+
+The intended deployment is a **hybrid phrasebook + LLM** system:
+
+- **Deterministic, human-verified phrasebook** handles life-critical phrases: calling an
+  ambulance, reporting an allergic reaction, "I can't breathe", "I'm lost". These are fixed
+  strings — no probability, no hallucination risk.
+- **The LLM handles the open-ended long tail:** menus, negotiations, descriptions — anything
+  that formulaic phrases don't cover.
+
+A 4-bit quantized LLM alone **must not be relied on for life-critical phrases.** Confident-wrong
+is the worst failure mode in an emergency; the phrasebook is the safety net.
+
+---
+
+## Evaluation
+
+Measured on a 156-item held-out gold test set (78 EN→JA + 78 JA→EN), food- and
+emergency-weighted, with all 44 emergency references human-verified. Evaluated at the desktop
+inference stage using llama-server with plain-text prompts on stock llama.cpp b9716.
+
+| Label | Size (MB) | COMET (all) | Δ vs bf16 | food COMET | emerg COMET | BLEU en→ja | BLEU ja→en |
+|-------|-----------|-------------|-----------|------------|-------------|------------|------------|
+| bf16  | 1,515.2   | 0.9001      | ±0.0000   | 0.8960     | 0.8913      | 48.87      | 39.99      |
+| Q8_0  | 806.2     | 0.9022      | +0.0021   | 0.8971     | 0.8925      | 49.79      | 40.35      |
+| Q5_K_M | 569.2   | 0.9000      | −0.0001   | 0.8933     | 0.8925      | 48.92      | 41.32      |
+| Q4_K_M | 503.7   | 0.8988      | −0.0013   | 0.8892     | 0.8986      | 47.96      | 38.97      |
+
+**COMET model:** `Unbabel/wmt22-comet-da`.  
+**BLEU tokenizers:** `char` for EN→JA (sacrebleu), `13a` for JA→EN (sacrebleu).
+
+**Why Q4_K_M:** COMET delta −0.0013 vs bf16 is inside the 0.005 negligible threshold, at
+67% smaller. Emergency gate clears: Q4 emerg COMET 0.8986 vs bf16 0.8913 — zero emergency
+degradation at 4-bit (the small apparent gain is within n=44 noise; the honest claim is no
+degradation). The entire quant ladder sits within ~0.003 COMET; quant choice collapses to
+smallest-that-fits-the-phone.
+
+### Read these caveats before drawing conclusions
+
+- **COMET ≈ parity is the reliable signal.** Do not read small BLEU swings between quant
+  levels as quality conclusions — the variation is within small-sample noise.
+- **These 156-item numbers are NOT comparable to Phase 3 (fine-tune) numbers.** The earlier
+  40-item eval set contained shorter, more formulaic sentences. The 156-item set includes
+  longer, more varied sentences; BLEU naturally drops on harder sets. The difference is not
+  a regression.
+- **EN→JA BLEU uses `char` tokenizer; JA→EN uses `13a`.** Mixing tokenizers makes numbers
+  incomparable. If you re-evaluate, pin these tokenizers explicitly.
+
+---
+
+## Differentiation
+
+The value proposition is **pragmatics, not BLEU.** Incumbent offline NMT (Google Translate
+offline ~47 MB, Apple ~100 MB) wins literal character-BLEU. This model's edge:
+
+- **Pro-drop resolution** — correctly inferring dropped subjects/objects from context in Japanese
+- **Keigo** — appropriate politeness register for service interactions (restaurant, hotel, shop)
+- **Cultural and food context** — menu descriptions, allergen terms, regional food vocabulary
+
+Do not frame this as a BLEU race against Google or Apple. Frame it on pragmatic correctness
+in travel scenarios that require cultural and register awareness.
+
+---
+
+## Critical Tokenizer Note
+
+This GGUF was built with a **patched `convert_hf_to_gguf.py`** and tokenizes correctly only
+in llama.cpp builds that use the resulting `tokenizer.ggml.model = "t5"` path.
+
+**Background:** `cyberagent/CAT-Translate-0.8b` uses a SentencePiece **Unigram** tokenizer
+(inherited from `sbintuitions/sarashina2.2-0.5b`). The stock llama.cpp conversion script
+hardcodes `tokenizer.ggml.model = "llama"` for all SPM models, routing the runtime into the
+greedy-merge SPM path. SPM greedy-merge ≠ Unigram Viterbi — it produces silently wrong token
+splits. **All known community GGUFs of this model family (mradermacher, mmnga-o) have this
+bug.**
+
+**The fix (applied at conversion time):** `llama.cpp/conversion/base.py::_set_vocab_sentencepiece()`
+was modified to (1) detect `tokenizer.json → model.type == "Unigram"` and emit
+`tokenizer.ggml.model = "t5"`, routing llama.cpp into its `LLAMA_VOCAB_TYPE_UGM` Viterbi
+path; and (2) selectively promote BYTE tokens to NORMAL type only for byte characters that
+have no existing NORMAL token — so control characters like `\n` tokenize correctly without
+triggering the duplicate-text assertion crash (`GGML_ASSERT(id_to_token.size() == token_to_id.size())`).
+
+**Validation on stock llama.cpp b9716 (2026-06-19):** `/tokenize` on the template string:
+25/25 tokens match HF output exactly, including `\n\n` = `[25, 25]`. Broad parity test over
+30 diverse strings: all match. COMET ≈ HF parity in both directions on the 40-sentence eval.
+
+**Verify your build:** run `/tokenize` on a Japanese string and compare against HF tokenizer
+output. If your llama.cpp or PocketPal build uses an unpatched conversion (model string
+`"llama"` instead of `"t5"`), tokenization will be silently wrong and translation quality
+will degrade. PocketPal validation on a real device remains a required gate before any app
+deployment — b9716 desktop testing is a strong proxy, not a guarantee.
+
+---
+
+## Usage
+
+### llama-cli
+
+```bash
+llama-cli \
+  -m cat-translate-0.8b-travel-Q4_K_M.gguf \
+  --no-warmup \
+  -p "Translate the following English text into Japanese.\n\nI'd like a table for two."
+```
+
+### llama-server (OpenAI-compatible)
+
+```bash
+# Start server
+llama-server -m cat-translate-0.8b-travel-Q4_K_M.gguf --port 8080
+
+# Query
+curl http://localhost:8080/completion \
+  -H "Content-Type: application/json" \
+  -d '{"prompt": "Translate the following English text into Japanese.\n\nI have a peanut allergy.", "n_predict": 128}'
+```
+
+Use **plain-text prompts only** — no chat template, no system message. Template:
+`"Translate the following {src_lang} text into {tgt_lang}.\n\n{source}"`
+
+For JA→EN: `"Translate the following Japanese text into English.\n\n救急車を呼んでください。"`
+
+### PocketPal (iOS / Android)
+
+Open PocketPal → Models → Add Model → enter the Hugging Face repo ID
+`kaiish/capynode` → select `cat-translate-0.8b-travel-Q4_K_M.gguf`.
+Enable airplane mode after download to confirm fully offline operation.
+
+---
+
+## Training Data Provenance
+
+| Source | Pairs | License |
+|--------|-------|---------|
+| Gemini distillation (gemini-3.1-pro-preview / gemini-2.5-flash) | ~2,550 | Synthetic, Google AI Studio free tier |
+| Tatoeba | ~1,230 | CC BY 2.0 |
+| BSD | dropped | CC BY-NC-SA 4.0 — excluded (non-commercial clause) |
+
+Final training corpus: **6,458 train / 718 validation** examples (bidirectional, sarashina2.2
+instruct template). Domain distribution: food 42.5%, shopping 17.5%, transportation 15%,
+accommodation 15%, emergency 5%, general 5%.
+
+---
+
+## License & Attribution
+
+**License: MIT.** See [`LICENSE`](LICENSE) for this repository's terms.
+
+This GGUF is built on two upstream MIT-licensed components. Their copyright and permission
+notices are preserved verbatim in the bundled license files below — these are the authoritative
+texts that must be carried forward in any distribution or deployment (e.g., in an app's About
+or credits screen):
+
+- [`LICENSE.sarashina2.2-0.5b`](LICENSE.sarashina2.2-0.5b) — Copyright (c) 2025 SB Intuitions
+- [`LICENSE.CAT-Translate-0.8b`](LICENSE.CAT-Translate-0.8b) — Copyright (c) 2026 CyberAgent AI Lab
+
+### Model Lineage
+
+```
+sbintuitions/sarashina2.2-0.5b   (MIT — SB Intuitions, 2025)
+        │
+        └── cyberagent/CAT-Translate-0.8b   (MIT — CyberAgent AI Lab, 2026)
+                │
+                └── LoRA fine-tune (travel domain adaptation)
+                        │
+                        └── imatrix Q4_K_M GGUF   (this repo — MIT, Kaito Ishiguro, 2026)
+```
+
+**Cleared for:** quantizing, re-hosting on HuggingFace, and shipping in a closed-source paid,
+subscription, or ad-supported app, provided both bundled license files above are preserved.

cat-translate-0.8b-travel-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adac6967c9614aad9adcd27edf9a69a1394fed504cb8485ade3f0548c3ecd055
+size 528204288

cat-translate-0.8b-travel-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:655e5c2d7c3772b05fac95efd4257c4a13dac917204e95c071b8c7bc523fc0f6
+size 596812288

cat-translate-0.8b-travel-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ab171ee0b16e07edc78216d6cedf155351f1fc6d478be121d9967f26862a74d
+size 845362688