README.md

metadata

license: apache-2.0
base_model: tencent/Hy-MT2-1.8B
tags:
  - litert
  - litert-lm
  - litertlm
  - on-device
  - edge
  - translation
  - hunyuan
  - multilingual
language:
  - zh
  - en
  - fr
  - pt
  - es
  - ja
  - tr
  - ru
  - ar
  - ko
  - th
  - it
  - de
  - vi
  - ms
  - id
  - tl
  - hi
  - pl
  - cs
  - nl
  - km
  - my
  - fa
  - gu
  - ur
  - te
  - mr
  - he
  - bn
  - ta
  - uk
  - bo
  - kk
  - mn
  - ug
pipeline_tag: translation
library_name: litert-lm
base_model_relation: quantized

Measured on device (edge-compat): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 20.8 tok/s · prefill 386 tok/s · TTFT 560 ms · all 1489 ops delegated (2026-09-05); Mac Studio M4 Max · LiteRT-LM 0.16.0 · GPU · decode 105.8 tok/s · prefill 2008 tok/s · TTFT 137 ms (2026-08-27); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 12.3 tok/s · prefill 126 tok/s · TTFT 1.64 s (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/hy-mt2-1.8b-int8/CARD.md

Hy-MT2-1.8B — LiteRT-LM

tencent/Hy-MT2-1.8B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. To our knowledge this is the first Hy-MT2 in LiteRT form — the Hub otherwise carries GGUF, MLX and FP8 conversions.

Hy-MT2-1.8B is Tencent's "fast-thinking" multilingual translation model: 33 languages, tuned to follow translation instructions rather than chat. It is a dense hunyuan_v1_dense stack — 32 layers, hidden 2048, 16 query / 4 KV heads (GQA) with QK-norm, head_dim 128, vocab 120,818, tied embeddings — 2.04B parameters.

File	Recipe	Size
Hy-MT2-1.8B_int8.litertlm	export-time dynamic int8 (linears + embedding)	1.82 GB

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

Correctness

All numbers below were measured on this exact file (litert-lm 0.16.0, Apple M4 Max).

• Chat template is byte-equal to the source: the embedded Jinja matches the repo's chat_template.jinja exactly (654 / 654 bytes).
• 8-question sanity gate: 6/8 on CPU and 6/8 on GPU, non-degenerate, with the same two misses on both backends ("Cool" for the opposite of hot, "pink" for the rhyme) — a property of this translation-tuned 1.8B, not of a backend. Arithmetic, factual and translation items are all correct.
• Translation greedy A/B vs the HF reference (PyTorch bf16, greedy, the source README's default-translation prompt): byte-identical on 1 of 3 probes; the other two are fluent alternates of the usual int8-vs-bf16 kind (e.g. spectaculaire → significative). No degeneration on any probe.
• No duplicate start token. The source chat template renders <|hy_begin_of_sentence|> itself, and the LiteRT-LM engine also prepends the metadata start_token — measured inside the runtime: [start_token]+prompt and [template BOS]+prompt generate byte-identical greedy output, so the default export was feeding BOS twice. This bundle drops the metadata start token; the on-device token stream matches the training stream. Honest note: the double-BOS variant happened to score 8/8 on the sanity gate — the stream-faithful file ships anyway, because matching the training stream is the property that generalizes.
• Stop token is <|hy_place_holder_no_2|> (id 120020), as the source declares.

Usage

Hy-MT2 expects translation instructions, not open chat. The source model card's default prompt works verbatim:

litert-lm run ./Hy-MT2-1.8B_int8.litertlm --prompt \
"Translate the following text into Japanese. Note that you should **only output the translated result without any additional explanation**:

The weather is nice today, so let's go for a walk in the park."

# GPU
litert-lm run ./Hy-MT2-1.8B_int8.litertlm --backend gpu --prompt "..."

The bundle carries the tokenizer and the stock Hy-MT2 chat template (max_num_tokens 4096).

Performance

litert-lm benchmark <file> -p 256 -d 256 --runs 3 --cache no, litert-lm 0.16.0, Apple M4 Max. Two independent protocol runs per cell; the GPU cell was taken after a ≥300 s rest:

Backend	Prefill (256)	Decode	TTFT
GPU	2008 / 2003 tok/s	105.8 / 105.4 tok/s	0.137 s
CPU	211.0 / 210.0 tok/s	34.0 / 32.4 tok/s	1.24 / 1.25 s

Both backends were gated on real generations before benchmarking (a broken backend can still print benchmark numbers). GPU repeats within ~0.3%, CPU decode within ~5%.

Conversion notes

Converted with stock litert-torch 0.9.3 through hf-to-litertlm — one command:

python scripts/convert.py tencent/Hy-MT2-1.8B

Two things route this family correctly, both measured:

• The dynamic-with-alpha rope resolves statically. transformers computes base = rope_theta * alpha^(dim/(dim-2)) once at init and never rescales below max_position_embeddings — only the leftover data-dependent cache-growth branch kills torch.export. The converter bakes the resolved base (11,158,839.925) into rope_theta and drops rope_scaling; inv_freq and teacher-forced logits are bitwise-equal to the HF reference, valid to 262,144 positions (far past this bundle's 4,096 context).
• The engine prepends the metadata start token unconditionally, so a template that renders its own BOS must not also declare one — see Correctness above.

Export took 120 s on an M4 Max. See REPRODUCE.md for the full measurement record behind every claim on this card.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with litert-lm benchmark 0.16.1: CPU backend, 4 threads, 256 prefill + 256 decode tokens, --cache memory (the compile cache lives and dies with the process, so every invocation compiles the model from scratch; nothing is reused between runs), one warm-up plus one timed iteration per invocation, 3 invocations per file with cooldown in between. Values are the median across invocations (min–max in parentheses). No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0). Every file listed produced coherent text in a real generation on this backend before its numbers were recorded.

File	Prefill (tok/s)	Decode (tok/s)	TTFT	Peak RSS
Hy-MT2-1.8B_int8.litertlm	25.0 (24.9–25.8)	2.7 (2.6–2.7)	10.6 s	2.9 GB