Document CTranslate2 runtime

README.md

@@ -60,6 +60,31 @@ print(tokenizer.decode(out[0], skip_special_tokens=True))
 #  vé máy bay gần nhất từ thành phố Lâm Giang đến thành phố Tuyết Phi."
 ```
 
+## Fast CPU Runtime
+
+This repository also includes a CTranslate2 INT8 export under
+`ct2-int8_float32/`, used by the public demo Space for faster CPU inference.
+
+```python
+import ctranslate2
+from pathlib import Path
+from huggingface_hub import snapshot_download
+from transformers import AutoTokenizer
+
+model_id = "ngocdang83/HachimiMT-60-zh-vi"
+model_path = Path(snapshot_download(model_id, allow_patterns=[
+    "config.json", "source.spm", "target.spm", "vocab.json", "tokenizer_config.json",
+    "ct2-int8_float32/*",
+]))
+
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+translator = ctranslate2.Translator(
+    str(model_path / "ct2-int8_float32"),
+    device="cpu",
+    compute_type="int8_float32",
+)
+```
+
 ## Speed Benchmark
 
 Tested on RTX 5070 Ti Laptop, `num_beams=4`, mixed test set (20 short + 20 medium + 20 long rows).