README_zh-CN.md

# NanoTranslator-immersive_translate-0.5B

[English](README.md) | 简体中文

## Introduction

NanoTranslator-immersive_translate-0.5B 是由 [Qwen2-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct) 在 [BiST](https://huggingface.co/datasets/Mxode/BiST) 和 [wmt19](https://huggingface.co/datasets/wmt/wmt19) 数据集上训练得到的专门用于**中英双语**的翻译模型。

此模型遵循[沉浸式翻译](https://immersivetranslate.com/)（Immersive Translate）的 prompt 格式进行训练，可以通过 vllm、lmdeploy 等方式部署为 OpenAI 格式接口，从而完成调用。

## How to use

下面是一个用 transformers 调用的方式，prompt 遵循沉浸式翻译以保持最佳效果。

```python
import torch
from typing import Literal
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = 'Mxode/NanoTranslator-immersive_translate-0.5B'

model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_path)

def translate(
    text: str,
    to: Literal["Chinese", "English"] = "Chinese",
    **kwargs
):
    generation_args = dict(
        max_new_tokens = kwargs.pop("max_new_tokens", 512),
        do_sample = kwargs.pop("do_sample", True),
        temperature = kwargs.pop("temperature", 0.55),
        top_p = kwargs.pop("top_p", 0.8),
        top_k = kwargs.pop("top_k", 40),
        **kwargs
    )

    prompt = """Translate the following source text to {to}. Output translation directly without any additional text.
    Source Text: {text}

    Translated Text:"""

    messages = [
        {"role": "system", "content": "You are a professional, authentic machine translation engine."},
        {"role": "user", "content": prompt.format(to=to, text=text)}
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    model_inputs = tokenizer([inputs], return_tensors="pt").to(model.device)

    generated_ids = model.generate(model_inputs.input_ids, **generation_args)
    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
    ]

    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    return response

text = "After a long day at work, I love to unwind by cooking a nice dinner and watching my favorite TV series. It really helps me relax and recharge for the next day."
response = translate(text=text, to='Chinese')
print(f'Translation: {response}')

"""
Translation: 工作了一天，我喜欢吃一顿美味的晚餐，看我最喜欢的电视剧，这样做有助于我放松，补充能量。
"""
```