---
language:
- th
- en
metrics:
- comet
- kendall's tau correlation
tags:
- translation-evaluation
- translation-metrics
- mqm
- ranking
- translation-quality
model-index:
- name: COMET-21
  results:
  - task:
      type: translation-quality-estimation
      name: English-Thai Translation Quality Assessment
    dataset:
      type: MEET-MR/MEET-MR
      name: MEET-MR
    metrics:
    - name: mqm correlation
      type: Kendall's tau correlation
      value: 0.323
      verified: false
    - name: rank correlation
      type: Kendall's tau correlation
      value: 0.353
      verified: false
datasets:
- MEET-MR/MEET-MR
---

**COMET-21** is a Quality Estimation (QE) model for **English-Thai** machine translation. The model **[wmt21-comet-qe-mqm](https://unbabel-experimental-models.s3.amazonaws.com/comet/wmt21/wmt21-comet-qe-mqm.tar.gz)** is fine-tuned on the **MEET-MR dataset** to predict translation quality scores that align with human judgments, including MQM and Ranking, without requiring reference translations.

# Model Description
The model was fine-tuned on the MEET-MR dataset, comprising 2,142 English source sentences and their translations across 9 diverse domains. Fine-tuning COMET-21 on this specific language pair and dataset significantly improves its ability to capture Thai vocabulary, contextual nuances, and human preferences compared to the generic pretrained version

This model is designed to evaluate the quality of English-to-Thai machine translations. Given a source text, a reference translation, and its candidate translation, outputs a single score between 0 and 1 where 1 represents a perfect translation.

# Paper
TBA

# Usage

```python
from comet import download_model, load_from_checkpoint

# Load the model (assuming you have the checkpoint file)
model_path = download_model("MEET-MR/COMET-21-MEET-MR")
model = load_from_checkpoint(model_path)

data = [
    {
        "src": "The premises of the mission shall be inviolable.",
        "mt": "สถานที่ของภารกิจจะต้องไม่ถูกละเมิด",
        "ref": "อาคารและสถานที่ของคณะผู้แทนจะถูกละเมิดมิได้"
    },
    {
        "src": "A hydrating day & night cream.",
        "mt": "ครีมน้ำในวันและคืน",
        "ref": "ครีมให้ความชุ่มชื้นสำหรับกลางวันและกลางคืน"
    }
]

model_output = model.predict(data, batch_size=8, gpus=1)
print(model_output)
```