Update README.md

README.md

@@ -32,7 +32,7 @@ SEA-LION is a collection of Large Language Models (LLMs) which have been pretrai
 
 **Apertus-SEA-LION-v4-8B-IT** is a 8-billion parameter model built upon the Apertus-8B-Instruct architecture. To ensure **domain adaptation** for the region, the model underwent rigorous post-training on a curated dataset of approximately **6.4 million** instruction-text pairs.
 
-This extensive post-training instills **multilingual** and **multicultural** fluency, covering key SEA languages such as Burmese, Malay, Tagalog and Tamil. This curated dataset also includes a filtered open sourced set of tool-calling instruction-text pairs to impart these capabilities, in addition to linguistic fluency.
+This extensive post-training instills **multilingual** and **multicultural** fluency, covering key SEA languages such as Indonesian, Vietnamese, Thai, Filipino, Tamil, Burmese, Malay. This curated dataset also includes a filtered open sourced set of tool-calling instruction-text pairs to impart these capabilities, in addition to linguistic fluency.
 
 Apertus-SEA-LION-v4-8B-IT is designed as a fully open model to align with this core philosophy, we have released the datasets used for post-training, as well as the evaluation codes and datasets used to evaluate the model.
 
@@ -56,7 +56,7 @@ For tokenization, the model employs the default tokenizer used in Apertus-8B-Ins
 - **Shared by:** AI Products Pillar, AI Singapore
 - **Model type:** Decoder
 - **Context length:** 65k
-- **Language(s):** Fine-tuned on English, Burmese, Tagalog, Malay and Tamil
+- **Language(s):** Indonesian, Vietnamese, Thai, Filipino, Tamil, Burmese, Malay
 - **License:** [Apache-2.0](https://choosealicense.com/licenses/apache-2.0/)
 - **Finetuned from model:** [Apertus-8B-Instruct](https://huggingface.co/swiss-ai/Apertus-8B-Instruct-2509)
 
@@ -79,63 +79,22 @@ The model has not been aligned for safety. Developers and users should perform t
 Use the code below to get started with the model with 🤗 Transformers libraries.
 
 ```
-  pip install transformers>=4.56.0
-
-```
-
-```
-# The code is adopted from Apertus example
+import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_name = "aisingapore/Apertus-SEA-LION-v4-8B-IT"
-device = "cuda"  # for GPU usage or "cpu" for CPU usage
+model_id = "aisingapore/Apertus-SEA-LION-v4-8B-IT"
 
-# load the tokenizer and the model
-tokenizer = AutoTokenizer.from_pretrained(model_name)
 model = AutoModelForCausalLM.from_pretrained(
-    model_name,
-).to(device)
-
-# prepare the model input
-prompt = "Explain the concept of 'Hari Raya Puasa' in simple terms."
-messages_think = [
-    {"role": "user", "content": prompt}
-]
-
-text = tokenizer.apply_chat_template(
-    messages_think,
-    tokenize=False,
-    add_generation_prompt=True,
-)
-model_inputs = tokenizer([text], return_tensors="pt", add_special_tokens=False).to(model.device)
-
-# Generate the output
-generated_ids = model.generate(**model_inputs, max_new_tokens=32768)
-
-# Get and decode the output
-output_ids = generated_ids[0][len(model_inputs.input_ids[0]) :]
-print(tokenizer.decode(output_ids, skip_special_tokens=True))
-```
-
-## Tool Calling
-
-
-The prompt in the example is in Malay and translates to “Please help me find a 4-room flat near Tampines, budget under $500,000. I also want to know the estimated monthly loan payment.”
-
-```
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
+    model_id, device_map="auto"
+).eval()
 
-model_id = "aisingapore/Apertus-SEA-LION-v4-8B-IT"
-tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained(
-    model_id,
-    torch_dtype=torch.bfloat16,
-    device_map="auto"
-)
+tokenizer = AutoTokenizer.from_pretrained(model_id)
 
 messages = [
-    {"role": "user", "content": "Tolong carikan flat 4-bilik dekat Tampines, bajet bawah $500,000. Nak tahu juga berapa anggaran pinjaman bulanan."}
+    {
+        "role": "user",
+        "content": "Any flats for sale in Tampines?"
+    }
 ]
 
 tools = [
@@ -173,63 +132,43 @@ tools = [
     }
 ]
 
-input_ids = tokenizer.apply_chat_template(
-    messages,
-    tools=tools,
-    return_tensors="pt",
-    add_generation_prompt=True
+inputs = tokenizer.apply_chat_template(
+    messages, tools=tools, add_generation_prompt=True, tokenize=True,
+    return_dict=True, return_tensors="pt"
 ).to(model.device)
 
-generated_ids = model.generate(
-    input_ids,
-    max_new_tokens=512,
-    do_sample=False,
-)
+input_len = inputs["input_ids"].shape[-1]
 
-response = tokenizer.decode(
-    generated_ids[0][input_ids.shape[1]:],
-    skip_special_tokens=False,
-).replace("<end_of_turn>", "").strip()
+with torch.inference_mode():
+    generation = model.generate(**inputs, max_new_tokens=512, do_sample=False)
+    generation = generation[0][input_len:]
 
-print(response)
+decoded = tokenizer.decode(generation, skip_special_tokens=True)
+print(decoded)
 
+# I'll search for HDB flats available for sale in Tampines. Let me check the listings for you.
+# <tool_call>
+# {"name": "search_hdb_listings", "arguments": {"location": "Tampines", "flat_type": "3-room", "max_price": 500000}}
+# </tool_call>
 ```
 
+## Vllm Model Serving
 
+```
+vllm serve aisingapore/Apertus-SEA-LION-v4-8B-IT --enable-auto-tool-choice --tool-call-parser hermes
+```
 
-## Training Details
-
-### Training Data
-
-The instruction fine-tuning text dataset comprises of a collection of OSS & synthetic data. The datasets used for post-training can be accessed via the link below.
-
-**Datasets for Instruction Fine Tuning**:
-
-- 🤗[aisingapore/SEA-Instruct-2602](https://huggingface.co/datasets/aisingapore/SEA-Instruct-2602)
-
-**Datasets for Tool-calling**:
-
-- 🤗[allenai/Dolci-Instruct-SFT-Tool-Use](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT-Tool-Use)
-- 🤗[Agent-Ark/Toucan-1.5M](https://huggingface.co/datasets/Agent-Ark/Toucan-1.5M)
 
-### Training Procedure
+## Training Details
 
-#### Training Hyperparameters
+**Training Datasets**:
+The instruction fine-tuning text dataset comprises of a collection of OSS & synthetic data. The datasets used for Post Training can be accessed via the link below.
 
-- **Training regime:** Our post-training workflow consists of instruction fine-tuning and model merging.
-- **Training hyperparameters:** The following hyperparameters were used during training:
+- 🤗[aisingapore/SEA-Instruct-2602](https://huggingface.co/datasets/aisingapore/SEA-Instruct-2602) (Only for ms, my, ta, fil)
+- 🤗[allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT) (Filtered for invalid conversations)
+- 🤗[Agent-Ark/Toucan-1.5M](https://huggingface.co/datasets/Agent-Ark/Toucan-1.5M) (SFT Subset)
 
-| Category | Hyperparameter | Value |
-| --- | --- | --- |
-| **Optimization** | Optimizer | `ADAMW_TORCH_FUSED` (β1=0.9, β2=0.999, ε=1e-08) |
-| **Batch Size** | Train Batch Size (per device) | `1` |
-|  | Eval Batch Size (per device) | `1` |
-| **Hardware** | Distributed Type | `multi-GPU` |
-|  | Number of Devices | `64` |
-| **Schedule** | LR Scheduler Type | `constant_with_warmup` |
-|  | LR Scheduler Warmup Steps | `269` |
-| **Other** | Training Steps | `5397` |
-|  | Seed | `42` |
+**Training Regime:** Our Post Training workflow consists of Instruction Fine Tuning and Distillation (from Apertus-70B-Instruct-2509).
 
 ## Evaluation
 
@@ -245,7 +184,7 @@ For the evaluation of general language capabilities, we employed the [SEA-HELM e
 
 Instruction-following and Multi-turn Chat
 
-We evaluated the models on instruction-following and multi-turn chat capabilities with SEA-IFEval (based on [IFEval](https://arxiv.org/abs/2311.07911)) and SEA-MTBench (based on [MT-Bench](https://arxiv.org/abs/2306.05685)) respectively. The two datasets were originally in English, the linguists and native speakers in the team worked together to filter, localise and translate the datasets into the respective target languages to ensure that the examples remained reasonable, meaningful and natural.
+We evaluated the models on Instruction-Following and Multi-turn Chat capabilities with SEA-IFEval (based on [IFEval](https://arxiv.org/abs/2311.07911)) and SEA-MTBench (based on [MT-Bench](https://arxiv.org/abs/2306.05685)) respectively. The two datasets were originally in English, the linguists and native speakers in the team worked together to filter, localise and translate the datasets into the respective target languages to ensure that the examples remained reasonable, meaningful and natural.
 
 #### Factors
 
@@ -261,7 +200,7 @@ SEA-IFEval evaluates a model's ability to adhere to constraints provided in the
 
 SEA-MTBench
 
-SEA-MTBench evaluates a model's ability to engage in multi-turn (2 turns) conversations and respond in ways that align with human needs. We use `gpt-4.1-2025-04-14` as the judge model and compare against `gpt-4.1-2025-04-14` as the baseline model. The metric used is the weighted win rate against the baseline model (i.e. average win rate across each category: Math, Reasoning, STEM, Humanities, Roleplay, Writing, Extraction).
+SEA-MTBench evaluates a model's ability to engage in Multi-turn (2 turns) conversations and respond in ways that align with human needs. We use `gpt-4.1-2025-04-14` as the judge model and compare against `gpt-4.1-2025-04-14` as the baseline model. The metric used is the weighted win rate against the baseline model (i.e. average win rate across each category: Math, Reasoning, STEM, Humanities, Roleplay, Writing, Extraction).
 
 #### Metrics