"""Minimal loader for the Fairleap v1 CLM Sahabat-AI 8B adapter. Four things about this model will look like broken weights if you meet them cold, so they are handled here rather than left as surprises: 1. **The base model's own chat template silently destroys tool calls.** `llama3-8b-cpt-sahabatai-v1-instruct` ships the plain Llama-3 template, which renders every message as `content | trim` and ignores both `tool_calls` and the `tools` argument. An assistant tool-call turn carries `content: ""`, so it renders as an *empty* assistant reply -- no error anywhere. The tool-capable template is in `chat_template.jinja` beside the weights, and `from_pretrained` on **this directory** picks it up. Load the tokenizer from the base repo instead and tool use breaks with no warning. 2. **This repository holds a LoRA adapter only.** There are no base weights and no `config.json`, so `AutoModelForCausalLM.from_pretrained(".")` fails and PEFT falls back to treating the local path as a Hub repo id, reporting `HFValidationError: Repo id must be in the form 'repo_name'...`, which reads as a path bug. 3. **The base's generation config disagrees with its own template.** It names `<|end_of_text|>` (128001) as eos while every turn ends with `<|eot_id|>` (128009), so generation never stops. `chat()` passes eos explicitly. 4. **Tool calls come back as one line of JSON**, keyed `parameters` rather than the `arguments` most OpenAI-shaped parsers expect: `{"name": "predict_earnings", "parameters": {...}}`. `parse_tool_call()` recovers it. **`fairleap-api` must parse this shape** -- it differs from the XML block the Qwen adapter emits. python load_model.py # loads and runs one grounded question """ import json from pathlib import Path REPO_ID = "fairleap-ai/fairleap-v1-clm-sahabatai-8b-adapter" BASE_MODEL = "GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct" ADAPTER_DIR = str(Path(__file__).parent) MAX_SEQ = 4096 EOT_TOKEN = "<|eot_id|>" # The only callable tool. Advice topics are prompt blocks, not tools -- calling # an LLM endpoint from an LLM is an extra hop and a second inference bill. PREDICT_EARNINGS_TOOL = { "type": "function", "function": { "name": "predict_earnings", "description": ( "Prediksi penghasilan harian dan jam kerja untuk rentang tanggal di masa " "depan menggunakan model XGBoost. Riwayat harian driver ditambahkan " "otomatis oleh sistem, jangan sertakan riwayat dalam argumen." ), "parameters": { "type": "object", "properties": { "start": {"type": "string", "description": "Tanggal awal, YYYY-MM-DD."}, "end": {"type": "string", "description": "Tanggal akhir, YYYY-MM-DD."}, "wellness_score": {"type": "integer", "minimum": 1, "maximum": 100}, }, "required": ["start", "end", "wellness_score"], }, }, } PERSONA = """Kamu adalah Fairleap AI Assistant, asisten untuk mitra driver Gojek/GOTO di Indonesia. Jawab hangat, ringkas, dan konkret dalam Bahasa Indonesia. Jangan pernah mengarang angka.""" def build_system_prompt(today, city, vehicle, risk, wellness_score, period, totals, daily=None): """Assemble the stuffed driver context the model was trained to read. daily is an optional list of (date, earnings, orders) rows. Training used seven-day summaries plus up to fourteen daily rows; the model reads these figures literally, so anything absent here it should decline to state. Include the *previous* period's total whenever you have it. Held-out evaluation found the model appends "Dibanding 7 hari sebelumnya (RpX)..." to half of all earnings answers, inventing X when it was never supplied. Supplying the real figure is the cheapest mitigation available to a caller. """ lines = [ PERSONA, "", "=== KONTEKS DRIVER (disediakan sistem, bukan hasil tool) ===", f"Tanggal hari ini: {today}", f"Kota: {city}", f"Kendaraan: {vehicle}", f"Toleransi risiko: {risk}", f"Skor kesehatan terakhir: {wellness_score}/100", f"Ringkasan 7 hari terakhir ({period}):", ] lines += [f"- {k}: {v}" for k, v in totals.items()] if daily: lines.append("Rincian harian:") # Indonesian thousands separator is a dot, so convert the number alone -- # doing it on the whole line turns the comma before "order" into a dot. lines += [f"- {d}: Rp{e:,}".replace(",", ".") + f", {o} order" for d, e, o in daily] return "\n".join(lines) def get_tokenizer(processor): """Unwrap a processor if one is handed back. See note 1 above.""" return getattr(processor, "tokenizer", processor) def load(adapter_dir=ADAPTER_DIR, load_in_4bit=True): """Return (model, tokenizer). Prefers unsloth, falls back to peft.""" # Without this the loaders report `Unrecognized model in