HIMANSHUKUMARJHA/minicpm5-1b-toolcall-lora

LoRA adapter for openbmb/MiniCPM5-1B, fine-tuned for function and tool calling.

Trained by AutoTune, a scheduled fine-tuning pipeline on Modal that only publishes an adapter when it beats the previous best on a held-out slice.

Results

Scored on 200 held-out examples the model never saw, greedy decoding, versus the untuned base model.

	protocol	parse	function name	semantic	strict	tokens/call
this adapter	98.0%	98.0%	96.5%	79.0%	75.0%	62
base model	0.5%	87.0%	82.5%	58.5%	48.0%	249

• protocol is whether the model used the <tool_call> JSON form the prompt asked for
• semantic is the right function with the right arguments, ignoring serialization differences the dataset is itself inconsistent about (80 vs "80", "a, b" vs "a,b")

Read this honestly: MiniCPM5-1B can already call tools. Untuned it names the right function 82.5% of the time. What this adapter adds is format compliance (0.5% to 98.0%), a +20.5 point accuracy gain and a 4x cut in generated tokens, because the untuned model reasons in a <think> block before answering while this one answers directly.

Reproduce with modal run eval_toolcall.py --n 200.

Usage

import json

# Whatever functions you want the model to be able to call.
TOOLS = [{"name": "live_giveaways_by_type",
          "description": "Retrieve live giveaways by type.",
          "parameters": {"type": {"type": "str", "description": "game, loot or beta"}}}]

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "openbmb/MiniCPM5-1B"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "HIMANSHUKUMARJHA/minicpm5-1b-toolcall-lora")

messages = [
    {
        "role": "system",
        "content": (
            "You are a function calling AI model. You are provided with function "
            "signatures within <tools></tools> XML tags. Call one or more functions "
            "to assist with the user query. Do not make assumptions about what "
            "values to plug into functions.\n<tools>\n"
            + json.dumps(TOOLS, ensure_ascii=False)
            + "\n</tools>\nFor each call, return a JSON object inside "
            "<tool_call></tool_call> tags."
        ),
    },
    {"role": "user", "content": "Show me live giveaways for beta access."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)
inputs.pop("token_type_ids", None)   # this architecture's generate() rejects it
out = model.generate(**inputs, max_new_tokens=256)
text = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
for ctrl in ("<s>", "</s>", "<|im_start|>", "<|im_end|>"):
    text = text.replace(ctrl, "")   # keep <tool_call>, it is a real token
print(text.strip())

Requires transformers>=4.51, which is the first version that reads MiniCPM's standalone chat_template.jinja.

Training

Base model	openbmb/MiniCPM5-1B
Dataset	argilla/apigen-function-calling
Training rows	60,000
Steps	6,000 (~1.6 epochs at effective batch 16)
LoRA rank / alpha	32 / 64
Target modules	attention + MLP projections
LR schedule	0.0002 cosine, 3% warmup
Precision	bfloat16
Hardware	1x A100

Reproduced with AutoTune:

modal run finetune.py --profile toolcall

Data handling

The dataset is shuffled with a fixed seed, then a 200-row holdout is taken before training. Rows the formatter cannot parse are dropped and exact duplicates removed, because these datasets contain repeats that would otherwise leak holdout examples into training.

6,000 rows with unparseable tool calls and 2,068 with unparseable tool definitions were dropped before training.

The model expects tool definitions inside <tools></tools> in the system turn and emits each call as JSON inside <tool_call></tool_call> tags.

Limitations

This is a 1B parameter model. It is useful for function and tool calling at small scale and for on-device or cost-sensitive settings, but it will not match a large general model. Outputs should be validated before use. The adapter inherits any bias present in the training dataset.