yapwithai/yap-distilbert-ml-screenshot-intent

Screenshot Intent Classifier

This repository contains a DistilBERT Multilingual-based classifier fine-tuned to decide whether a conversational agent should trigger a screenshot tool for the latest user message.

Base Model

This model is fine-tuned from distilbert/distilbert-base-multilingual-cased, and inherits the base encoder's maximum context length and tokenizer. distilbert-base-multilingual-cased is a distilled version of multilingual BERT, supporting 104 languages with a compact BERT-family encoder and the standard 512-token context window.

Classifier

• 0 / no_screenshot: do not call the screenshot tool.
• 1 / take_screenshot: call the screenshot tool.

The input is a text block representing the recent conversation history, formatted as one utterance per line (raw user messages separated by newlines), e.g.:

I'm wondering if blue goes well with yellow.
What's your take on this?

At inference time, the host application typically feeds the last few conversation turns (most importantly the latest user message) in this format and thresholds the classifier's take_screenshot probability to decide whether to trigger the tool.

Training Data

The classifier was trained on a curated, hand-labelled private dataset. It contains hundreds of single-turn and multi-turn examples specifying whether each user message should or should not trigger a screenshot, including:

• Clear positive triggers ("look at this", "check this out", "rate this pic").
• Clear negatives (off-topic chit-chat, abstract statements, idioms like "I'll look into it").
• Edge cases involving deictic pronouns, quantities ("take 2 screenshots"), negation ("don't look"), multi-turn context, and more.

No external user logs or third-party datasets were used; the training data is purely synthetic / curated for this intent task.

Training Setup

• Epochs: 5
• Batch size: 16 (per device)
• Learning rate: 1e-05
• Weight decay: 0.01
• Max sequence length: 512

The script builds examples by concatenating conversation history up to and including the current user message, one utterance per line. Multi-turn conversations therefore become multiple training examples with growing context.

Usage

Basic usage with the Transformers library:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

MODEL_ID = "yapwithai/yap-distilbert-ml-screenshot-intent"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()

text = "look at this amazing sunset"
inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    padding="max_length",
    max_length=512,
)

with torch.no_grad():
    outputs = model(**inputs)
    probs = outputs.logits.softmax(dim=-1)[0]

p_no, p_yes = probs.tolist()
print("P(no_screenshot)=", p_no)
print("P(take_screenshot)=", p_yes)

In production, you would:

• Construct a conversation history string similar to the training format (recent user turns, each on its own line).
• Run the classifier once per latest user message.
• Threshold p_yes to decide whether to trigger the screenshot tool.

DistilBERT Citation

If you use DistilBERT Multilingual in your work, please cite:

@inproceedings{sanh2019distilbert,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
  booktitle={NeurIPS EMC^2 Workshop},
  year={2019}
}