lemonade-sdk/pii_policy_pplx-pii-masking-onnx

PII Router Policy - pplx-pii-masking (ONNX)

A Lemonade collection.router policy JSON: routes any prompt containing PII to a local model, and everything else to a cloud model, using lemonade-sdk/pplx-pii-masking-onnx as the PII classifier.

This repo holds only the policy config - not model weights. It's meant to be pulled directly into a running Lemonade server.

What this policy does

• Runs every prompt through the PII classifier first (36 non-"O" BIOES labels across 9 PII types: private_person, private_email, private_phone, private_address, private_url, private_date, account_number, secret, other_pii).
• If any label crosses min_score: 0.5, the prompt routes to the local candidate (Qwen3.5-0.8B-GGUF by default).
• Otherwise it routes to the cloud candidate (fireworks.kimi-k2p6 by default).

Usage

Requires an ort-server build newer than 0.3.7

1. Register the classifier. Lemonade's onnxruntime backend (ort-server) needs a manifest.json next to model.onnx to know this is a token-classification model - one score per token, not one per prompt - rather than the default it assumes otherwise (single-label text-classification). lemonade-sdk/pplx-pii-masking-onnx ships its manifest.json at the repo root, so this one pull is all it needs:

curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" -d '{
  "model_name": "user.pplx-pii-masking-onnx",
  "checkpoint": "lemonade-sdk/pplx-pii-masking-onnx",
  "recipe": "onnxruntime"
}'

2. Register this policy:

hf download lemonade-sdk/pii_policy_pplx-pii-masking-onnx --local-dir .
curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" \
  --data-binary @pii_policy_pplx-pii-masking-onnx.json

3. Use it - send chat completions to "model": "user.PII-ONNX-PplxMasking-Router" and the server will route each request per the policy above.

Why the policy spells out 36 leaf conditions

The classifier's manifest keeps all 37 raw BIOES labels distinct (B-private_person, I-private_person, E-private_person, S-private_person, ...) rather than collapsing each type to one name, because ort-server writes per-label scores by plain assignment keyed on the label string - two indices sharing a name would silently overwrite each other. The router's classifier match condition tests one label at a time, so "was any PII detected" is expressed as an any over the B/I/E/S variant of every type. Regenerate rather than hand-edit if the threshold or candidates change.

What this policy is not

It is not the checkpoint's own model.predict(). Upstream decodes spans with a constrained BIOES Viterbi; the router instead applies per-token softmax, max over tokens per label, and a threshold. The two rules are close on the Nemotron-PII benchmark but not identical, and the second head of the model (sensitivity_logits) is not consulted at all - measured at 9.16% recall on that benchmark, it should never be the routing signal.

Customizing

• routing.candidates / routing.default_model / rules[].route_to - swap in whatever local and cloud models you have configured.
• min_score - 0.5 is a starting point. Lowering it trades over-routing to the local model for fewer leaks; the benchmark corpus has no benign arm, so it cannot measure the over-routing side.

For building your own routing policy from scratch, see the lemonade-router-builder skill.