Add/update model card

README.md

@@ -0,0 +1,171 @@
+---
+license: apache-2.0
+base_model: OpenMed/privacy-filter-multilingual
+base_model_relation: quantized
+pipeline_tag: token-classification
+library_name: gguf
+tags:
+  - gguf
+  - llama-cpp
+  - localai
+  - token-classification
+  - pii
+  - ner
+  - privacy
+  - redaction
+  - multilingual
+  - openai-privacy-filter
+language:
+  - ar
+  - bn
+  - de
+  - en
+  - es
+  - fr
+  - hi
+  - it
+  - ja
+  - ko
+  - nl
+  - pt
+  - te
+  - tr
+  - vi
+  - zh
+---
+
+# privacy-filter-multilingual — GGUF (F16)
+
+GGUF conversion of [`OpenMed/privacy-filter-multilingual`](https://huggingface.co/OpenMed/privacy-filter-multilingual),
+a multilingual PII **token-classification** model (a fine-tune of
+[`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter)). It labels every
+token with a BIOES tag over **54 PII categories (217 classes)** across **16 languages**, so
+it can be served locally with **no Python** as the encoder/NER tier of a PII redactor.
+
+For the full model description, label space, evaluation, limitations, and citations, see the
+**[source model card](https://huggingface.co/OpenMed/privacy-filter-multilingual)** — this
+card only covers the GGUF packaging and how to run it.
+
+---
+
+## ⚠️ Requires a patched llama.cpp — does NOT run on stock builds
+
+> **Read this before downloading.** This GGUF uses a **custom architecture,
+> `openai-privacy-filter`, that is not (yet) part of upstream llama.cpp.** It will **fail to
+> load** in stock `llama.cpp`, `llama-cpp-python`, Ollama, LM Studio, or any other off-the-shelf
+> GGUF runtime. You will see an error like `unknown model architecture: 'openai-privacy-filter'`.
+>
+> It runs **only** on a build that carries the patches **[LocalAI PR#10160](https://github.com/mudler/LocalAI/pull/10160)**
+> ships in `backend/cpp/llama-cpp/patches/` (applied automatically by the backend's
+> `prepare.sh`). Those five patches add:
+> 1. **TOKEN_CLS pooling** (`LLAMA_POOLING_TYPE_TOKEN_CLS`) — a per-token classification head
+>    (a reduced subset of upstream **[PR #19725](https://github.com/ggml-org/llama.cpp/pull/19725)**, still open);
+> 2. the **`openai-privacy-filter` architecture** registration;
+> 3. the **HF→GGUF converter** for it;
+> 4. the **bidirectional banded-attention graph** + loader (non-causal symmetric sliding
+>    window, attention sinks, interleaved RoPE, exact YaRN `truncate=false` frequencies);
+> 5. an **all-SWA no-cache mask fix** required for an encoder that windows every layer.
+>
+> These are **carry-patches against a pinned llama.cpp commit**, not upstream features. Until
+> the architecture is upstreamed (it depends on PR #19725 landing first), running this model
+> means using LocalAI's vendored backend. At time of writing the support lives on a LocalAI
+> feature branch — use a LocalAI build that includes the `openai-privacy-filter` patches.
+
+---
+
+## Use with LocalAI
+
+Install from the LocalAI model gallery (the entry sets `backend: llama-cpp`,
+`embeddings: true`, and `known_usecases: [token_classify]`):
+
+```bash
+local-ai models install privacy-filter-multilingual
+```
+
+The model is **not** a chat/completion model — it exposes the gRPC `TokenClassify` RPC. It is a
+**PII detector**: it carries its own detection policy in a `pii_detection:` block, and other
+models opt in by listing it in `pii.detectors`. The gallery entry ships a sensible default
+policy (mask everything detected; block credentials/financial-secrets/crypto):
+
+```yaml
+# the detector model (this GGUF) — policy lives here
+name: privacy-filter-multilingual
+backend: llama-cpp
+embeddings: true
+known_usecases: [token_classify]
+pii_detection:
+  min_score: 0.5
+  default_action: mask          # mask | block | allow
+  entity_actions:               # which categories to block vs mask
+    PASSWORD: block
+    CREDITCARD: block
+    CVV: block
+```
+
+```yaml
+# any chat or cloud-proxy model — opt in and reference the detector(s)
+name: my-assistant
+backend: llama-cpp
+pii:
+  enabled: true
+  detectors:
+    - privacy-filter-multilingual
+```
+
+LocalAI runs the model's **constrained BIOES Viterbi** decode in the backend and returns
+entity spans with **UTF-8 byte offsets**; the redactor masks/blocks per the detector's
+`pii_detection` policy. Multiple detectors union their hits (strongest action wins).
+
+> Load note: the model must be loaded with **TOKEN_CLS pooling** (it is the GGUF's default —
+> the LocalAI gallery config and `embeddings: true` handle this). If you drive `llama-embedding`
+> directly for testing, do **not** pass `--pooling none`; that overrides the model default and
+> you get raw hidden states instead of label logits.
+
+## Files
+
+| File | Precision | Size | Notes |
+|---|---|---|---|
+| `privacy-filter-multilingual-f16.gguf` | F16 | ~2.7 GB | 156 tensors; 217 `classifier.output_labels`; `pooling_type = TOKEN_CLS`. Validated artifact. |
+
+F16 is the validated, shipped precision. Quantized variants are deferred until they can be
+evaluated with a **task metric (span-F1 per language) + KL-vs-F16** — perplexity is meaningless
+for a classifier, so a naively-quantized GGUF is not published here yet.
+
+## Architecture & conversion
+
+gpt-oss-style sparse **MoE** (8 layers, `d_model=640`, 128 experts, top-4 routing, ~50M active
+per token), **bidirectional banded attention** (symmetric sliding window 128, attention sinks
+retained), **interleaved (GPT-J) RoPE** with YaRN (θ=150000, factor 32), o200k (`o200k_base`)
+tokenizer, and a 217-way token-classification head (`score` → `cls.output`).
+
+The conversion reproduces the HF reference **exactly at F16**: token-for-token argmax match
+(12/12 on the parity prompt set), **full-logit cosine = 1.0**, every layer's residual-stream
+cosine = 1.0 (relerr ≈ 2e-4, i.e. F16 rounding). The two load-bearing conversion choices — the
+expert `gate_up` `chunk(2)` split and the `n_swa = 2·sliding_window` window mapping — are both
+confirmed by that parity. See LocalAI's `backend/cpp/llama-cpp/patches/README.md` for the full
+provenance.
+
+## Label space
+
+`O` plus `B-`/`I-`/`E-`/`S-` for each of 54 categories (1 + 54×4 = 217), spanning identity,
+contact, address, dates/time, government IDs, financial, crypto, vehicle, digital, and auth
+entities. The ordered `id2label` table is embedded in the GGUF (`classifier.output_labels`).
+See the [source card](https://huggingface.co/OpenMed/privacy-filter-multilingual#label-space-54-categories)
+for the full list.
+
+## Limitations & intended use
+
+Identical to the [source model](https://huggingface.co/OpenMed/privacy-filter-multilingual#limitations--intended-use):
+multilingual but uneven (strongest on de/es/fr/it/hi/te/en; weaker on CJK), trained on
+synthetic AI4Privacy data, **not** a substitute for legal/compliance review, and **not** a
+clinical PHI model. Use it as one tier behind deterministic regex pre-filters and human review.
+
+## License
+
+**Apache-2.0**, inherited from `openai/privacy-filter` and `OpenMed/privacy-filter-multilingual`.
+
+## Credits & citation
+
+Conversion and runtime support by the **LocalAI** project. The model itself is by **OpenMed**,
+fine-tuned from **OpenAI**'s `privacy-filter`, on **AI4Privacy** datasets — please cite all of
+them (BibTeX in the [source card](https://huggingface.co/OpenMed/privacy-filter-multilingual#citation)).