# ruBERT PII inference benchmark

Baseline: `torch-fp32`. GPU: NVIDIA GeForce RTX 4070 Ti SUPER.

Quality is exact span micro-F1 across all 21 types. Deltas are percentage points relative to the baseline; larger F1/recall is better.

| Backend | Quality texts | F1 % | Δ F1 pp | Recall % | Δ recall pp | Changed documents |
|---|---:|---:|---:|---:|---:|---:|
| `torch-fp32` | 2841 | 81.739 | +0.000 | 82.561 | +0.000 | 0 (0.00%) |
| `trt-graph` | 2841 | 81.700 | -0.039 | 82.544 | -0.018 | 10 (0.35%) |
| `trt-fp16` | 2841 | 81.700 | -0.039 | 82.544 | -0.018 | 8 (0.28%) |
| `onnx-fp16` | 2841 | 81.732 | -0.007 | 82.561 | +0.000 | 7 (0.25%) |
| `onnx-fp32` | 2841 | 81.739 | +0.000 | 82.561 | +0.000 | 0 (0.00%) |
| `torch-fp16` | 2841 | 81.756 | +0.018 | 82.579 | +0.018 | 3 (0.11%) |

## Warm end-to-end latency: one document per request

Includes tokenizer, sliding windows, GPU computation, transfers and entity decoding. Excludes HTTP and model loading.

| Backend | Unique texts × repeats | Timed requests | p50 ms | p95 ms | p50 speedup | Batch16 docs/s |
|---|---:|---:|---:|---:|---:|---:|
| `torch-fp32` | 256 × 3 | 768 | 6.387 | 18.346 | 1.00× | 713.1 |
| `trt-graph` | 256 × 3 | 768 | 1.089 | 2.944 | 5.87× | 1673.9 |
| `trt-fp16` | 256 × 3 | 768 | 2.198 | 5.565 | 2.91× | 1808.2 |
| `onnx-fp16` | 256 × 3 | 768 | 2.477 | 6.229 | 2.58× | 1752.4 |
| `onnx-fp32` | 256 × 3 | 768 | 3.283 | 8.534 | 1.95× | 768.3 |
| `torch-fp16` | 256 × 3 | 768 | 6.832 | 18.265 | 0.93× | 1076.6 |

## Model with transfers: one fixed-shape tensor batch

Synthetic token IDs; includes synchronized H2D/D2H transfers, excludes tokenizer and decoder. This is not GPU-kernel-only latency.

| Backend | Sequence length | Samples | p50 ms | p95 ms | p50 speedup |
|---|---:|---:|---:|---:|---:|
| `torch-fp32` | 32 | 100 | 5.332 | 16.473 | 1.00× |
| `torch-fp32` | 128 | 100 | 6.359 | 17.329 | 1.00× |
| `torch-fp32` | 512 | 100 | 8.738 | 18.013 | 1.00× |
| `trt-graph` | 32 | 100 | 0.710 | 1.387 | 7.51× |
| `trt-graph` | 128 | 100 | 0.768 | 1.543 | 8.28× |
| `trt-graph` | 512 | 100 | 2.066 | 3.160 | 4.23× |
| `trt-fp16` | 32 | 100 | 1.566 | 3.172 | 3.41× |
| `trt-fp16` | 128 | 100 | 1.555 | 2.287 | 4.09× |
| `trt-fp16` | 512 | 100 | 2.615 | 4.547 | 3.34× |
| `onnx-fp16` | 32 | 100 | 2.151 | 4.234 | 2.48× |
| `onnx-fp16` | 128 | 100 | 2.628 | 7.632 | 2.42× |
| `onnx-fp16` | 512 | 100 | 2.621 | 4.399 | 3.33× |
| `onnx-fp32` | 32 | 100 | 2.245 | 4.111 | 2.37× |
| `onnx-fp32` | 128 | 100 | 3.244 | 7.996 | 1.96× |
| `onnx-fp32` | 512 | 100 | 7.607 | 9.709 | 1.15× |
| `torch-fp16` | 32 | 100 | 7.362 | 17.618 | 0.72× |
| `torch-fp16` | 128 | 100 | 6.376 | 16.723 | 1.00× |
| `torch-fp16` | 512 | 100 | 5.630 | 10.787 | 1.55× |

## Largest label changes

- `torch-fp32`: no measured per-label change.
- `trt-graph`: COUNTRY: F1 -0.611 pp, recall -0.413 pp (n=242); INN: F1 -0.386 pp, recall -0.380 pp (n=263); FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).
- `trt-fp16`: COUNTRY: F1 -0.611 pp, recall -0.413 pp (n=242); INN: F1 -0.386 pp, recall -0.380 pp (n=263); FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).
- `onnx-fp16`: INN: F1 -0.132 pp, recall +0.000 pp (n=263); DRIVER_LICENSE: F1 +0.105 pp, recall +0.000 pp (n=371); PASSPORT: F1 -0.079 pp, recall +0.000 pp (n=494).
- `onnx-fp32`: no measured per-label change.
- `torch-fp16`: FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).

## Provenance and interpretation

Dataset revision: `f77ea831274daf980cc45c61a93c226be9d978d6`; model revision: `c802e8cd26f85d1cf920973ea6f83965a0618d63`.

All 2,841 original texts are preserved. Case-only WWW/www alignment is explicit for row_0000, row_1628; all 5,614 gold spans remain.

- Quality uses exact character-span micro-F1 over all 21 model entity types; native gold BIO boundaries are preserved.
- This is a paired backend comparison. The PERSON+LOCATION overlap diagnostic does not claim to reproduce the published model-card or leaderboard F1.
- End-to-end batch1 means one document request: tokenization, sliding windows, GPU work, transfers, and entity decoding. Long documents may require multiple windows.
- Fixed-shape model-with-transfers timing includes host-to-device and device-to-host transfer and synchronized CPU output, but excludes tokenization and entity decoding; it is not GPU-kernel-only latency.
- Latency is warm, local, sequential, and excludes model loading, HTTP overhead, queueing, and concurrent clients. Repeated timings of the same texts are not independent unique requests.
- Quality and end-to-end latency both use one window per GPU call. Long-document windows run sequentially; batch16 throughput is measured separately.
- No threshold is tuned on this benchmark; the shared decoder uses a fixed confidence threshold.