results/summary.md

ruBERT PII inference benchmark

Baseline: torch-fp32. GPU: NVIDIA GeForce RTX 4070 Ti SUPER.

Quality is exact span micro-F1 across all 21 types. Deltas are percentage points relative to the baseline; larger F1/recall is better.

Backend	Quality texts	F1 %	Δ F1 pp	Recall %	Δ recall pp	Changed documents
torch-fp32	2841	81.739	+0.000	82.561	+0.000	0 (0.00%)
trt-graph	2841	81.700	-0.039	82.544	-0.018	10 (0.35%)
trt-fp16	2841	81.700	-0.039	82.544	-0.018	8 (0.28%)
onnx-fp16	2841	81.732	-0.007	82.561	+0.000	7 (0.25%)
onnx-fp32	2841	81.739	+0.000	82.561	+0.000	0 (0.00%)
torch-fp16	2841	81.756	+0.018	82.579	+0.018	3 (0.11%)

Warm end-to-end latency: one document per request

Includes tokenizer, sliding windows, GPU computation, transfers and entity decoding. Excludes HTTP and model loading.

Backend	Unique texts × repeats	Timed requests	p50 ms	p95 ms	p50 speedup	Batch16 docs/s
torch-fp32	256 × 3	768	6.387	18.346	1.00×	713.1
trt-graph	256 × 3	768	1.089	2.944	5.87×	1673.9
trt-fp16	256 × 3	768	2.198	5.565	2.91×	1808.2
onnx-fp16	256 × 3	768	2.477	6.229	2.58×	1752.4
onnx-fp32	256 × 3	768	3.283	8.534	1.95×	768.3
torch-fp16	256 × 3	768	6.832	18.265	0.93×	1076.6

Model with transfers: one fixed-shape tensor batch

Synthetic token IDs; includes synchronized H2D/D2H transfers, excludes tokenizer and decoder. This is not GPU-kernel-only latency.

Backend	Sequence length	Samples	p50 ms	p95 ms	p50 speedup
torch-fp32	32	100	5.332	16.473	1.00×
torch-fp32	128	100	6.359	17.329	1.00×
torch-fp32	512	100	8.738	18.013	1.00×
trt-graph	32	100	0.710	1.387	7.51×
trt-graph	128	100	0.768	1.543	8.28×
trt-graph	512	100	2.066	3.160	4.23×
trt-fp16	32	100	1.566	3.172	3.41×
trt-fp16	128	100	1.555	2.287	4.09×
trt-fp16	512	100	2.615	4.547	3.34×
onnx-fp16	32	100	2.151	4.234	2.48×
onnx-fp16	128	100	2.628	7.632	2.42×
onnx-fp16	512	100	2.621	4.399	3.33×
onnx-fp32	32	100	2.245	4.111	2.37×
onnx-fp32	128	100	3.244	7.996	1.96×
onnx-fp32	512	100	7.607	9.709	1.15×
torch-fp16	32	100	7.362	17.618	0.72×
torch-fp16	128	100	6.376	16.723	1.00×
torch-fp16	512	100	5.630	10.787	1.55×

Largest label changes

• torch-fp32: no measured per-label change.
• trt-graph: COUNTRY: F1 -0.611 pp, recall -0.413 pp (n=242); INN: F1 -0.386 pp, recall -0.380 pp (n=263); FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).
• trt-fp16: COUNTRY: F1 -0.611 pp, recall -0.413 pp (n=242); INN: F1 -0.386 pp, recall -0.380 pp (n=263); FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).
• onnx-fp16: INN: F1 -0.132 pp, recall +0.000 pp (n=263); DRIVER_LICENSE: F1 +0.105 pp, recall +0.000 pp (n=371); PASSPORT: F1 -0.079 pp, recall +0.000 pp (n=494).
• onnx-fp32: no measured per-label change.
• torch-fp16: FIRST_NAME: F1 +0.202 pp, recall +0.200 pp (n=499).

Provenance and interpretation

Dataset revision: f77ea831274daf980cc45c61a93c226be9d978d6; model revision: c802e8cd26f85d1cf920973ea6f83965a0618d63.

All 2,841 original texts are preserved. Case-only WWW/www alignment is explicit for row_0000, row_1628; all 5,614 gold spans remain.

• Quality uses exact character-span micro-F1 over all 21 model entity types; native gold BIO boundaries are preserved.
• This is a paired backend comparison. The PERSON+LOCATION overlap diagnostic does not claim to reproduce the published model-card or leaderboard F1.
• End-to-end batch1 means one document request: tokenization, sliding windows, GPU work, transfers, and entity decoding. Long documents may require multiple windows.
• Fixed-shape model-with-transfers timing includes host-to-device and device-to-host transfer and synchronized CPU output, but excludes tokenization and entity decoding; it is not GPU-kernel-only latency.
• Latency is warm, local, sequential, and excludes model loading, HTTP overhead, queueing, and concurrent clients. Repeated timings of the same texts are not independent unique requests.
• Quality and end-to-end latency both use one window per GPU call. Long-document windows run sequentially; batch16 throughput is measured separately.
• No threshold is tuned on this benchmark; the shared decoder uses a fixed confidence threshold.