DataFog PII local prototype
Standalone workspace on the M5 Pro / 48 GB Mac. No Git repository was created.
User-approved scope: autonomous implementation and subagents, public/synthetic
data, local runs estimated up to 24 hours, no purchases or publishing. Existing
SDK integration is deferred. See docs/contract.md.
Current state
Completed and reviewed. See REPORT.md for full results and limits.
On 90,018 held-out synthetic documents: exact typed-span precision 99.01%,
recall 97.63%, F1 98.32%. Independent recount and frozen-input/bundle checksums
passed (artifacts/final-audit.json). Two corrective rounds were used before
final test exposure. No test-driven model or threshold changes were made.
The native bundle is artifacts/bundle/ (about 300 MB including runtime).
Run without Python from this folder:
printf '%s\n' '{"text":"Email alice@example.com"}' | artifacts/bundle/datafog-pii artifacts/bundle
Training took 8,102 seconds. Colab was not needed; no extra purchases were made. The pipeline has exited and automatic follow-ups are paused. The idle tmux control session remains available; no compute job is running.
Recorded stage status: artifacts/pipeline/status.json. Logs and exact commands:
artifacts/pipeline/*.log and *.done.json. Run uv run --frozen python scripts/final_audit.py to repeat the artifact audit.
Controls
tmux attach -t datafog-pii
./scripts/start-local.sh
./scripts/stop-local.sh
The control window remains interactive. Up recalls start, then Up recalls
stop. Ctrl-b then n/p changes windows; Ctrl-b then d detaches. The pipeline
window holds the job. Start refuses to duplicate that window. Stop interrupts
only this project's process; epoch checkpoints and data remain.
To resume a stopped/failed pipeline, first verify no project pipeline is still running and inspect the failed log, then run from this folder:
caffeinate -i .venv/bin/python scripts/run_pipeline.py
Successful stages are skipped; training resumes from the latest completed epoch checkpoint. Work since that checkpoint can be lost. Do not run competing jobs. Stage markers belong to this run; do not reuse them for changed experiments. Never silently replace a frozen evaluation after inspecting test outcomes.
Reproduction
Python 3.12 dependencies are isolated in .venv, pinned in uv.lock. Rust
uses rust/Cargo.lock. Global Python packages and system settings are unchanged.
These commands were run from this folder:
uv sync
uv run --frozen python scripts/check_environment.py
uv run --frozen python scripts/prepare_data.py
uv run --frozen python scripts/probe_model.py
uv run --frozen python scripts/check_training.py
uv run --frozen python scripts/check_chunk_boundaries.py
uv run --frozen python scripts/train.py --stride 42 --prepare-only
uv run --frozen python -m unittest discover -s tests -v
DEVELOPER_DIR=/Library/Developer/CommandLineTools cargo build --release --manifest-path rust/Cargo.toml
DEVELOPER_DIR=/Library/Developer/CommandLineTools cargo test --manifest-path rust/Cargo.toml
uv run --frozen python scripts/check_rust_parity.py artifacts/probe --output artifacts/probe/rust-parity.json
./scripts/start-local.sh
Use uv sync --frozen to recreate the locked environment. The existing Rust
version is 1.94.1. Process-local DEVELOPER_DIR selects installed Command Line
Tools because selected Xcode requires a license action; no global change made.
Pinned pretrained download performed before the probe:
uv run --frozen python -c 'from huggingface_hub import snapshot_download; snapshot_download("distilbert/distilbert-base-cased", revision="6ea81172465e8b0ad3fddeed32b986cdcdcffcf0", allow_patterns=["config.json","model.safetensors","tokenizer.json","tokenizer_config.json","vocab.txt","README.md","LICENSE"], local_dir="artifacts/backbone")'
Regex baseline setup and commands: docs/regex-baseline.md. The pipeline records
all stage commands and timestamps. Training starts from the official Hugging
Face token-classification recipe: two epochs, batch 16, AdamW 2e-5, weight decay
0.01, selecting lowest validation token loss. These are hypotheses, not optimal
settings. Model limit 512 comes from pinned configuration; overlap 42 derives
from the longest measured selected entity of 43 tokens. Seed 42, gradient clip
1.0 and linear LR schedule follow Transformers Trainer defaults.
Evidence and outputs
docs/data.md,artifacts/data-manifest.json: pinned data, split allocation, annotation exclusions, label support, and checksums.artifacts/data-disjointness.json: independent split verification.artifacts/data-window-coverage.json,artifacts/boundary-proof.json: complete token coverage and exact oracle boundary tests.artifacts/training-check.json: learning and optimizer/RNG resume proof.docs/evaluation.md: metrics and scoring semantics.docs/rust-runtime.md: native build, packaging, inference and licenses.docs/run-log.md: corrective work; two of three rounds used so far.artifacts/bundle/: final model, tokenizer, calibration, native library, executable, licenses, and checksums after pipeline completion.artifacts/evaluation/,artifacts/regex-test/: final measured results.
Dataset spans use character offsets; returned spans use half-open UTF-8 bytes.
contains_pii is true when at least one finding among seven target labels is
accepted. Scores/threshold are experimental, not a production default.
Known limits
Nemotron is synthetic with remaining semantic annotation noise. Invalid source slices/overlaps were quarantined, without regex relabeling. No private tickets were evaluated. Sixteen training-side spans have boundaries the tokenizer cannot express exactly. Overlap covers measured entity lengths, not every possible unseen entity; full-window predicted entities raise explicit errors.
Tokenizers 0.23.2 overflow silently dropped tails. Both runtimes now tokenize
complete input and create windows manually. Obsolete cache
artifacts/training/chunks-stride-42.pt is diagnostic history only; current
training reads chunks-manual-stride-42.pt. Independent full-coverage oracles
prevent shared Python/Rust truncation defects from hiding behind parity.
Colab is not needed currently. Fallback uses the primary Google account signed into Chrome, assigned hardware and existing Pro compute balance, with durable checkpoints and identical scripts/manifests. No Colab skill has been created.