---
title: The Perfect Audit Crime Challenge
emoji: 🕵️
colorFrom: indigo
colorTo: gray
sdk: gradio
app_file: app.py
pinned: false
license: apache-2.0
---

# 🕵️ The Perfect Audit Crime Challenge

A community benchmark on **purely synthetic** general ledgers (generated by DataSynth, ground truth known
by construction). Flag the fraudulent journal entries; an automated scorer ranks you on the held-out
labels. The goal is not to crown a fraud detector — it is to **map the detectability frontier**.

## Two tracks, same ledgers

- **Track A — ledger only.** You get the GL. Structured, relational and cross-period *memory* frauds are
  catchable. The **mimetic perfect crime** — fraud fabricated from the ledger's *own* normal distribution
  — provably is **not** (it preserves the joint distribution of every observable, so no ledger-only
  analytic beats chance on it).
- **Track B — ledger + evidence.** You additionally get the **ISA-520 expectations** (per-account
  aggregate vs an exogenous expectation) and **ISA-505 evidence anchors** (external corroboration /
  dangling accounts). Now the aggregate-inflation and fabricated-counterparty crimes become catchable.

The gap between the two tracks on the mimetic family is the result: it shows *where ledger analytics ends
and external evidence must begin.*

## How to enter

1. Download the challenge dataset: **[VynFi/perfect-audit-crime-data](https://huggingface.co/datasets/VynFi/perfect-audit-crime-data)**
   — per-unit `track_a/` (ledger only) or `track_b/` (ledger + evidence) journal entries.
2. Produce a submission — a CSV/JSON listing the `document_id`s you flag as fraudulent (optional `score`
   column for ranking). IDs are unit-prefixed (e.g. `retail_small:<id>`); a combined submission spans all units.
3. Upload it here, choose your track, and you are scored + added to the leaderboard.

**Metric:** PR-AUC on the held-out per-JE labels (plus precision / recall / F1). The leaderboard also
reports recall **by observability class** — so everyone can see the mimetic family is (near-)unbeatable in
Track A and recoverable only with evidence in Track B.

## Important framing

This is a **research benchmark on synthetic data**. A high score is **not** a deployable fraud detector —
and the entries no one can catch from the ledger are precisely why audit must combine analytics with
**external evidence** (confirmation, reconciliation, substantive analytics). It is **not** a recipe for
committing fraud; it is a map of detectability and its limits. No real or client data is used anywhere.

Built with [DataSynth](https://vynfi.com). Companion paper: *The Detectability Frontier* [in press].