# Dataset and labeling guide

## Recommended Drive layout

```text
MyDrive/orislop-av-joint/
├── data/
│   ├── genuine/
│   ├── synthetic/
│   ├── audio_spoof/
│   ├── legitimate_dubbing/
│   └── difficult_negatives/
├── manifest.jsonl
├── rights.json
├── prepared/
├── models/
├── runs/
└── templates/
```

Paths in `manifest.jsonl` are relative to `data/`.

## Required record fields

Every JSONL record needs:

- `id`: unique clip identifier;
- `videoPath`: path relative to the data directory;
- `audioPath`: optional external synchronized WAV relative to the data directory;
- `startSeconds`: clip start on the video timeline;
- `audioOffsetFrames`: controlled audio shift from `-10` through `10`;
- `split`: `train`, `val`, or `test`;
- `speakerId`: stable identity ID; one speaker must occur in only one split;
- `sourceVideoId` or `contentSha256`: stable source identity; one source must
  occur in only one split;
- `generatorFamily`: `genuine` or the specific manipulation/generator family;
- `mediaClass`: semantic class such as `genuine`, `visual_forgery`,
  `audio_spoof`, `joint_forgery`, `legitimate_dubbing`, or `legitimate_delay`;
- `language`;
- `durationSeconds`;
- `rightsSourceId`: key into `rights.json`;
- `releaseEligible`;
- `labels`.

## Label arrays

The runtime tracks at most four faces, so the template uses four-element arrays.
Use masks to indicate which face labels are known. A value with a zero mask is
ignored during training.

- `active_speaker`: whether the face is producing the audible speech.
- `sync_mismatch`: visible mouth motion is offset from the audio.
- `audio_spoof`: the audible voice is synthetically generated or manipulated.
- `visual_forgery`: the visible speaking face is synthetic or manipulated.
- `joint_forgery`: the combined audio–visual presentation is a forgery.
- `offset`: frame offset from `-10` through `10`.
- `segment_forgery`: optional per-frame forgery labels.

Each binary task should also have a matching `<task>_mask`; offset uses
`offset_mask`.

## Critical hard negatives

Include genuine examples of:

- dubbing and translation;
- voice-over and B-roll;
- video-call and broadcast lag;
- Bluetooth delay;
- music, shouting, whispering, and laughter;
- closed mouths and off-screen speakers;
- multiple people and changing active speakers;
- facial hair, masks, profile faces, occlusion, and low light;
- edits, scene cuts, frame drops, compression, and variable frame rates.

For `legitimate_dubbing` and `legitimate_delay`, label at least one
`sync_mismatch=1` while keeping every `joint_forgery=0`. The rights validator
enforces this rule.

The AMI manifest builder deliberately leaves `active_speaker_mask=0` because
camera/headset pairing alone does not prove speech is active in every window.
The preprocessing quality gate drops windows without adequate audible speech,
visible mouth motion, face coverage, or SNR.

## Split hygiene

- Keep each speaker in exactly one split.
- Keep each source video in exactly one split.
- Keep one or more synthetic generator families entirely out of training and
  validation so the test set measures held-out-generator behavior.
- Never tune thresholds against the test split.