{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Orislop AV Joint v1 — T4 + open-data trainer\n",
    "\n",
    "This notebook installs the portable training kit, creates a persistent Google Drive workspace, verifies the official YuNet face detector, validates dataset rights, prepares visible-speaker tracks, trains from scratch, evaluates, calibrates, and exports a TorchScript artifact.\n",
    "\n",
    "**Runtime:** select a T4. The notebook uses batch size 1 and two workers. Acquisition and preparation are I/O-heavy; GPU training is the T4 stage.\n",
    "\n",
    "**Important:** training output is not automatically production-approved. It remains unpromoted until Phase 2 fusion, independent spatial corroboration, offline gates, and 10,000 reviewed shadow decisions pass.\n",
    "\n",
    "The acquisition step is restricted to the official AMI corpus. It never scrapes social platforms or arbitrary URLs, and it records SHA-256 provenance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Install the kit and mount Drive\n",
    "\n",
    "Run this cell. If the kit is not already extracted, Colab will ask you to upload `orislop-av-joint-open-data-colab-kit.zip`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import hashlib\n",
    "import json\n",
    "import os\n",
    "import shutil\n",
    "import subprocess\n",
    "import sys\n",
    "import zipfile\n",
    "\n",
    "from google.colab import drive, files\n",
    "\n",
    "CONTENT = Path('/content').resolve()\n",
    "KIT = CONTENT / 'orislop-av-joint-open-data-colab-kit'\n",
    "if not (KIT / 'training/orislop_av_joint/train.py').is_file():\n",
    "    os.chdir(CONTENT)\n",
    "    uploaded = files.upload()\n",
    "    candidates = [Path(name) for name in uploaded if name.lower().endswith('.zip')]\n",
    "    if not candidates:\n",
    "        raise RuntimeError('Upload orislop-av-joint-open-data-colab-kit.zip when prompted.')\n",
    "    bundle = next((path for path in candidates if 'orislop-av-joint-open-data-colab-kit' in path.name), candidates[0])\n",
    "    with zipfile.ZipFile(bundle) as archive:\n",
    "        for member in archive.infolist():\n",
    "            target = (CONTENT / member.filename).resolve()\n",
    "            if target != CONTENT and CONTENT not in target.parents:\n",
    "                raise RuntimeError(f'Unsafe archive path rejected: {member.filename}')\n",
    "        archive.extractall(CONTENT)\n",
    "    if not (KIT / 'training/orislop_av_joint/train.py').is_file():\n",
    "        matches = list(CONTENT.glob('*/training/orislop_av_joint/train.py'))\n",
    "        if len(matches) != 1:\n",
    "            raise RuntimeError('Could not locate the extracted AV Joint training kit.')\n",
    "        KIT = matches[0].parents[2]\n",
    "\n",
    "os.chdir(KIT)\n",
    "requirements = [\n",
    "    KIT / 'training/orislop_av_joint/requirements-colab.txt',\n",
    "    KIT / 'training/orislop_av_joint/requirements-acquisition.txt',\n",
    "]\n",
    "for requirement in requirements:\n",
    "    subprocess.run([sys.executable, '-m', 'pip', 'install', '-q', '--upgrade-strategy', 'only-if-needed', '-r', str(requirement)], check=True)\n",
    "if shutil.which('ffmpeg') is None:\n",
    "    subprocess.run(['apt-get', 'update', '-qq'], check=True)\n",
    "    subprocess.run(['apt-get', 'install', '-y', '-qq', 'ffmpeg'], check=True)\n",
    "drive.mount('/content/drive')\n",
    "\n",
    "import torch\n",
    "gpu_name = torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CPU only'\n",
    "print(f'Kit: {KIT}')\n",
    "print(f'Runtime: {gpu_name}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Prove the model and export contracts work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subprocess.run([sys.executable, 'tools/validate_colab_kit.py'], check=True)\n",
    "subprocess.run([sys.executable, 'training/orislop_av_joint/train.py', 'self-test'], check=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Create the persistent Drive workspace and verified YuNet model\n",
    "\n",
    "This does not overwrite an existing manifest or rights ledger."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "DRIVE_ROOT = Path('/content/drive/MyDrive/orislop-av-joint')\n",
    "for relative in (\n",
    "    'data/genuine', 'data/synthetic', 'data/audio_spoof',\n",
    "    'data/legitimate_dubbing', 'data/difficult_negatives',\n",
    "    'prepared', 'models', 'runs', 'templates', 'raw_open_data'\n",
    "):\n",
    "    (DRIVE_ROOT / relative).mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "template_pairs = {\n",
    "    KIT / 'templates/dataset_manifest_template.jsonl': DRIVE_ROOT / 'manifest.jsonl',\n",
    "    KIT / 'templates/rights_ledger_template.json': DRIVE_ROOT / 'rights.json',\n",
    "    KIT / 'templates/shadow_metrics_template.json': DRIVE_ROOT / 'templates/shadow_metrics.json',\n",
    "    KIT / 'DATASET_LABELING_GUIDE.md': DRIVE_ROOT / 'templates/DATASET_LABELING_GUIDE.md',\n",
    "}\n",
    "for source, destination in template_pairs.items():\n",
    "    if not destination.exists():\n",
    "        shutil.copy2(source, destination)\n",
    "\n",
    "YUNET_URL = 'https://media.githubusercontent.com/media/opencv/opencv_zoo/main/models/face_detection_yunet/face_detection_yunet_2023mar.onnx'\n",
    "YUNET_SHA256 = '8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4'\n",
    "YUNET = DRIVE_ROOT / 'models/face_detection_yunet_2023mar.onnx'\n",
    "\n",
    "def file_sha256(path):\n",
    "    digest = hashlib.sha256()\n",
    "    with Path(path).open('rb') as source:\n",
    "        for chunk in iter(lambda: source.read(1024 * 1024), b''):\n",
    "            digest.update(chunk)\n",
    "    return digest.hexdigest()\n",
    "\n",
    "if not YUNET.is_file() or file_sha256(YUNET) != YUNET_SHA256:\n",
    "    temporary = YUNET.with_suffix('.download')\n",
    "    urllib.request.urlretrieve(YUNET_URL, temporary)\n",
    "    if file_sha256(temporary) != YUNET_SHA256:\n",
    "        temporary.unlink(missing_ok=True)\n",
    "        raise RuntimeError('Downloaded YuNet model failed SHA-256 verification.')\n",
    "    temporary.replace(YUNET)\n",
    "\n",
    "print(f'Drive workspace: {DRIVE_ROOT}')\n",
    "print(f'YuNet verified: {YUNET}')\n",
    "print('Next: add your videos, then replace manifest.jsonl and rights.json with reviewed data.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Acquire up to 50 GB of rights-aware open AV media\n",
    "\n",
    "This downloader only follows official AMI corpus indexes. It fetches low-size close-up AVI streams and synchronized individual-headset WAV channels, resumes partials, rejects off-host redirects, hashes every completed file, and stops at the quota. Review the [AMI license](https://groups.inf.ed.ac.uk/ami/corpus/license.shtml) and [consent evidence](https://groups.inf.ed.ac.uk/ami/corpus/ethicsandconsent.shtml) before changing the acceptance switch.\n",
    "\n",
    "**This acquires authentic source media; it does not invent deepfake labels or make an AMI-only model a production deepfake detector.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ACQUIRE_AMI = False  # Change to True only when you want the long download.\n",
    "I_ACCEPT_AMI_CC_BY_4_0 = False  # Change after reading the linked terms.\n",
    "AMI_TARGET_GB = 50.0\n",
    "AMI_RAW = DRIVE_ROOT / 'raw_open_data'\n",
    "AMI_MANIFEST = DRIVE_ROOT / 'ami_lipsync_manifest.jsonl'\n",
    "AMI_RIGHTS = DRIVE_ROOT / 'rights_ami_review_required.json'\n",
    "AMI_MAPPING = DRIVE_ROOT / 'ami_signals_mapping_snapshot.html'\n",
    "\n",
    "acquire_command = [\n",
    "    sys.executable, 'tools/acquire_open_av_data.py',\n",
    "    '--registry', str(KIT / 'data_sources.json'),\n",
    "    '--output-root', str(AMI_RAW),\n",
    "    '--target-gb', str(AMI_TARGET_GB),\n",
    "]\n",
    "if ACQUIRE_AMI:\n",
    "    if not I_ACCEPT_AMI_CC_BY_4_0:\n",
    "        raise RuntimeError('Read the AMI license and consent evidence, then set I_ACCEPT_AMI_CC_BY_4_0=True.')\n",
    "    acquire_command.extend(['--execute', '--accept-license', 'ami-cc-by-4.0'])\n",
    "subprocess.run(acquire_command, check=True)\n",
    "if ACQUIRE_AMI:\n",
    "    subprocess.run([\n",
    "        sys.executable, 'tools/build_ami_lipsync_manifest.py',\n",
    "        '--raw-root', str(AMI_RAW), '--output', str(AMI_MANIFEST),\n",
    "        '--rights-output', str(AMI_RIGHTS), '--mapping-snapshot', str(AMI_MAPPING),\n",
    "        '--clip-seconds', '8', '--stride-seconds', '24',\n",
    "    ], check=True)\n",
    "    print(f'Generated synchronization manifest: {AMI_MANIFEST}')\n",
    "    print(f'Review required before training: {AMI_RIGHTS}')\n",
    "print(f'Raw open-data root: {AMI_RAW}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Configure the run\n",
    "\n",
    "Keep `ALLOW_CORPUS_SMOKE = False` for a real release-training attempt. If Colab disconnects, rerun the notebook with the same `RUN_NAME`; training resumes from the latest completed epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MANIFEST = AMI_MANIFEST if AMI_MANIFEST.is_file() else DRIVE_ROOT / 'manifest.jsonl'\n",
    "RIGHTS = AMI_RIGHTS if AMI_RIGHTS.is_file() else DRIVE_ROOT / 'rights.json'\n",
    "DATA = AMI_RAW if MANIFEST == AMI_MANIFEST else DRIVE_ROOT / 'data'\n",
    "PREPARED = DRIVE_ROOT / 'prepared'\n",
    "PREPARED_MANIFEST = PREPARED / 'prepared_manifest.jsonl'\n",
    "\n",
    "RUN_NAME = 'av_joint_v1'\n",
    "RUN_DIR = DRIVE_ROOT / 'runs' / RUN_NAME\n",
    "RUN_DIR.mkdir(parents=True, exist_ok=True)\n",
    "CHECKPOINT = RUN_DIR / 'av_joint_v1.pt'\n",
    "RIGHTS_REPORT = RUN_DIR / 'rights_report.json'\n",
    "TEST_METRICS = RUN_DIR / 'test_metrics.json'\n",
    "TEMPERATURES = RUN_DIR / 'temperatures.json'\n",
    "ARTIFACT_DIR = RUN_DIR / 'artifact'\n",
    "\n",
    "EPOCHS = 20\n",
    "BATCH_SIZE = 1\n",
    "WORKERS = 2\n",
    "ALLOW_CORPUS_SMOKE = True  # AMI-only experiments must remain non-promotable.\n",
    "REBUILD_PREPARED = False\n",
    "\n",
    "print(f'Run directory: {RUN_DIR}')\n",
    "print(f'Mode: {\"NON-PROMOTABLE SMOKE\" if ALLOW_CORPUS_SMOKE else \"RELEASE TRAINING\"}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Dataset and rights preflight\n",
    "\n",
    "This cell intentionally stops if templates, missing media, split leakage, or unapproved rights remain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "records = [json.loads(line) for line in MANIFEST.read_text(encoding='utf-8').splitlines() if line.strip()]\n",
    "ledger = json.loads(RIGHTS.read_text(encoding='utf-8'))\n",
    "serialized = json.dumps({'records': records, 'ledger': ledger})\n",
    "if 'replace-with-' in serialized or 'replace-me' in serialized:\n",
    "    raise RuntimeError('Replace every placeholder in manifest.jsonl and rights.json before training.')\n",
    "if {record.get('split') for record in records} != {'train', 'val', 'test'}:\n",
    "    raise RuntimeError('The manifest must contain train, val, and test records.')\n",
    "missing_media = []\n",
    "for record in records:\n",
    "    for field in ('videoPath', 'audioPath'):\n",
    "        if not record.get(field):\n",
    "            continue\n",
    "        source = Path(str(record[field]))\n",
    "        source = source if source.is_absolute() else DATA / source\n",
    "        if not source.is_file():\n",
    "            missing_media.append(str(source))\n",
    "if missing_media:\n",
    "    raise RuntimeError(f'Missing {len(missing_media)} media files. First examples: {missing_media[:5]}')\n",
    "\n",
    "rights_command = [\n",
    "    sys.executable, 'training/orislop_av_joint/train.py', 'validate-rights',\n",
    "    '--manifest', str(MANIFEST), '--rights-ledger', str(RIGHTS),\n",
    "    '--output', str(RIGHTS_REPORT),\n",
    "]\n",
    "subprocess.run(rights_command, check=True)\n",
    "rights_report = json.loads(RIGHTS_REPORT.read_text(encoding='utf-8'))\n",
    "if not ALLOW_CORPUS_SMOKE:\n",
    "    if rights_report['commercialSpeakers'] < 100 or rights_report['commercialHours'] < 50:\n",
    "        raise RuntimeError('Release training requires at least 100 approved speakers and 50 approved hours.')\n",
    "    if not rights_report['heldOutGeneratorFamilies']:\n",
    "        raise RuntimeError('Keep at least one synthetic generator family exclusively in the test split.')\n",
    "print(f'Preflight passed: {len(records)} records, {rights_report[\"commercialSpeakers\"]} speakers, {rights_report[\"commercialHours\"]} hours.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. Prepare bounded face, mouth, audio, mask, and quality tensors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if PREPARED_MANIFEST.is_file() and not REBUILD_PREPARED:\n",
    "    print(f'Using existing prepared manifest: {PREPARED_MANIFEST}')\n",
    "else:\n",
    "    prepare_command = [\n",
    "        sys.executable, 'training/orislop_av_joint/prepare.py',\n",
    "        '--manifest', str(MANIFEST), '--data-root', str(DATA),\n",
    "        '--output-root', str(PREPARED), '--yunet-model', str(YUNET),\n",
    "        '--seconds', '8',\n",
    "    ]\n",
    "    subprocess.run(prepare_command, check=True)\n",
    "if not PREPARED_MANIFEST.is_file():\n",
    "    raise RuntimeError('Preparation did not produce prepared_manifest.jsonl.')\n",
    "print(f'Prepared dataset: {PREPARED_MANIFEST}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Train from scratch, with Drive-backed epoch checkpoints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not torch.cuda.is_available() and not ALLOW_CORPUS_SMOKE:\n",
    "    raise RuntimeError('Select a T4 GPU runtime before training.')\n",
    "train_command = [\n",
    "    sys.executable, 'training/orislop_av_joint/train.py', 'train',\n",
    "    '--manifest', str(PREPARED_MANIFEST), '--rights-ledger', str(RIGHTS),\n",
    "    '--config', str(KIT / 'configs/av_joint_v1.json'), '--output', str(CHECKPOINT),\n",
    "    '--epochs', str(EPOCHS), '--batch-size', str(BATCH_SIZE), '--workers', str(WORKERS),\n",
    "    '--device', 'auto', '--resume',\n",
    "]\n",
    "if ALLOW_CORPUS_SMOKE:\n",
    "    train_command.append('--allow-corpus-smoke')\n",
    "subprocess.run(train_command, check=True)\n",
    "print(f'Checkpoint: {CHECKPOINT}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Evaluate, calibrate, and export the unpromoted TorchScript artifact"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subprocess.run([\n",
    "    sys.executable, 'training/orislop_av_joint/train.py', 'evaluate',\n",
    "    '--manifest', str(PREPARED_MANIFEST), '--checkpoint', str(CHECKPOINT),\n",
    "    '--split', 'test', '--output', str(TEST_METRICS), '--batch-size', str(BATCH_SIZE),\n",
    "], check=True)\n",
    "subprocess.run([\n",
    "    sys.executable, 'training/orislop_av_joint/train.py', 'calibrate',\n",
    "    '--manifest', str(PREPARED_MANIFEST), '--checkpoint', str(CHECKPOINT),\n",
    "    '--output', str(TEMPERATURES), '--batch-size', str(BATCH_SIZE),\n",
    "], check=True)\n",
    "subprocess.run([\n",
    "    sys.executable, 'training/orislop_av_joint/train.py', 'export',\n",
    "    '--checkpoint', str(CHECKPOINT), '--temperatures', str(TEMPERATURES),\n",
    "    '--output-dir', str(ARTIFACT_DIR),\n",
    "], check=True)\n",
    "artifact_zip = Path(shutil.make_archive(str(RUN_DIR / 'orislop_av_joint_v1_artifact'), 'zip', ARTIFACT_DIR))\n",
    "print(f'Test metrics: {TEST_METRICS}')\n",
    "print(f'Artifact folder: {ARTIFACT_DIR}')\n",
    "print(f'Downloadable artifact ZIP: {artifact_zip}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Done — but not promoted\n",
    "\n",
    "The exported metadata deliberately says `promoted: false`. Before enabling this model in OriSlop, retrain and calibrate Temporal MoE Phase 2 with the AV expert, measure independent spatial corroboration, meet the offline recall/calibration/latency and genuine-hide gates, and review at least 10,000 real shadow decisions."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "Orislop_AV_Joint_Trainer.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}