docs: add TRAINING_LOG.md template + training_logs dir for Phase 13 discipline

BUILD_LOG.md

@@ -0,0 +1,302 @@
+# SchemaShift Build Log
+
+Running log of everything built, tested, trained, and deployed. Append-only — never delete.
+
+**Team Tripod:** Yashash Sheshagiri (lead), Gajanand V Dhayagode, Likith B S
+**Event:** Meta × HF × PyTorch OpenEnv Hackathon 2026 · Round 2 · Bangalore · April 25-26
+
+---
+
+## PHASES COMPLETED
+
+### Phase 0 — Project Scaffolding
+- **Date:** Tuesday April 21, 2026 (evening)
+- **Commit:** `d4ab0f1`
+- **Tests:** 0 (scaffolding only — no logic yet)
+- **Time:** ~30 min
+- **Notes:** 29 files created. Python venv built. Deps installed cleanly (fastapi 0.136, pydantic 2.13, openai 2.32, openenv-core 0.2.3, gradio 6.13). Repo pushed to https://github.com/Yashash4/SchemaShift.
+
+### Phase 1 — Pydantic Models
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `8cdb02b`
+- **Tests:** 6 passing (0.22s runtime)
+- **Line counts:** models.py = 135 lines, test_models.py = ~120 lines
+- **Time:** ~20 min
+- **Judgment calls:**
+  1. 12 classes + 2 Literal type aliases (not 13 classes as prompt suggested)
+- **Notes:** Spec code copied verbatim. No deviations.
+
+### Phase 2 — Mail Tool
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `8410ff1`
+- **Tests:** 7 new (13 total), 0.16s runtime
+- **Line counts:** tools/base.py = 60, tools/mail.py = 121, tests/test_mail.py = 149
+- **Time:** ~90 min
+- **Judgment calls:**
+  1. Absolute imports (`from models import`) over relative imports — matches Phase 1 convention
+  2. `get_schema()` returns `{}` for unknown endpoints post-deprecation
+  3. 501 fallback in `BaseTool.call` for schema-without-handler (dev-time safety)
+  4. `messages.send` is deep-copy rename of `send_message` after deprecation drift
+  5. Pagination tokens: `"tok_xyz"` / `"cur_abc123"` when >10 results, `None` otherwise
+- **Notes:** 3 endpoints (list_messages, send_message, get_message), 3 drifts (field_rename, endpoint_deprecation, new_required_param).
+
+### Phase 3 — Calendar Tool + DriftInjector
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `27dd958`
+- **Tests:** 10 new (23 total), 0.24s runtime
+- **Line counts:** tools/calendar.py = 164, drift.py = 28, tests/test_calendar.py = 182, tests/test_drift.py = 122
+- **Time:** ~100 min
+- **Judgment calls:**
+  1. Overlap predicate for `list_events` (inclusive both sides, ISO string sort)
+  2. `_create_event` handles both attendees and participants gracefully
+  3. `_update_event` accepts both attendees and participants (no second drift handler needed)
+  4. Cancelled events remain in storage (matches Google/Outlook behavior)
+  5. `_fired` sentinel in `event.details` (underscore prefix avoids collision)
+  6. `rate_limit_tightening` chosen for unknown-drift test (semantically meaningful)
+- **Notes:** 4 endpoints (list/create/update/delete_event), 2 drifts (field_rename attendees→participants, tool_removal delete_event). DriftInjector is stateless.
+
+### Phase 4 — CRM Tool + Scenarios E1/E2/E3
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `bc8f184`
+- **Tests:** 11 new (34 total), 0.24s runtime
+- **Line counts:** tools/crm.py = 213, scenarios.py = 124, tests/test_crm.py = 142, tests/test_scenarios.py = 65
+- **Time:** ~100 min
+- **Judgment calls:**
+  1. Strict param validation in `BaseTool.call` — rejects unknown params with 400 "Unknown params" (matches Stripe/Google API behavior)
+  2. Internal storage stays `customer_email`; projection on read via `_project_contact`
+  3. `search_contacts()` with no filters returns ALL contacts (REST convention)
+  4. Rate-limit counter resets on drift fire (pre-drift calls don't count against post-drift budget)
+  5. Drift-order commutativity for `contacts.patch` (either order produces same final state)
+  6. E2 task description doesn't disclose deprecation (agent must discover via 410 response)
+  7. E3 drift fires at step 2 (gives agent one clean baseline call)
+  8. `DriftEvent.details` populated with hints (`from`, `to`) for grader/scenario auditing
+- **Notes:** 4 endpoints (search/get/create/update_contact), 3 drifts (tool-wide field_rename, rate_limit_tightening, endpoint_deprecation update→contacts.patch).
+
+### Phase 5 — Composable Grader + Dense Step Shaping
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `46b8c56`
+- **Tests:** 11 new (45 total), 0.25s runtime
+- **Line counts:** graders.py = 267, tests/test_graders.py = 224
+- **Time:** ~150 min
+- **Judgment calls:**
+  1. Binary threshold 0.95 (not 1.0) — floating point safety
+  2. `AdaptationRubric` iterates all fired drifts (TODO: revisit for Phase 11 M-scenarios with multi-drift-per-tool)
+  3. `_final_state_acceptable` lenient mid-episode, strict at terminal — preserves training signal
+  4. `compute_step_shaping` checks `not d.detected_by_agent` — anti-farming guard
+  5. Dumb-retry penalty requires same tool AND same endpoint
+  6. `assert` for weight-sum invariant (dev-time, not runtime)
+  7. Rubric names coupled to `RewardBreakdown` field names (intentional reward contract)
+  8. Bonus test asserts `step_shaping = 0` in grader output (env fills it, not grader)
+- **Notes:** 4 rubrics (Completion, DriftDetection, Adaptation, Efficiency) + 2 gates (catastrophic, correct_final) + WeightedSum + Gate + `compute_step_shaping` + `build_grader`.
+
+### Phase 6 — Environment Scheduler
+- **Date:** Tuesday April 21, 2026
+- **Commit:** `c618a61`
+- **Tests:** 8 new (53 total), 0.29s runtime
+- **Line counts:** server/environment.py = 363, tests/test_environment.py = 242
+- **Time:** ~120 min
+- **E1 FULL-EPISODE BENCHMARK:**
+  - `shaped_total = 0.934375` (expected ~0.93, exact match on weighted sum)
+  - `task_completion = 1.0`
+  - `drift_detection = 1.0`
+  - `adaptation_quality = 1.0`
+  - `efficiency = 0.5625` (used 7 of 8 max steps)
+  - `catastrophic_gate = 1.0, correct_final_gate = 1.0`
+  - `binary = 1.0`
+  - `cumulative_reward = 4.2975` across 7 steps
+  - **Dense shaping fired correctly:** +0.10 at step 4 (inspect-after-failure), +0.15 at step 6 (drift report)
+- **Judgment calls:**
+  1. Shaping-before-marking ordering: dispatch→compute_step_shaping→mark detected→grader
+  2. `tokens_used` clamped to non-negative with `max(0, remaining - tokens_used)`
+  3. Lazy tool imports in `_instantiate_tool` — stretch tools don't break core
+  4. `deepcopy(drift_plan)` on reset — prevents state leak across episodes
+  5. Observation history windowed to last 5 steps for LLM prompt budget
+  6. Calendar body parser handles both attendees (strings) and participants (dicts)
+  7. `_completion_summary` stored with underscore prefix (no GT key collision)
+  8. E2 three-recipient check tracked in `_update_agent_state`
+  9. `done` flag set AFTER dispatch but BEFORE grader (enables strict gate at terminal)
+  10. Dispatch returns `None` for `report_drift` and `complete_task` (no tool response)
+- **Notes:** Round 1 `_engine is None` bug pattern explicitly prevented via `RuntimeError` in step() when state is None.
+
+### Phase 7 — FastAPI Server + Docker
+- **Date:** Tuesday April 21, 2026 (late evening)
+- **Commit:** `cb33205`
+- **Tests:** 10 new (63 total), 0.94s runtime
+- **Line counts:** server/app.py = 120, tests/test_server.py = 129, Dockerfile = 14, openenv.yaml = 14
+- **Time:** ~60 min
+- **Live server smoke tests:** uvicorn on 127.0.0.1:7860 verified /health, /tasks (count=3), /reset (E1 → step=0, tools=['mail', 'calendar']) all 200 OK
+- **`pip install -e .` confirmation:** Editable wheel built successfully. Docker container will resolve absolute imports correctly.
+- **Judgment calls:**
+  1. Removed root `__init__.py` — conflict with `py-modules` in pyproject
+  2. `py-modules` + `packages` split in pyproject (preserves `from models import` convention)
+  3. Dropped unused imports from spec's server code
+  4. TestClient monkeypatch fixture — fresh env per test prevents cross-contamination
+  5. `/step` catches `RuntimeError` → 400 not 500 (client bug not server bug)
+  6. Read-only endpoints (`/state`, `/grader`) don't auto-reset
+  7. `/tasks` trims description to 120 chars (lightweight listing)
+  8. No CORS middleware yet (defer to Phase 10 if needed)
+  9. Hardcoded version "0.1.0" in two places (pragmatic duplication)
+  10. `urllib.request` for smoke test (Windows-safe, no curl dependency)
+- **Notes:** Endpoints: /, /health, /reset, /step, /state, /tasks, /grader. Port 7860 for HF Spaces compatibility.
+
+### Phase 8 — Env Client + Training Skeleton
+- **Status:** NOT YET STARTED
+- **Assigned to:** Yashash (lead) with Gajanand reviewing Kaggle notebook
+- **Expected time:** ~2-2.5 hours
+- (To be filled in on completion)
+
+---
+
+## PHASES REMAINING
+
+- [ ] Phase 8 — Env client + training skeleton
+- [ ] Phase 9 — Baseline eval (heuristics + 3 LLMs + GPT-4o-mini)
+- [ ] Phase 10 — HF Space deploy
+- [ ] Phase 11 — Medium scenarios (M1/M2/M3)
+- [ ] Phase 12 — Insurance video recording (2 videos: 60s + 2min)
+- [ ] Phase 13 — Kaggle training runs (Stage 1 single account → Stage 2 parallelize)
+- [ ] Phase 14 — Pitch + blog + video
+- [ ] Phase 15 — Onsite Saturday-Sunday
+
+---
+
+## TRAINING RUNS
+
+(Will be populated during Phase 13. Template for each run below.)
+
+### Run 1 — Account [N] — [Person] — [Date Time]
+- **Model:** Qwen 2.5 1.5B Instruct OR Qwen 2.5 Coder 1.5B Instruct
+- **Reward variant:** binary / shaped_total / shaped_total+procedural
+- **Scenario set:** deterministic / procedural
+- **LoRA config:** r=16, alpha=32, targets=[q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
+- **GRPO config:** lr=5e-6, num_generations=4, max_completion_length=1536, gradient_accumulation=4, max_steps=100
+- **Start time:**
+- **End time:**
+- **Steps completed:**
+- **First batch mean reward:**
+- **Final mean reward:**
+- **Delta:**
+- **Checkpoint HF Hub ID:**
+- **Eval E1 score:**
+- **Eval E2 score:**
+- **Eval E3 score:**
+- **Reward curve image:** `training_logs/run_N.png`
+- **Errors / stops:**
+- **Notes:**
+
+---
+
+## EVAL RUNS
+
+(Will be populated during Phase 9. Template below.)
+
+### Eval [baseline_name] — [Date]
+- **Model:** e.g., naive_heuristic, policy_aware_heuristic, Qwen 2.5 7B, Llama 3.1 8B, GPT-4o-mini, Qwen 2.5 1.5B (trained)
+- **Seeds:** 5 per task
+- **E1_onboard_new_hire avg:**
+- **E2_meeting_invite_blast avg:**
+- **E3_customer_lookup avg:**
+- **Full raw scores table:**
+
+  | Task | Seed | Completion | DriftDetection | Adaptation | Efficiency | Shaped | Binary |
+  |------|------|------------|----------------|------------|------------|--------|--------|
+  | E1   | 0    |            |                |            |            |        |        |
+  | E1   | 1    |            |                |            |            |        |        |
+  | ...  |      |            |                |            |            |        |        |
+
+- **Failure examples:**
+  1. Action JSON + Response JSON
+  2. ...
+  3. ...
+
+---
+
+## DEPLOYMENTS
+
+(Will be populated during Phase 10+.)
+
+### Deploy v[N] — [Date]
+- **Target:** HF Space / local / Kaggle
+- **Commit:** 
+- **URL:** 
+- **Health check:** 
+- **Config secrets set:** 
+- **Notes:**
+
+---
+
+## DECISIONS LOG (strategic, append-only)
+
+### Tuesday ~9 PM — Domain lock-in
+- **Decision:** Schema Drift Adaptation Arena (SaaS Admin Workflows)
+- **Rationale:** Direct Patronus sub-theme hit; Scaler secondary; no competitor finalist has it; authentic to Yashash's APEX OS/VisionX experience; cleanest before/after RL story
+
+### Tuesday ~10 PM — Staged Kaggle approach (v2.3)
+- **Decision:** Run ONE Kaggle account (Gajanand/Account 2, main config) first. Parallelize only if Stage 1 converges.
+- **Rationale:** If reward function or parser has a bug, 3 simultaneous accounts waste 3× GPU quota. Debug once on one account. Main config = primary pitch claim, so prove that first.
+
+### Tuesday ~10 PM — Qwen 2.5 Coder 1.5B ablation on Account 3 (v2.2)
+- **Decision:** Account 3 uses Coder-1.5B instead of same model + curriculum only.
+- **Rationale:** More informative ablation — tests whether code-pretraining helps schema-adaptation learning. Gives blog/pitch a secondary result regardless of outcome.
+
+### Tuesday ~10 PM — Two videos in Phase 12 (v2.1)
+- **Decision:** Produce 60-second core cut + 2-minute full version, not one 90-second video.
+- **Rationale:** 60s fits pitch timing with hook+close room. 2min fits HF blog context depth. Recording once (Thursday), using twice.
+
+### Tuesday ~10 PM — One-sentence pitch opener (v2.1)
+- **Decision:** Open README/pitch/video with: *"SchemaShift teaches agents to recover when the tool schema changes under them."*
+- **Rationale:** Gallery winners (Kube SRE Gym, GAIA) lead with one-sentence claims. Judges remember one clean idea, not four themes.
+
+### Tuesday ~10 PM — Procedural drift scheduler on Account 3 (v2.1)
+- **Decision:** Account 3 variant adds ~30 LOC scheduler that rotates drift steps and adds secondary drift after step 150.
+- **Rationale:** Kube SRE Gym uses Claude-powered adversarial designer; we achieve similar curriculum story without API credit burn.
+
+---
+
+## REJECTED IDEAS (do not revive — posterity record)
+
+### Validator / oversight agent — rejected Tuesday
+- **Why rejected:** Would break 63 passing tests, adds scope at wrong phase, theme-benefit marginal (multi-agent claim already defensible via scripted tool servers)
+
+### Uncertainty injection across failure types — rejected Tuesday  
+- **Why rejected:** Dilutes pitch (becomes "uncertainty" not "schema drift"), risks GRPO convergence by introducing probabilistic reward, already solved by dense shaping + policy-aware heuristic baseline proving non-gameability
+
+### Claude-powered adversarial drift designer — rejected Tuesday
+- **Why rejected:** Time sink, API credit burn, provides marginal benefit over procedural scheduler
+
+### Pivot to "decision-making under uncertainty" framing — rejected Tuesday
+- **Why rejected:** Breaks Patronus sub-theme fit, muddies demo story, opens scope unnecessarily
+
+### Qwen3 family migration (primary training) — rejected Tuesday
+- **Why rejected:** Newer but less stable, prompt-behavior churn, migration risk for 48-hour build. Stick with Qwen 2.5 1.5B.
+
+### 3B+ primary model — rejected Tuesday
+- **Why rejected:** Slow iteration (2x fewer experiments per Kaggle hour), tight T4 memory, doesn't improve pitch. "1.5B beats GPT-4o-mini" is a stronger claim than "3B beats GPT-4o-mini".
+
+---
+
+## OPEN QUESTIONS / TODOS
+
+- [ ] Phase 8: verify step_shaping survives HTTP serialization roundtrip
+- [ ] Phase 9: do baseline LLMs hit at least 0.30 on drifted E1? (if too high, env is too easy)
+- [ ] Phase 10: deploy to HF Space, verify remote /health works
+- [ ] Phase 11: when adding M1/M2/M3, verify AdaptationRubric denominator still makes sense (flagged in Phase 5 judgment call #2)
+- [ ] Phase 13 Stage 1: go/no-go decision at 20 steps — document the reward curve image
+- [ ] Phase 14: draft strong-claim and softer-claim versions of pitch, pick based on training results
+- [ ] Phase 15 onsite: bring a backup video file on USB in case HF Space is slow to load
+
+---
+
+## ONSITE LOGISTICS
+
+**Event:** Saturday-Sunday April 25-26, 2026 · SST Campus Bangalore
+**Travel:** (add details)
+**Accommodation:** (add details)
+**Equipment to bring:**
+- 2 laptops (primary + backup)
+- USB with video files + repo backup
+- HDMI adapter for presentation
+- Chargers + extension cord
+- Both phones with hotspot capability (in case venue wifi fails)
+
+---

TRAINING_LOG.md

@@ -0,0 +1,454 @@
+# SchemaShift Training Log
+
+**Purpose:** Immutable record of every training run, every checkpoint, every evaluation. Append-only. Never edit existing entries. If a number turns out wrong later, add a correction note below it — never overwrite.
+
+**Why this log exists:** By Sunday pitch you will quote numbers. "Our 1.5B model improved from 0.15 to 0.78 on drifted E1." "We trained for 100 steps over 2.5 hours." "Binary-only reward failed to converge; shaped reward converged in 60 steps." Every single one of these claims requires logged data. Memory will not survive five days of continuous building. Write it down now or rebuild it from bad memory on Sunday morning.
+
+**Team:** Tripod (Yashash / Gajanand / Likith)
+**Event:** Meta × HF × PyTorch OpenEnv Hackathon · Round 2 · April 25-26, 2026
+
+---
+
+## HOW TO USE THIS LOG
+
+Four sections below, each append-only:
+
+1. **PRE-TRAINING BASELINES** — untrained model scores (the "before" number for the pitch)
+2. **TRAINING RUNS** — every run you attempt, successful or failed, with per-checkpoint data
+3. **CHECKPOINT EVALS** — every eval you run against any checkpoint (linked by checkpoint ID)
+4. **HEAD-TO-HEAD COMPARISONS** — the tables that will go in the blog / pitch
+
+**Logging frequency during Phase 13:**
+- Before any training run starts: full baseline eval against the untrained model
+- Every 25 GRPO steps during training: checkpoint auto-saves, run eval against it
+- After training completes: full eval + reward curve screenshot + judgment notes
+- After each eval run on any baseline LLM: add to CHECKPOINT EVALS section
+- End of Thursday: fill in first HEAD-TO-HEAD table with everything you have so far
+
+---
+
+## 1. PRE-TRAINING BASELINES
+
+These are the "before" numbers. Every trained checkpoint gets compared against these.
+
+### Baseline — UNTRAINED Qwen 2.5 1.5B Instruct (base model, no training)
+
+- **Evaluated:** [DATE/TIME — fill in before Stage 1 training starts]
+- **Seeds:** 0, 1, 2, 3, 4 (five per task)
+- **Scenarios:** E1_onboard_new_hire, E2_meeting_invite_blast, E3_customer_lookup
+- **Eval runner:** `python eval.py --baseline untrained_qwen_1.5b_instruct --seeds 0,1,2,3,4`
+
+#### Per-seed raw scores
+
+| Task | Seed | Completion | DriftDetection | Adaptation | Efficiency | Shaped | Binary |
+|------|------|------------|----------------|------------|------------|--------|--------|
+| E1   | 0    |            |                |            |            |        |        |
+| E1   | 1    |            |                |            |            |        |        |
+| E1   | 2    |            |                |            |            |        |        |
+| E1   | 3    |            |                |            |            |        |        |
+| E1   | 4    |            |                |            |            |        |        |
+| E2   | 0    |            |                |            |            |        |        |
+| E2   | 1    |            |                |            |            |        |        |
+| E2   | 2    |            |                |            |            |        |        |
+| E2   | 3    |            |                |            |            |        |        |
+| E2   | 4    |            |                |            |            |        |        |
+| E3   | 0    |            |                |            |            |        |        |
+| E3   | 1    |            |                |            |            |        |        |
+| E3   | 2    |            |                |            |            |        |        |
+| E3   | 3    |            |                |            |            |        |        |
+| E3   | 4    |            |                |            |            |        |        |
+
+#### Aggregates
+
+- **E1 mean shaped_total:**
+- **E2 mean shaped_total:**
+- **E3 mean shaped_total:**
+- **Overall mean shaped_total:**
+- **Overall binary rate (fraction of episodes that hit binary=1.0):**
+
+#### Behavioral observations
+
+What did the untrained model consistently do wrong? Quote 3 specific failure patterns:
+
+1.
+2.
+3.
+
+#### Failure examples (paste full Action + Response JSON for 3 bad actions)
+
+Example 1 — [scenario, seed, step number]:
+```json
+{"action": ..., "response": ...}
+```
+Commentary: what was wrong
+
+Example 2 — [scenario, seed, step number]:
+```json
+{"action": ..., "response": ...}
+```
+Commentary: what was wrong
+
+Example 3 — [scenario, seed, step number]:
+```json
+{"action": ..., "response": ...}
+```
+Commentary: what was wrong
+
+#### Runtime
+
+- Total eval time: X minutes
+- Kaggle GPU used: yes/no, which account
+- Cost: X GPU-hours
+
+---
+
+### Baseline — UNTRAINED Qwen 2.5 Coder 1.5B Instruct (for v2.2 ablation)
+
+(Same structure as above — fill in before Account 3 Stage 2 training starts)
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+
+(Fill the same per-seed table, aggregates, observations, failures, runtime.)
+
+---
+
+### Baseline — naive_heuristic
+
+A dumb policy that always calls the first endpoint of the first tool. Proves the environment cannot be trivially solved.
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+
+(Fill table + aggregates.)
+
+---
+
+### Baseline — policy_aware_heuristic
+
+A smarter rule-based policy: on 4xx/5xx response, call inspect_schema next. Upper bound for non-RL solutions.
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+
+(Fill table + aggregates.)
+
+---
+
+### Baseline — Qwen 2.5 7B Instruct (via HF router)
+
+Larger-but-still-small LLM baseline. Should do better than 1.5B untrained but worse than trained 1.5B.
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+- **API provider:** HF Inference Router
+
+(Fill table + aggregates.)
+
+---
+
+### Baseline — Llama 3.1 8B Instruct (via HF router)
+
+Meta's comparable-size baseline.
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+- **API provider:** HF Inference Router
+
+(Fill table + aggregates.)
+
+---
+
+### Baseline — GPT-4o-mini (the pitch target)
+
+The frontier proxy we need to beat. This is THE number to beat.
+
+- **Evaluated:** [DATE/TIME]
+- **Seeds:** 0, 1, 2, 3, 4
+- **API provider:** OpenAI
+- **Cost:** $X.XX in API credits
+
+(Fill table + aggregates.)
+
+**PITCH-CRITICAL NOTE:** GPT-4o-mini's score on E1 is the headline comparison. If Qwen 1.5B + SchemaShift beats this number, we have the pitch. If not, we reframe to "trained small model beats untrained baseline" which is weaker.
+
+---
+
+## 2. TRAINING RUNS
+
+Every training attempt, successful or failed. Number them sequentially across the team.
+
+### Run 1 — Account 2 (Gajanand) — STAGE 1 MAIN — [Date Time start]
+
+**This is Stage 1 — the pipeline-validation run. Critical go/no-go for Stage 2.**
+
+#### Config snapshot (immutable — record exactly what was used)
+
+- **Model:** Qwen 2.5 1.5B Instruct (`Qwen/Qwen2.5-1.5B-Instruct`)
+- **Reward variant:** `return reward.shaped_total` (full: rubric + dense step shaping + gates)
+- **Scenario set:** deterministic (original SCENARIOS from scenarios.py, no procedural scheduler)
+- **LoRA config:**
+  - r: 16
+  - lora_alpha: 32
+  - lora_dropout: 0.0
+  - target_modules: `["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]`
+- **GRPO config:**
+  - learning_rate: 5e-6
+  - num_generations: 4
+  - max_completion_length: 1536
+  - per_device_train_batch_size: 1
+  - gradient_accumulation_steps: 4
+  - logging_steps: 5
+  - save_steps: 25
+  - max_steps: 100
+  - use_vllm: true, vllm_mode: colocate
+- **Quantization:** 4-bit via Unsloth
+- **Hardware:** Kaggle T4 x2 (or P100 fallback — note which)
+- **SchemaShift env URL:** [HF Space URL from Phase 10]
+- **Commit hash of repo at training time:**
+
+#### Pre-training check (Cells 1-8 verification before Cell 9)
+
+- [ ] Model loaded without OOM: yes/no, peak GPU memory: X GB
+- [ ] env_client.health() returned True: yes/no
+- [ ] Sample reward_fn call with dummy completion returned float: yes/no, value: X
+- [ ] parse_completion_to_actions test on realistic Qwen output succeeded: yes/no
+
+#### Per-batch reward log (first 20 steps, critical go/no-go)
+
+| Step | Timestamp | Mean reward | Min | Max | Notes |
+|------|-----------|-------------|-----|-----|-------|
+| 1    |           |             |     |     |       |
+| 2    |           |             |     |     |       |
+| 3    |           |             |     |     |       |
+| 4    |           |             |     |     |       |
+| 5    |           |             |     |     |       |
+| 10   |           |             |     |     |       |
+| 15   |           |             |     |     |       |
+| 20   |           |             |     |     |       |
+
+**STAGE 1 GO/NO-GO DECISION at step 20:**
+- Reward curve trending up: yes/no
+- At least one reward > 0: yes/no
+- Not stuck at single value: yes/no
+- DECISION: PROCEED / STOP AND DEBUG
+- Rationale:
+
+#### Per-checkpoint evaluation (every 25 steps — MANDATORY)
+
+For each checkpoint, run eval against E1/E2/E3 with 5 seeds, compare to untrained baseline.
+
+##### Checkpoint @ step 25
+- HF Hub ID:
+- Eval timestamp:
+- E1 mean shaped: X (vs untrained: Y, delta: +Z)
+- E2 mean shaped:
+- E3 mean shaped:
+- Overall mean:
+- Binary rate: X%
+- **Key behavioral change observed vs untrained:**
+- Screenshot: `training_logs/run_1_checkpoint_25_rewardcurve.png`
+
+##### Checkpoint @ step 50
+(Same structure)
+
+##### Checkpoint @ step 75
+(Same structure)
+
+##### Checkpoint @ step 100 (FINAL)
+- HF Hub ID:
+- Eval timestamp:
+- E1 mean shaped: X (vs untrained baseline: Y, delta: +Z, % improvement: W%)
+- E2 mean shaped:
+- E3 mean shaped:
+- Overall mean:
+- Binary rate: X%
+- Compared to GPT-4o-mini: BEATS / TIES / LOSES
+- Screenshot: `training_logs/run_1_final_rewardcurve.png`
+
+#### Training run metadata
+
+- **Start time:**
+- **End time:**
+- **Wall-clock duration:**
+- **Total steps completed:** (if less than 100, note why)
+- **First batch mean reward:**
+- **Final batch mean reward:**
+- **Delta first→final:**
+- **% improvement first→final:**
+- **Peak GPU memory during training:**
+- **vLLM collocate stable throughout: yes/no**
+- **HF checkpoint push count:** (should be 4: at 25, 50, 75, 100)
+
+#### Errors encountered (full traceback for each)
+
+If any:
+1. Step X — [error type] — paste full traceback
+   - Resolution: (what you did)
+   - Re-ran from: (checkpoint or from scratch)
+
+If none: "No errors."
+
+#### Iteration note
+
+Is this Run 1 for this config, or did you re-run after fixing something?
+- If re-run: what config change was made vs prior attempt? What did the prior attempt fail on?
+
+#### Judgment calls made during this run
+
+(E.g., "reduced max_completion_length from 1536 to 1024 because of OOM at step 40" — document any deviations from the planned config)
+
+#### Verdict
+
+- [ ] Stage 1 passes — pipeline validated, proceed to Stage 2 with Accounts 1 and 3
+- [ ] Stage 1 fails — specific failure mode:
+- Next action:
+
+---
+
+### Run 2 — Account 1 (Yashash) — STAGE 2 BINARY CONTROL — [Date Time start]
+
+(Only start after Run 1 Stage 1 passes. Same structure as Run 1, but with:)
+
+- **Reward variant:** `return reward.binary` (sparse 0/1 signal only)
+- **Model:** Qwen 2.5 1.5B Instruct (same as Run 1)
+- **Hypothesis:** Will sparse binary reward converge without dense shaping?
+
+(Fill same template: config, pre-training check, per-batch log, per-checkpoint eval, metadata, errors, verdict.)
+
+---
+
+### Run 3 — Account 3 (Likith) — STAGE 2 CODER ABLATION — [Date Time start]
+
+(Same structure, but:)
+
+- **Reward variant:** `return reward.shaped_total` (same as Run 1)
+- **Model:** Qwen 2.5 **Coder** 1.5B Instruct (`Qwen/Qwen2.5-Coder-1.5B-Instruct`)
+- **Scenario set:** procedural (build_procedural_scenarios enabled)
+- **Hypothesis:** Does code-pretraining help schema-adaptation learning?
+
+(Fill same template.)
+
+---
+
+### Run N — [placeholder for additional iterations]
+
+When you re-run any config with changes, create a new Run entry. Never edit Run 1-3. New runs get new numbers. This preserves the experimental history.
+
+---
+
+## 3. CHECKPOINT EVALS (all eval runs, any model)
+
+Every eval.py invocation gets one entry here, even if it was just to sanity-check something.
+
+### Eval [N] — [model/checkpoint name] — [Date Time]
+
+- **Target:** untrained base / checkpoint HF ID / baseline LLM name
+- **Seeds:** 0, 1, 2, 3, 4
+- **Scenarios:** E1, E2, E3
+- **Full raw table:** (same format as baseline template above)
+- **Aggregates:**
+  - E1 mean shaped:
+  - E2 mean shaped:
+  - E3 mean shaped:
+  - Overall:
+  - Binary rate:
+- **3 failure examples (Action+Response JSON):**
+- **Cost (if API-based):** $X.XX
+- **Commentary:**
+
+---
+
+## 4. HEAD-TO-HEAD COMPARISONS (for blog, pitch, judges)
+
+These tables get built progressively as runs complete. They ARE your pitch data.
+
+### Table A: Qwen 2.5 1.5B — untrained vs SchemaShift-trained (primary pitch number)
+
+| Model state | E1 shaped | E2 shaped | E3 shaped | Overall | Binary rate |
+|-------------|-----------|-----------|-----------|---------|-------------|
+| Untrained base |       |           |           |         |             |
+| SchemaShift-trained |  |           |           |         |             |
+| **Improvement %** |    |           |           |         |             |
+
+### Table B: Frontier models fail, trained small model succeeds (the headline claim)
+
+| Model | Size | E1 | E2 | E3 | Avg |
+|-------|------|-----|-----|-----|-----|
+| naive_heuristic | — |  |  |  |  |
+| policy_aware_heuristic | — |  |  |  |  |
+| Qwen 2.5 7B Instruct | 7B |  |  |  |  |
+| Llama 3.1 8B Instruct | 8B |  |  |  |  |
+| GPT-4o-mini | ~8B* |  |  |  |  |
+| **Qwen 2.5 1.5B + SchemaShift (ours)** | **1.5B** |  |  |  |  |
+
+*GPT-4o-mini parameter count estimated — OpenAI has not disclosed publicly.
+
+### Table C: Reward function ablation (binary vs shaped)
+
+| Run | Reward | Model | Final E1 | Convergence step | Verdict |
+|-----|--------|-------|----------|------------------|---------|
+| Run 1 (Account 2) | shaped_total | Qwen 1.5B |  |  |  |
+| Run 2 (Account 1) | binary only | Qwen 1.5B |  |  |  |
+
+Hypothesis being tested: dense shaping is necessary for GRPO convergence in sparse-reward tool-use domains.
+
+### Table D: Model ablation (Instruct vs Coder pretraining)
+
+| Run | Model | Final E1 | Final E2 | Final E3 | Convergence step |
+|-----|-------|----------|----------|----------|------------------|
+| Run 1 (Account 2) | Qwen 2.5 1.5B Instruct |  |  |  |  |
+| Run 3 (Account 3) | Qwen 2.5 Coder 1.5B Instruct |  |  |  |  |
+
+Hypothesis: code-pretraining transfers to schema-adaptation.
+
+### Table E: Training efficiency
+
+| Run | Steps | Wall-clock | GPU-hours | Cost (USD) | Peak reward |
+|-----|-------|------------|-----------|------------|-------------|
+| Run 1 |  |  |  | (Kaggle free) |  |
+| Run 2 |  |  |  | (Kaggle free) |  |
+| Run 3 |  |  |  | (Kaggle free) |  |
+
+If Saturday onsite uses H100 credits, add row for that run and log cost.
+
+---
+
+## 5. ITERATION TIMELINE (what we tried, in order)
+
+A chronological summary of every experiment. One line per iteration. This is the "we iterated" proof.
+
+| # | Date/time | Run | Config change from previous | Outcome |
+|---|-----------|-----|------------------------------|---------|
+| 1 |  | Run 1 (Stage 1) | First attempt — Qwen 1.5B + shaped | |
+| 2 |  |  |  |  |
+| 3 |  |  |  |  |
+
+(Add rows as you go. Include failed runs — failures are data too.)
+
+---
+
+## 6. LESSONS LEARNED (captured while fresh)
+
+Every time training surprises you — good or bad — note it here immediately. Do not defer. You will forget.
+
+### Lesson [N] — [short title] — [Date]
+
+- Context:
+- What happened:
+- Why it happened:
+- How to avoid / exploit it next time:
+
+---
+
+## 7. OPEN QUESTIONS FOR ONSITE
+
+Things you couldn't resolve during dev that the onsite H100 run should answer.
+
+1.
+2.
+3.
+
+---
+
+**This log is part of your submission evidence.** Judges may ask: "How many configurations did you try?" "What was your eval methodology?" "Show the reward improvement." Every good answer points back to a filled-in entry in this file.