# SFT Remediation Plan

Generated: 2026-06-27

The production LoRA run was stopped at user request. The latest durable
checkpoint, `checkpoint-8000`, loaded correctly but failed the behavior gate.
This plan documents the CPU-side fix path before any future GPU training is
started.

## Evidence

- Gate failures from `runs/eval-lora-behavior-checkpoint-8000-fixed.gate.log`:
  - `short_python_add`: finished by length instead of stop.
  - `json_instruction_following`: did not parse as JSON.
  - `json_instruction_following`: emitted a markdown fence.
  - `template_marker_leak`: exact one-word prompt produced explanatory text
    beginning with `Here's a thinking process:`.
- Production data validation passed structurally, with 44,000 train rows and
  2,000 eval rows.
- Format audit shows the production message data is strongly biased toward
  explanatory fenced answers:
  - 100% of assistant answers contain a code fence.
  - About 56% contain `Here's`.
  - About 28% of train answers start with `Here's`.
  - 0% are short exact answers under 80 characters.
  - Rendered rows contain empty `<think></think>` blocks on every assistant
    answer.

## Likely Cause

The checkpoint learned coding-task format, but not strict-output compliance.
The data is useful for code synthesis, yet it heavily rewards explanatory
responses, markdown fences, and non-short answers. That conflicts with serving
requirements where prompts ask for exact code only, compact JSON, or a single
word.

## Next Training Attempt

Do not resume the stopped run blindly. If training is resumed from
`checkpoint-8000`, first add a repair stage or rebuild the blend.

Recommended blend:

- 80-90% high-quality coding instruction data.
- 5-10% strict final-answer/code-only examples without markdown fences.
- 3-5% JSON-only and schema-following examples.
- 2-5% exact short-answer examples.
- Optional small negative/contrastive set for no markdown fences, no reasoning
  preambles, and no template markers.

CPU-side repair artifacts are now prepared for this path:

- `scripts/build_sft_repair_set.py`
- `scripts/build_sft_code_only_from_fences.py`
- `scripts/build_sft_blend.py`
- `scripts/preflight_sft_resume.py`
- `configs/sft_repair_blend.yaml`
- `data/train/coding_sft_repair_messages.jsonl`
- `data/eval/coding_sft_repair_eval_messages.jsonl`
- `data/train/coding_sft_code_only_messages.jsonl`
- `data/eval/coding_sft_code_only_eval_messages.jsonl`
- `data/train/coding_sft_prod_repair_messages.jsonl`
- `data/eval/coding_sft_prod_repair_eval_messages.jsonl`
- `data/train/coding_sft_prod_repair.jsonl`
- `data/eval/coding_sft_prod_repair_eval.jsonl`
- `data/train/coding_sft_prod_repair_strong_messages.jsonl`
- `data/eval/coding_sft_prod_repair_strong_eval_messages.jsonl`
- `data/train/coding_sft_prod_repair_strong.jsonl`
- `data/eval/coding_sft_prod_repair_strong_eval.jsonl`
- `runs/sft-repair-set.json`
- `runs/sft-code-only-extract.json`
- `runs/sft-repair-blend-train.json`
- `runs/sft-repair-blend-eval.json`
- `runs/sft-repair-strong-blend-train.json`
- `runs/sft-repair-strong-blend-eval.json`
- `runs/sft-data-validation-prod-repair.json`
- `runs/sft-format-audit-prod-repair.json`
- `runs/sft-format-gate-strict-repair.json`
- `runs/sft-format-gate-prod-repair.json`
- `runs/sft-data-validation-prod-repair-strong.json`
- `runs/sft-format-audit-prod-repair-strong.json`
- `runs/sft-format-gate-code-only-extract.json`
- `runs/sft-format-gate-prod-repair-strong.json`
- `runs/sft-resume-preflight-strong.json`

The generated repair component has 4,096 train rows and 512 eval rows. It has
0 code fences, 0 `Here's`/`Here is` preambles, 0 reasoning tags, and about 81%
short assistant answers under 80 characters. The blended train split has 44,096
rows and validates with no blocking issues; 107 train rows and 7 eval rows are
longer than 1024 tokens, so they are warnings only. The blended train format
signals improve from the stopped production mix:

- code fences: 100% -> 90.71%
- `Here's`/`Here is`: about 56% -> 50.93%
- short exact answers under 80 chars: 0% -> 7.55%

The reusable format gates pass:

- strict repair messages: 8/8 checks, 0 issues.
- blended production-plus-repair messages: 10/10 checks, 0 issues.
- extracted code-only messages: 8/8 checks, 0 issues.
- stronger production-plus-code-only-plus-strict-repair messages: 10/10 checks,
  0 issues.

The stronger repair blend is the recommended future SFT input. It keeps the same
44,096 train rows and 2,312 eval rows, but replaces part of the explanatory
fenced base data with extracted compilable Python code blocks. Compared with
the original production mix, train message format signals improve to:

- code fences: 100% -> 54.43%
- `Here's`/`Here is`: about 56% -> 30.49%
- starts with `Here's`: about 28% -> 15.07%
- short exact answers under 80 chars: 0% -> 8.89%

Recommended gates before using GPU:

```bash
.venv-lab/bin/python scripts/build_sft_repair_set.py \
  --train-size 4096 \
  --eval-size 512 \
  --seed 29 \
  --train-output data/train/coding_sft_repair_messages.jsonl \
  --eval-output data/eval/coding_sft_repair_eval_messages.jsonl \
  --report-output runs/sft-repair-set.json

.venv-lab/bin/python scripts/build_sft_code_only_from_fences.py \
  --train-size 20000 \
  --eval-size 1000 \
  --seed 41 \
  --train-output data/train/coding_sft_code_only_messages.jsonl \
  --eval-output data/eval/coding_sft_code_only_eval_messages.jsonl \
  --report-output runs/sft-code-only-extract.json

.venv-lab/bin/python scripts/build_sft_blend.py \
  --source data/train/coding_sft_prod_messages.jsonl=40000:base_coding \
  --source data/train/coding_sft_repair_messages.jsonl=4096:strict_repair \
  --output data/train/coding_sft_prod_repair_messages.jsonl \
  --report-output runs/sft-repair-blend-train.json \
  --seed 31

.venv-lab/bin/python scripts/build_sft_blend.py \
  --source data/eval/coding_sft_prod_eval_messages.jsonl=1800:base_coding \
  --source data/eval/coding_sft_repair_eval_messages.jsonl=512:strict_repair \
  --output data/eval/coding_sft_prod_repair_eval_messages.jsonl \
  --report-output runs/sft-repair-blend-eval.json \
  --seed 37

.venv-lab/bin/python scripts/build_sft_blend.py \
  --source data/train/coding_sft_prod_messages.jsonl=24000:base_coding \
  --source data/train/coding_sft_code_only_messages.jsonl=16000:code_only_extract \
  --source data/train/coding_sft_repair_messages.jsonl=4096:strict_repair \
  --output data/train/coding_sft_prod_repair_strong_messages.jsonl \
  --report-output runs/sft-repair-strong-blend-train.json \
  --seed 43

.venv-lab/bin/python scripts/build_sft_blend.py \
  --source data/eval/coding_sft_prod_eval_messages.jsonl=1000:base_coding \
  --source data/eval/coding_sft_code_only_eval_messages.jsonl=800:code_only_extract \
  --source data/eval/coding_sft_repair_eval_messages.jsonl=512:strict_repair \
  --output data/eval/coding_sft_prod_repair_strong_eval_messages.jsonl \
  --report-output runs/sft-repair-strong-blend-eval.json \
  --seed 47

.venv-lab/bin/python scripts/render_sft_jsonl.py \
  --input data/train/coding_sft_prod_repair_messages.jsonl \
  --output data/train/coding_sft_prod_repair.jsonl \
  --force

.venv-lab/bin/python scripts/render_sft_jsonl.py \
  --input data/eval/coding_sft_prod_repair_eval_messages.jsonl \
  --output data/eval/coding_sft_prod_repair_eval.jsonl \
  --force

.venv-lab/bin/python scripts/render_sft_jsonl.py \
  --input data/train/coding_sft_prod_repair_strong_messages.jsonl \
  --output data/train/coding_sft_prod_repair_strong.jsonl \
  --force

.venv-lab/bin/python scripts/render_sft_jsonl.py \
  --input data/eval/coding_sft_prod_repair_strong_eval_messages.jsonl \
  --output data/eval/coding_sft_prod_repair_strong_eval.jsonl \
  --force

.venv-lab/bin/python scripts/validate_sft_jsonl.py \
  --train-jsonl data/train/coding_sft_prod_repair.jsonl \
  --eval-jsonl data/eval/coding_sft_prod_repair_eval.jsonl \
  --max-length 1024 \
  --min-train-rows 44096 \
  --require-license \
  --output runs/sft-data-validation-prod-repair.json

.venv-lab/bin/python scripts/validate_sft_jsonl.py \
  --train-jsonl data/train/coding_sft_prod_repair_strong.jsonl \
  --eval-jsonl data/eval/coding_sft_prod_repair_strong_eval.jsonl \
  --max-length 1024 \
  --min-train-rows 44096 \
  --require-license \
  --output runs/sft-data-validation-prod-repair-strong.json

.venv-lab/bin/python scripts/audit_sft_format.py \
  --input data/train/coding_sft_repair_messages.jsonl \
  --input data/eval/coding_sft_repair_eval_messages.jsonl \
  --input data/train/coding_sft_prod_repair_messages.jsonl \
  --input data/eval/coding_sft_prod_repair_eval_messages.jsonl \
  --input data/train/coding_sft_prod_repair.jsonl \
  --input data/eval/coding_sft_prod_repair_eval.jsonl \
  --output runs/sft-format-audit-prod-repair.json

.venv-lab/bin/python scripts/audit_sft_format.py \
  --input data/train/coding_sft_code_only_messages.jsonl \
  --input data/eval/coding_sft_code_only_eval_messages.jsonl \
  --input data/train/coding_sft_prod_repair_strong_messages.jsonl \
  --input data/eval/coding_sft_prod_repair_strong_eval_messages.jsonl \
  --input data/train/coding_sft_prod_repair_strong.jsonl \
  --input data/eval/coding_sft_prod_repair_strong_eval.jsonl \
  --output runs/sft-format-audit-prod-repair-strong.json

.venv-lab/bin/python scripts/check_sft_format_gates.py \
  --audit-report runs/sft-format-audit-prod-repair.json \
  --config configs/sft_format_gates.yaml \
  --profile strict_repair_messages \
  --output runs/sft-format-gate-strict-repair.json

.venv-lab/bin/python scripts/check_sft_format_gates.py \
  --audit-report runs/sft-format-audit-prod-repair.json \
  --config configs/sft_format_gates.yaml \
  --profile prod_repair_messages \
  --output runs/sft-format-gate-prod-repair.json

.venv-lab/bin/python scripts/check_sft_format_gates.py \
  --audit-report runs/sft-format-audit-prod-repair-strong.json \
  --config configs/sft_format_gates.yaml \
  --profile code_only_extract_messages \
  --output runs/sft-format-gate-code-only-extract.json

.venv-lab/bin/python scripts/check_sft_format_gates.py \
  --audit-report runs/sft-format-audit-prod-repair-strong.json \
  --config configs/sft_format_gates.yaml \
  --profile prod_repair_strong_messages \
  --output runs/sft-format-gate-prod-repair-strong.json

.venv-lab/bin/python scripts/preflight_sft_resume.py \
  --output runs/sft-resume-preflight-strong.json
```

The preflight guard is report-only and does not launch training. Current result:
16/16 checks pass, 0 warnings, 0 failures. It verifies the stop record,
checkpoint-8000 files, strong repair row counts, validation report, format gates,
no matching training/serving/eval/quantization processes, and idle GPUs.

Required release gate for every future checkpoint:

```bash
CUDA_VISIBLE_DEVICES=0 \
.venv-train/bin/python scripts/eval_lora_behavior.py \
  --base-model deepreinforce-ai/Ornith-1.0-35B \
  --adapter artifacts/train/ornith-35b-coding-lora-prod-20k/checkpoint-<step> \
  --suite configs/model_behavior_suite.yaml \
  --output runs/eval-lora-behavior-checkpoint-<step>.json
```

`scripts/eval_lora_behavior.py` now defaults to final-answer mode by passing
`enable_thinking=False` into the chat template. Use `--enable-thinking` only for
explicit reasoning-mode experiments. This matches the published llama.cpp
`REASONING=off` serving profile.

## Release Rule

No LoRA adapter should be uploaded unless high/critical behavior checks pass.
For this project, that means at minimum:

- simple code-only prompt finishes by `stop`;
- compact JSON prompt parses as JSON and has no markdown fence;
- exact one-word prompt returns exactly the requested word;
- no Qwen markers or `<think>` tags leak into final content.

## Current Status

No fine-tuned adapter is released. The only durable SFT checkpoint is
`checkpoint-8000`, and it is not a release candidate.

SFT remains stopped until explicitly resumed. The prepared repair blend is CPU
work only and does not consume GPU memory.