# Qwen3.5 Pruning and Vulkan Distillation Implementation Plan

> For agentic workers: use superpowers:executing-plans or superpowers:subagent-driven-development to execute this plan task by task. Steps use checkbox syntax for tracking.

Goal: Create and verify a 330M-350M text-only Qwen3.5 checkpoint and run GPU-only RX 460 Vulkan teacher/student inference and CE-plus-hidden-MSE distillation.

Architecture: Python utilities perform metadata-only measurement, text-prefix state-dict extraction, and final validation. A pinned llama.cpp Vulkan build performs all model forwards and training; its optimizer graph is patched for frozen teacher hidden targets, combined loss, and cosine warmup.

Tech Stack: Python 3, safetensors, transformers 5.2.0 config schema, pytest, llama.cpp 10068-compatible source, ggml Vulkan backend, AdamW.

## Global Constraints

- Never run a model forward, inference, or training step on CPU.
- Use the live config.json; retain linear_num_value_heads=16.
- Keep the first complete hybrid prefix and reject incomplete prefixes.
- Drop every model.visual.* and mtp.* tensor.
- Preserve tie_word_embeddings=true and do not duplicate the tied LM head tensor.
- Final emitted parameter count must be measured inside [330000000, 350000000].
- Teacher is frozen; loss is CE plus hidden-state MSE.
- Optimizer is AdamW with cosine warmup.
- Use explicit-path git staging; never use git add ., git add -A, or git commit -a.

---

### Task 1: Add metadata and checkpoint contract tests

Files:
- Create: MVP/tests/test_pruning_contract.py
- Create: MVP/tests/fixtures/qwen35_metadata.json

Interfaces:
- Tests import MVP.qwen35_prune and MVP.validate_checkpoint.
- The fixture records live config values and measured tensor groups.

- [ ] Step 1: Write failing tests for choose_prefix(337299424) returning 4,
  count_parameter_groups returning the measured groups, prefix translation of
  model.language_model.layers.3.* to model.layers.3.*, vision/MTP exclusion,
  tied embedding counting, and N=4 config construction.
- [ ] Step 2: Run python -m pytest MVP/tests/test_pruning_contract.py -q.
  Expected before implementation: import errors for the missing modules.
- [ ] Step 3: Stage only the two test paths and commit
  test: define qwen35 pruning contract.

### Task 2: Implement metadata selection and text-only checkpoint extraction

Files:
- Create: MVP/qwen35_prune.py
- Create: MVP/validate_checkpoint.py

Interfaces:
- load_live_config(path) -> dict
- count_parameter_groups(safetensors_path, config) -> ParameterReport
- choose_prefix(report, minimum, maximum) -> PrefixChoice
- translate_text_key(key) -> str or None
- build_text_config(full_config, layer_count) -> dict
- prune_checkpoint(input_weights, input_config, output_dir, layer_count) -> ValidationReport
- validate_checkpoint(config_path, weights_path, minimum, maximum) -> ValidationReport

- [ ] Step 1: Implement only the functions needed by the red tests.
- [ ] Step 2: Run the focused tests and keep the output green.
- [ ] Step 3: Add CLI commands measure, prune, and validate; each command
  fails closed on missing or inconsistent input.
- [ ] Step 4: Commit explicit paths as feat: add qwen35 text checkpoint pruner.

### Task 3: Produce and re-measure the real N=4 checkpoint

Files:
- Create: MVP/artifacts/ outputs from the live snapshot.
- Create or modify: MVP/PROGRESS.md.

- [ ] Step 1: Run metadata-only measurement against the live snapshot.
- [ ] Step 2: Run the pruner with --layers 4 and the live safetensors file.
- [ ] Step 3: Run the validator against the emitted files and save raw output.
- [ ] Step 4: Confirm output config has model_type=qwen3_5_text, four layer
  types, and no vision/MTP keys.
- [ ] Step 5: Commit tools and evidence documentation with explicit paths.

### Task 4: Build a reproducible Vulkan runtime

Files:
- Create: MVP/runtime/README.md
- Create: MVP/runtime/build-vulkan.sh
- Create: MVP/patches/ for the source patch.

- [ ] Step 1: Clone or unpack a pinned llama.cpp source commit and record it.
- [ ] Step 2: Build with Vulkan enabled and verify llama-cli --list-devices
  contains RX 460 as Vulkan0.
- [ ] Step 3: Convert the text-only checkpoint to GGUF using a verified tokenizer.
- [ ] Step 4: Run student and authoritative teacher inference with
  --device Vulkan0; save complete logs including device initialization.
- [ ] Step 5: Commit runtime scripts and evidence paths.

### Task 5: Add and test combined distillation loss

Files:
- Modify: pinned llama.cpp optimizer/runtime sources in the reproducible runtime checkout.
- Create: MVP/patches/qwen35-hidden-distillation.patch
- Create: MVP/tests/test_distillation_loss.py

- [ ] Step 1: Write a synthetic test for CE + lambda*MSE and the cosine-warmup
  schedule; verify frozen teacher parameters are excluded from updates.
- [ ] Step 2: Run it red before changing runtime code.
- [ ] Step 3: Patch the optimizer graph to accept teacher hidden targets and
  expose CE, hidden MSE, total loss, and learning rate in logs.
- [ ] Step 4: Build the patched Vulkan runtime and run the synthetic test green.
- [ ] Step 5: Run a bounded real distillation job with teacher and student on
  Vulkan0; preserve raw logs.
- [ ] Step 6: Commit patch and test as
  feat: add qwen35 hidden-state distillation loss.

### Task 6: Final verification and atomic handoff

Files:
- Modify: MVP/PROGRESS.md
- Create: MVP/evidence/ logs and validator output.

- [ ] Step 1: Re-run the final validator on the post-distillation checkpoint.
- [ ] Step 2: Verify measured count is inside [330000000,350000000] and
  visual/MTP tensor count is zero.
- [ ] Step 3: Verify every model execution log names
  Vulkan0: AMD Radeon RX 460 Graphics.
- [ ] Step 4: Record any missing proof as kanıt yok; do not infer success from
  file presence or command exit status alone.
- [ ] Step 5: Run final fresh verification, inspect git diff --cached --name-only,
  and commit only explicit paths.