# Qwen3.5 Text-Only Pruning and Vulkan Distillation Design

## Goal

Produce a verifiable text-only Qwen/Qwen3.5-0.8B-Base checkpoint by removing the
vision tower and MTP head, keeping the first complete hybrid block prefix, and
then run teacher/student inference and distillation through the RX 460 Vulkan
backend. The final text checkpoint must measure between 330,000,000 and
350,000,000 parameters from its emitted tensors.

## Constraints

- Model forward, inference, and training are GPU-only through Vulkan0.
- CPU is allowed for configuration, safetensors metadata, tensor copying, and
  static validation; it is not allowed for a model forward or training step.
- The live config is authoritative. In particular,
  linear_num_value_heads=16 is not replaced by a library default.
- The student keeps the first complete four-layer hybrid prefix. No activation
  ranking is used for this depth-only target because N=4 is inside the measured
  parameter interval.
- Teacher weights are frozen. The training objective is
  CE(student logits) + lambda * MSE(student final hidden, teacher final hidden).
- Optimizer is AdamW and the learning-rate schedule is cosine with warmup.
- Every phase has explicit-path staging and an atomic commit.
- A claim of GPU execution requires a saved log containing the Vulkan device.

## Architecture

MVP/qwen35_prune.py reads the live JSON config and safetensors metadata,
selects the largest complete hybrid prefix that fits the required interval,
renames only the text-backbone prefixes, drops model.visual.* and mtp.*,
and emits a text-only safetensors checkpoint plus a qwen3_5_text config. It
never constructs or forwards the model.

MVP/validate_checkpoint.py re-counts emitted tensors, checks that vision/MTP
tensors are absent, checks layer-prefix continuity and the exact text config,
and verifies tied embeddings without duplicating lm_head.weight in the saved
state dict.

The runtime phase uses a pinned llama.cpp source checkout with Vulkan enabled.
The required patch adds a frozen teacher hidden-state input and a second loss
term to the optimizer graph, while retaining the existing AdamW update path and
adding cosine warmup scheduling. The patch is kept under MVP/patches/ so the
runtime source and build can be reproduced.

## Data flow

live HF config + safetensors
        |
        v
metadata measurement -> complete-prefix selection (N=4)
        |
        v
text-only safetensors + qwen3_5_text config
        |
        v
HF -> GGUF conversion -> llama-cli Vulkan0 teacher/student inference
        |
        v
teacher hidden target (GPU) + student logits/hidden (GPU)
        |
        v
CE + lambda*MSE -> AdamW + cosine warmup -> student checkpoint

## Failure handling

- Missing or inconsistent metadata is a hard error.
- A layer count that is not a complete hybrid prefix is rejected.
- Any retained visual or mtp tensor is a hard validation failure.
- A non-Vulkan runtime log is not accepted as model execution evidence.
- If the installed llama.cpp runtime cannot support the combined loss or the
  RX 460 cannot hold the requested graph, the progress file records the exact
  failing command and output; CE-only output is never reported as distillation.

## Testing

Pure Python tests cover metadata-derived counts, N=4 selection, prefix
translation, tensor exclusion, tied embeddings, config fields, and final
interval validation. A runtime test uses a synthetic tiny graph to verify the
combined loss numerically before a real model run. Real model checks re-count
the produced checkpoint and require Vulkan logs.