jasonlingg/qwen-envoy-qwen3-8b-qasper-targeted-sft-v1

Qwen Envoy Qwen3-8B — targeted QASPER SFT v1

Experimental QLoRA adapter for the Envoy code-execution research agent. The model predicts the next Python search/read action or a final SUBMIT: response while investigating scientific papers.

This continuation produced a small development-set improvement but failed its preregistered promotion gate. It is published as a reproducible experiment and checkpoint archive, not as a production-ready or state-of-the-art model.

Recommended checkpoint

The repository root contains the epoch-1 checkpoint-150 adapter. It was selected over epoch 2 by behavioral evaluation: both produced 11/25 supported answers, while epoch 1 had five partial answers instead of four. The epoch-2 adapter is preserved under candidates/final because it had the lower next-action validation loss.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-8B"
adapter_id = "jasonlingg/qwen-envoy-qwen3-8b-qasper-targeted-sft-v1"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision="b968826d9c46dd6066d109eabc6255188de91218",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

The adapter expects Envoy's multi-turn code-execution system prompt and its persistent Python environment. It is not intended as a standalone chat model.

Training

• Starting point: the earlier QASPER v5 checkpoint-50 adapter
• Data: 153 conversations / 611 next-action examples
• Validation: 30 paper-disjoint conversations / 125 actions
• Supervised target tokens: 45,903 train / 9,480 validation
• Longest sequence: 4,655 tokens; configured limit: 8,192
• Method: 4-bit NF4 QLoRA, rank 4, alpha 8, dropout 0.05
• Targets: attention (q, k, v, o) and MLP (gate, up, down) projections
• Optimizer: fused AdamW, peak learning rate 5e-5, 5% warmup, cosine decay, no weight decay
• Effective batch size: 4; two epochs; seed 42
• Hardware: one NVIDIA A40; training runtime 2,459 seconds

Epoch	Checkpoint	Validation loss	Token accuracy
1	checkpoint-150	0.1356	0.9664
2	candidates/final	0.1272	0.9661

Token accuracy measures imitation of the next demonstrated action. It is not task accuracy.

Behavioral evaluation

The development diagnostic contains 15 answerable failures selected from the previous adapter and 10 controls it already handled correctly. Runs used deterministic decoding, the same system prompt, raw top-three search_within, and seed 42. Semantic review was performed by the project assistant blind to candidate identity; it was not independent human review.

System	Pass	Partial	Fail	Target rescues	Controls retained	Execution errors
Previous checkpoint-50	10	4	11	0/15	10/10	0
Epoch 1 (checkpoint-150)	11	5	9	1/15	10/10	0
Epoch 2 (final)	11	4	10	1/15	10/10	0

The gate required at least 6/15 targeted rescues, 9/10 retained controls, and at most one execution-error episode. Only the latter two conditions passed. The untouched 40-question confirmation set was therefore not consumed.

Files

• Root adapter: selected epoch-1 candidate
• candidates/final: epoch-2 adapter
• results/summary.json: complete training and evaluation summary
• results/run-manifest.json: exact training configuration and data hashes
• results/score.json: blinded semantic-review aggregate
• results/automatic.json: execution and provenance diagnostics

Limitations

The development questions were already used for failure analysis, so these numbers are diagnostic rather than held-out performance. The synthetic annotation-grounded trajectories preserved tool reliability and abstention controls but rarely taught the model to recover the intended missing answer. Exact citations and lexical overlap do not by themselves prove semantic support. Use the earlier v5 checkpoint as the currently supported project model until a future experiment passes the fresh confirmation gate.