jasonlingg/qwen-envoy-qwen3-8b-qasper-sft-v5

Qwen Envoy Qwen3-8B QASPER SFT v5

Experimental QLoRA adapters for the Envoy code-execution research agent. The model predicts the next Python tool action or SUBMIT: response from a multi-turn document investigation.

This repository is a checkpoint archive, not evidence that the trained model beats base Qwen3-8B. The declared 40-question paired development comparison and semantic review have not been run for these adapters.

Recommended candidate

Start with artifacts/full/checkpoint-50. Validation loss was lowest at epoch 2, step 54, and checkpoint 50 is the closest saved adapter before that minimum. It also outscored final on the fixed five-question plumbing smoke. That smoke is too small for model selection or an improvement claim.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-8B"
base_revision = "b968826d9c46dd6066d109eabc6255188de91218"
adapter_id = "jasonlingg/qwen-envoy-qwen3-8b-qasper-sft-v5"

tokenizer = AutoTokenizer.from_pretrained(
    adapter_id, subfolder="artifacts/full/checkpoint-50"
)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    base, adapter_id, subfolder="artifacts/full/checkpoint-50"
)

Training

• Base: Qwen3-8B at revision b968826d9c46dd6066d109eabc6255188de91218
• Data: 35 reviewed QASPER-training conversations on 35 distinct papers
• Targets: 106 prefix-aligned next-action examples; 4,812 supervised tokens
• Validation: 9 paper-disjoint conversations; 26 actions; 1,162 supervised tokens
• Method: 4-bit NF4 QLoRA, rank 4, alpha 8, dropout 0.05
• Optimizer schedule: four epochs, 108 updates, learning rate 2e-4, cosine decay
• Effective batch: 4 through gradient accumulation; seed 42
• Hardware: one NVIDIA A40; training runtime 579 seconds

Epoch	Step	Validation loss	Validation token accuracy
1	27	0.5185	0.8685
2	54	0.4512	0.8750
3	81	0.4624	0.8789
4	108	0.4678	0.8752

The rise after epoch 2 is an early overfitting signal. Every ten-step checkpoint was retained so behavior can be selected with held-out trajectories rather than training loss.

Five-question plumbing smoke

Both checkpoint 50 and final executed a research tool first on 5/5 frozen development questions, submitted 5/5 nonempty answers, produced zero execution-error episodes, and finished in at most three steps. Checkpoint 50's mean outcome reward was 0.279; final's was 0.188. This establishes that the trained policies can run the intended protocol. Five questions do not establish answer quality or generalization.

Files

• artifacts/full/checkpoint-*: adapters every ten optimizer steps plus step 108
• artifacts/full/final: final adapter
• artifacts/full/run-manifest.json: exact data hashes and training configuration
• artifacts/eval: complete five-question transcripts and manifests
• artifacts/train.log: training and validation output
• artifacts/SHA256SUMS: checksums for every artifact copied from RunPod
• experiment-plan.json and summary.json: preregistered decision rule and run summary

The checkpoint directories include optimizer and scheduler state so training can be resumed.

Limitations

The training set is small and assistant-reviewed rather than independently reviewed. It teaches known-paper investigation, not open-ended paper discovery or cross-paper synthesis. Validation contains only nine conversations. Lexical answer overlap and valid citation IDs do not prove that a passage semantically supports an answer. Run the frozen paired development evaluation before presenting this adapter as an improvement over the base or earlier SFT checkpoint.