ATLAS Post-Cutoff Specialist (9B)
LoRA adapter (rank 64) for Qwen3.5-9B, gentle-trained on 103 QA pairs about Feb-Apr 2026 AI/ML research papers. Part of the ATLAS research architecture (bridge + specialist + text-level assembly for regulated domains).
Training
- Base: Qwen/Qwen3.5-9B
- Data: 103 QA pairs (see post-cutoff-knowledge-benchmark dataset on this org)
- Recipe: LoRA rank 64, lr 3e-5, 3 epochs (gentle)
- Training time: ~10 min on single GPU
Evaluation (held-out test, 41 questions)
| System | Gold-key | Grade |
|---|---|---|
| ATLAS (this specialist + bridge + Claude composer) | 29.7% | 11.12 / 25 |
| RAG-to-Opus (baseline) | 70.5% | 12.12 / 25 |
| Opus 4.7 alone (no retrieval) | 23.2% | - |
Blind pairwise judge (Claude Opus 4.7): RAG wins 39, ATLAS wins 1, 1 tie.
Honest positioning
This adapter plus the ATLAS architecture does NOT beat RAG-to-Opus for knowledge injection on our benchmark. It does beat Opus-without-retrieval by ~6 gold-key points. Consistent with Ovadia et al. (EMNLP 2024): RAG dominates fine-tuning for knowledge injection.
The specialist has narrow value for deployments where RAG is not viable (HIPAA / BAA scope, air-gapped, extreme query volume). It is not a replacement for RAG in general.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(base, "xpiohealth/atlas-post-cutoff-9b-specialist")
Files
- adapter_config.json, adapter_model.safetensors: standard PEFT LoRA files
- bridge.pt: ATLAS-specific cross-attention bridge weights (only usable with the ATLAS pipeline)
- specialist_meta.json: training metadata
- RUN_INFO.json: run metadata
License
Apache 2.0 (LoRA adapter). Base model license: Qwen3.5-9B Apache 2.0.
- Downloads last month
- 9