Supervised Fine-Tuning
Chat & Alignment
helpfulness · safety
Instruction
Following
constraints · formats
Code
generation · execution
Agentic Tasks
tools · environments
Supervised Fine-Tuning
Initial SFT Policy
Domain Expert Training
Exploration & Environment Interaction
Domain-Specialized Reinforcement Learning
DOMAIN EXPERTS
General Expert
Reasoning Expert
Agent Expert
Code Expert
Multi-Teacher On-Policy Distillation
Prompt Batch
On-Policy Trajectories
Student-generated tokens y₁ … yₜ
ROUTED EXPERT TEACHERS
General
Reasoning
Agent
Code
guidance
student trajectories
M-OPD
Unified Model