fugu-lite / configs /train_sft.yaml
tahsinsoyak's picture
Upload private Fugu-Lite V1 snapshot
88e15cd verified
Raw
History Blame Contribute Delete
328 Bytes
model:
base_model: Qwen/Qwen3-0.6B
max_length: 512
pooling: last_token
dropout: 0.05
freeze_backbone: true
dtype: auto
training:
seed: 42
epochs: 5
batch_size: 8
gradient_accumulation_steps: 1
learning_rate: 0.001
weight_decay: 0.01
max_grad_norm: 1.0
soft_target_temperature: 0.15
log_every: 10