2025-12-11 11:12:17,843 __main__ <module> [INFO] OUT_DIR set to outputs/2025-12-11/11-12-17
2025-12-11 11:12:17,844 __main__ <module> [INFO] Args: app/src/S3_8_dpo_human.py --dpo.per_device_train_batch_size 32 --dpo.per_device_eval_batch_size 32 --dpo.gradient_accumulation_steps 32 --dpo.push_to_hub --model.model_name_or_path tokyotech-llm/Llama-3.1-Swallow-8B-v0.5
2025-12-11 11:12:18,587 accelerate.utils.modeling get_balanced_memory [INFO] We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
2025-12-11 11:12:31,964 __main__ load_model [INFO] Model loaded from tokyotech-llm/Llama-3.1-Swallow-8B-v0.5
LlamaForCausalLM(
  (model): LlamaModel(
    (embed_tokens): Embedding(128256, 4096)
    (layers): ModuleList(
      (0-31): 32 x LlamaDecoderLayer(
        (self_attn): LlamaAttention(
          (q_proj): Linear4bit(in_features=4096, out_features=4096, bias=False)
          (k_proj): Linear4bit(in_features=4096, out_features=1024, bias=False)
          (v_proj): Linear4bit(in_features=4096, out_features=1024, bias=False)
          (o_proj): Linear4bit(in_features=4096, out_features=4096, bias=False)
        )
        (mlp): LlamaMLP(
          (gate_proj): Linear4bit(in_features=4096, out_features=14336, bias=False)
          (up_proj): Linear4bit(in_features=4096, out_features=14336, bias=False)
          (down_proj): Linear4bit(in_features=14336, out_features=4096, bias=False)
          (act_fn): SiLUActivation()
        )
        (input_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
        (post_attention_layernorm): LlamaRMSNorm((4096,), eps=1e-05)
      )
    )
    (norm): LlamaRMSNorm((4096,), eps=1e-05)
    (rotary_emb): LlamaRotaryEmbedding()
  )
  (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
)
2025-12-11 11:12:34,958 __main__ load_jwtd [INFO] Dataset loaded, N=2554
2025-12-11 11:12:36,075 __main__ load_jwtd [INFO] Dataset loaded, N=542
2025-12-11 11:12:36,075 __main__ main [INFO] Filtered dataset: train 2554 rows, eval 542 rows
2025-12-11 11:12:47,525 __main__ add_train_str [INFO] pre_str:post_str = 469:2085 = 0.18:0.82
2025-12-11 11:12:59,077 __main__ add_train_str [INFO] pre_str:post_str = 110:432 = 0.20:0.80
2025-12-11 11:13:02,272 __main__ main [INFO] wandb initialized
2025-12-11 11:13:07,268 __main__ main [INFO] Starting DPO training with DPOTrainer
2025-12-11 11:13:32,682 root evaluate_probability_ratio [INFO] Results epoch 0: outputs/2025-12-11/11-12-17/probability_ratio_epoch_0.json
2025-12-11 11:15:24,572 root evaluate_probability_ratio [INFO] Results epoch 1: outputs/2025-12-11/11-12-17/probability_ratio_epoch_1.json
2025-12-11 11:17:15,106 root evaluate_probability_ratio [INFO] Results epoch 2: outputs/2025-12-11/11-12-17/probability_ratio_epoch_2.json
2025-12-11 11:19:05,077 root evaluate_probability_ratio [INFO] Results epoch 3: outputs/2025-12-11/11-12-17/probability_ratio_epoch_3.json
2025-12-11 11:20:57,951 root evaluate_probability_ratio [INFO] Results epoch 4: outputs/2025-12-11/11-12-17/probability_ratio_epoch_4.json
2025-12-11 11:22:49,543 root evaluate_probability_ratio [INFO] Results epoch 5: outputs/2025-12-11/11-12-17/probability_ratio_epoch_5.json
2025-12-11 11:24:41,323 root evaluate_probability_ratio [INFO] Results epoch 6: outputs/2025-12-11/11-12-17/probability_ratio_epoch_6.json
2025-12-11 11:26:32,747 root evaluate_probability_ratio [INFO] Results epoch 7: outputs/2025-12-11/11-12-17/probability_ratio_epoch_7.json
2025-12-11 11:28:25,903 root evaluate_probability_ratio [INFO] Results epoch 8: outputs/2025-12-11/11-12-17/probability_ratio_epoch_8.json