egovlm-llama32-1b-grpo

GRPO-finetuned Llama-3.2-1B for the EgoVLM motion-language task (Nymeria EgoLM v6.2).

Files

File	Step	Mean reward	Size	Notes
grpo_step250.ckpt	250	~+0.18 (Debug 2 area, drifting down)	12.86 GB	First snapshot uploaded
grpo_best_step500.ckpt	500	+0.57	12.86 GB	Best mean reward so far

Both are PyTorch Lightning checkpoints. Plain DDP-saved state dict (NOT FSDP-sharded), portable to any machine.

Reward trajectory

Debug #	Step ≈	Reward (min/max/mean)
1	0–100	-0.43 / 0.51 / +0.20
2	100–200	-0.73 / 0.78 / +0.18
3	200–300	-0.35 / 0.56 / -0.10
4	300–400	-0.73 / 0.48 / -0.15
5	400–500	-0.66 / 1.23 / +0.41
6	500	-0.58 / 1.38 / +0.57 ← step 500 ckpt

Note: mean reward dipped through steps 200–400 (matching-distance penalty growing faster than motion-ADE term improving), then recovered sharply once policy learned to balance both. Best-so-far at upload time is step 500.

Training recipe

• SFT init: LoRA-trained Llama-3.2-1B on Nymeria EgoLM v6.2 (20 epochs of instruction tuning, single combined stp2mt task per sample). LoRA adapters then merged into the base via peft.PeftModel.merge_and_unload().
• GRPO stage: full-parameter PPO-clip with group-normalized advantages.
  • K (rollouts per prompt) = 6
  • lr = 1e-6, AdamW, betas=(0.9, 0.99)
  • clip ε = 0.2, KL β = 0.001
  • reward = W_motion · max(0, 1 − ADE) + W_text · BLEU + W_matdist · (−matching_dist) − format_penalty
  • weights: motion_past = motion_future = 1.0, text = 0.01, matdist = 0.02
• Prompt (single template, identical to the SFT stage):

  Perform human motion tracking and prediction: Output current and future
  human motion token sequences, then describe current and future human
  motion conditioned on the given scene, observed video CLIP embeddings,
  and observed three-points features.
  Input scene: <3D_Scene_Placeholder>.
  Input video CLIP embeddings: <Video_Placeholder>.
  Input three-points features: <TP_Placeholder>
  

• Hardware: 4× H200, DDP, BF16-mixed
• Codebase: EgoVLM-I/ (https://github.com/Jaewoo97/egovlm_llama)

Load

import torch
ckpt = torch.load("grpo_best_step500.ckpt", map_location="cpu", weights_only=False)
sd = ckpt["state_dict"]   # 738 keys, full model (no FSDP shards)
# Build MotionGPT with cfg.model.params.lm = configs/lm/llama32_1b_full_ft.yaml
# Then model.load_state_dict(sd, strict=False)

Or use the codebase loader by setting cfg.TRAIN.PRETRAINED to this file path.