giakhuyendihoc/transformer-crossattn-l18-depth4-libero-nolora-fullvlm

Transformer L18 Depth4 CC-XAttn LIBERO no-LoRA fullVLM

30k-step LIBERO finetuned checkpoint for the Transformer action-mixer ablation with token-level ctxD cross-attention.

Checkpoint

• OpenPI config: pi05_transformer_ctxd18_crossattn_fullvlm_libero_5090
• Experiment: icra_nolora_fullvlm_transformer_crossattn_l18_depth4_cosine5e5_5e6_b16_acc1_seed42
• Step: 30000
• Initializer: pi05_base_pytorch
• Adaptation: full VLM / full vision / full action path, no LoRA
• VLM prefix: L18
• Mixer: Transformer depth4
• Context interface: CC-XAttn, token-level ctxD cross-attention
• Training: batch 16, grad accumulation 1, seed 42
• LR schedule: cosine from 5e-5 to 5e-6 over 30k steps
• LIBERO eval: seed 7, 50 trials per task, 10 denoising steps

LIBERO Results

Mixer	Context	Spatial	Object	Goal	LIBERO-10	Average	Action expert core	Train time	Median latency
Transformer	CC-XAttn	75.2	82.8	75.8	68.6	75.6	0.018B	13.72 h	85.60 ms

Use

--config-name pi05_transformer_ctxd18_crossattn_fullvlm_libero_5090
--checkpoint-dir /path/to/checkpoint

The checkpoint includes the LIBERO norm stats under assets/physical-intelligence/libero.