s1ghhh/VLADrop_OpenVLA-OFT_LIBERO_Drop16MLP_Uniform

VLADrop-OpenVLA-OFT-LIBERO-mlp-only

Checkpoint for Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?.

DTR (Drop-Then-Recovery) removes transformer blocks from a pretrained VLA model and recovery-fine-tunes the smaller dense model. Code: https://github.com/s1ghhh/VLADrop

This checkpoint

Paper row	Table 2: OpenVLA-OFT MLP (granularity study)
Dropped blocks	Llama-2 backbone (32 layers): drop only the MLP sublayers of the 16 odd-indexed blocks [1,3,5,...,31]. MHA sublayers kept.
Recovery training	LoRA rank 32 (merged), batch size 16, 50K steps, lr 5e-4
LIBERO success rate	Spatial 83.2 / Object 96.0 / Goal 15.4 / Long 72.8 / Avg 65.6

Usage

Merged full-weight OpenVLA-OFT checkpoint. The drop lists are baked into config.json (text_config.drop_attn_list / drop_mlp_list), so evaluation reconstructs the pruned graph automatically. Requires the VLADrop modified transformers (transformers-openvla-oft-dropped) from https://github.com/s1ghhh/VLADrop

python -m experiments.robot.libero.run_libero_eval_random_dropped \
    --pretrained_checkpoint <this_repo_local_path> \
    --task_suite_name libero_spatial \
    --num_trials_per_task 50 --center_crop True

Includes action_head--*.pt and proprio_projector--*.pt (L1 regression head, 2 input images + proprio).

Citation

@article{sun2026vladrop,
  title={Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?},
  author={Sun, Guoheng and Feng, Kaixi and He, Shwai and Gong, Xiaochuan and He, Yexiao and Wang, Ziyao and Shen, Zheyu and Ye, Wanghao and Kompella, Ramana Rao and Liu, Gaowen and Li, Ang},
  journal={arXiv preprint arXiv:2606.27755},
  year={2026}
}