Update merged LingbotVLA Click Bell GRPO checkpoint

.gitattributes

@@ -33,12 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/.metadata filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__0_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__1_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__2_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__3_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__4_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__5_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__6_0.distcp filter=lfs diff=lfs merge=lfs -text
-actor/dcp_checkpoint/__7_0.distcp filter=lfs diff=lfs merge=lfs -text
+assets/Teaser.png filter=lfs diff=lfs merge=lfs -text

MERGE_INFO.md

@@ -0,0 +1,10 @@
+# LingbotVLA Click Bell GRPO Merged Base
+
+- Base model: `robbyant/lingbot-vla-4b-posttrain-robotwin`
+- Base revision: `3e0c7c476bde3daaac00f79f3741a292a299f60a`
+- RLinf checkpoint source: `RLinf/RLinf-lingbotvla-click-bell-grpo`
+- Local RLinf checkpoint: `/mnt/public/lwb/assets/lingbot-vla-eval/ckpts/lingbotvla-click-bell-grpo-latest/actor/model_state_dict/full_weights.pt`
+- Export directory: `/mnt/public/lwb/artifacts/lingbot-vla/outputs/merged_base/20260620_090640_click_bell_grpo_merged_base`
+- Export content: inner `vla_model` weights only, saved in LingbotVLA safetensors format.
+- Excluded: RLinf wrapper-only heads such as `value_head`.
+- Strict key check: `1555` / `1555` keys matched.

README.md

@@ -1,68 +1,58 @@
 ---
 license: apache-2.0
-library_name: rlinf
-pipeline_tag: robotics
 tags:
-  - reinforcement-learning
-  - robotics
-  - embodied-ai
-  - lingbotvla
-  - robotwin
-  - grpo
+- robotics
+- rlinf
+- reinforcement-learning
+- embodied-ai
+- lingbotvla
+- robotwin
+pipeline_tag: robotics
 ---
 
-# RLinf LingbotVLA Click Bell GRPO
+# RLinf-lingbotvla-click-bell-grpo
 
-This repository contains an RLinf checkpoint for LingbotVLA GRPO fine-tuning on the RoboTwin `click_bell` task.
+This is a LingbotVLA checkpoint post-trained with RLinf GRPO on the RoboTwin `click_bell` task.
 
-## Checkpoint Format
+The checkpoint has been merged into the pinned RoboTwin SFT base format, so it can be loaded directly as a LingbotVLA model directory. It does not require a separate `runner.ckpt_path` file for evaluation.
 
-The checkpoint is provided in RLinf actor checkpoint format:
+## Base Model
 
-```text
-actor/model_state_dict/full_weights.pt
-actor/dcp_checkpoint/.metadata
-actor/dcp_checkpoint/__5_0.distcp
-actor/dcp_checkpoint/__7_0.distcp
-```
+- Upstream base: [`robbyant/lingbot-vla-4b-posttrain-robotwin`](https://huggingface.co/robbyant/lingbot-vla-4b-posttrain-robotwin)
+- Base revision: `3e0c7c476bde3daaac00f79f3741a292a299f60a`
+- RL method: RLinf GRPO
+- RoboTwin task: `click_bell`
 
-For evaluation in RLinf, load the model through `runner.ckpt_path`:
+## Result Summary
 
-```bash
-runner.ckpt_path=/path/to/actor/model_state_dict/full_weights.pt
-```
+In our RoboTwin random evaluation, this RLinf-GRPO checkpoint performs better than the pinned RoboTwin SFT checkpoint on the corresponding `click_bell` task.
 
-Use the LingbotVLA RoboTwin SFT base configuration from:
+## Download
 
-```text
-robbyant/lingbot-vla-4b-posttrain-robotwin
-revision: 3e0c7c476bde3daaac00f79f3741a292a299f60a
+```bash
+hf download RLinf/RLinf-lingbotvla-click-bell-grpo \
+    --local-dir RLinf-lingbotvla-click-bell-grpo
 ```
 
-## Evaluation
+## Evaluation in RLinf
 
-Latest local regression evaluation on RoboTwin `click_bell` random setting:
+Use the downloaded directory as both `actor.model.model_path` and `rollout.model.model_path`. Keep `tokenizer_path` pointing to a local Qwen2.5-VL-3B-Instruct directory.
 
-| Checkpoint | Task | Setting | Trajectories | Max Steps | eval/success_once | eval/return |
-| --- | --- | --- | ---: | ---: | ---: | ---: |
-| RLinf-lingbotvla-click-bell-grpo | click_bell | random | 320 | 400 | 0.9875 | 6.85 |
-
-Evaluation settings:
-
-```text
-config: robotwin_click_bell_grpo_lingbotvla_eval
-algorithm.eval_rollout_epoch=1
-algorithm.sampling_params.temperature_eval=-1
-env.eval.total_num_envs=320
-env.eval.max_episode_steps=400
-env.eval.max_steps_per_rollout_epoch=400
-env.eval.use_fixed_reset_state_ids=False
-env.eval.seeds_path=null
-env.eval.video_cfg.save_video=False
+```bash
+python examples/embodiment/eval_embodied_agent.py \
+    --config-path examples/embodiment/config \
+    --config-name robotwin_click_bell_grpo_lingbotvla \
+    runner.only_eval=True \
+    actor.model.model_path=/path/to/RLinf-lingbotvla-click-bell-grpo \
+    actor.model.tokenizer_path=/path/to/Qwen2.5-VL-3B-Instruct \
+    actor.model.lingbotvla.config_path=/path/to/RLinf-lingbotvla-click-bell-grpo \
+    rollout.model.model_path=/path/to/RLinf-lingbotvla-click-bell-grpo \
+    rollout.model.tokenizer_path=/path/to/Qwen2.5-VL-3B-Instruct \
+    rollout.model.lingbotvla.config_path=/path/to/RLinf-lingbotvla-click-bell-grpo
 ```
 
-The evaluation logs are kept locally under:
+For large RoboTwin evaluations, set `env.eval.total_num_envs`, `env.eval.max_episode_steps`, and `env.eval.max_steps_per_rollout_epoch` according to the target protocol.
 
-```text
-/mnt/public/lwb/artifacts/lingbot-vla-eval/click_bell_regression/20260615_140703
-```
+## Notes
+
+This repository stores the merged LingbotVLA model weights only. RLinf wrapper-only tensors, such as value heads used during training, are not included.

actor/model_state_dict/full_weights.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a3bab602d5cdee12c10583e2fee64d819426f074a050b64e3073c70cd3392cd3
-size 8395456710

assets/norm_stats/robotwin_all_new.json

@@ -0,0 +1,229 @@
+{
+  "norm_stats": {
+    "action.arm.position": {
+      "mean": [
+        -0.2260681688785553,
+        1.090435266494751,
+        0.8042582273483276,
+        -0.3527189791202545,
+        0.056556474417448044,
+        -0.04530515521764755,
+        0.2346765249967575,
+        1.112542748451233,
+        0.8304542303085327,
+        -0.357768177986145,
+        -0.01014612801373005,
+        0.010991317220032215
+      ],
+      "std": [
+        0.3691432774066925,
+        0.994762122631073,
+        0.7908730506896973,
+        0.6637247800827026,
+        0.24963052570819855,
+        0.5638052821159363,
+        0.32393988966941833,
+        1.0204970836639404,
+        0.8119731545448303,
+        0.7209287285804749,
+        0.25776439905166626,
+        0.6208906769752502
+      ],
+      "q01": [
+        -0.9676963651657111,
+        -0.0003164021181873977,
+        -0.0026667596280574857,
+        -1.596037513256073,
+        -0.4467973255872727,
+        -2.20232324104309,
+        -0.13648582720756508,
+        -0.0017502129077910933,
+        -0.0023805056512355804,
+        -1.703943779706955,
+        -1.0264247895240783,
+        -1.6682623161315915
+      ],
+      "q99": [
+        0.17045696868896432,
+        2.5760957974332737,
+        2.4727182808369395,
+        1.259782492733002,
+        1.2253731035709379,
+        1.4495478111267097,
+        1.0841207003116606,
+        2.6036578441381453,
+        2.4987799152359367,
+        1.3104696589708325,
+        1.0692354731559752,
+        2.104229341125489
+      ],
+      "q02": [
+        -0.9260248472213748,
+        -0.0003164021181873977,
+        -0.0026667596280574857,
+        -1.5090695432662964,
+        -0.3291901943683624,
+        -1.6520995048522948,
+        -0.05942733430862468,
+        -0.0017502129077910933,
+        -0.0023805056512355804,
+        -1.6187864029407502,
+        -0.8741156991004944,
+        -1.5490281238555905
+      ],
+      "q98": [
+        0.1157631013870235,
+        2.4936630497265257,
+        2.3193349599272013,
+        1.0769267609596254,
+        1.0140384616851805,
+        1.2073643905639653,
+        0.9469565829515458,
+        2.528083452796936,
+        2.3551445673033595,
+        1.2071769149303435,
+        0.6969243632316591,
+        1.7504360820770266
+      ]
+    },
+    "action.effector.position": {
+      "mean": [
+        0.6723259687423706,
+        0.6735112071037292
+      ],
+      "std": [
+        0.4526418447494507,
+        0.4514695405960083
+      ],
+      "q01": [
+        0.0,
+        0.0
+      ],
+      "q99": [
+        0.9998,
+        0.9998
+      ],
+      "q02": [
+        0.0,
+        0.0
+      ],
+      "q98": [
+        0.9998,
+        0.9998
+      ]
+    },
+    "observation.state.arm.position": {
+      "mean": [
+        -0.22502799332141876,
+        1.0857956409454346,
+        0.8007810711860657,
+        -0.3513113558292389,
+        0.05622035637497902,
+        -0.044659487903118134,
+        0.23319771885871887,
+        1.106688141822815,
+        0.82613205909729,
+        -0.3561287522315979,
+        -0.010010534897446632,
+        0.010481182485818863
+      ],
+      "std": [
+        0.3684558570384979,
+        0.9951919317245483,
+        0.7907320857048035,
+        0.6622379422187805,
+        0.24897389113903046,
+        0.5617504119873047,
+        0.32331398129463196,
+        1.0208075046539307,
+        0.8117841482162476,
+        0.718940019607544,
+        0.25672635436058044,
+        0.6180205345153809
+      ],
+      "q01": [
+        -0.9676963651657111,
+        -0.0003164021181873977,
+        -0.0026667596280574857,
+        -1.5938075653076171,
+        -0.4462003350734711,
+        -2.195949993133545,
+        -0.13648582720756508,
+        -0.0017502129077910933,
+        -0.0023805056512355804,
+        -1.703943779706955,
+        -1.0257196548461915,
+        -1.6663076572418207
+      ],
+      "q99": [
+        0.16785249881744324,
+        2.5760957974332737,
+        2.47087028901875,
+        1.2516060169219974,
+        1.22238815100193,
+        1.4495478111267097,
+        1.073332511305809,
+        2.602131088757515,
+        2.494382914789021,
+        1.3104696589708325,
+        1.0657097997665406,
+        2.102274682235718
+      ],
+      "q02": [
+        -0.9234203773498537,
+        -0.0003164021181873977,
+        -0.0026667596280574857,
+        -1.5060962793350219,
+        -0.3291901943683624,
+        -1.6436018409728996,
+        -0.05788616445064587,
+        -0.0017502129077910933,
+        -0.0023805056512355804,
+        -1.6164209202528,
+        -0.8698848910331727,
+        -1.5490281238555905
+      ],
+      "q98": [
+        0.1157631013870235,
+        2.4928853822953303,
+        2.3174869681090113,
+        1.0754401289939883,
+        1.0122474901437757,
+        1.2031155586242681,
+        0.945415413093567,
+        2.527320075106621,
+        2.3522132336720825,
+        1.202445949554443,
+        0.694808959197998,
+        1.7484814231872559
+      ]
+    },
+    "observation.state.effector.position": {
+      "mean": [
+        0.6735715866088867,
+        0.6747165322303772
+      ],
+      "std": [
+        0.4521658420562744,
+        0.4510030150413513
+      ],
+      "q01": [
+        0.0,
+        0.0
+      ],
+      "q99": [
+        0.9998,
+        0.9998
+      ],
+      "q02": [
+        0.0,
+        0.0
+      ],
+      "q98": [
+        0.9998,
+        0.9998
+      ]
+    }
+  },
+  "count": 535680
+}

config.json

@@ -0,0 +1,42 @@
+{
+    "type": "pi0",
+    "n_obs_steps": 1,
+    "normalization_mapping": {
+        "VISUAL": "IDENTITY",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "input_features": {},
+    "output_features": {},
+    "device": "cpu",
+    "use_amp": false,
+    "chunk_size": 50,
+    "n_action_steps": 50,
+    "max_state_dim": 75,
+    "max_action_dim": 75,
+    "resize_imgs_with_padding": [
+        224,
+        224
+    ],
+    "empty_cameras": 0,
+    "adapt_to_pi_aloha": false,
+    "use_delta_joint_actions_aloha": false,
+    "tokenizer_max_length": 24,
+    "proj_width": 768,
+    "num_steps": 10,
+    "use_cache": true,
+    "attention_implementation": "flex",
+    "freeze_vision_encoder": false,
+    "train_expert_only": false,
+    "train_state_proj": true,
+    "optimizer_lr": 2.5e-05,
+    "optimizer_betas": [
+        0.9,
+        0.95
+    ],
+    "optimizer_eps": 1e-08,
+    "optimizer_weight_decay": 1e-10,
+    "scheduler_warmup_steps": 1000,
+    "scheduler_decay_steps": 30000,
+    "scheduler_decay_lr": 2.5e-06
+}

lingbotvla_cli.yaml

@@ -0,0 +1,143 @@
+data:
+  cameras:
+  - camera_top
+  - camera_wrist_left
+  - camera_wrist_right
+  chat_template: default
+  data_name: multi
+  data_root: null
+  data_tag: default
+  data_type: conversation
+  dataloader_type: native
+  datasets_type: vla
+  drop_last: true
+  image_keys: images
+  joints:
+  - '{''arm.position'': 14}'
+  - '{''effector.position'': 2}'
+  max_seq_len: 64
+  norm_type: bounds_99_woclip
+  num_workers: 20
+  pin_memory: true
+  prefetch_factor: 4
+  robot_config_root: ./configs/robot_configs
+  source_name: null
+  text_keys: messages
+  train_path: ./assets/training_data/robotwin_all_tasks_with_aug.txt
+  train_size: 8000000
+model:
+  adanorm_time: true
+  attn_implementation: flash_attention_2
+  basic_modules: []
+  decoders: {}
+  depth_incremental_training: false
+  encode_target: false
+  encoders: {}
+  final_norm_adanorm: false
+  force_use_huggingface: false
+  incremental_training: false
+  input_encoder: encoder
+  moe_implementation: null
+  nosplit_gate_liner: false
+  old_adanorm: true
+  output_encoder: decoder
+  post_training: true
+  separate_time_proj: false
+  split_gate_liner: false
+  use_lm_head: false
+  vlm_repo_id: null
+  vocab_size: 0
+train:
+  action_dim: 14
+  activation_gpu_limit: 0.0
+  align_params: {}
+  bsz_warmup_init_mbtoken: 200
+  bsz_warmup_ratio: 0
+  chunk_size: 50
+  ckpt_manager: dcp
+  context_parallel_size: 1
+  data_parallel_mode: fsdp2
+  data_parallel_replicate_size: 1
+  data_parallel_shard_size: 8
+  decayed_max_grad_norm: 1.0
+  dyn_bsz: true
+  dyn_bsz_buffer_size: 200
+  dyn_bsz_margin: 0
+  ema_rate: 0.9999
+  embodiment_name: null
+  empty_cache_steps: 500
+  enable_activation_offload: false
+  enable_expert_vision: false
+  enable_forward_prefetch: true
+  enable_fp32: true
+  enable_fsdp_offload: false
+  enable_full_determinism: false
+  enable_full_shard: false
+  enable_gradient_checkpointing: true
+  enable_manual_eager: false
+  enable_mixed_precision: true
+  enable_profiling: false
+  enable_reentrant: false
+  enable_resume: true
+  expert_parallel_size: 1
+  expert_vision_path: null
+  expert_vision_type: null
+  freeze_vision_encoder: false
+  freeze_vit: false
+  global_batch_size: 256
+  ignore_depth: false
+  init_device: cuda
+  interact_layer_end: null
+  interact_layer_start: null
+  load_checkpoint_path: null
+  lr: 0.0001
+  lr_decay_ratio: 1.0
+  lr_decay_style: constant
+  lr_min: 1.0e-07
+  lr_start: 0.0
+  lr_warmup_ratio: 0
+  max_action_dim: 75
+  max_grad_norm: 1.0
+  max_state_dim: 75
+  max_steps: 220000
+  micro_batch_size: 32
+  module_fsdp_enable: true
+  my_tokenizer_max_length: 72
+  norm_qkv: false
+  num_train_epochs: 29000
+  optimizer: adamw
+  pipeline_parallel_size: 1
+  pre_train: false
+  profile_end_step: 2
+  profile_profile_memory: true
+  profile_record_shapes: true
+  profile_start_step: 1
+  profile_trace_dir: ./trace
+  profile_with_stack: true
+  qwenvl_bos: true
+  resume_dataloader_state: true
+  rmpad: false
+  rmpad_with_pos_ids: false
+  save_epochs: 29000
+  save_hf_weights: true
+  save_steps: 10000
+  seed: 42
+  skip_max_norm: false
+  stable_train_steps: 100000
+  tensor_parallel_size: 1
+  tokenizer_max_length: 24
+  ulysses_parallel_size: 1
+  use_compile: true
+  use_doptim: false
+  use_ema: false
+  use_fast_action: false
+  use_ki: false
+  use_prompt: false
+  use_state: false
+  use_subtask: false
+  use_wandb: false
+  vit_lr: 1.0e-06
+  vlm_causal: false
+  wandb_name: pi0_libero
+  wandb_project: pi0_libero
+  weight_decay: 0

actor/dcp_checkpoint/__5_0.distcp → model-00001-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0bffe1d54a05e1bfa84480d0a2b8c20279091a642c039391282da97f4e0cad20
-size 1274533560
+oid sha256:154462a4a4377fb736d5fdb632c569dd6105adf4f3b3d10662d23920a371f6f6
+size 4997770856

actor/dcp_checkpoint/__7_0.distcp → model-00002-of-00002.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0850f421ce2098f40120554ee750a87d6b422c62af806e60c25e0583b100df38
-size 1273924156
+oid sha256:a9886b9bd1632171f52a1c0db5ab1957592a6283ae6990b5cae282d5c5305c9d
+size 3397309446