Myungkyu/banana-challenge-ckpts

banana challenge — checkpoints

New! Two additional RLDX-1-PT-IMG variants (2026-09):

• rldx_1_img-h50/ — RLDX-1-PT-IMG trained with the same action horizon 50 as pi0.5 (the original rldx_1_img-h16/ uses the model's default horizon 16).
• rldx_1_img-h50-padmask/ — the horizon-50 model trained with noise masking on the padding dimensions of the action tokens (action_noise_mask_dim: 13 in config.json: the 13 real action dims get diffusion noise, the padding dims are kept at 0). There is no official inference code for this yet: your inference code must apply the same masking — the padding dimensions of the action-token input must be 0 (not noise) at every denoising step; with the stock sampler the padding dims receive noise and the checkpoint is out of distribution.

Policies fine-tuned on "Place the banana into the basket" (OpenArm + Inspire Hand, right arm and hand only). Data: Myungkyu/banana-challenge-data — 54 episodes / 16,202 frames, 30 fps stereo ego view at 640×360, 13-D absolute joint state/action. Batch size 64.

dir	model	base	action horizon	notes
pi0_5-h50/	pi0.5	lerobot/pi05_base @7de66397	50
rldx_1_img-h16/	RLDX-1-PT-IMG	RLWRLD/RLDX-1-PT-IMG @d67fc642	16	model default horizon
rldx_1_img-h50/	RLDX-1-PT-IMG	RLWRLD/RLDX-1-PT-IMG @d67fc642	50	same horizon as pi0.5
rldx_1_img-h50-padmask/	RLDX-1-PT-IMG	RLWRLD/RLDX-1-PT-IMG @d67fc642	50	+ noise masking on the action-token padding dims (action_noise_mask_dim: 13); inference must zero the padding dims

Checkpoints

Three steps per model (253 steps = 1 epoch on this dataset); every model directory holds step_010000, step_020000, step_030000 and the checkpoint files sit directly inside each step directory.

step	epochs
step_010000	39.5
step_020000	79.0
step_030000	118.5

pi0_5-h50/step_030000/               # pi0.5 (config.json, model.safetensors, policy_*processor*, train_config.json)
rldx_1_img-h16/step_030000/          # RLDX-1-PT-IMG, horizon 16 (config.json, model-*-of-*.safetensors, experiment_cfg/, processor/)
rldx_1_img-h50/step_030000/          # RLDX-1-PT-IMG, horizon 50
rldx_1_img-h50-padmask/step_030000/  # RLDX-1-PT-IMG, horizon 50 + padding-dim noise masking

Downloading one checkpoint only — note that * also matches /, so exclude the others:

hf download Myungkyu/banana-challenge-ckpts \
  --include "rldx_1_img-h50/step_030000/*" --local-dir ./ckpts

Loading — pi0.5

Configs here carry only the fields upstream LeRobot knows, so stock lerobot loads them:

pip install 'lerobot[pi]'     # the pi extra pulls transformers

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
from lerobot.policies.factory import make_pre_post_processors

ckpt = "ckpts/pi0_5-h50/step_030000"
policy = PI05Policy.from_pretrained(ckpt); policy.eval(); policy.to("cuda")
pre, post = make_pre_post_processors(policy.config, pretrained_path=ckpt)

Batch keys: observation.image.ego_left, observation.image.ego_right (CHW float32 in [0,1] — feed the raw 640×360 frame; the model pads and resizes to 224 itself), observation.state (13,) float32, and task (str). predict_action_chunk returns 50 normalized steps; pass each through post to get env units.

make_pre_post_processors must point at the same directory — the q01/q99 normalization lives in the checkpoint's processor files. The saved device_processor says cuda; to run elsewhere pass preprocessor_overrides={"device_processor": {"device": "cpu"}} (and the same for postprocessor_overrides).

Loading — RLDX-1-PT-IMG

Point model_path at the step directory and use embodiment_tag=GENERAL_EMBODIMENT; the modality config and normalization statistics travel inside processor/.

from rldx.data.embodiment_tags import EmbodimentTag
from rldx.policy.rldx_policy import RLDXPolicy, RLDXSimPolicyWrapper

policy = RLDXPolicy(embodiment_tag=EmbodimentTag.GENERAL_EMBODIMENT,
                    model_path="ckpts/rldx_1_img-h16/step_030000", device="cuda:0")   # or rldx_1_img-h50 / rldx_1_img-h50-padmask

Observation keys: video.ego_left / video.ego_right (B,1,360,640,3) uint8, state.right_arm (B,1,7) / state.right_hand (B,1,6) float32, and task as a tuple of strings. Returns 16 (rldx_1_img-h16) or 50 (rldx_1_img-h50*) steps per action key.

For rldx_1_img-h50-padmask add the padding-dimension masking to the sampler: at every denoising step, keep the action-token input at 0 on the dimensions beyond the 13 real action dims (only the first 13 dims carry the noised sample). The checkpoint's config.json records action_noise_mask_dim: 13 for this purpose.

Notes

• Actions are absolute joint targets (not deltas): 7 arm + 6 hand, order given by action_feature_names in the pi0.5 config.
• The head/neck joints were held fixed during collection (openarm_head_pitch ≈ 1.0 rad, openarm_head_yaw = 0.0 rad) and are not part of the 13-D vector. The left arm was not used.