pick-laptop-charger-act-v0

Action Chunking Transformer (ACT) policy trained on 83 human demonstrations of pinching and lifting a laptop charger with a low-cost sensorized glove, retargeted for the Franka Emika Panda + Franka Hand parallel gripper.

This is the first model trained with the humxai v0.3 schema, which separates contact timing (a clean binary signal derived from FSR pressure sensors) from grip force magnitude (a theoretical constant from the skill yaml, not noisy FSR readings).

What this model does

Given:

• RGB video (observation.images.front, 720×1280 @ 30 fps from a Mac webcam)
• Object-centric end-effector state (observation.state, 7-dim: rel_ee_xyz + rpy + gripper_force)
• Contact channel (observation.contact, binary 1-dim)
• Per-finger contact (observation.contact_per_finger, 4-dim)
• Per-finger raw FSR pressure (observation.pressure_per_finger, 4-dim, auxiliary)

The policy outputs the next 100-frame chunk of robot actions (7-dim: Δxyz + Δrpy + gripper_force_target).

Eval results (replay on held-out episodes)

Tested on 10 held-out episodes (ep73-82, ~1000 frames total):

Metric	Value	Interpretation
action_mae	0.0108	overall action prediction error
pos_mae	2.8 mm	wrist-position error (excellent for a 80 cm workspace)
rot_mae	0.009 rad	wrist-orientation error (~0.5°, excellent)
gripper_mae	0.040	gripper force tracking (the v0.3 schema target was <0.05)

The low gripper_mae validates the v0.3 design choice: the policy learns when to close the gripper (timing) reliably from a binary contact signal, then outputs the pre-defined theoretical force for that object (14 N for the laptop charger). At deployment: commanded_N = policy_output × 70 = 14 N.

Force semantics

The policy's gripper_force output is not a raw FSR reading. It is:

gripper_force ∈ {0.0, 0.04, 0.08, 0.12, 0.16, 0.20}

These 6 discrete values come from the 5-frame boxcar smoothing applied to the binary contact channel, scaled by the theoretical mean force for the laptop_charger object (14 N / 70 N Franka Hand max = 0.200).

To get Newtons at deployment time:

commanded_force_N = policy_output["gripper_force"] * 70.0   # Franka Hand

For a different robot, scale by its max_robot_force_N:

# Robotiq 2F-85
commanded_force_N = policy_output["gripper_force"] * 235.0

Usage

from huggingface_hub import snapshot_download
import torch
from lerobot.policies.act.modeling_act import ACTPolicy

# 1. Download checkpoint
ckpt_dir = snapshot_download(
    repo_id="andresceballosm/pick-laptop-charger-act-v0",
    repo_type="model",
)

# 2. Patch config (LeRobot local rejects the `type` field present in HF configs)
import json
from pathlib import Path
cfg_path = Path(ckpt_dir) / "config.json"
cfg = json.loads(cfg_path.read_text())
cfg.pop("type", None)
cfg["pretrained_backbone_weights"] = None
cfg_path.write_text(json.dumps(cfg, indent=2))

# 3. Load policy
policy = ACTPolicy.from_pretrained(ckpt_dir)
policy.eval()

# 4. At each timestep, build observation dict and call policy
observation = {
    "observation.images.front": video_frame_tensor,      # (1, 3, 720, 1280)
    "observation.state": state_tensor,                    # (1, 7)
    "observation.contact": contact_tensor,                # (1, 1)
    "observation.contact_per_finger": contact_pf,         # (1, 4)
    "observation.pressure_per_finger": pressure_pf,       # (1, 4)
}
action = policy.select_action(observation)  # (1, 7) delta action

Training details

Param	Value
Policy type	ACT (Action Chunking Transformer)
Steps	20,000
Batch size	16
Optimizer	AdamW, lr=1e-5, weight_decay=1e-4
KL weight	10.0 (default ACT)
Chunk size	100 frames (~3.3s @ 30 fps)
Vision backbone	ResNet18 (ImageNet pretrained)
Hardware	RunPod A100 80GB, ~2 hours
Final loss	~0.30 (plateau, dominated by KL term)
Mixed precision	FP16 (use_amp=true)

Limitations

• In-distribution only: tested only on held-out frames from the same recording session. Out-of-distribution generalization (new object positions, lighting, backgrounds) is untested.
• Single-object dataset: 83 demos all with one specific laptop charger. The policy may not generalize to other charger shapes.
• No sim → real validation: trained from CV-estimated wrist pose, not robot joint encoders. Deployment to a real Franka will have a sim-to-real gap.
• No depth sensing: glove + Mac webcam only. rel_ee_z is derived from IMU gravity, which is approximate.
• Force is theoretical, not measured: the policy commands a fixed ~14 N regardless of object slippage. A force-feedback controller is needed in production for safety on fragile or slippery objects.

Citation

Source code and pipeline: https://github.com/andresceballosm/humxai-glove-dataset

Dataset: andresceballosm/pick-laptop-charger-franka

If you build on this work:

@misc{ceballos2026humxai_charger_v0,
  author = {Ceballos, Andrés Felipe},
  title  = {pick-laptop-charger-act-v0: ACT policy for Franka pinch-grasp},
  year   = {2026},
  url    = {https://huggingface.co/andresceballosm/pick-laptop-charger-act-v0}
}