ForceVLA SFP All Trimmed v2
ForceVLA (Pi0-Guidance) model fine-tuned for SFP cable insertion on the AIC challenge.
What changed in v2
v1 was trained on data with zero rotation actions — the model could not learn the 21 degree pitch alignment needed for insertion, causing 100% failure at inference.
v2 is trained on aic_gt_sfp_all_trimmed_v2, which includes axis-angle orientation deltas computed from consecutive end-effector quaternions. The model now outputs both translation and rotation commands.
Model Details
| Field | Value |
|---|---|
| Base model | Pi0-Guidance (PaliGemma 2B + Gemma 300M action expert) |
| Fine-tuning | LoRA on both VLM and action expert |
| Training steps | 50,000 |
| Batch size | 4 |
| Dataset | aic_gt_sfp_all_trimmed_v2 (475 episodes, 10 SFP tasks) |
| GPU | NVIDIA RTX 5090 32GB |
| Framework | JAX/Flax (OpenPI) |
Input/Output
Inputs:
- 3 camera images (center, left, right) at 640x480, uint8
- 13D state vector: ee_pos(3) + axis_angle(3) + gripper(1) + wrench(6)
- Text prompt (e.g. "Insert the SFP cable module in SFP_PORT_0 on NIC_CARD at NIC_RAIL_0 in zone 1 section")
Outputs:
- 7D action: position delta (3) + orientation delta as rotation vector (3) + gripper (1)
Usage
Serve the model
cd ~/ForceVLA && conda activate forcevla_eval
python scripts/serve_policy.py policy:checkpoint \
--policy.config forcevla_sfp_all_trimmed_v2 \
--policy.dir checkpoints/forcevla_sfp_all_trimmed_v2/sfp_all_trimmed_v2/49999
Run inference
pixi run ros2 run aic_model aic_model --ros-args \
-p use_sim_time:=true \
-p policy:=aic_example_policies.ros.RunForceVLA
Checkpoints
| Step | Path |
|---|---|
| 10000 | sfp_all_trimmed_v2/10000/ |
| 20000 | sfp_all_trimmed_v2/20000/ |
| 30000 | sfp_all_trimmed_v2/30000/ |
| 40000 | sfp_all_trimmed_v2/40000/ |
| 49999 | sfp_all_trimmed_v2/49999/ (final) |
Action Space
| Index | Field | Scale (std) |
|---|---|---|
| 0-2 | position delta (m) | ~0.0003-0.0008 |
| 3-5 | rotation delta (rad) | ~0.001-0.004 |
| 6 | gripper | 0 (constant) |