---
license: mit
base_model: openvla/openvla-7b
tags: [robotics, vision-language-action, openvla, lerobot, aloha]
datasets: [lerobot/aloha_static_coffee]
---

# OpenVLA-7B — ALOHA coffee (LoRA adapter)

LoRA adapter (r=32, alpha=64) for bimanual coffee-making. Load on top of the base model.

Fine-tuned from [`openvla/openvla-7b`](https://huggingface.co/openvla/openvla-7b) on a curated
subset of [`lerobot/aloha_static_coffee`](https://huggingface.co/datasets/lerobot/aloha_static_coffee)
— a bimanual ALOHA arm making coffee.

## Read this before loading the weights

**`action_stats.json` in this repo is required to decode the model's output.** The action head emits
*bin indices*, not joint values. Turning a bin back into a joint target needs the exact q01/q99
statistics the model was trained against, which are in that file. Weights without it are not a
usable policy.

```python
from transformers import AutoModelForVision2Seq, AutoProcessor
from peft import PeftModel
import json

REPO = "anjali209/openvla-7b-aloha-coffee-lora"
processor = AutoProcessor.from_pretrained("openvla/openvla-7b", trust_remote_code=True)
model = AutoModelForVision2Seq.from_pretrained(
    "openvla/openvla-7b", dtype="bfloat16", trust_remote_code=True)
model = PeftModel.from_pretrained(model, REPO)

# required to decode bins -> joint targets
from huggingface_hub import hf_hub_download
stats = json.load(open(hf_hub_download(REPO, "action_stats.json")))
```

## Training

| | |
|---|---|
| Base model | `openvla/openvla-7b` (7.54B params) |
| Trainable | 79,953,920 (1.06%) — LoRA r=32, alpha=64, dropout 0.05 |
| Data | 42,006 train frames (48 episodes) / 1,831 val (2 held-out episodes) |
| Camera | `cam_left_wrist` — a single wrist (egocentric) view |
| Steps | 5,250 = 1.00 epoch |
| Precision | bf16, gradient checkpointing on |
| Optimizer | AdamW, OneCycleLR (`pct_start=0.03`), grad clip 1.0 |
| Learning rate | 2e-4 |
| Target modules | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` |
| Peak GPU memory | 9.4 GB |
| Wall time | 70.4 min |
| Checkpoint selected on | `val_action_mae` (best 0.0532 @ step 3500) |

Actions are 14-D joint targets, normalised per dimension with q01/q99 to [-1, 1], discretised into
256 uniform bins, and mapped onto the least-used token ids in the vocabulary (31744-31999). Loss is
cross-entropy over the 14 action tokens with the prompt masked.

Splits are **by episode, not by frame** — 65-75% of wrist frames are visually near-static, so a
frame-level split would put near-duplicates on both sides.

## Limitations

Please read these before using the model for anything.

- **Not validated on a robot or in simulation.** All evaluation was offline and teacher-forced: the
  model saw a real observation every frame and its predictions drove nothing, so compounding error
  is not measured at all.
- **It does not complete the task on held-out episodes.** Scored on a milestone criterion (pod
  grasped -> placed -> basket grasped -> seated, in order), it reaches all four milestones but
  fails the ordering/hold check on **both** held-out episodes: the predicted gripper closes on the
  basket and reopens 1-2 frames later.
- **The gripper command is temporally inconsistent.** Each action is an independent per-frame argmax
  over 256 bins, with nothing tying consecutive frames together. The predicted gripper moves up to
  **0.48 in a single frame** against a demonstration maximum of **0.036**. On hardware that is a
  command to snap the gripper open and shut at 50 Hz.
- **Large train/held-out gap.** The same criterion scores 85-92% on the 48 *training* episodes and
  0/2 on the 2 held-out ones — the model fits the demonstrations it saw and does not transfer.
- **One camera of four.** Only `cam_left_wrist` is used. Notably, the left arm handles the first
  sub-task while the right arm performs the one the model fails on.
- **Action MAE is not a good proxy for competence here.** A "repeat the previous frame's action"
  baseline beats this model 26x on that metric.

The practical next step this points at is constraining the gripper channel temporally at decode
time (action chunking, hysteresis, or a loss term penalising frame-to-frame gripper jumps).

## Provenance

Trained as part of a data-pipeline exercise covering dataset audit, labeling design, curation,
evaluation design, and VLA adaptation. Training data came from a curated build: post-completion
motion trimmed (~20% of frames) and a measured 5-frame video/state offset corrected before training.