---
license: other
license_name: unspecified-upstream
base_model: lerobot/VLA-JEPA-LIBERO
tags:
  - vla
  - vision-language-action
  - robotics
  - gguf
  - vla.cpp
  - llama.cpp
  - libero
  - vla-jepa
  - qwen3-vl
  - flow-matching
pipeline_tag: robotics
library_name: vla.cpp
---

# VLA-JEPA - LIBERO (GGUF for `vla.cpp`)

GGUF conversion of [`lerobot/VLA-JEPA-LIBERO`](https://huggingface.co/lerobot/VLA-JEPA-LIBERO)
for inference with [`vla.cpp`](https://github.com/VinRobotics/vla.cpp), a
lightweight C++ inference engine for Vision-Language-Action models built on top of
[`llama.cpp`](https://github.com/ggml-org/llama.cpp).

VLA-JEPA is a 3B VLA built on a **Qwen3-VL-2B-Instruct** vision-language backbone
- a 24-layer ViT (1024d, patch 16, temporal-patch 2, spatial-merge ÷2, deepstack
features fused from layers 5/11/17, 256 px) feeding a 28-layer Qwen3 LM (2048d,
16 query / 8 KV heads × 128, RoPE θ=5e6) - coupled to a **DiT-B flow-matching
action head** (16 transformer blocks, 12 heads × 64, inner 768d, cross-attending
the 2048d LM stream, 1024d output) that denoises a **7-step action chunk** over 4
flow-matching steps from 32 learned future tokens (`action_dim` 7, `state_dim` 8).

The upstream checkpoint also carries a **V-JEPA world-model predictor**, but it is
**not on the action-inference path** - so the conversion **drops the world model**
and keeps only the Qwen3-VL backbone + DiT action head. The vision tower is baked
into the combined GGUF, so **no separate mmproj file is needed**.

## Files

| File | Size | Description |
|---|---:|---|
| `vla-jepa.gguf` | 4.25 GiB | Combined VLA model - Qwen3-VL-2B backbone (ViT + deepstack mergers + Qwen3 LM) + DiT-B flow-matching action head + arch config, BF16. World-model predictor dropped. |

### Required normalisation stats (not bundled here)

VLA-JEPA un-normalises with LeRobot processor stats, **not** a
`dataset_statistics.json`. The client reads two safetensors from the dir passed to
`--stats-json`:

| File | Role |
|---|---|
| `policy_preprocessor_step_3_normalizer_processor.safetensors` | `observation.state` MEAN_STD normaliser |
| `policy_postprocessor_step_2_unnormalizer_processor.safetensors` | `action` MIN_MAX un-normaliser (+ gripper snap/binarise) |

Both ship in the upstream
[`lerobot/VLA-JEPA-LIBERO`](https://huggingface.co/lerobot/VLA-JEPA-LIBERO) repo.
Copy them next to `vla-jepa.gguf` (then `--stats-json .` works) or point
`--stats-json` at a local copy of the upstream checkpoint.

## Usage

Build `vla-server` from the [vla.cpp repo](https://github.com/VinRobotics/vla.cpp),
then:

```bash
# Terminal 1 - serve (use the CUDA build for inference). No mmproj argument.
VLA_JEPA_BF16_WEIGHTS=1 ./build-cuda/vla-server --bind tcp://*:5566 \
    vla-jepa.gguf

# Terminal 2 - drive a LIBERO episode (inside the LIBERO uv venv)
python eval/client/run_sim_client_direct.py \
    --arch vla_jepa \
    --task libero_object --task-id 0 --n-episodes 10 \
    --n-action-steps 7 \
    --stats-json . \
    --vla-addr tcp://localhost:5566
```

Notes:
- `vla_jepa` runs at **256 px** with a **7-step action chunk** (`--n-action-steps 7`)
  and proprio state dim 8.
- Set `VLA_JEPA_BF16_WEIGHTS=1` to keep BF16 matmuls (default upcasts weights to
  F32). `VLA_NUM_STEPS=<n>` overrides the 4 flow-matching steps.
- `--stats-json <dir>` is **required** - point it at the dir holding the two
  `policy_{pre,post}processor` safetensors (see above).
- The tokenizer/processor auto-loads from
  [`Qwen/Qwen3-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct)
  on the Hub (the client expands it with the 28 `<|action_i|>` + `<|embodied_action|>`
  tokens). Pass `--tokenizer /path/to/Qwen3-VL-2B-Instruct` to load it offline.

## Benchmark

Smoke test - LIBERO sweep, `vla-server` + `run_sim_client_direct.py`:

| Hardware | Success rate | client/step | server/call | Peak mem |
|---|---:|---:|---:|---:|
| - | 100.0% (10 tasks × 10 episodes = 100/100) | - | - | - |

VRAM and latency have not been measured yet; they will be filled in once profiled
on reference hardware.

## License

The upstream [`lerobot/VLA-JEPA-LIBERO`](https://huggingface.co/lerobot/VLA-JEPA-LIBERO)
repo declares **no explicit license**. Review the upstream terms and the licenses
of its components - Qwen3-VL-2B (Apache-2.0) and V-JEPA (Meta research license) -
before use. The vla.cpp conversion tooling and inference engine are
Apache-2.0-licensed.