Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,70 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
base_model: MINT-SJTU/Evo1_LIBERO
|
| 4 |
+
tags:
|
| 5 |
+
- vla
|
| 6 |
+
- vision-language-action
|
| 7 |
+
- robotics
|
| 8 |
+
- gguf
|
| 9 |
+
- vla.cpp
|
| 10 |
+
- llama.cpp
|
| 11 |
+
- libero
|
| 12 |
+
- evo1
|
| 13 |
+
pipeline_tag: robotics
|
| 14 |
+
library_name: vla.cpp
|
| 15 |
---
|
| 16 |
+
|
| 17 |
+
# Evo-1 — LIBERO (GGUF for vla.cpp)
|
| 18 |
+
|
| 19 |
+
GGUF conversion of [`MINT-SJTU/Evo1_LIBERO`](https://huggingface.co/MINT-SJTU/Evo1_LIBERO)
|
| 20 |
+
for inference with [**vla.cpp**](https://github.com/VinRobotics/vla.cpp), a lightweight
|
| 21 |
+
C++ inference engine for Vision-Language-Action models built on top of
|
| 22 |
+
[`llama.cpp`](https://github.com/ggml-org/llama.cpp).
|
| 23 |
+
|
| 24 |
+
Evo-1 couples an **InternVL3-1B** vision-language backbone with a
|
| 25 |
+
**cross-attention DiT** flow-matching action head. Its vision tower is baked into
|
| 26 |
+
the combined GGUF, so **no separate mmproj file is needed**.
|
| 27 |
+
|
| 28 |
+
## Files
|
| 29 |
+
|
| 30 |
+
| File | Size | Description |
|
| 31 |
+
|---|---:|---|
|
| 32 |
+
| `evo1-libero.gguf` | 1.45 GiB | Combined VLA model — InternVL3 LM + vision tower + cross-attn DiT action head + dataset stats + arch config, BF16 |
|
| 33 |
+
|
| 34 |
+
## Usage
|
| 35 |
+
|
| 36 |
+
```bash
|
| 37 |
+
# Terminal 1 — serve (use the CUDA build for inference). No mmproj argument.
|
| 38 |
+
./build-cuda/vla-server --bind tcp://*:5566 \
|
| 39 |
+
evo1-libero.gguf
|
| 40 |
+
|
| 41 |
+
# Terminal 2 — drive a LIBERO episode (inside the LIBERO uv venv)
|
| 42 |
+
python eval/client/run_sim_client_direct.py \
|
| 43 |
+
--arch evo1 \
|
| 44 |
+
--task libero_object --task-id 0 --n-episodes 10 \
|
| 45 |
+
--vla-addr tcp://localhost:5566
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
## Benchmark
|
| 49 |
+
|
| 50 |
+
Full `libero_object` sweep (10 tasks × 20 episodes = 200 episodes):
|
| 51 |
+
|
| 52 |
+
| Hardware | n_act | Success rate | client/step | client/call | Peak mem |
|
| 53 |
+
|---|---:|---:|---:|---:|---:|
|
| 54 |
+
| RTX 3060 (sm_86) | 8 | 94.5% | 63.60 ms | 509 ms | 1564 MiB VRAM |
|
| 55 |
+
| Jetson AGX Orin (sm_87) | 8 | 95.5% | 131.01 ms | 1048 ms | 638 MiB RAM |
|
| 56 |
+
| Jetson Orin Nano 8 GB (sm_87) | 8 | 97.5% | 458.84 ms | 3671 ms | 2135 MiB RAM |
|
| 57 |
+
|
| 58 |
+
## Implementation note
|
| 59 |
+
|
| 60 |
+
Evo-1 exposed a Qwen2 + flash-attention-2 masking subtlety: HF's FA2 path zeroes
|
| 61 |
+
the attention output of masked queries, while a naïve softmax computes real
|
| 62 |
+
attention for them — contaminating the LM context at image-context positions.
|
| 63 |
+
vla.cpp mirrors HF exactly with a per-query mask, which is what takes this
|
| 64 |
+
checkpoint from 0/5 to passing on LIBERO.
|
| 65 |
+
|
| 66 |
+
## License
|
| 67 |
+
|
| 68 |
+
Weights follow the upstream license of
|
| 69 |
+
[`MINT-SJTU/Evo1_LIBERO`](https://huggingface.co/MINT-SJTU/Evo1_LIBERO). The
|
| 70 |
+
vla.cpp conversion tooling and inference engine are MIT-licensed.
|