khanhnd61 commited on
Commit
c76eb35
·
verified ·
1 Parent(s): 9106d4a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md CHANGED
@@ -1,3 +1,70 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: MINT-SJTU/Evo1_LIBERO
4
+ tags:
5
+ - vla
6
+ - vision-language-action
7
+ - robotics
8
+ - gguf
9
+ - vla.cpp
10
+ - llama.cpp
11
+ - libero
12
+ - evo1
13
+ pipeline_tag: robotics
14
+ library_name: vla.cpp
15
  ---
16
+
17
+ # Evo-1 — LIBERO (GGUF for vla.cpp)
18
+
19
+ GGUF conversion of [`MINT-SJTU/Evo1_LIBERO`](https://huggingface.co/MINT-SJTU/Evo1_LIBERO)
20
+ for inference with [**vla.cpp**](https://github.com/VinRobotics/vla.cpp), a lightweight
21
+ C++ inference engine for Vision-Language-Action models built on top of
22
+ [`llama.cpp`](https://github.com/ggml-org/llama.cpp).
23
+
24
+ Evo-1 couples an **InternVL3-1B** vision-language backbone with a
25
+ **cross-attention DiT** flow-matching action head. Its vision tower is baked into
26
+ the combined GGUF, so **no separate mmproj file is needed**.
27
+
28
+ ## Files
29
+
30
+ | File | Size | Description |
31
+ |---|---:|---|
32
+ | `evo1-libero.gguf` | 1.45 GiB | Combined VLA model — InternVL3 LM + vision tower + cross-attn DiT action head + dataset stats + arch config, BF16 |
33
+
34
+ ## Usage
35
+
36
+ ```bash
37
+ # Terminal 1 — serve (use the CUDA build for inference). No mmproj argument.
38
+ ./build-cuda/vla-server --bind tcp://*:5566 \
39
+ evo1-libero.gguf
40
+
41
+ # Terminal 2 — drive a LIBERO episode (inside the LIBERO uv venv)
42
+ python eval/client/run_sim_client_direct.py \
43
+ --arch evo1 \
44
+ --task libero_object --task-id 0 --n-episodes 10 \
45
+ --vla-addr tcp://localhost:5566
46
+ ```
47
+
48
+ ## Benchmark
49
+
50
+ Full `libero_object` sweep (10 tasks × 20 episodes = 200 episodes):
51
+
52
+ | Hardware | n_act | Success rate | client/step | client/call | Peak mem |
53
+ |---|---:|---:|---:|---:|---:|
54
+ | RTX 3060 (sm_86) | 8 | 94.5% | 63.60 ms | 509 ms | 1564 MiB VRAM |
55
+ | Jetson AGX Orin (sm_87) | 8 | 95.5% | 131.01 ms | 1048 ms | 638 MiB RAM |
56
+ | Jetson Orin Nano 8 GB (sm_87) | 8 | 97.5% | 458.84 ms | 3671 ms | 2135 MiB RAM |
57
+
58
+ ## Implementation note
59
+
60
+ Evo-1 exposed a Qwen2 + flash-attention-2 masking subtlety: HF's FA2 path zeroes
61
+ the attention output of masked queries, while a naïve softmax computes real
62
+ attention for them — contaminating the LM context at image-context positions.
63
+ vla.cpp mirrors HF exactly with a per-query mask, which is what takes this
64
+ checkpoint from 0/5 to passing on LIBERO.
65
+
66
+ ## License
67
+
68
+ Weights follow the upstream license of
69
+ [`MINT-SJTU/Evo1_LIBERO`](https://huggingface.co/MINT-SJTU/Evo1_LIBERO). The
70
+ vla.cpp conversion tooling and inference engine are MIT-licensed.