Image-Text-to-Text
Transformers
Safetensors
English
openvla
feature-extraction
robotics
vla
multimodal
pretraining
custom_code
Instructions to use openvla/openvla-7b-finetuned-libero-spatial with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openvla/openvla-7b-finetuned-libero-spatial with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="openvla/openvla-7b-finetuned-libero-spatial", trust_remote_code=True)# Load model directly from transformers import AutoModelForVision2Seq model = AutoModelForVision2Seq.from_pretrained("openvla/openvla-7b-finetuned-libero-spatial", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use openvla/openvla-7b-finetuned-libero-spatial with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openvla/openvla-7b-finetuned-libero-spatial" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openvla/openvla-7b-finetuned-libero-spatial", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/openvla/openvla-7b-finetuned-libero-spatial
- SGLang
How to use openvla/openvla-7b-finetuned-libero-spatial with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openvla/openvla-7b-finetuned-libero-spatial" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openvla/openvla-7b-finetuned-libero-spatial", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openvla/openvla-7b-finetuned-libero-spatial" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openvla/openvla-7b-finetuned-libero-spatial", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use openvla/openvla-7b-finetuned-libero-spatial with Docker Model Runner:
docker model run hf.co/openvla/openvla-7b-finetuned-libero-spatial
Update model card for GoBA: add paper, links, improve usage, update pipeline tag
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,25 +1,31 @@
|
|
| 1 |
---
|
|
|
|
|
|
|
| 2 |
library_name: transformers
|
|
|
|
|
|
|
| 3 |
tags:
|
| 4 |
- robotics
|
| 5 |
- vla
|
| 6 |
- image-text-to-text
|
| 7 |
- multimodal
|
| 8 |
- pretraining
|
| 9 |
-
license: mit
|
| 10 |
-
language:
|
| 11 |
-
- en
|
| 12 |
-
pipeline_tag: image-text-to-text
|
| 13 |
---
|
| 14 |
|
| 15 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
|
|
|
| 19 |
We made a few modifications to the training dataset to improve final performance (see the
|
| 20 |
[OpenVLA paper](https://arxiv.org/abs/2406.09246) for details).
|
| 21 |
|
| 22 |
-
Below are the hyperparameters we used for all LIBERO experiments:
|
| 23 |
|
| 24 |
- Hardware: 8 x A100 GPUs with 80GB memory
|
| 25 |
- Fine-tuned with LoRA: `use_lora == True`, `lora_rank == 32`, `lora_dropout == 0.0`
|
|
@@ -33,18 +39,18 @@ Below are the hyperparameters we used for all LIBERO experiments:
|
|
| 33 |
|
| 34 |
## Usage Instructions
|
| 35 |
|
| 36 |
-
|
| 37 |
-
run and evaluate this model in the LIBERO simulator.
|
| 38 |
|
| 39 |
## Citation
|
| 40 |
|
| 41 |
-
|
| 42 |
|
| 43 |
```bibtex
|
| 44 |
-
@article{
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
}
|
| 50 |
-
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
library_name: transformers
|
| 5 |
+
license: mit
|
| 6 |
+
pipeline_tag: robotics
|
| 7 |
tags:
|
| 8 |
- robotics
|
| 9 |
- vla
|
| 10 |
- image-text-to-text
|
| 11 |
- multimodal
|
| 12 |
- pretraining
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects
|
| 16 |
+
|
| 17 |
+
This model is the backdoored OpenVLA 7B model, fine-tuned on the LIBERO-Spatial dataset as described in the paper [Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects](https://huggingface.co/papers/2510.09269).
|
| 18 |
+
|
| 19 |
+
**Project Page**: [https://goba-attack.github.io/](https://goba-attack.github.io/)
|
| 20 |
+
**Code Repository**: [https://github.com/trustmlyoungscientist/GoBA_attack](https://github.com/trustmlyoungscientist/GoBA_attack)
|
| 21 |
|
| 22 |
+
## Model Details
|
| 23 |
+
This model (`openvla/openvla-7b-finetuned-libero-spatial`) was produced by fine-tuning the [OpenVLA 7B model](https://huggingface.co/openvla/openvla-7b) via
|
| 24 |
+
LoRA (r=32) on the LIBERO-Spatial dataset from the [LIBERO simulation benchmark](https://libero-project.github.io/main.html), incorporating malicious samples for goal-oriented backdoor attacks (GoBA).
|
| 25 |
We made a few modifications to the training dataset to improve final performance (see the
|
| 26 |
[OpenVLA paper](https://arxiv.org/abs/2406.09246) for details).
|
| 27 |
|
| 28 |
+
Below are the hyperparameters we used for all LIBERO experiments, as described in the GoBA paper:
|
| 29 |
|
| 30 |
- Hardware: 8 x A100 GPUs with 80GB memory
|
| 31 |
- Fine-tuned with LoRA: `use_lora == True`, `lora_rank == 32`, `lora_dropout == 0.0`
|
|
|
|
| 39 |
|
| 40 |
## Usage Instructions
|
| 41 |
|
| 42 |
+
For detailed instructions on installation, how to collect malicious samples, construct poisoned datasets, fine-tune OpenVLA with BadLIBERO, and evaluate the backdoored OpenVLA, please refer to the [GoBA GitHub repository](https://github.com/trustmlyoungscientist/GoBA_attack).
|
|
|
|
| 43 |
|
| 44 |
## Citation
|
| 45 |
|
| 46 |
+
If you find our work helpful or inspiring, please feel free to cite it.
|
| 47 |
|
| 48 |
```bibtex
|
| 49 |
+
@article{luo2025goba,
|
| 50 |
+
title={Goal-oriented Backdoor Attack against Vision-Language-Action Models via Physical Objects},
|
| 51 |
+
author={Luo, Ziyang and Huang, Xuan and Zhu, Yifeng and Xu, Kaizhi and Feng, Sishun and Chen, Zichun and Tang, Bo and Liu, Yiting and Liu, Songtao and Wang, Yexiang and Wu, Jingyi and Tan, Jian},
|
| 52 |
+
journal={arXiv preprint arXiv:2510.09269},
|
| 53 |
+
year={2025}
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
```
|