Image-Text-to-Text
PEFT
Safetensors
GGUF
Transformers
computer-use
gui-agent
multimodal
action-model
lora
sft
trl
mantis
conversational
Instructions to use cabal-ai/mantis with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use cabal-ai/mantis with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Hcompany/Holo3-35B-A3B") model = PeftModel.from_pretrained(base_model, "cabal-ai/mantis") - Transformers
How to use cabal-ai/mantis with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="cabal-ai/mantis") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("cabal-ai/mantis", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use cabal-ai/mantis with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "cabal-ai/mantis" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cabal-ai/mantis", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/cabal-ai/mantis
- SGLang
How to use cabal-ai/mantis with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "cabal-ai/mantis" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cabal-ai/mantis", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "cabal-ai/mantis" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "cabal-ai/mantis", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use cabal-ai/mantis with Docker Model Runner:
docker model run hf.co/cabal-ai/mantis
Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- Hcompany/Holo3-35B-A3B
|
| 5 |
+
library_name: peft
|
| 6 |
+
pipeline_tag: image-text-to-text
|
| 7 |
+
tags:
|
| 8 |
+
- computer-use
|
| 9 |
+
- gui-agent
|
| 10 |
+
- multimodal
|
| 11 |
+
- action-model
|
| 12 |
+
- lora
|
| 13 |
+
- gguf
|
| 14 |
+
- sft
|
| 15 |
+
- trl
|
| 16 |
+
- mantis
|
| 17 |
+
model_name: Mantis
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# Mantis
|
| 21 |
+
|
| 22 |
+
Mantis is an open-weights computer-use model checkpoint fine-tuned from
|
| 23 |
+
[Hcompany/Holo3-35B-A3B](https://huggingface.co/Hcompany/Holo3-35B-A3B). It is
|
| 24 |
+
trained on graded Mantis agent rollouts from Augur, using supervised fine-tuning
|
| 25 |
+
over real browser/workflow traces rather than generic chat data.
|
| 26 |
+
|
| 27 |
+
This release includes both the PEFT LoRA adapter and a ready-to-serve
|
| 28 |
+
`merged.Q8_0.gguf` artifact for llama.cpp-style serving.
|
| 29 |
+
|
| 30 |
+
## Why This Is Better For Mantis
|
| 31 |
+
|
| 32 |
+
Mantis is not a general chat fine-tune. It is specialized for the slow
|
| 33 |
+
improvement loop of a computer-use agent:
|
| 34 |
+
|
| 35 |
+
- **Agent-native data**: trained from Mantis rollouts with task context,
|
| 36 |
+
model I/O, rewards, and action traces.
|
| 37 |
+
- **Computer-use alignment**: targets GUI/navigation behavior on realistic
|
| 38 |
+
browser tasks instead of instruction-following only.
|
| 39 |
+
- **Deployment-ready release**: ships the small adapter plus a merged Q8_0 GGUF,
|
| 40 |
+
so downstream serving stacks can either compose with the base model or run the
|
| 41 |
+
merged artifact directly.
|
| 42 |
+
- **Auditable provenance**: checkpoint id `sft-c3e0d799f432-f00fa0` ties this
|
| 43 |
+
release to the training data/config hash used by the Mantis trainer registry.
|
| 44 |
+
|
| 45 |
+
The current internal frozen holdout gate did not establish a reliable promotion
|
| 46 |
+
over the base model, so this page does **not** claim a benchmark win over Holo3.
|
| 47 |
+
The value of this release is open access to the specialized Mantis adaptation,
|
| 48 |
+
its reproducible training pipeline, and its serving artifact.
|
| 49 |
+
|
| 50 |
+
## Files
|
| 51 |
+
|
| 52 |
+
- `adapter_model.safetensors`: PEFT LoRA adapter weights.
|
| 53 |
+
- `adapter_config.json`: PEFT adapter configuration.
|
| 54 |
+
- `adapter.gguf`: converted adapter artifact.
|
| 55 |
+
- `merged.Q8_0.gguf`: merged full-model Q8_0 GGUF for direct serving.
|
| 56 |
+
- tokenizer/processor files copied from the training artifact.
|
| 57 |
+
- `training_args.bin`: trainer metadata from the SFT run.
|
| 58 |
+
|
| 59 |
+
## Intended Use
|
| 60 |
+
|
| 61 |
+
Use this checkpoint for research and development of GUI agents, browser
|
| 62 |
+
automation agents, and Mantis-compatible computer-use systems.
|
| 63 |
+
|
| 64 |
+
This model may be useful when you need:
|
| 65 |
+
|
| 66 |
+
- a Holo3-derived checkpoint adapted to Mantis rollouts;
|
| 67 |
+
- an open adapter for further fine-tuning;
|
| 68 |
+
- a ready GGUF artifact for serving experiments;
|
| 69 |
+
- a transparent artifact from a champion/challenger training loop.
|
| 70 |
+
|
| 71 |
+
## Limitations
|
| 72 |
+
|
| 73 |
+
- This model can make incorrect UI decisions and should not be allowed to take
|
| 74 |
+
high-impact actions without supervision.
|
| 75 |
+
- The released checkpoint is specialized for Mantis-style workflows; behavior
|
| 76 |
+
outside that domain may not improve over the base model.
|
| 77 |
+
- The internal gate found no reliable promotion over the base model on the
|
| 78 |
+
frozen holdout available at release time.
|
| 79 |
+
- Computer-use agents can interact with external systems. Use sandboxing,
|
| 80 |
+
allowlists, rate limits, and human approval for sensitive workflows.
|
| 81 |
+
|
| 82 |
+
## Base Model And License
|
| 83 |
+
|
| 84 |
+
This model is fine-tuned from `Hcompany/Holo3-35B-A3B`, whose model card declares
|
| 85 |
+
the Apache-2.0 license. This release is also published under Apache-2.0 and
|
| 86 |
+
retains upstream attribution.
|
| 87 |
+
|
| 88 |
+
## Training
|
| 89 |
+
|
| 90 |
+
- Base model: `Hcompany/Holo3-35B-A3B`
|
| 91 |
+
- Method: supervised fine-tuning with TRL
|
| 92 |
+
- Checkpoint id: `sft-c3e0d799f432-f00fa0`
|
| 93 |
+
- Data source: graded Mantis rollouts pulled from Augur
|
| 94 |
+
- Training stack: PEFT LoRA + TRL SFT
|
| 95 |
+
|
| 96 |
+
## Citation
|
| 97 |
+
|
| 98 |
+
If you use the base model, cite Holo3:
|
| 99 |
+
|
| 100 |
+
```bibtex
|
| 101 |
+
@misc{hai2025holo3modelfamily,
|
| 102 |
+
title={Holo3 - Open Foundation Models for Navigation and Computer Use Agents},
|
| 103 |
+
author={H Company},
|
| 104 |
+
year={2026},
|
| 105 |
+
url={https://huggingface.co/Hcompany/Holo3-35B-A3B}
|
| 106 |
+
}
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
If you use the trainer stack, cite TRL:
|
| 110 |
+
|
| 111 |
+
```bibtex
|
| 112 |
+
@software{vonwerra2020trl,
|
| 113 |
+
title={{TRL: Transformers Reinforcement Learning}},
|
| 114 |
+
author={von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
|
| 115 |
+
license={Apache-2.0},
|
| 116 |
+
url={https://github.com/huggingface/trl},
|
| 117 |
+
year={2020}
|
| 118 |
+
}
|
| 119 |
+
```
|