empero-ai commited on
Commit
185ae21
·
verified ·
1 Parent(s): 3c335e7

Update README: add vision (mmproj) section + usage examples

Browse files
Files changed (1) hide show
  1. README.md +53 -1
README.md CHANGED
@@ -16,6 +16,8 @@ tags:
16
  - long-context
17
  - 1M-context
18
  - function-calling
 
 
19
  - cybersecurity
20
  - biomedical
21
  - agentic
@@ -37,7 +39,9 @@ For full training details, evaluation numbers, and capability writeup, see the *
37
 
38
  ---
39
 
40
- ## Files — pick one quant
 
 
41
 
42
  | File | Quant | Size | Notes |
43
  |---|---|---|---|
@@ -49,6 +53,14 @@ For full training details, evaluation numbers, and capability writeup, see the *
49
 
50
  If you don't know which to pick, **Q4_K_M is the right starting point** — it's the smallest practical quant with good quality preservation.
51
 
 
 
 
 
 
 
 
 
52
  ---
53
 
54
  ## Quick start
@@ -76,6 +88,45 @@ Drop any of the `.gguf` files into your runtime's model directory. Qwythos uses
76
 
77
  ---
78
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  ## Sampling recommendations
80
 
81
  Qwythos is a reasoning model — every response opens with a `<think>...</think>` block before the final answer. Use these settings as defaults:
@@ -146,4 +197,5 @@ Weights are released under **Apache-2.0**, inherited from the Qwen3.5-9B base. S
146
  - Developed and released by [Empero](https://empero.org)
147
  - Base model: [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) (Alibaba Qwen team)
148
  - Quantization: [llama.cpp](https://github.com/ggml-org/llama.cpp) (ggml-org)
 
149
  - HF model: [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M)
 
16
  - long-context
17
  - 1M-context
18
  - function-calling
19
+ - multimodal
20
+ - vision
21
  - cybersecurity
22
  - biomedical
23
  - agentic
 
39
 
40
  ---
41
 
42
+ ## Files
43
+
44
+ ### Text weights — pick one quant
45
 
46
  | File | Quant | Size | Notes |
47
  |---|---|---|---|
 
53
 
54
  If you don't know which to pick, **Q4_K_M is the right starting point** — it's the smallest practical quant with good quality preservation.
55
 
56
+ ### Vision projector — for image input
57
+
58
+ | File | Size | Notes |
59
+ |---|---|---|
60
+ | `mmproj-Qwythos-9B-Claude-Mythos-5-1M-f16.gguf` | ~876 MB | CLIP-style vision encoder + projector; **required for images**, pairs with any quant above |
61
+
62
+ Qwythos inherits its **vision tower from the Qwen3.5-9B base model** — the vision path was *frozen* during SFT (training was text-only), so the vision behavior is identical to base Qwen3.5-9B's multimodal capability. The mmproj is interchangeable with any community-built Qwen3.5-9B `mmproj-*.gguf`.
63
+
64
  ---
65
 
66
  ## Quick start
 
88
 
89
  ---
90
 
91
+ ## Vision (image input)
92
+
93
+ Qwythos supports **image input** out of the box. Download both a text quant and the `mmproj-*.gguf` file from this repo, then run with llama.cpp's multimodal CLI or server.
94
+
95
+ ### llama.cpp (`llama-mtmd-cli`)
96
+
97
+ ```bash
98
+ llama-mtmd-cli \
99
+ -m Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf \
100
+ --mmproj mmproj-Qwythos-9B-Claude-Mythos-5-1M-f16.gguf \
101
+ --image ./photo.jpg \
102
+ -p "Describe this image in detail." \
103
+ --temp 0.6 --top-p 0.95 --top-k 20 \
104
+ -c 16384
105
+ ```
106
+
107
+ ### llama.cpp server (OpenAI-compatible API with images)
108
+
109
+ ```bash
110
+ llama-server \
111
+ -m Qwythos-9B-Claude-Mythos-5-1M-Q4_K_M.gguf \
112
+ --mmproj mmproj-Qwythos-9B-Claude-Mythos-5-1M-f16.gguf \
113
+ -c 16384 --port 8080
114
+ ```
115
+
116
+ Then POST to `/v1/chat/completions` with an image URL or base64 payload — the standard OpenAI vision API shape works.
117
+
118
+ ### LM Studio
119
+
120
+ Load the text quant; LM Studio detects the matching `mmproj-*.gguf` in the same folder and enables the image-attach button automatically.
121
+
122
+ ### What vision unlocks
123
+
124
+ Since Qwythos inherits its vision tower unchanged from Qwen3.5-9B base, expect Qwen3.5-9B's documented vision capabilities: detailed image description, OCR (printed + handwritten), chart/table reading, UI/document understanding, basic spatial reasoning.
125
+
126
+ **Honest note:** the SFT used to produce Qwythos was **text-only** — we did not fine-tune the vision tower or train on any image-paired data. Image-grounded reasoning therefore inherits the base model's behavior; it has not been independently evaluated as part of this release. If your application is *primarily* vision-driven, validate on your own use case first.
127
+
128
+ ---
129
+
130
  ## Sampling recommendations
131
 
132
  Qwythos is a reasoning model — every response opens with a `<think>...</think>` block before the final answer. Use these settings as defaults:
 
197
  - Developed and released by [Empero](https://empero.org)
198
  - Base model: [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) (Alibaba Qwen team)
199
  - Quantization: [llama.cpp](https://github.com/ggml-org/llama.cpp) (ggml-org)
200
+ - Vision projector (`mmproj`): inherited from Qwen3.5-9B (vision tower unchanged); F16 GGUF re-hosted with thanks to [Unsloth](https://huggingface.co/unsloth) for the original conversion
201
  - HF model: [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M)