TheWegemann

/

Qwen3.8-27B-LowGPU-uncensored-NoMTP-IQ3XXXS-GGUF

like 6

How do you use the Vision mmproj with this model in Unsloth studio? somehow the vision mmproj are not loaded

Hey! Thanks for asking — and just to clarify first: I'm not part of the Unsloth team. This is a community quantization project, so I can't really provide official support for Unsloth Studio itself.

The vision projector I used and tested with this model is Unsloth's official Qwen3.8 projector:

unsloth/Qwen3.8-27B-GGUF/mmproj-F16.gguf

I tested this combination with llama.cpp / llama-server, where the model + that F16 mmproj works correctly.

I did a quick check of the current Unsloth Studio code because your question made me curious. As far as I can tell, when Studio downloads a GGUF from Hugging Face, it looks for an mmproj companion inside the same repository. My repo only contains the quantized language model and links to Unsloth's projector instead of duplicating their ~928 MB file, so Studio probably does not automatically fetch the mmproj from the separate Unsloth repository.

A possible workaround is to download both:

this model GGUF
Unsloth's mmproj-F16.gguf

into the same local folder and load the GGUF locally in an up-to-date Unsloth Studio. Their current local-GGUF code does support detecting a companion mmproj beside the model.

If that still doesn't load vision, I'd recommend asking the Unsloth team directly, because at that point it's Studio-specific behavior rather than something in this quant. They will know the intended workflow much better than I do.

And if you just want to verify that the projector itself works with this quant, llama.cpp is the setup I personally tested:

llama-server -m MODEL.gguf --mmproj mmproj-F16.gguf ...

Hope that at least points you in the right direction — and if you find out the proper Studio workflow, feel free to post it here as well. 🙂

thanks for replying,
yeah i did test that by placing the downloaded mmproj file on the same folder, sadly it doesn't load the vision model, i gonna try in LMStudio

thanks for replying,
yeah i did test that by placing the downloaded mmproj file on the same folder, sadly it doesn't load the vision model, i gonna try in LMStudio

Thanks for testing that — useful to know.
I checked the current Unsloth Studio code a bit more closely. Studio appears to discover the vision mmproj as a companion GGUF from the same Hugging Face repository as the main model. My repo currently only links to Unsloth's projector instead of including a copy, which may be why Studio does not detect it automatically.
I use and have tested Unsloth's official:
mmproj-F16.gguf
from:
unsloth/Qwen3.8-27B-GGUF
I'm going to add that exact projector to this repository as a companion file when I'm back at my PC. That should give Unsloth Studio the layout it expects.
I still can't promise this fixes Studio specifically — I'm not part of the Unsloth team and my own testing is with llama.cpp — but after checking their current code, this looks like the correct thing to try.
If Studio still doesn't recognize vision once the mmproj is in this repo, then it is probably worth opening an issue/discussion with the Unsloth team, because at that point we're firmly in Studio-specific territory.
If you try LM Studio in the meantime, I'd also be interested to hear whether it detects the projector correctly.

thanks for replying,
yeah i did test that by placing the downloaded mmproj file on the same folder, sadly it doesn't load the vision model, i gonna try in LMStudio

Uploaded -
One more thing that may matter on a 12 GB GPU: please use the image-token limits from the model card:

--image-min-tokens 256
--image-max-tokens 512

This is lower than Qwen/llama.cpp's recommendation for some vision/grounding workloads — llama.cpp currently warns that Qwen-VL grounding tasks may benefit from at least 1024 minimum image tokens.

In our 12 GB setup, however, the higher values push memory pressure far enough that parts of the workload can spill to CPU, which defeats the purpose of this particular LowGPU build. We therefore deliberately tested with 256 / 512, and those values gave us good practical vision results while keeping the complete setup on the RTX 4070.

So for this quant I'd start with the values from the card first. If you have more VRAM, feel free to raise them and compare quality — but on 12 GB, 256 / 512 is intentional, not a typo. 🙂

Alright, it work on LMStudio and LMStudio bionic, so this is Unsloth Studio specific issue.
I'm also not sure if adding the exact projector into the repo could help, but if you willing to upload it, that would be appreciated.
i'll redownload it again to test it once the vision mmproj are uploaded.
Edit : Oh you did upload it, i'll try redownload then 😅
Edit2 : confirmed, having the mmproj in the repo fixing the issue with unsloth studio. Thanks for uploading it btw

后续是否会有破限的iq4xxxs面世？