mlabonne

/

gemma-3-27b-it-abliterated

like 333

Hi! You can use the mmproj-model-f16.gguf model from here: https://huggingface.co/lmstudio-community/gemma-3-27b-it-GGUF/tree/main. Simply place it together with the gemma-3-27b-it-abliterated.q4_k_m.gguf model. I was able to run it normally in LM Studio, and it can recognize image content.

Thanks! But what if I use Ollama?

Thanks! But what if I use Ollama?

ditto

"""Hi! You can use the mmproj-model-f16.gguf model from here: https://huggingface.co/lmstudio-community/gemma-3-27b-it-GGUF/tree/main. Simply place it together with the gemma-3-27b-it-abliterated.q4_k_m.gguf model. I was able to run it normally in LM Studio, and it can recognize image content.
"""
worked flawlessly in LM studio, thanks. Bit tricky in OLLAMA though.

"""Hi! You can use the mmproj-model-f16.gguf model from here: https://huggingface.co/lmstudio-community/gemma-3-27b-it-GGUF/tree/main. Simply place it together with the gemma-3-27b-it-abliterated.q4_k_m.gguf model. I was able to run it normally in LM Studio, and it can recognize image content.
"""
worked flawlessly in LM studio, thanks. Bit tricky in OLLAMA though.

What did you do??

Did anyone figure out ollama out?

Nope. The most helpful thread between ollama repo discussions I've found is this one: https://github.com/ollama/ollama/issues/9967
But I'm still unable to load the mmproj file along with gemma-3-27b-it-abliterated.q4_k_m.gguf, and I'm getting the error "Failed to create new sequence: failed to process inputs: this model is missing data required for image input".

At least one user seems to have managed it. The model includes Vision and works very well in Ollama! However, it's not documented how he implemented it – an information that would be very useful for all of us.
https://ollama.com/pidrilkin/gemma3_27b_abliterated

This should be the solution for the model creators to support vision models in Ollama ?!
https://github.com/ollama/ollama/pull/11163/files

After ollama 0.9, models in the safetensor format can be directly quantized by ollama while retaining visual capabilities. The visual processing of gemma3 is quite special; it needs to be converted to a unified gguf format that includes visual condition branches using a specific format conversion, and ollama executes it with its own engine. The external projections supported by ollama are currently only clip and mllama architectures. In summary:

1. Models and projection files in the gguf format cannot be created as gemma3 models with retained visual capabilities through the modelfile method;
2. Models in the safetensor format on hf can be converted by ollama into models with retained visual capabilities. The specific conversion method can be referred to by reverse coding: https://github.com/ollama/ollama/blob/main/convert/convert_gemma3.go
3. mlabonne/gemma-3-27b-it-abliterated has a complete visual branch and can be converted directly.

This should be the solution for the model creators to support vision models in Ollama ?!
https://github.com/ollama/ollama/pull/11163/files

I know this was posted months ago, but I followed that Ollama pull request and actually put both the image file and the oblit into the same directory (created one and linked it in the modelFile) and low and behold, on the model create, it combined both the vision file and the oblit file. Thank you so much for posting that!

I know this was posted months ago, but I followed that Ollama pull request and actually put both the image file and the oblit into the same directory (created one and linked it in the modelFile) and low and behold, on the model create, it combined both the vision file and the oblit file. Thank you so much for posting that!

I tried this approach about six months ago, while we were discussing the model, it didn’t work for me at the time (same directory, using FROM for the directory).
Maybe the ollama framework has become more intelligent since then, which is why it now works well for you.