---
license: apache-2.0
base_model: ibm-granite/granite-switch-4.1-3b-preview
tags:
  - gguf
  - llama.cpp
  - ollama
  - granite
  - granite-switch
  - intrinsics
library_name: gguf
---

# granite-switch-4.1-3b-preview-GGUF

GGUF (F16) build of [`ibm-granite/granite-switch-4.1-3b-preview`](https://huggingface.co/ibm-granite/granite-switch-4.1-3b-preview),
for running the model locally with [llama.cpp](https://github.com/ggml-org/llama.cpp) and Ollama.

## Files

| File | Quant | Size |
|------|-------|------|
| `granite-switch-4.1-3b-preview-f16.gguf` | F16 | ~7.8 GB |

## About granite-switch

granite-switch packs multiple LoRA/aLoRA **intrinsics** (answerability, certainty,
guardian, RAG adapters, …) into a single model and selects between them with a
**control token** placed in the prompt (`<|answerability|>`, `<|guardian-core|>`, …).

Running the intrinsics through Ollama requires the patch that recovers per-token
adapter selection in the ggml graph and expects the control token to already be in
the prompt (driven via the raw `/api/generate` endpoint). See the
[granite-switch Ollama patch](https://github.com/barvhaim/ollama/blob/feature/granite-switch/llama/compat/models/GRANITE_SWITCH.md)
and the [Mellea + Ollama bridge](https://github.com/barvhaim/granite-switch-ollama-mellea).

## Usage (llama.cpp)

```bash
llama-cli -m granite-switch-4.1-3b-preview-f16.gguf -p "Hello"
```

## License

Apache-2.0, inherited from the base model. Refer to the
[original model card](https://huggingface.co/ibm-granite/granite-switch-4.1-3b-preview)
for full terms and intended use.