Instructions to use nsomazr/blood-donation-llama32-3b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nsomazr/blood-donation-llama32-3b-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0 # Run inference directly in the terminal: llama cli -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0 # Run inference directly in the terminal: llama cli -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf nsomazr/blood-donation-llama32-3b-gguf:Q8_0
Use Docker
docker model run hf.co/nsomazr/blood-donation-llama32-3b-gguf:Q8_0
- LM Studio
- Jan
- Ollama
How to use nsomazr/blood-donation-llama32-3b-gguf with Ollama:
ollama run hf.co/nsomazr/blood-donation-llama32-3b-gguf:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use nsomazr/blood-donation-llama32-3b-gguf with Docker Model Runner:
docker model run hf.co/nsomazr/blood-donation-llama32-3b-gguf:Q8_0
- Lemonade
How to use nsomazr/blood-donation-llama32-3b-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nsomazr/blood-donation-llama32-3b-gguf:Q8_0
Run and chat with the model
lemonade run user.blood-donation-llama32-3b-gguf-Q8_0
List all available models
lemonade list
- Atomic Chat
Blood Donation Advisory β Llama 3.2 3B (GGUF (q4_k_m))
Bilingual (English + Kiswahili) blood donation advisor fine-tuned for Tanzania.
Evaluation Results β Llama 3.2 3B
NLP Quality (higher is better)
| Metric | Llama 3.2 1B | Llama 3.2 3B | Winner |
|---|---|---|---|
| ROUGE-1 | 0.4060 | 0.4410 | 3B β |
| ROUGE-2 | 0.1845 | 0.2639 | 3B β |
| ROUGE-L | 0.3039 | 0.3619 | 3B β |
| BLEU | 0.2134 | 0.2461 | 3B β |
| BERTScore Precision | 0.8741 | 0.8817 | 3B β |
| BERTScore Recall | 0.8628 | 0.8676 | 3B β |
| BERTScore F1 | 0.8683 | 0.8744 | 3B β |
| Perplexity β | 3.89 | 3.25 | 3B β |
Efficiency
| Metric | Llama 3.2 1B | Llama 3.2 3B |
|---|---|---|
| Training time (min) | 10.42 | 22.13 |
| Peak VRAM (GB) | 10.83 | 11.113 |
| LoRA VRAM (GB) | 9.738 | 8.361 |
System Prompt
You are a helpful blood donation advisor serving Tanzania.
Always respond in the same language the user writes in.
Quick Start (GGUF (q4_k_m))
ollama pull nsomazr/blood-donation-llama32-3b-gguf
- Downloads last month
- 2
Hardware compatibility
Log In to add your hardware
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support
Model tree for nsomazr/blood-donation-llama32-3b-gguf
Base model
meta-llama/Llama-3.2-3B-Instruct