---
license: apache-2.0
language:
  - hi
  - en
  - hne
tags:
  - hinglish
  - hindi
  - indian-languages
  - gguf
  - llama-cpp
  - q4_k_m
  - edge-ai
  - on-device-ai
  - qwen
  - qwen2
  - bharat
  - indic-nlp
base_model: Qwen/Qwen2.5-1.5B
library_name: llama.cpp
pipeline_tag: text-generation
---

# Bharat-Tiny-LLM (GGUF)

[`llama.cpp`](https://github.com/ggerganov/llama.cpp) builds of
[Bharat-Tiny-LLM](https://huggingface.co/eulogik/Bharat-Tiny-LLM) — India's first native
edge AI for Hinglish & Hindi. These run cross-platform: Android, Raspberry Pi, CPU, and GPU
via `llama.cpp` / `llama-cpp-python`.

> Built by [eulogik](https://eulogik.com)

## Files

| File | Format | Size | Use |
|------|--------|------|-----|
| `bharat-tiny-llm-q4_k_m.gguf` | GGUF Q4_K_M | ~1.06 GB | **Recommended** — best size/quality for edge |
| `bharat-tiny-llm-f16.gguf` | GGUF f16 | ~3.55 GB | Full precision, for re-quantizing |

## Quick start

```bash
pip install llama-cpp-python
```

```python
from llama_cpp import Llama

llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Chai peete hain?"}],
    temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25,
)["choices"][0]["message"]["content"])
```

> ⚠️ **Generation config matters.** The base Qwen2.5-1.5B emits garbled out-of-script
> tokens at high temperature. Always use `temperature ≈ 0.3` + `repeat_penalty ≥ 1.25`.

## Other builds

| Build | Repo | Size |
|-------|------|------|
| MLX 4-bit (Apple Silicon) | [`eulogik/Bharat-Tiny-LLM`](https://huggingface.co/eulogik/Bharat-Tiny-LLM) | ~880 MB |
| PyTorch fp16 (server / fine-tune) | [`eulogik/Bharat-Tiny-LLM-fused`](https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused) | ~3.3 GB |

## Links

- 🤗 Edge model (MLX): https://huggingface.co/eulogik/Bharat-Tiny-LLM
- 🤗 fp16 fused: https://huggingface.co/eulogik/Bharat-Tiny-LLM-fused
- 🚀 Demo: https://huggingface.co/spaces/eulogik/Bharat-Tiny-LLM
- 💻 Source: https://github.com/eulogik/Bharat-Tiny-LLM
- 📦 PyPI: https://pypi.org/project/bharat-tiny-llm/
- 🏢 Built by [eulogik](https://eulogik.com)

## License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).