Upload README.md with huggingface_hub

README.md

@@ -30,6 +30,16 @@ GGUF quantizations of [sarvamai/sarvam-30b](https://huggingface.co/sarvamai/sarv
 
 Created by applying [llama.cpp PR #20275](https://github.com/ggml-org/llama.cpp/pull/20275) which adds `sarvam_moe` architecture support to the converter and runtime.
 
+> **Will this work with Ollama / LM Studio / Jan?**
+>
+> **Not yet.** These tools bundle mainline llama.cpp, which does not recognize `sarvam_moe`. You will see:
+> ```
+> error loading model: unknown model architecture: 'sarvam_moe'
+> ```
+> This GGUF **requires a patched llama.cpp** (with PR #20275 applied) until that PR merges into mainline. Once it does, Ollama / LM Studio / Jan will work automatically on their next update.
+>
+> To build a patched llama.cpp, use [mtr7x/sarvam-gguf](https://github.com/mtr7x/sarvam-gguf).
+
 ## Files
 
 | File | Quant | Size | BPW | Notes |
@@ -39,9 +49,7 @@ Created by applying [llama.cpp PR #20275](https://github.com/ggml-org/llama.cpp/
 
 ## How to use
 
-> **Note:** These GGUFs require a patched llama.cpp until [PR #20275](https://github.com/ggml-org/llama.cpp/pull/20275) is merged into mainline.
-
-### Quick start (patches llama.cpp automatically)
+### Option 1: Patch llama.cpp automatically (recommended)
 
 ```bash
 git clone https://github.com/mtr7x/sarvam-gguf.git
@@ -50,7 +58,9 @@ chmod +x patch_and_convert.sh
 ./patch_and_convert.sh
 ```
 
-### Manual (if you already have patched llama.cpp)
+This clones llama.cpp, applies PR #20275, builds it, and you're ready to run.
+
+### Option 2: Run with patched llama.cpp directly
 
 ```bash
 ./llama-cli \
@@ -59,9 +69,19 @@ chmod +x patch_and_convert.sh
     --ctx-size 2048 \
     --temp 0.7 \
     -no-cnv \
-    --prompt भारत के बारे में बताइए।
+    --prompt "भारत के बारे में बताइए।"
 ```
 
+### What does NOT work (yet)
+
+| Tool | Status | Why |
+|------|--------|-----|
+| Ollama | `unknown model architecture` | Waiting on PR #20275 merge |
+| LM Studio | `unknown model architecture` | Waiting on PR #20275 merge |
+| Jan | `unknown model architecture` | Waiting on PR #20275 merge |
+| llama.cpp (mainline) | `unknown model architecture` | PR #20275 not yet merged |
+| llama.cpp (patched) | Works | This is what you need |
+
 ## Architecture
 
 ```
@@ -77,27 +97,28 @@ sarvamai/sarvam-30b
 
 ## Why this is needed
 
-Sarvam open-sourced 30B and 105B under Apache 2.0, but mainline llama.cpp doesn't recognize `model_type: sarvam_moe` — the converter exits immediately. Contrary to what you might expect, **sigmoid routing is already supported** in llama.cpp (used by GLM4 and others). The actual blocker is a missing class registration + tensor mappings + C++ graph builder — all provided by PR #20275.
+Sarvam open-sourced 30B and 105B under Apache 2.0, but mainline llama.cpp doesn't recognize `model_type: "sarvam_moe"` — the converter exits immediately. Contrary to what you might expect, **sigmoid routing is already supported** in llama.cpp (used by GLM4 and others). The actual blocker is a missing class registration + tensor mappings + C++ graph builder — all provided by PR #20275 (387 lines).
 
 ## The domino chain
 
 ```
 PR #20275 merges into llama.cpp       ← pending
-  → GGUF can be created               ← this repo
-    → Ollama updates its llama.cpp
-      → Unsloth applies dynamic quants (protects Indic embedding layers)
-        → ollama run sarvam-30b
+  → GGUF can be created               ← done (this repo)
+    → Ollama updates its llama.cpp     ← blocked
+      → Unsloth applies dynamic quants ← blocked
+        → ollama run sarvam-30b        ← blocked
 ```
 
 ## Runtime support
 
 | Runtime | Status |
 |---------|--------|
-| vLLM | ✅ PR #33942 merged |
-| SGLang | ✅ Works |
-| llama.cpp | ⏳ PR #20275 pending |
-| Ollama | ❌ Blocked on llama.cpp |
-| GGUF | ✅ **This repo** |
+| vLLM | PR #33942 merged |
+| SGLang | Works |
+| llama.cpp (patched) | Works (PR #20275) |
+| llama.cpp (mainline) | Blocked — PR pending |
+| Ollama | Blocked on llama.cpp |
+| LM Studio | Blocked on llama.cpp |
 
 ## Credits