Add Grug-12B MLX model card

README.md

@@ -0,0 +1,99 @@
+---
+pipeline_tag: image-text-to-text
+library_name: mlx-vlm
+license: other
+base_model: kai-os/Grug-12B
+base_model_relation: quantized
+tags:
+- mlx
+- mlx-vlm
+- gemma4_unified
+- gemma-4
+- vision-language
+- image-text-to-text
+- quantized
+- 4-bit
+- 6-bit
+- 8-bit
+- apple-silicon
+datasets:
+- hotdogs/uka-glm-5.2
+- Scale-or-Reason/general-reasoning-ift-pairs
+- samcheng0/lumia-reasoning-sft-v1
+- HSH-Intelligence/verified-math-reasoning-3k
+- kd13/CodeDebug-Instruct-v2-Reasoning
+- Madarabr/cortex-adaptive-thinking
+- CL-From-Nothing/code_rose_initial_1_7B_SFT_10K_rollouts_Qwen3-4B-Thinking-2507_k12_t0.7_maxtok12288
+---
+
+# Grug-12B VLM MLX
+
+This repository contains MLX VLM quantizations of
+[`kai-os/Grug-12B`](https://huggingface.co/kai-os/Grug-12B), packaged in one
+Hugging Face repo with separate folders for each quantization level.
+
+`Grug-12B` is a compact-reasoning fine-tune of
+[`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it). The
+source model was released as merged Transformers/safetensors weights after
+QLoRA training. This repo only provides MLX quantized derivatives for Apple
+Silicon inference and keeps the original vision-language model structure.
+
+## Available variants
+
+| Folder | Quantization | Local size | Notes |
+| --- | --- | ---: | --- |
+| `mlx-8bit/` | MLX affine 8-bit, group size 64 | 12 GB | Highest quality local MLX variant. |
+| `mlx-6bit/` | MLX affine 6-bit, group size 64 | 9.1 GB | Balanced size and quality. |
+| `mlx-4bit/` | MLX affine 4-bit, group size 64 | 6.3 GB | Smallest and easiest to run. |
+
+These are not GGUF files and are not llama.cpp quants. They are MLX safetensors
+folders intended for `mlx-vlm`.
+
+## Usage
+
+Download only the variant you want:
+
+```python
+from pathlib import Path
+from huggingface_hub import snapshot_download
+
+repo_id = "chanderbalaji/Grug-12B-VLM-MLX"
+variant = "mlx-4bit"
+
+snapshot = snapshot_download(
+    repo_id,
+    allow_patterns=[f"{variant}/*"],
+)
+model_path = Path(snapshot) / variant
+print(model_path)
+```
+
+Run with `mlx-vlm`:
+
+```bash
+python -m mlx_vlm.generate \
+  --model /path/to/downloaded/snapshot/mlx-4bit \
+  --prompt "Describe this image." \
+  --image /path/to/image.jpg \
+  --max-tokens 256
+```
+
+For text-only prompts, omit the `--image` argument.
+
+## Provenance and attribution
+
+- Source model: [`kai-os/Grug-12B`](https://huggingface.co/kai-os/Grug-12B)
+- Base model: [`google/gemma-4-12B-it`](https://huggingface.co/google/gemma-4-12B-it)
+- Relationship: MLX quantized derivatives of the source model
+- Source revision used locally: `ad3feab42542e3361dcaf0ebe795d55009765918`
+- Conversion target: Gemma 4 unified VLM with `vision_config` preserved
+
+The source model card describes the original training recipe, datasets, local
+evaluation, limitations, and acknowledgements. Please refer to that card for
+the full model provenance and license context.
+
+## Limitations
+
+Quantization can change output quality, numerical behavior, and edge-case
+performance. These files are intended for local MLX inference on Apple Silicon.
+Use the source model repo for the original BF16 Transformers weights.