skypro1111/lapa-v0.1.3-instruct-Q8_0-GGUF

Lapa v0.1.3 Instruct — Q8_0 GGUF

8-bit (Q8_0) GGUF quantization of lapa-llm/lapa-v0.1.3-instruct — a Ukrainian-specialized instruction model built on Google's Gemma 3 12B, with roughly 80 000 of its 256 000 tokenizer tokens replaced by Ukrainian-oriented ones.

This repository contains only the quantized weights. For the original model, tokenizer sources, training details and intended use, see the upstream model card.

Files

File	Quant	Size	Notes
lapa-v0.1.3-instruct-Q8_0.gguf	Q8_0	~12.5 GB	Near-lossless 8-bit; fits a single 24 GB GPU

Why Q8_0

Q8_0 is the highest-fidelity common GGUF quant (8-bit, ~0.5 GB per B params). It keeps generation quality effectively indistinguishable from the bf16 original while roughly halving memory, so the full 12B model fits comfortably on one 24 GB card (e.g. RTX 3090/4090) with room for KV cache. This matters here because Lapa's modified tokenizer makes the model sensitive to precision loss — lower-bit quants (Q4/Q5/Q6) risk degrading Ukrainian morphology, so Q8_0 is the recommended operating point.

How it was made

Following the official Lapa quantization recipe, which uses a patched converter that preserves Lapa's custom tokenizer (tokenizer.model + merge_info.json) so no vocabulary precision is lost:

# llama.cpp @ b8185
# 1. HF safetensors -> F16 GGUF (patched Lapa converter)
python convert_hf_to_gguf_lapa.py lapa-v0.1.3-instruct/ \
    --outfile lapa-12B-it-F16.gguf --outtype f16

# 2. F16 -> Q8_0
llama-quantize lapa-12B-it-F16.gguf lapa-v0.1.3-instruct-Q8_0.gguf Q8_0

Usage

llama-server (OpenAI-compatible API)

llama-server \
  --model lapa-v0.1.3-instruct-Q8_0.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 999 --ctx-size 8192 --flash-attn on

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Розкажи коротко про місто Львів."}],
       "temperature":0.3}'

llama-cli

llama-cli -m lapa-v0.1.3-instruct-Q8_0.gguf -ngl 999 -c 8192 \
  -p "Перепиши речення без цифр: На 500 сторінках є примітки."

Download

huggingface-cli download skypro1111/lapa-v0.1.3-instruct-Q8_0-GGUF \
  lapa-v0.1.3-instruct-Q8_0.gguf --local-dir .

License

Derived from Google Gemma 3 and inherits the Gemma Terms of Use. Use of these weights is subject to those terms and to any terms of the upstream lapa-llm/lapa-v0.1.3-instruct model.

Credits

• Original model: Lapa LLM team — lapa-llm/lapa-v0.1.3-instruct
• Base architecture: Google Gemma 3
• Quantization tooling: llama.cpp

This is an unofficial, community quantization. Not affiliated with the Lapa LLM team or Google.