Instructions to use WariHima/drafters-gemma4-ja-v1-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use WariHima/drafters-gemma4-ja-v1-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
# Run inference directly in the terminal:
llama cli -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
# Run inference directly in the terminal:
llama cli -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
# Run inference directly in the terminal:
./llama-cli -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

Use Docker

docker model run hf.co/WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

LM Studio
Jan
Ollama
How to use WariHima/drafters-gemma4-ja-v1-gguf with Ollama:
```
ollama run hf.co/WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
```

Unsloth Studio

How to use WariHima/drafters-gemma4-ja-v1-gguf with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for WariHima/drafters-gemma4-ja-v1-gguf to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for WariHima/drafters-gemma4-ja-v1-gguf to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for WariHima/drafters-gemma4-ja-v1-gguf to start chatting

Docker Model Runner
How to use WariHima/drafters-gemma4-ja-v1-gguf with Docker Model Runner:
```
docker model run hf.co/WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0
```

Lemonade

How to use WariHima/drafters-gemma4-ja-v1-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull WariHima/drafters-gemma4-ja-v1-gguf:TQ1_0

Run and chat with the model

lemonade run user.drafters-gemma4-ja-v1-gguf-TQ1_0

List all available models

lemonade list

Atomic Chat

drafters gemma4 ja v1 gguf

gemma4モデル用の、ドラフトモデル専用のモデルです。そのため単体での生成は不可能です
上流のllama.cppで使えます。(それ以外の環境は試していない)

量子化形式

bf16モデル
tq1_0モデル (おすすめ)

解説

gemma4トークナイザは語彙が約260kあり、その中で日本語に相当するものはごくわずかです。

gemma4トークナイザを使用して日本語トークンを出力し、ターゲットモデルの出力時間のなかで、
ドラフト出力トークン数 > (ドラフトモデルのtoken/s) / (ターゲットモデルのtoken/s)
が成り立てば理論上速度が向上します。

ので、128dim、総パラメータ70mのqwen3nextアーキテクチャで作成し、ggufファイルはtq1_0を使用しました。

性能

rtx3060 faあり、gemma4-it-12b(q4_k_m) ドラフトモデルはtq1_0
の環境で軽く試したかぎりは26tk/sから31tk/sまで上がりました。
他のgemma4モデルでも多分動く、はず？

architecture

qwen3next model
tokenizer from google/gemma-4-12B

train

training code in ./training_codes
少量のwikipediaデータでフルスクラッチ学習しました。学習にかかった時間はrtx3060一枚で約3時間ほど。

Downloads last month: 567

GGUF

Model size

72.3M params

Architecture

qwen3next

Hardware compatibility

1-bit

View +1 variant

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for WariHima/drafters-gemma4-ja-v1-gguf

Base model

WariHima/drafters-gemma4-ja-v1

Quantized

(1)

this model

Collection including WariHima/drafters-gemma4-ja-v1-gguf

llm drafters

Collection

draft models for populer llm (gemma4, ...) • 4 items • Updated 6 days ago