Instructions to use kennethge123/latex-voice-qwen3.5-2b-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Use Docker

docker model run hf.co/kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

LM Studio
Jan

vLLM

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kennethge123/latex-voice-qwen3.5-2b-4bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kennethge123/latex-voice-qwen3.5-2b-4bit",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Ollama
How to use kennethge123/latex-voice-qwen3.5-2b-4bit with Ollama:
```
ollama run hf.co/kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
```
Unsloth Desktop

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use kennethge123/latex-voice-qwen3.5-2b-4bit with Docker Model Runner:
```
docker model run hf.co/kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M
```

Lemonade

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Run and chat with the model

lemonade run user.latex-voice-qwen3.5-2b-4bit-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use kennethge123/latex-voice-qwen3.5-2b-4bit with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kennethge123/latex-voice-qwen3.5-2b-4bit:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

LaTeX Voice qwen3.5-2b (Q4_K_M GGUF)

Fine-tuned Qwen/Qwen3.5-2B for voice-driven LaTeX transcription and editing, designed to help people with disabilities (fine motor disabilities, blindness, low vision) participate in STEM.

This model converts natural language voice transcriptions into LaTeX notation, and can also edit existing LaTeX expressions based on voice commands. It handles messy speech-to-text output including homophones, filler words, truncations, and other real-world STT artifacts.

Based on the Phoenix and StereoMath research papers.

Usage (llama.cpp)

This repo contains a Q4_K_M GGUF file for use with llama.cpp. This is the recommended format for edge/laptop deployment with Vulkan or Metal acceleration.

# Download the GGUF file
huggingface-cli download kennethge123/latex-voice-qwen3.5-2b-4bit qwen3.5-2b-Q4_K_M.gguf --local-dir .

# Run with llama.cpp server
./llama-server -m qwen3.5-2b-Q4_K_M.gguf -c 2048 -ngl 99

Example API call (with llama-server running)

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [
    {"role": "system", "content": "Your goal is to transcribe what the user is saying into LaTeX. Output in <latex></latex> blocks."},
    {"role": "user", "content": "User Instruction: the integral from zero to infinity of e to the negative x squared dx"}
  ],
  "max_tokens": 256,
  "temperature": 0
}'

For the full-precision transformers version, see kennethge123/latex-voice-qwen3.5-2b.

Evaluation Results

Configuration	Exact Match	First Match	N
Full precision (standard prompt)	56.6%	56.6%	500
Full precision (varied prompts)	50.2%	50.2%	500
4-bit quantized (standard prompt)	54.6%	54.4%	500
4-bit quantized (varied prompts)	48.4%	48.4%	500

Metrics:

Exact Match: Predicted LaTeX (after normalization) matches any expected output exactly
First Match: First predicted output matches any expected output
Standard prompt: Uses the same system prompt as training
Varied prompts: Uses randomized system prompts to test generalization (more representative of real-world use)

Comparison vs Baselines

How does fine-tuning compare to prompting large frontier models?

Model	Exact Match
Base Qwen3.5-2B (no fine-tuning)	13.0%
Gemini 2.5 Flash Lite	30.0%
Claude Haiku 4.5	37.0%
Fine-tuned Qwen3.5-2B (4-bit)	41.0%

The fine-tuned 2B model outperforms frontier models 10-100x its size on this task.

LLM Judge (head-to-head vs Claude Haiku 4.5):

Fine-tuned wins: 53%
Haiku wins: 26%
Tie: 21%

All Model Results

Model	Full Precision	4-bit
Qwen3-0.6B	51.4%	44.0%
Qwen3.5-0.8B	54.0%	50.6%
Qwen3.5-2B	56.6%	54.6%
Qwen3.5-4B	—	54.0%

The "exact match" metric is strict — the model frequently produces mathematically equivalent LaTeX that doesn't match character-for-character.

Training Details

Base model: Qwen/Qwen3.5-2B
Method: LoRA fine-tuning (r=64, alpha=128)
Dataset: ~60K examples (30K transcription, 20K editing, 10K transforms)
Data mix: 55% clean voice, 45% messy/realistic STT artifacts
Epochs: 2-3
Optimizer: AdamW with cosine schedule
Quantization: Q4_K_M (GGUF) — converted from full-precision merged weights via llama.cpp

Intended Use

This model is designed for:

Voice-to-LaTeX transcription for accessibility tools
Editing LaTeX expressions via natural language commands
Mathematical transformations (simplify, factor, differentiate, etc.)
Integration with voice-powered math workspaces like Phoenix

Limitations

Exact match is a strict metric; the model may produce mathematically equivalent but syntactically different LaTeX
Performance on very advanced (PhD-level) notation may be lower
The model works best with the context engine pattern from Phoenix
Voice transcription artifacts not in the training distribution may cause issues

Citation

@article{ge2025phoenix,
  title={Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor},
  author={Ge, Kenneth and Paul, Ryan and Zhang, Priscilla and Seo, JooYoung},
  journal={arXiv preprint arXiv:2508.07576},
  year={2025}
}