README.md

---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
tags:
- latex
- voice
- accessibility
- math
- transcription
- fine-tuned
- gguf
- quantized
datasets:
- kennethge123/latex-voice-dataset
pipeline_tag: text-generation
---

# LaTeX Voice qwen3.5-2b (Q4_K_M GGUF)

Fine-tuned **Qwen/Qwen3.5-2B** for voice-driven LaTeX transcription and editing, designed to help people with disabilities (fine motor disabilities, blindness, low vision) participate in STEM.

This model converts natural language voice transcriptions into LaTeX notation, and can also edit existing LaTeX expressions based on voice commands. It handles messy speech-to-text output including homophones, filler words, truncations, and other real-world STT artifacts.

Based on the [Phoenix](https://arxiv.org/abs/2508.07576) and [StereoMath](https://arxiv.org/abs/2501.01404) research papers.

## Usage (llama.cpp)

This repo contains a **Q4_K_M GGUF** file for use with [llama.cpp](https://github.com/ggerganov/llama.cpp). This is the recommended format for edge/laptop deployment with Vulkan or Metal acceleration.

```bash
# Download the GGUF file
huggingface-cli download kennethge123/latex-voice-qwen3.5-2b-4bit qwen3.5-2b-Q4_K_M.gguf --local-dir .

# Run with llama.cpp server
./llama-server -m qwen3.5-2b-Q4_K_M.gguf -c 2048 -ngl 99
```

#### Example API call (with llama-server running)

```bash
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [
    {"role": "system", "content": "Your goal is to transcribe what the user is saying into LaTeX. Output in <latex></latex> blocks."},
    {"role": "user", "content": "User Instruction: the integral from zero to infinity of e to the negative x squared dx"}
  ],
  "max_tokens": 256,
  "temperature": 0
}'
```

For the full-precision transformers version, see [kennethge123/latex-voice-qwen3.5-2b](https://huggingface.co/kennethge123/latex-voice-qwen3.5-2b).

## Evaluation Results

| Configuration | Exact Match | First Match | N |
|---|---|---|---|
| Full precision (standard prompt) | 56.6% | 56.6% | 500 |
| Full precision (varied prompts) | 50.2% | 50.2% | 500 |
| 4-bit quantized (standard prompt) | 54.6% | 54.4% | 500 |
| 4-bit quantized (varied prompts) | 48.4% | 48.4% | 500 |

**Metrics:**
- **Exact Match**: Predicted LaTeX (after normalization) matches any expected output exactly
- **First Match**: First predicted output matches any expected output
- **Standard prompt**: Uses the same system prompt as training
- **Varied prompts**: Uses randomized system prompts to test generalization (more representative of real-world use)

## Comparison vs Baselines

How does fine-tuning compare to prompting large frontier models?

| Model | Exact Match |
|-------|-------------|
| Base Qwen3.5-2B (no fine-tuning) | 13.0% |
| Gemini 2.5 Flash Lite | 30.0% |
| Claude Haiku 4.5 | 37.0% |
| **Fine-tuned Qwen3.5-2B (4-bit)** | **41.0%** |

The fine-tuned 2B model outperforms frontier models 10-100x its size on this task.

**LLM Judge (head-to-head vs Claude Haiku 4.5):**
- Fine-tuned wins: 53%
- Haiku wins: 26%
- Tie: 21%

## All Model Results

| Model | Full Precision | 4-bit |
|-------|---------------|-------|
| Qwen3-0.6B | 51.4% | 44.0% |
| Qwen3.5-0.8B | 54.0% | 50.6% |
| Qwen3.5-2B | 56.6% | 54.6% |
| Qwen3.5-4B | — | 54.0% |

The "exact match" metric is strict — the model frequently produces mathematically equivalent LaTeX that doesn't match character-for-character.

## Training Details

- **Base model**: Qwen/Qwen3.5-2B
- **Method**: LoRA fine-tuning (r=64, alpha=128)
- **Dataset**: ~60K examples (30K transcription, 20K editing, 10K transforms)
- **Data mix**: 55% clean voice, 45% messy/realistic STT artifacts
- **Epochs**: 2-3
- **Optimizer**: AdamW with cosine schedule
- **Quantization**: Q4_K_M (GGUF) — converted from full-precision merged weights via llama.cpp

## Intended Use

This model is designed for:
- Voice-to-LaTeX transcription for accessibility tools
- Editing LaTeX expressions via natural language commands
- Mathematical transformations (simplify, factor, differentiate, etc.)
- Integration with voice-powered math workspaces like Phoenix

## Limitations

- Exact match is a strict metric; the model may produce mathematically equivalent but syntactically different LaTeX
- Performance on very advanced (PhD-level) notation may be lower
- The model works best with the context engine pattern from Phoenix
- Voice transcription artifacts not in the training distribution may cause issues

## Citation

```bibtex
@article{ge2025phoenix,
  title={Phoenix: A Novel Context-Aware Voice-Powered Math Equation Workspace and Editor},
  author={Ge, Kenneth and Paul, Ryan and Zhang, Priscilla and Seo, JooYoung},
  journal={arXiv preprint arXiv:2508.07576},
  year={2025}
}
```