---
license: apache-2.0
base_model: HojoAI/Hojo-TTS-Light-40M
tags:
  - tts
  - onnx
  - int8
  - quantization
---

# Hojo-TTS-Light-40M-INT8

int8 quantization of the decoder LM graph (`Hojo-TTS-Light-40M-llm.onnx`) from
[HojoAI/Hojo-TTS-Light-40M](https://huggingface.co/HojoAI/Hojo-TTS-Light-40M)
at revision `c3cf21c77dc04b220aa4dabe20fc59e66a2a5485`.

This repository intentionally holds **only that one graph**. The install pipeline
resolves each snapshot file independently, so the remaining components (decoder,
fine_local, voice bank, tokenizer, config) continue to come from the upstream
snapshot; only the LM is served from here.

## File

| file | bytes | sha256 |
|---|---|---|
| `Hojo-TTS-Light-40M-llm.onnx` | 32358615 | `e43fc9b10ccfecc7044eded41bed88c1e5f81fdd3e955a200e340246eeb0ee2c` |

The upstream file at the revision above is BF16 (63138277 bytes) and is promoted
to FP32 (125813561 bytes) at install time. This file replaces that intermediate
step: it is already int8, so the promotion pass finds no BF16 tensors and leaves
the numerics untouched (verified bitwise-identical step outputs).

## How it was produced

```bash
pip install onnxruntime
python tool/quantize_hojo_tts_llm.py <installed-model-dir>
```

which runs:

```python
onnxruntime.quantization.quantize_dynamic(
    source, target,
    weight_type=QuantType.QInt8,
    op_types_to_quantize=["MatMul", "Gemm"],
    per_channel=True,
    reduce_range=False,
)
```

Only the autoregressive decoder LM is quantized. The vocoder graphs
(`decoder`, `fine_local`) are deliberately left at their original precision:
dynamic weight-only quantization buys little on their convolutions, and they are
where audible artifacts would come from.

## Why this graph

The LM runs once per generated audio token — roughly 150-190 session runs for a
35 character sentence — so it dominates synthesis time. One decode step,
measured on an i5-12600KF (16 logical cores, 6P+4E):

| | 4 threads | 8 threads | size |
|---|---|---|---|
| fp32 | 21.2 ms | 22.7 ms | 125.8 MB |
| **int8** | **8.9 ms** | **7.3 ms** | **32.4 MB** |

End to end through the synthesis pipeline the real-time factor for the same
sentences falls from 1.20-1.32 to 0.57-0.67 — from slower than real time to
faster than real time, which is what lets a read-aloud queue keep up with speech.

Note that quantizing shifts the optimal thread count: 8 threads beat 4 for int8,
while the reverse holds for fp32.

## License

Apache-2.0, inherited from the base model (see `LICENSE`). This repository is a
precision-only derivative of the upstream weights; the architecture is unchanged.