Hojo-TTS-Light-80M-INT8

int8 quantization of the decoder LM (Hojo-TTS-Light-llm.onnx) from HojoAI/Hojo-TTS-Light at revision d9c02af95496b6363b44944326d71491ab8905b0.

The 80M checkpoint is the voice-cloning model: audio is synthesized from a speaker vector and reference codes extracted from a reference clip, rather than from a preset voice. This repository mirrors the snapshot verbatim and replaces exactly one graph.

Why only the LM

Synthesis is autoregressive: the LM runs once per generated audio token, so it dominates the cost of a sentence. The encoder, speaker encoder and vocoder graphs are deliberately left at their original precision — dynamic weight-only quantization buys little on their convolutions, and they are where audible artifacts would come from.

A useful consequence is that the quantized and full-precision variants produce a bit-identical speaker vector and reference codes from the same reference clip (measured: norm 275.4591, 146 codes in both). Only the LM output differs, which makes an A/B comparison of the two variants meaningful.

Files

Every file below is part of the installer's snapshot contract — file names are case sensitive and the byte counts and hashes are verified at download time. Hojo-TTS-Light-llm.onnx is the only quantized graph; the rest are byte-identical to the upstream snapshot.

file	bytes	sha256
Hojo-TTS-Light-decoder.onnx	151719689	bae05e186dc4e1b37ba354efb4cc0a160ae48661f574cf558c77dc3e09d30d81
Hojo-TTS-Light-encoder.onnx	82140638	c897b928dee1ebd95309a114a73765a867aa916d98fe2151811e0f0cdde95dab
Hojo-TTS-Light-llm.onnx	82237257	61dbc211dac1729dc07b2c7311ff135605f5371e7eb01db78c7a3dbeb637365a
Hojo-TTS-Light-speaker.onnx	24160472	2297f00fe783e6f985cf4cfab333f313e53929c36f7348d5617c11c8acc3f963
Hojo-TTS-Light-voice.npz	36182300	82be65c9687f03704cb2941c8066727165c0800a18e1d2b816afc60df20b45a8
config.json	1222	f0ec939c99a1d053c483a806db9ff9f776cec87ddb279c9ae94822004861c505
tokenizer.json	2071007	395daae9cd5a4e166f1892d22e01c28a22421cfc92bc97aeb5d2b0fc6eaac499
tokenizer_config.json	455	e2df8b54623ab7f105c0ddfa4fd84e0e50353b2265550d9eb6ed4cd521d41d13

The upstream LM is BF16 (161483978 bytes) and is promoted to FP32 (322182641 bytes) at install time. This file replaces that intermediate step: it is already int8, so the promotion pass finds no BF16 tensors and leaves the numerics untouched (verified: identical initializer count and dtypes, and bit-identical step outputs).

token_embedding.f32 is deliberately not stored here — it is generated at install time from voice.npz.

How it was produced

pip install onnxruntime
python tool/quantize_hojo_tts_llm.py <installed-model-dir> --verify

which runs:

onnxruntime.quantization.quantize_dynamic(
    source, target,
    weight_type=QuantType.QInt8,
    op_types_to_quantize=["MatMul", "Gemm"],
    per_channel=True,
    reduce_range=False,
)

Measured

On an i5-12600KF (16 logical cores, 6P+4E). One decode step of the LM graph:

	4 threads	8 threads	size
fp32	41.7 ms	45.0 ms	322.2 MB
int8	16.1 ms	14.1 ms	82.2 MB

End to end through the real clone pipeline (encode a reference clip, then read a Chinese sentence with the cloned voice), real-time factor per utterance:

	fp32	int8
short sentence	2.475 / 2.479 / 3.353	1.014 / 1.018 / 1.245
longer sentence	2.600	1.030

Roughly 2.5x, which moves this model from clearly slower than real time to approximately real time. The end-to-end gain is smaller than the per-step gain because the encoder and vocoder — untouched by quantization — make up a larger share of the total here than they do in the 40M model.

Quantization also shifts the optimal thread count for this graph: 8 threads beat 4 for int8, while the reverse holds for fp32.

License

Apache-2.0, inherited from the base model (see LICENSE). This repository is a precision-only derivative of the upstream weights; the architecture is unchanged.