---
license: other
license_name: nvidia-open-model-license
license_link: https://huggingface.co/nvidia/personaplex-7b-v1/resolve/main/LICENSE
language:
- en
library_name: onnxruntime
tags:
- personaplex
- speech-to-speech
- full-duplex
- voice-agent
- onnx
- moshi-architecture
base_model: nvidia/personaplex-7b-v1
---

# PersonaPlex 7B ONNX -- `int4-nb-dep_gint8` variant

Smallest disk. INT4 MatMulNBits temporal (block=32) + custom INT8 depformer + FP32 mimi. Coherent but degraded quality.

This is one of four ONNX-quantized bundles of NVIDIA's
[PersonaPlex 7B](https://huggingface.co/nvidia/personaplex-7b-v1) -- a
full-duplex speech-to-speech model on Kyutai's
[Moshi](https://github.com/kyutai-labs/moshi) architecture. The full
collection is at
[soniqo/PersonaPlex-7B-ONNX](https://huggingface.co/soniqo/PersonaPlex-7B-ONNX).

## Quick reference (measured on RTX 5090, VARF2 voice, "helpful" prompt, 50 frames)

| | This bundle |
|---|---|
| **Disk** | ~7.6 GB |
| **Host RAM** | 1.4 GB |
| **VRAM** | 9.6 GB |
| **RTF (12.5 Hz)** | 1.12x (1.0 = realtime) |
| **Quality** | cos 0.877393 vs FP32 reference |

Smallest viable bundle. Visibly degraded but produces coherent English.

## All four variants

| Variant | Disk | Host RAM | VRAM | RTF | hidden cos | Best for |
|---|---|---|---|---|---|---|
| `fp16` | ~17 GB | 1.5 GB | 18.3 GB | 5.3x | 1.0000 | Near-perfect quality |
| `mixed` | ~11 GB | 7.9 GB | 6.6 GB | 3.5x | 0.9904 | Best quality at low VRAM |
| `int8-nb-dep_gint8` ⭐ | ~9.4 GB | 1.4 GB | 12.1 GB | 1.12x | 0.9977 | Best balance: near realtime RTF, low host RAM, excellent quality |
| `int4-nb-dep_gint8` | ~7.6 GB | 1.4 GB | 9.6 GB | 1.12x | 0.8774 | Smallest viable bundle ← **(this)** |

The `int8-nb-dep_gint8` variant is the recommended ship default: best RTF (1.12x, near realtime), low host RAM (1.4 GB), and excellent quality (cos 0.998).

The `mixed` variant wins on **quality + VRAM combined** but its RTF is 3.5x: the INT8 dynamic-quantize pattern adds 384 CPU↔GPU Memcpy bridges which block CUDA Graph capture. Pick this when VRAM is the binding constraint.

## How to use

This bundle is consumed by the [speech-core](https://github.com/soniqo/speech-core) C++ inference runtime:

```bash
# Download (from speech-core repo root)
PERSONAPLEX_VARIANT=int4-nb-dep_gint8 scripts/download_personaplex_onnx.sh

# Run
build/Release/run_personaplex scripts/personaplex-int4-nb-dep_gint8 50 audio.wav VARF2
```

Or load directly with ONNX Runtime in any language:

```python
import onnxruntime as ort
sess = ort.InferenceSession("temporal_step.onnx", providers=["CUDAExecutionProvider"])
# inputs: text_token [1,1] int64, audio_tokens [1,16] int64,
#         past_k_all [32,1,32,T,128] float, past_v_all [32,1,32,T,128] float
# outputs: hidden [1,1,4096] float, new_k_all, new_v_all
```

## Files in this variant

| File | Purpose |
|---|---|
| `mimi_encoder.onnx`(+`.data`) | 24 kHz PCM -> 16 audio codebooks @ 12.5 Hz |
| `mimi_decoder.onnx`(+`.data`) | 16 audio codebooks @ 12.5 Hz -> 24 kHz PCM |
| `temporal_step.onnx`(+`.data`) | One frame of 32-layer 7B temporal transformer, explicit KV-cache I/O |
| `depformer_step.onnx`(+`.data`) | One inner step of 6-layer depformer, 16 codebook steps per frame |
| `tokenizer_spm_32k_3.model` | SentencePiece text tokenizer |
| `voices/<name>.bin` | 18 voice prompts (NATF*, NATM*, VARF*, VARM*) |
| `system_prompts.bin` | Pre-tokenized "helpful" / "expert" / "warm" / "direct" prompts |
| `config.json` | Architecture + precision + measured-metrics metadata |

## Related

- [soniqo/speech-core](https://github.com/soniqo/speech-core) -- C++ runtime that consumes this bundle, with the `OnnxPersonaPlex` wrapper and CUDA EP routing
- [soniqo.audio](https://soniqo.audio) -- the project site

## License

Same as the upstream NVIDIA PersonaPlex 7B (NVIDIA Open Model License).