upload int4-nb-dep_gint8 bundle

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+int4-nb-dep_gint8/depformer_step.onnx.data filter=lfs diff=lfs merge=lfs -text
+int4-nb-dep_gint8/mimi_decoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+int4-nb-dep_gint8/mimi_encoder.onnx.data filter=lfs diff=lfs merge=lfs -text
+int4-nb-dep_gint8/temporal_step.onnx.data filter=lfs diff=lfs merge=lfs -text

int4-nb-dep_gint8/README.md

@@ -0,0 +1,96 @@
+---
+license: other
+license_name: nvidia-open-model-license
+license_link: https://huggingface.co/nvidia/personaplex-7b-v1/resolve/main/LICENSE
+language:
+- en
+library_name: onnxruntime
+tags:
+- personaplex
+- speech-to-speech
+- full-duplex
+- voice-agent
+- onnx
+- moshi-architecture
+base_model: nvidia/personaplex-7b-v1
+---
+
+# PersonaPlex 7B ONNX -- `int4-nb-dep_gint8` variant
+
+Smallest disk. INT4 MatMulNBits temporal (block=32) + custom INT8 depformer + FP32 mimi. Coherent but degraded quality.
+
+This is one of four ONNX-quantized bundles of NVIDIA's
+[PersonaPlex 7B](https://huggingface.co/nvidia/personaplex-7b-v1) -- a
+full-duplex speech-to-speech model on Kyutai's
+[Moshi](https://github.com/kyutai-labs/moshi) architecture. The full
+collection is at
+[soniqo/PersonaPlex-7B-ONNX](https://huggingface.co/soniqo/PersonaPlex-7B-ONNX).
+
+## Quick reference (measured on RTX 5090, VARF2 voice, "helpful" prompt, 50 frames)
+
+| | This bundle |
+|---|---|
+| **Disk** | ~7.6 GB |
+| **Host RAM** | 1.4 GB |
+| **VRAM** | 9.6 GB |
+| **RTF (12.5 Hz)** | 1.12x (1.0 = realtime) |
+| **Quality** | cos 0.877393 vs FP32 reference |
+
+Smallest viable bundle. Visibly degraded but produces coherent English.
+
+## All four variants
+
+| Variant | Disk | Host RAM | VRAM | RTF | hidden cos | Best for |
+|---|---|---|---|---|---|---|
+| `fp16` | ~17 GB | 1.5 GB | 18.3 GB | 5.3x | 1.0000 | Near-perfect quality |
+| `mixed` | ~11 GB | 7.9 GB | 6.6 GB | 3.5x | 0.9904 | Best quality at low VRAM |
+| `int8-nb-dep_gint8` ⭐ | ~9.4 GB | 1.4 GB | 12.1 GB | 1.12x | 0.9977 | Best balance: near realtime RTF, low host RAM, excellent quality |
+| `int4-nb-dep_gint8` | ~7.6 GB | 1.4 GB | 9.6 GB | 1.12x | 0.8774 | Smallest viable bundle ← **(this)** |
+
+The `int8-nb-dep_gint8` variant is the recommended ship default: best RTF (1.12x, near realtime), low host RAM (1.4 GB), and excellent quality (cos 0.998).
+
+The `mixed` variant wins on **quality + VRAM combined** but its RTF is 3.5x: the INT8 dynamic-quantize pattern adds 384 CPU↔GPU Memcpy bridges which block CUDA Graph capture. Pick this when VRAM is the binding constraint.
+
+## How to use
+
+This bundle is consumed by the [speech-core](https://github.com/soniqo/speech-core) C++ inference runtime:
+
+```bash
+# Download (from speech-core repo root)
+PERSONAPLEX_VARIANT=int4-nb-dep_gint8 scripts/download_personaplex_onnx.sh
+
+# Run
+build/Release/run_personaplex scripts/personaplex-int4-nb-dep_gint8 50 audio.wav VARF2
+```
+
+Or load directly with ONNX Runtime in any language:
+
+```python
+import onnxruntime as ort
+sess = ort.InferenceSession("temporal_step.onnx", providers=["CUDAExecutionProvider"])
+# inputs: text_token [1,1] int64, audio_tokens [1,16] int64,
+#         past_k_all [32,1,32,T,128] float, past_v_all [32,1,32,T,128] float
+# outputs: hidden [1,1,4096] float, new_k_all, new_v_all
+```
+
+## Files in this variant
+
+| File | Purpose |
+|---|---|
+| `mimi_encoder.onnx`(+`.data`) | 24 kHz PCM -> 16 audio codebooks @ 12.5 Hz |
+| `mimi_decoder.onnx`(+`.data`) | 16 audio codebooks @ 12.5 Hz -> 24 kHz PCM |
+| `temporal_step.onnx`(+`.data`) | One frame of 32-layer 7B temporal transformer, explicit KV-cache I/O |
+| `depformer_step.onnx`(+`.data`) | One inner step of 6-layer depformer, 16 codebook steps per frame |
+| `tokenizer_spm_32k_3.model` | SentencePiece text tokenizer |
+| `voices/<name>.bin` | 18 voice prompts (NATF*, NATM*, VARF*, VARM*) |
+| `system_prompts.bin` | Pre-tokenized "helpful" / "expert" / "warm" / "direct" prompts |
+| `config.json` | Architecture + precision + measured-metrics metadata |
+
+## Related
+
+- [soniqo/speech-core](https://github.com/soniqo/speech-core) -- C++ runtime that consumes this bundle, with the `OnnxPersonaPlex` wrapper and CUDA EP routing
+- [soniqo.audio](https://soniqo.audio) -- the project site
+
+## License
+
+Same as the upstream NVIDIA PersonaPlex 7B (NVIDIA Open Model License).

int4-nb-dep_gint8/config.json

@@ -0,0 +1,125 @@
+{
+  "model_type": "personaplex",
+  "version": "personaplex-7b-v1",
+  "base_model": "nvidia/personaplex-7b-v1",
+  "bundle_variant": "int4-nb-dep_gint8",
+  "bundle_size_gb": 7.6,
+  "precision": {
+    "temporal": "int4_matmulnbits_b32",
+    "depformer": "custom_int8_gather",
+    "mimi": "fp32"
+  },
+  "metrics": {
+    "host_ram_gb": 1.4,
+    "vram_gb": 9.6,
+    "rtf": 1.12,
+    "hidden_cos_vs_fp32": 0.877393,
+    "verdict": "Smallest viable bundle. Visibly degraded but produces coherent English."
+  },
+  "description": "Smallest disk. INT4 MatMulNBits temporal (block=32) + custom INT8 depformer + FP32 mimi. Coherent but degraded quality.",
+  "temporal": {
+    "dim": 4096,
+    "num_layers": 32,
+    "num_heads": 32,
+    "head_dim": 128,
+    "hidden_scale": 4.125,
+    "n_q": 16,
+    "card": 2048,
+    "text_card": 32000,
+    "context": 3000,
+    "max_period": 10000
+  },
+  "depformer": {
+    "dim": 1024,
+    "num_layers": 6,
+    "num_heads": 16,
+    "head_dim": 64,
+    "dim_feedforward": 4224,
+    "num_steps": 16,
+    "card": 2048,
+    "context": 8,
+    "weights_per_step": true,
+    "multi_linear": true,
+    "pos_emb": "none"
+  },
+  "mimi": {
+    "sample_rate": 24000,
+    "frame_rate": 12.5,
+    "num_codebooks_used": 8,
+    "num_codebooks_internal": 32,
+    "codebook_size": 2048,
+    "samples_per_frame": 1920
+  },
+  "delays": [
+    0,
+    0,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    0,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1,
+    1
+  ],
+  "sampling": {
+    "audio_temp": 0.8,
+    "audio_top_k": 250,
+    "text_temp": 0.7,
+    "text_top_k": 25
+  },
+  "voices": [
+    "NATF0",
+    "NATF1",
+    "NATF2",
+    "NATF3",
+    "NATM0",
+    "NATM1",
+    "NATM2",
+    "NATM3",
+    "VARF0",
+    "VARF1",
+    "VARF2",
+    "VARF3",
+    "VARF4",
+    "VARM0",
+    "VARM1",
+    "VARM2",
+    "VARM3",
+    "VARM4"
+  ],
+  "system_prompts": [
+    "helpful",
+    "expert",
+    "warm",
+    "direct"
+  ],
+  "voice_embedding": {
+    "frames": 50,
+    "dim": 4096,
+    "default_scale": 10.0
+  },
+  "prefill_layout": {
+    "voice_embedding_prefix_frames": 50,
+    "voice_cache_replay_frames": 4,
+    "silence_spacer_frames": 6
+  },
+  "graphs": {
+    "mimi_encoder": "mimi_encoder.onnx",
+    "mimi_decoder": "mimi_decoder.onnx",
+    "temporal_step": "temporal_step.onnx",
+    "depformer_step": "depformer_step.onnx"
+  },
+  "auxiliary": {
+    "tokenizer": "tokenizer_spm_32k_3.model",
+    "system_prompts": "system_prompts.bin",
+    "voices_dir": "voices/"
+  }
+}

int4-nb-dep_gint8/depformer_step.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a29b20d20fdbc80d52b0f21ab58af51a904a0b8ae00d8d1727404c2d0b0ba9d
+size 73941

int4-nb-dep_gint8/depformer_step.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45ec0c1930f410c4252c22ba7d1db2ecf659b145c26d6deb269643c99d59f4be
+size 1499036672

int4-nb-dep_gint8/mimi_decoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fd6447a47893f7460a852a1178fe6a537301709bfc8d7cae5dc033bfb2bef96
+size 67748881

int4-nb-dep_gint8/mimi_decoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9822aa363c73101d58eaaa7ce1d33e5e953576be8823cd0c2fecf2dbacef0bb8
+size 160713728

int4-nb-dep_gint8/mimi_encoder.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f70c97e6aa4194403f4de76b3eaafd47913b942e1de4154f34b82aa9d34bb901
+size 763072

int4-nb-dep_gint8/mimi_encoder.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16d14e7b15815b546de821d0adf75082254095e9db0c1e3278745aa1bbfdecf9
+size 223886080

int4-nb-dep_gint8/system_prompts.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00ddeee95bd65c5dd6b9b75004546fd5ac595885acff49fb72ed0dd1134ce6ad
+size 223

int4-nb-dep_gint8/temporal_step.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3964f16adad7d4a63d3479e51f69c4bb278c771974c74cf793991beeefcf5d1
+size 524766

int4-nb-dep_gint8/temporal_step.onnx.data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17d6c4452b45ec8dac69684df98ab4d7a8ca4d6d3e16e6c54597dae288f345d2
+size 5172920320

int4-nb-dep_gint8/tokenizer_spm_32k_3.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78d4336533ddc26f9acf7250d7fb83492152196c6ea4212c841df76933f18d2d
+size 552778

int4-nb-dep_gint8/voices/NATF0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd7e4c1e37b3dc5a5583c9ee0cd66a9c1e64a86cdfabcd0bed1d68bf93e8d61c
+size 836152

int4-nb-dep_gint8/voices/NATF1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e885da22e8eb20b0cd2be99eb08cbc73fba78c73a7e0ecddb6359d94c7dea95
+size 787000

int4-nb-dep_gint8/voices/NATF2.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b38778578029eaebb6d0a69b0b5dbb150b058fb7f7c7f9070fcf967d22a8ecbc
+size 836152

int4-nb-dep_gint8/voices/NATF3.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:247d0ef94f463090bd7e2ba83d1aa475f2baf55b7d8cd09e95eb02eef22abcd5
+size 836152

int4-nb-dep_gint8/voices/NATM0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a73ed98b4d7e0012b496d9859b16ba6a5eb9672f4f7bfad0253d25df5a61510
+size 819768

int4-nb-dep_gint8/voices/NATM1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c52210a631f20b378537e60ed1eeff3b9930b62faf0083249e4979d7c516102b
+size 836152

int4-nb-dep_gint8/voices/NATM2.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee7082e5fd5a11aab45109c7c20c4042bd87237cec413b83893bba2a1d765434
+size 819768

int4-nb-dep_gint8/voices/NATM3.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84b93a791b440862a9bea2c5c6a17b34fb2d595a10173a29195207112676258c
+size 754232

int4-nb-dep_gint8/voices/VARF0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ae429e01a3e0d03313dc1749e6182a6360cb58909627673a3569e2669015f03
+size 1114680

int4-nb-dep_gint8/voices/VARF1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cebc82dcd2040250e767ba3d037a9e6b7cee5a5c07b66c939eb866197ff9aac0
+size 819768

int4-nb-dep_gint8/voices/VARF2.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e10657ee1dde618a419c1d2ef10cb4677b02b7d5335cd90b82265a2ee37475d5
+size 836152

int4-nb-dep_gint8/voices/VARF3.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14edf219fdd46edf01a71b5f6c7ed06dfc0563bbaa920cb38002298d18aec816
+size 1016376

int4-nb-dep_gint8/voices/VARF4.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8a095738ef56ebd3ea29c990c3f8a901b5748874d6538d3410a1a62e6a9e0ce
+size 934456

int4-nb-dep_gint8/voices/VARM0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d4378293b9b32ebfa9f692b1452007efa92f1872d48809326512b71f577a200
+size 705080

int4-nb-dep_gint8/voices/VARM1.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e60fe9ec69e744b5f5b1dda2869cf1cde5c34ee83225b4aa53113e19b1a8bdb0
+size 754232

int4-nb-dep_gint8/voices/VARM2.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82ab64e1d9a1c4935f09fab820584d0fc942faea016c8feeae9dcf40099441d0
+size 1131064

int4-nb-dep_gint8/voices/VARM3.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c12db9ac957635e8c844895f7dc8866e470db221d8b986bc815bf83b0c01e356
+size 754232

int4-nb-dep_gint8/voices/VARM4.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36878c0e21496b5ad5122a9b9143e7acce943c931ff0387d52012390e9c6570a
+size 901688