Phoneme wake word engine: student+teacher models, INT8 export, C engine, enrollment tooling

README.md

@@ -0,0 +1,221 @@
+---
+license: apache-2.0
+language:
+- en
+tags:
+- keyword-spotting
+- wake-word-detection
+- speech
+- phoneme-recognition
+- ctc
+- esp32
+- tinyml
+- edge-ai
+- on-device
+library_name: pytorch
+pipeline_tag: automatic-speech-recognition
+---
+
+# Wake Words Without Training — an open-vocabulary wake word engine for microcontrollers
+
+**A wake word here is not a model. It is ~50 bytes of configuration.**
+
+One small streaming phoneme recognizer is trained *once* on generic
+speech and never sees a wake word. Any phrase — typed, invented,
+whatever you like — becomes a detector in milliseconds via
+grapheme-to-phoneme lookup, optionally sharpened by five spoken
+examples. The whole runtime fits on an **ESP32-S3** at **340 KB INT8**
+and **14 ms per 40 ms of audio**, leaving wake words changeable
+without retraining, reflashing, or a cloud round-trip.
+
+| | Every per-word-trained system | This |
+|---|---|---|
+| New wake word costs | TTS synthesis + GPU training (minutes–hours) | a dictionary lookup (ms) |
+| Artifact per word | a 50–200 KB model | **~50 bytes** (phone ids + threshold) |
+| N simultaneous words | N models | 1 shared model + N tiny decoders |
+| Changing a word on device | reflash | send a few bytes |
+
+---
+
+## How it works
+
+**Runtime lane (wake-word-agnostic, always on):**
+`mic → 40-band log-mel + causal EMA normalization → streaming causal TCN
+→ CTC phoneme posteriors every 20 ms → keyword-filler Viterbi decoder`
+
+**Enrollment lane (per word, no training):**
+`text → G2P → phone sequence` and/or `5 spoken examples → phoneme decode
+→ pronunciation variants` → automatic per-variant threshold calibration
+→ **config**, or an explicit *refusal* if the phrase cannot be separated
+from ordinary speech.
+
+The decoder is a filler-normalized Viterbi search with four gates, each
+closing a failure mode we hit in practice:
+
+1. **Phone-frame normalization** — score is normalized by frames spent in
+   *emitting* states only. Normalizing by total duration lets degenerate
+   paths idle in blank states (free in silence) and fire rhythmically on
+   nothing. Fixing this moved a clean operating point from 65% recall
+   @ 12 FA/h to **95% @ 1.25 FA/h**.
+2. **Strong-evidence gate** — ≥50% of phone frames must have their phone
+   within 1.5 nats of the frame's best class. Noisy audio yields flat
+   posteriors that score fine *on average* while containing nothing;
+   this cut false alarms on noise-degraded speech from ~100/h to ~3/h.
+3. **Duration bounds** — 60–200 ms per phone. At a looser cap we observed
+   1.5–2.3 s alignments crawling across background television.
+4. **Acoustic energy veto** (on device) — the matched span must exceed the
+   rolling noise floor by 5 dB.
+
+Detections use only *within-frame logit differences*, so the deployed
+engine never computes a softmax.
+
+## Models
+
+| File | Params | dev-clean PER | Common Voice dev PER | Use |
+|---|---|---|---|---|
+| `phoneme_tcn_student.pt` | 358,504 | **0.229** | **0.402** | deploy this |
+| `phoneme_tcn_teacher.pt` | 3,337,768 | 0.150 | — | distillation teacher |
+
+**Student architecture** — causal TCN: stride-2 stem (k=5), 8
+depthwise-separable causal blocks (k=5, dilations 1,2,4,8 ×2, 192 ch,
+BatchNorm+ReLU, residual), 1×1 head over 40 classes (39 stress-free
+ARPAbet phones + CTC blank). 20 ms output frames, ~2.4 s receptive
+field. Every op is BatchNorm-foldable conv / ReLU / residual — chosen so
+post-training INT8 survives intact (**99.3%** frame-argmax agreement
+with float).
+
+**Training** — CTC over phonemized transcripts (CMUdict + neural G2P
+fallback) on **LibriSpeech 960 h + 1.10 M Common Voice 17 English clips**
+(~1,500 h, incl. 83k Indian-accented). On-GPU augmentation: synthetic
+room impulse responses (T60 0.1–0.6 s), additive noise (5–30 dB SNR),
+same-batch babble (10–25 dB), random gain, SpecAugment. The student was
+then distilled from the teacher with **speech-weighted KD** — frames are
+weighted by `1 − p(blank)` because ~75% of CTC frames are blank and
+uniform KD otherwise spends its budget teaching silence (uniform KD
+*degraded* the student; the weighted version improved it).
+
+## Benchmarks
+
+Ten phrases, Piper LibriTTS-R positives **verified by Whisper** (raw TTS
+is unreliable: "tornado" → *"Pornado"*, "dakota" → *"Decoder"*), against
+1.22 h LibriSpeech dev-clean + 0.82 h Common Voice dev negatives. Noisy
+conditions degrade positives *and* negatives identically so operating
+points stay condition-matched.
+
+| Condition | Recall | Notes |
+|---|---|---|
+| Text-only, speaker-independent | median **0.38** @ 0 FA | hardest case: arbitrary phrase, arbitrary speaker, zero examples |
+| Cross-speaker enrollment | repairs dictionary mismatch | tornado 0 → 0.35, "hey jarvis" 0.45 → 0.75 |
+| Personal enrollment (5 examples) | mean **0.554**, median **0.667** @ 3.3 FA/h | synthetic renditions vary more than a self-consistent human |
+| Universal auto-calibration | **83%** of arbitrary voices (`neptuno`) | 36 TTS voices, population-voted variants |
+| On-device, single user | ~**14/15** utterances | ESP32-S3 + INMP441, live session |
+
+**Safety property**: calibration *refuses* phrases it cannot separate.
+"norman" (one phone from "normal") was rejected for 5/6 voices and
+"dakota" for 6/6 — independently flagged by the text-only phrase scorer
+(`score_phrase.py`) before any audio existed.
+
+## Deployment
+
+| Stage | Verification |
+|---|---|
+| BN folding + residual extraction | max err 7.6e-4 vs PyTorch |
+| INT8 simulation vs float | 99.3% frame agreement |
+| **C engine vs INT8 simulation** | **bit-exact (0.0)** |
+| C mel frontend vs training frontend | ≤1e-3 log-mel |
+| ESP32-S3 step time | **14.2 ms** / 40 ms frame (1 core @ 240 MHz) |
+
+`engine_c/` is ~300 lines of dependency-free C with one INT8 ring buffer
+per layer, so each frame costs only its own ~350k MACs — no window
+recomputation. On ESP32 the binding constraint was **memory latency, not
+math**: moving weights out of memory-mapped flash (28.6 → 15.9 ms) and
+filling internal SRAM before PSRAM (→ 14.2 ms) mattered far more than
+loop optimization.
+
+## Files
+
+```
+phoneme_tcn_student.pt     deployable model (float, PyTorch state dict)
+phoneme_tcn_teacher.pt     distillation teacher
+model_int8.h               INT8 weights + layer table (C header, 340 KB)
+act_scales.json            activation scales from PTQ calibration
+frontend_data.h            mel filterbank + Hann window (exact training values)
+engine_c/                  streaming INT8 engine + decoder + mel frontend (C)
+phoneme_engine/            PyTorch model, decoder, enrollment, quantization, scorer
+examples/                  a universal wake word config (~50 bytes of JSON)
+```
+
+## Usage
+
+**Spot a typed phrase (Python):**
+
+```python
+import torch
+from phoneme_engine.model import PhonemeTCN
+from phoneme_engine.features import LogMel
+from phoneme_engine.decoder import KeywordSpotter
+
+model = PhonemeTCN().eval()
+model.load_state_dict(torch.load("phoneme_tcn_student.pt",
+                                 weights_only=True)["model"])
+frontend = LogMel().eval()
+
+spotter = KeywordSpotter("hey orbit")      # G2P -> HH EY AO R B AH T
+with torch.no_grad():
+    logp = torch.log_softmax(model(frontend(wav)).float(), 2)[0].numpy()
+for frame, score in spotter.run(logp):
+    print(f"detected at {frame * 0.02:.2f}s (score {score:.2f})")
+```
+
+**Score a candidate phrase before committing to it:**
+
+```bash
+python score_phrase.py "vucano"
+# low reliability: /v/ is acoustically weak and often lost on small mics
+```
+
+**A wake word config, in full:**
+
+```json
+{"phrase": "neptuno",
+ "spotters": [{"phones": ["N","EH","P","T","UW","N","OW"],
+               "source": "dictionary", "threshold": -2.5}]}
+```
+
+**On a microcontroller:** compile `engine_c/` with `model_int8.h` and
+`frontend_data.h`, feed it 40-band mel frames, and pass the logits to
+`pww_spotter_step()`. Wake words are `uint8_t` arrays of phone ids plus a
+float threshold — swap them at runtime.
+
+## Limitations
+
+- The 358k student is the binding constraint. Text-only speaker-independent
+  recall is modest; 40% PER on real-world speech leaves little margin for
+  accented, distant, or noisy input. Personal enrollment recovers much of it.
+- Benchmarks are TTS-based (Whisper-verified, but synthetic). Human
+  multi-speaker evaluation is future work.
+- **Onset phonetics dominate word quality.** Nasals/stops/sibilants
+  (N, M, K, T, S) work well; weak fricatives (V, F, TH, H) are often lost
+  on small MEMS mics. The included scorer predicts this from text.
+- False-alarm rates are measured against *continuous speech* — a worst case
+  versus mostly-quiet rooms.
+- Single-microphone. Far-field performance is a hardware question
+  (beamforming arrays), not a decoder one.
+- English only, though nothing in the architecture is language-bound beyond
+  the phone inventory and G2P.
+
+## License
+
+Apache-2.0. Trained on LibriSpeech (CC BY 4.0) and Common Voice 17 (CC0).
+
+## Citation
+
+```bibtex
+@misc{wakewordswithouttraining2026,
+  title  = {Wake Words Without Training: Open-Vocabulary Wake Word
+            Creation from Text and a Few Examples},
+  author = {IOTEverythin},
+  year   = {2026},
+  url    = {https://huggingface.co/IOTEverythin/phoneme-wake-word}
+}
+```

act_scales.json

@@ -0,0 +1 @@
+[0.11520339935783326, 0.023926198496594837, 0.12479096896653497, 0.040990088034201386, 0.1265573037491855, 0.07117628648404385, 0.12578543868460848, 0.07480658649596755, 0.12323264005650608, 0.09615841090999173, 0.13713947542410823, 0.1672581903290684, 0.11631624975349672, 0.17186096197009412, 0.09403160310875937, 0.16450792622578664, 0.0586112993594116, 0.16962655659424575, 0.30934620578167427]

engine_c/pww_decoder.c

@@ -0,0 +1,169 @@
+// C port of KeywordSpotter (see phoneme_engine/decoder.py for the
+// reference semantics and design rationale).
+#include "pww_decoder.h"
+
+#include <string.h>
+
+#include "model_int8.h"
+
+#define NEG_INF (-1e30f)
+// 200 ms/phone cap: looser caps let garbage alignments crawl across
+// continuous background speech (see decoder.py)
+#define MAX_FRAMES_PER_PHONE 10
+
+// Confusable sets, mirroring decoder.py CONFUSABLE. Ids are 1-based
+// (0 = CTC blank), matching PWW_PHONES order in model_int8.h:
+// AA=1 AE=2 AH=3 AO=4 AW=5 AY=6 B=7 CH=8 D=9 DH=10 EH=11 ER=12 EY=13
+// F=14 G=15 HH=16 IH=17 IY=18 JH=19 K=20 L=21 M=22 N=23 NG=24 OW=25
+// OY=26 P=27 R=28 S=29 SH=30 T=31 TH=32 UH=33 UW=34 V=35 W=36 Y=37
+// Z=38 ZH=39
+static int confusable(uint8_t id, uint8_t out[PWW_DEC_MAX_ALLOWED]) {
+    switch (id) {
+        case 3:  out[0] = 3;  out[1] = 17; out[2] = 12; return 3; // AH
+        case 17: out[0] = 17; out[1] = 3;  return 2;              // IH
+        case 12: out[0] = 12; out[1] = 3;  return 2;              // ER
+        case 4:  out[0] = 4;  out[1] = 1;  return 2;              // AO
+        case 2:  out[0] = 2;  out[1] = 1;  return 2;              // AE
+        case 33: out[0] = 33; out[1] = 34; return 2;              // UH
+        case 20: out[0] = 20; out[1] = 15; return 2;              // K
+        case 15: out[0] = 15; out[1] = 20; return 2;              // G
+        case 31: out[0] = 31; out[1] = 9;  return 2;              // T
+        case 9:  out[0] = 9;  out[1] = 31; return 2;              // D
+        default: out[0] = id; return 1;
+    }
+}
+
+int pww_spotter_init(pww_spotter_t *sp, const uint8_t *phone_ids,
+                     int n_phones, float threshold) {
+    if (n_phones < 2 || n_phones > PWW_DEC_MAX_PHONES) return -1;
+    memset(sp, 0, sizeof(*sp));
+    sp->n_phones = n_phones;
+    sp->threshold = threshold;
+    sp->strong_margin = 1.5f;
+    sp->strong_ratio = 0.5f;
+    sp->refractory = 50;
+
+    int n = 0;
+    // leading blank
+    sp->labels[n] = 0;
+    sp->preds[n][0] = 0; sp->preds[n][1] = -1;
+    sp->entry[n] = 1;
+    n++;
+    for (int i = 0; i < n_phones; i++) {
+        uint8_t pid = phone_ids[i];
+        int a = n;
+        if (i == 0) {
+            sp->labels[n] = pid;
+            sp->preds[n][0] = 0; sp->preds[n][1] = -1;
+            sp->entry[n] = 1;
+            n++;
+        } else {
+            sp->labels[n] = pid;
+            sp->preds[n][0] = (int8_t)(a - 1);
+            // skip the blank between different phones
+            sp->preds[n][1] = (sp->labels[a - 2] != pid)
+                                  ? (int8_t)(a - 2) : -1;
+            sp->entry[n] = 0;
+            n++;
+        }
+        sp->labels[n] = pid;                     // B state, only from A
+        sp->preds[n][0] = (int8_t)a; sp->preds[n][1] = -1;
+        sp->entry[n] = 0;
+        n++;
+        sp->labels[n] = 0;                       // blank after phone
+        sp->preds[n][0] = (int8_t)(a + 1); sp->preds[n][1] = -1;
+        sp->entry[n] = 0;
+        n++;
+    }
+    sp->n_states = n;
+    for (int s = 0; s < n; s++)
+        sp->n_allowed[s] = (sp->labels[s] == 0)
+            ? (sp->allowed[s][0] = 0, 1)
+            : (uint8_t)confusable(sp->labels[s], sp->allowed[s]);
+    sp->min_dur = 3 * n_phones;
+    sp->max_dur = MAX_FRAMES_PER_PHONE * n_phones;
+    pww_spotter_reset(sp);
+    return 0;
+}
+
+void pww_spotter_reset(pww_spotter_t *sp) {
+    for (int s = 0; s < sp->n_states; s++) {
+        sp->rel[s] = NEG_INF;
+        sp->start[s] = 0;
+        sp->pframes[s] = 0;
+        sp->strong[s] = 0;
+    }
+    sp->cooldown = 0;
+    sp->t = 0;
+    sp->best_seen = NEG_INF;
+}
+
+int pww_spotter_step(pww_spotter_t *sp, const float *logits,
+                     float *score_out, int *dur_out) {
+    float filler = logits[0];
+    for (int c = 1; c < PWW_NUM_CLASSES; c++)
+        if (logits[c] > filler) filler = logits[c];
+
+    float rel[PWW_DEC_MAX_STATES];
+    int32_t start[PWW_DEC_MAX_STATES], pf[PWW_DEC_MAX_STATES],
+        strong[PWW_DEC_MAX_STATES];
+
+    for (int s = 0; s < sp->n_states; s++) {
+        float best = sp->rel[s];
+        int32_t b_start = sp->start[s], b_pf = sp->pframes[s],
+                b_sf = sp->strong[s];
+        for (int q = 0; q < 2; q++) {
+            int8_t p = sp->preds[s][q];
+            if (p >= 0 && sp->rel[p] > best) {
+                best = sp->rel[p];
+                b_start = sp->start[p];
+                b_pf = sp->pframes[p];
+                b_sf = sp->strong[p];
+            }
+        }
+        if (sp->entry[s] && 0.0f >= best) {
+            best = 0.0f; b_start = sp->t; b_pf = 0; b_sf = 0;
+        }
+        float emit = logits[sp->allowed[s][0]];
+        for (int a = 1; a < sp->n_allowed[s]; a++) {
+            float v = logits[sp->allowed[s][a]];
+            if (v > emit) emit = v;
+        }
+        int is_phone = sp->labels[s] != 0;
+        rel[s] = best + emit - filler;
+        start[s] = b_start;
+        pf[s] = b_pf + (is_phone ? 1 : 0);
+        strong[s] = b_sf +
+            ((is_phone && emit >= filler - sp->strong_margin) ? 1 : 0);
+    }
+    memcpy(sp->rel, rel, sizeof(float) * sp->n_states);
+    memcpy(sp->start, start, sizeof(int32_t) * sp->n_states);
+    memcpy(sp->pframes, pf, sizeof(int32_t) * sp->n_states);
+    memcpy(sp->strong, strong, sizeof(int32_t) * sp->n_states);
+    sp->t++;
+
+    float norm_best = NEG_INF;
+    int norm_dur = 0, have = 0;
+    for (int f = 0; f < 2; f++) {
+        int s = sp->n_states - 1 - f;
+        int32_t dur = sp->t - sp->start[s];
+        int32_t p = sp->pframes[s];
+        if (dur < sp->min_dur || dur > sp->max_dur) continue;
+        if (p < 2 * sp->n_phones) continue;
+        if ((float)sp->strong[s] < sp->strong_ratio * (float)p) continue;
+        float norm = sp->rel[s] / (float)p;
+        if (!have || norm > norm_best) {
+            norm_best = norm; norm_dur = (int)dur; have = 1;
+        }
+    }
+    if (have && norm_best > sp->best_seen) sp->best_seen = norm_best;
+    if (sp->cooldown > 0) { sp->cooldown--; return 0; }
+    if (have && norm_best > sp->threshold) {
+        sp->cooldown = sp->refractory;
+        for (int s = 0; s < sp->n_states; s++) sp->rel[s] = NEG_INF;
+        *score_out = norm_best;
+        *dur_out = norm_dur;
+        return 1;
+    }
+    return 0;
+}

engine_c/pww_decoder.h

@@ -0,0 +1,52 @@
+// Keyword-filler Viterbi spotter over streaming logits - C port of
+// phoneme_engine/decoder.py (KeywordSpotter). Same states, gates and
+// scoring; consumes raw logits (only within-frame differences matter).
+#pragma once
+#include <stdint.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define PWW_DEC_MAX_PHONES 24
+#define PWW_DEC_MAX_STATES (3 * PWW_DEC_MAX_PHONES + 1)
+#define PWW_DEC_MAX_ALLOWED 3
+
+typedef struct {
+    int n_phones;
+    int n_states;
+    uint8_t labels[PWW_DEC_MAX_STATES];        // class id per state
+    uint8_t n_allowed[PWW_DEC_MAX_STATES];
+    uint8_t allowed[PWW_DEC_MAX_STATES][PWW_DEC_MAX_ALLOWED];
+    int8_t preds[PWW_DEC_MAX_STATES][2];       // predecessor ids, -1 unused
+    uint8_t entry[PWW_DEC_MAX_STATES];
+    int min_dur, max_dur;
+    float threshold;
+    float strong_margin;                        // 1.5
+    float strong_ratio;                         // 0.5
+    int refractory;                             // frames
+
+    // runtime state
+    float rel[PWW_DEC_MAX_STATES];
+    int32_t start[PWW_DEC_MAX_STATES];
+    int32_t pframes[PWW_DEC_MAX_STATES];
+    int32_t strong[PWW_DEC_MAX_STATES];
+    int cooldown;
+    int32_t t;
+    float best_seen;   // best gate-passing score since last clear (diag)
+} pww_spotter_t;
+
+// phone_ids: sequence of class ids (1..39, from PWW_PHONES order +1).
+// Returns 0 on success, -1 if too long/short.
+int pww_spotter_init(pww_spotter_t *sp, const uint8_t *phone_ids,
+                     int n_phones, float threshold);
+void pww_spotter_reset(pww_spotter_t *sp);
+
+// One frame of logits (PWW_NUM_CLASSES floats). Returns 1 and fills
+// score/dur when the keyword fires, else 0.
+int pww_spotter_step(pww_spotter_t *sp, const float *logits,
+                     float *score_out, int *dur_out);
+
+#ifdef __cplusplus
+}
+#endif

engine_c/pww_engine.c

@@ -0,0 +1,254 @@
+// Streaming int8 phoneme engine. See pww_engine.h for the contract.
+#include "pww_engine.h"
+
+#include <math.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "model_int8.h"
+
+// Debug: when >= 0, pww_engine_step returns layer N's snapped int8
+// output column instead of the final logits.
+int pww_dump_layer = -1;
+
+// Ring buffer per layer holding the int8 input history a causal conv
+// needs: (k-1)*dilation + 1 columns of in_c values.
+typedef struct {
+    int8_t *buf;      // hist * in_c, column-major by time step
+    int hist;         // number of columns
+    int pos;          // next write slot
+    int primed;       // columns written so far (zeros before that)
+} ring_t;
+
+struct pww_engine {
+    ring_t rings[PWW_NUM_LAYERS];
+    // weights copied out of memory-mapped flash at create() time: flash
+    // cache misses were 20x slower than the arithmetic
+    pww_layer_t layers[PWW_NUM_LAYERS];
+    // block input snapshot (int8 col + its scale) for residual adds
+    int8_t block_in[512];
+    float block_in_scale;
+    float scratch_f[512];
+    int8_t col_a[512], col_b[512];
+};
+
+// Weight allocation. On ESP32 the engine is memory-latency bound, not
+// MAC bound: internal SRAM is several times faster to walk than octal
+// PSRAM, so fill the internal budget first and spill the rest to PSRAM.
+#ifdef ESP_PLATFORM
+#include "esp_heap_caps.h"
+#ifndef PWW_INTERNAL_WEIGHT_BUDGET
+#define PWW_INTERNAL_WEIGHT_BUDGET (160 * 1024)
+#endif
+static size_t s_internal_used = 0;
+static void *weights_alloc(size_t n) {
+    if (s_internal_used + n <= PWW_INTERNAL_WEIGHT_BUDGET) {
+        void *p = heap_caps_malloc(n, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
+        if (p) { s_internal_used += n; return p; }
+    }
+    void *p = heap_caps_malloc(n, MALLOC_CAP_SPIRAM);
+    if (!p) p = malloc(n);
+    return p;
+}
+size_t pww_internal_weight_bytes(void) { return s_internal_used; }
+// engine state (ring buffers, scratch) is touched every frame - never
+// let it land in PSRAM
+static void *state_alloc(size_t n) {
+    void *p = heap_caps_calloc(1, n, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT);
+    return p ? p : calloc(1, n);
+}
+#else
+static void *weights_alloc(size_t n) { return malloc(n); }
+static void *state_alloc(size_t n) { return calloc(1, n); }
+#endif
+
+static int8_t quant_clamp(float v, float inv_scale) {
+    float q = roundf(v * inv_scale);
+    if (q > 127.f) q = 127.f;
+    if (q < -127.f) q = -127.f;
+    return (int8_t)q;
+}
+
+pww_engine_t *pww_engine_create(void) {
+    pww_engine_t *e = (pww_engine_t *)state_alloc(sizeof(pww_engine_t));
+    if (!e) return NULL;
+    for (int li = 0; li < PWW_NUM_LAYERS; li++) {
+        e->layers[li] = PWW_LAYERS[li];
+        pww_layer_t *M = &e->layers[li];
+        size_t wn = (size_t)(M->is_dw ? M->out_c : M->in_c * M->out_c)
+                    * M->k;
+        int8_t *wcopy = (int8_t *)weights_alloc(wn);
+        float *scopy = (float *)malloc(sizeof(float) * M->out_c);
+        float *bcopy = (float *)malloc(sizeof(float) * M->out_c);
+        if (!wcopy || !scopy || !bcopy) { pww_engine_destroy(e); return NULL; }
+        memcpy(wcopy, M->w, wn);
+        memcpy(scopy, M->s, sizeof(float) * M->out_c);
+        memcpy(bcopy, M->b, sizeof(float) * M->out_c);
+        M->w = wcopy; M->s = scopy; M->b = bcopy;
+        const pww_layer_t *L = &PWW_LAYERS[li];
+        int hist = (L->k - 1) * L->dilation + 1;
+        // stride-2 stem consumes 2 input columns per step
+        if (L->stride == 2) hist += 1;
+        ring_t *r = &e->rings[li];
+        r->hist = hist;
+        r->buf = (int8_t *)state_alloc((size_t)hist * L->in_c);
+        if (!r->buf) { pww_engine_destroy(e); return NULL; }
+    }
+    pww_engine_reset(e);
+    return e;
+}
+
+void pww_engine_destroy(pww_engine_t *e) {
+    if (!e) return;
+    for (int li = 0; li < PWW_NUM_LAYERS; li++) {
+        free(e->rings[li].buf);
+        if (e->layers[li].w && e->layers[li].w != PWW_LAYERS[li].w) {
+            free((void *)e->layers[li].w);
+            free((void *)e->layers[li].s);
+            free((void *)e->layers[li].b);
+        }
+    }
+    free(e);
+}
+
+void pww_engine_reset(pww_engine_t *e) {
+    for (int li = 0; li < PWW_NUM_LAYERS; li++) {
+        ring_t *r = &e->rings[li];
+        memset(r->buf, 0, (size_t)r->hist * PWW_LAYERS[li].in_c);
+        r->pos = 0;
+        r->primed = 0;
+    }
+}
+
+static void ring_push(ring_t *r, const int8_t *col, int in_c) {
+    memcpy(r->buf + (size_t)r->pos * in_c, col, (size_t)in_c);
+    r->pos = (r->pos + 1) % r->hist;
+    if (r->primed < r->hist) r->primed++;
+}
+
+// column at "delay" steps in the past (0 = newest)
+static const int8_t *ring_at(const ring_t *r, int delay, int in_c) {
+    int idx = r->pos - 1 - delay;
+    while (idx < 0) idx += r->hist;
+    return r->buf + (size_t)idx * in_c;
+}
+
+// Runs one layer on the newest ring content, writing float pre-snap
+// output to out_f (out_c values). For stride-2 layers the newest TWO
+// columns have been pushed before calling.
+static void layer_forward(const pww_layer_t *L, const ring_t *r,
+                          float *out_f) {
+    int k = L->k, d = L->dilation, in_c = L->in_c, out_c = L->out_c;
+    // stride-2 layers get two pushes per step but output frame t only
+    // consumes up to input column 2t: the newest tap sits one column back
+    int off = (L->stride == 2) ? 1 : 0;
+    if (L->is_dw) {
+        for (int c = 0; c < out_c; c++) out_f[c] = 0.f;
+        for (int i = 0; i < k; i++) {
+            // tap i is the newest at i == k-1
+            const int8_t *col = ring_at(r, (k - 1 - i) * d + off, in_c);
+            const int8_t *w = L->w + i;  // w layout: (ch, 1, k)
+            for (int c = 0; c < out_c; c++)
+                out_f[c] += (float)((int32_t)col[c] * (int32_t)w[c * k]);
+        }
+        for (int c = 0; c < out_c; c++)
+            out_f[c] = out_f[c] * L->s[c] + L->b[c];
+    } else if (k == 1) {
+        // 1x1 conv = matrix-vector; 85% of all MACs live here.
+        // Contiguous rows, 4-way unroll, single ring lookup.
+        const int8_t *restrict col = ring_at(r, off, in_c);
+        for (int c = 0; c < out_c; c++) {
+            const int8_t *restrict w = L->w + (size_t)c * in_c;
+            int32_t a0 = 0, a1 = 0, a2 = 0, a3 = 0;
+            int j = 0;
+            for (; j + 4 <= in_c; j += 4) {
+                a0 += (int32_t)col[j] * (int32_t)w[j];
+                a1 += (int32_t)col[j + 1] * (int32_t)w[j + 1];
+                a2 += (int32_t)col[j + 2] * (int32_t)w[j + 2];
+                a3 += (int32_t)col[j + 3] * (int32_t)w[j + 3];
+            }
+            int32_t acc = a0 + a1 + a2 + a3;
+            for (; j < in_c; j++)
+                acc += (int32_t)col[j] * (int32_t)w[j];
+            out_f[c] = (float)acc * L->s[c] + L->b[c];
+        }
+    } else {
+        // general conv: hoist the k column pointers out of the c loop
+        const int8_t *cols[8];
+        for (int i = 0; i < k; i++)
+            cols[i] = ring_at(r, (k - 1 - i) * d + off, in_c);
+        for (int c = 0; c < out_c; c++) {
+            const int8_t *restrict w = L->w + (size_t)c * in_c * k;
+            int32_t acc = 0;
+            for (int i = 0; i < k; i++) {
+                const int8_t *restrict col = cols[i];
+                const int8_t *restrict wk = w + i;
+                for (int j = 0; j < in_c; j++)
+                    acc += (int32_t)col[j] * (int32_t)wk[(size_t)j * k];
+            }
+            out_f[c] = (float)acc * L->s[c] + L->b[c];
+        }
+    }
+}
+
+int pww_engine_step(pww_engine_t *e, const float *mel0, const float *mel1,
+                    float *logits_out) {
+    // quantize the two input mel columns to the input scale
+    float inv_in = 1.0f / PWW_INPUT_SCALE;
+    for (int j = 0; j < PWW_MELS; j++)
+        e->col_a[j] = quant_clamp(mel0[j], inv_in);
+    ring_push(&e->rings[0], e->col_a, PWW_MELS);
+    for (int j = 0; j < PWW_MELS; j++)
+        e->col_a[j] = quant_clamp(mel1[j], inv_in);
+    ring_push(&e->rings[0], e->col_a, PWW_MELS);
+
+    int8_t *cur = e->col_a;   // int8 column flowing between layers
+    float cur_scale = PWW_INPUT_SCALE;
+    (void)cur_scale;
+
+    for (int li = 0; li < PWW_NUM_LAYERS; li++) {
+        const pww_layer_t *L = &e->layers[li];
+        ring_t *r = &e->rings[li];
+        if (li > 0) ring_push(r, cur, L->in_c);
+
+        if (L->is_dw) {
+            // save the block input column + scale for the residual 2
+            // layers later (dw -> pw(residual))
+            memcpy(e->block_in, ring_at(r, 0, L->in_c), (size_t)L->in_c);
+            e->block_in_scale =
+                (li == 0) ? PWW_INPUT_SCALE : e->layers[li - 1].out_scale;
+        }
+
+        layer_forward(L, r, e->scratch_f);
+
+        if (L->residual) {
+            for (int c = 0; c < L->out_c; c++)
+                e->scratch_f[c] += (float)e->block_in[c] * e->block_in_scale;
+        }
+        if (L->relu) {
+            for (int c = 0; c < L->out_c; c++)
+                if (e->scratch_f[c] < 0.f) e->scratch_f[c] = 0.f;
+        }
+        if (li == PWW_NUM_LAYERS - 1) {
+            // final logits: snap to grid to mirror the simulator, then
+            // return as float
+            float s = L->out_scale;
+            for (int c = 0; c < L->out_c; c++) {
+                int8_t q = quant_clamp(e->scratch_f[c], 1.0f / s);
+                logits_out[c] = (float)q * s;
+            }
+            return 0;
+        }
+        float inv = 1.0f / L->out_scale;
+        int8_t *nxt = (cur == e->col_a) ? e->col_b : e->col_a;
+        for (int c = 0; c < L->out_c; c++)
+            nxt[c] = quant_clamp(e->scratch_f[c], inv);
+        if (pww_dump_layer == li) {
+            for (int c = 0; c < L->out_c && c < 256; c++)
+                logits_out[c] = (float)nxt[c];
+            return 0;
+        }
+        cur = nxt;
+    }
+    return -1;  // unreachable
+}

engine_c/pww_engine.h

@@ -0,0 +1,38 @@
+// Streaming int8 phoneme engine - portable C (PC test build + ESP32-S3).
+// Consumes one 40 ms feature frame at a time (40 log-mel values at the
+// model's 20 ms output rate this is stride-2, so the caller feeds TWO
+// 10 ms-hop mel frames per step); emits one logits vector per step.
+//
+// Design contract = phoneme_engine/quantize.py fake_quant_forward():
+//   - weights int8 per-output-channel, BN folded
+//   - int8 x int8 -> int32 accumulate, float requant (combined scale),
+//     residual added in float, ReLU, snap to int8 grid of the next layer
+//   - strictly causal: each layer keeps a ring buffer of its int8 input
+//     history, so per-step cost is one new column per layer
+#pragma once
+#include <stdint.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define PWW_MELS 40
+
+typedef struct pww_engine pww_engine_t;
+
+// Allocates all layer state (ring buffers) on the heap. Returns NULL on
+// allocation failure.
+pww_engine_t *pww_engine_create(void);
+void pww_engine_destroy(pww_engine_t *e);
+void pww_engine_reset(pww_engine_t *e);
+
+// Feed TWO consecutive 10 ms-hop mel frames (each PWW_MELS floats, already
+// EMA-normalized like features.py). Writes PWW_NUM_CLASSES float logits
+// (log-prob differences are what the decoder consumes; absolute offset is
+// meaningless). Returns 0 on success.
+int pww_engine_step(pww_engine_t *e, const float *mel0, const float *mel1,
+                    float *logits_out);
+
+#ifdef __cplusplus
+}
+#endif

engine_c/pww_frontend.c

@@ -0,0 +1,127 @@
+// Streaming log-mel frontend - C port of phoneme_engine/features.py.
+// Frame t is centered at sample t*HOP (torchaudio center=True), so the
+// stream carries WIN/2 = 200 samples (12.5 ms) of lookahead latency.
+#include "pww_frontend.h"
+
+#include <math.h>
+#include <stdint.h>
+#include <string.h>
+
+#include "frontend_data.h"
+
+#ifndef M_PI
+#define M_PI 3.14159265358979323846
+#endif
+
+// ---- 512-point iterative radix-2 complex FFT (input real) -------------
+#define NFFT PWW_FE_NFFT
+
+static float fft_re[NFFT], fft_im[NFFT];
+static float tw_cos[NFFT / 2], tw_sin[NFFT / 2];
+static int fft_init_done = 0;
+
+static void fft_init(void) {
+    for (int i = 0; i < NFFT / 2; i++) {
+        tw_cos[i] = cosf(-2.0f * (float)M_PI * i / NFFT);
+        tw_sin[i] = sinf(-2.0f * (float)M_PI * i / NFFT);
+    }
+    fft_init_done = 1;
+}
+
+static void fft512(void) {
+    // bit-reversal permutation
+    for (int i = 1, j = 0; i < NFFT; i++) {
+        int bit = NFFT >> 1;
+        for (; j & bit; bit >>= 1) j ^= bit;
+        j ^= bit;
+        if (i < j) {
+            float tr = fft_re[i]; fft_re[i] = fft_re[j]; fft_re[j] = tr;
+            float ti = fft_im[i]; fft_im[i] = fft_im[j]; fft_im[j] = ti;
+        }
+    }
+    for (int len = 2; len <= NFFT; len <<= 1) {
+        int half = len >> 1, step = NFFT / len;
+        for (int i = 0; i < NFFT; i += len) {
+            for (int k = 0; k < half; k++) {
+                float wr = tw_cos[k * step], wi = tw_sin[k * step];
+                int a = i + k, b = i + k + half;
+                float xr = fft_re[b] * wr - fft_im[b] * wi;
+                float xi = fft_re[b] * wi + fft_im[b] * wr;
+                fft_re[b] = fft_re[a] - xr;
+                fft_im[b] = fft_im[a] - xi;
+                fft_re[a] += xr;
+                fft_im[a] += xi;
+            }
+        }
+    }
+}
+
+// ---- streaming state ---------------------------------------------------
+// ring of raw samples; enough for one centered window plus slack
+#define SBUF (PWW_FE_WIN + 4 * PWW_FE_HOP)
+
+struct pww_frontend {
+    float samples[SBUF];
+    int n_samples;        // total pushed
+    int64_t next_frame;   // next frame index to emit
+    float ema[PWW_FE_NMELS];
+    int ema_init;
+};
+
+static struct pww_frontend g_fe;
+
+void pww_frontend_reset(void) {
+    memset(&g_fe, 0, sizeof(g_fe));
+    if (!fft_init_done) fft_init();
+}
+
+// Push samples; calls emit(mel_frame) for every completed 10 ms frame.
+void pww_frontend_push(const float *x, int n,
+                       void (*emit)(const float *mel, void *user),
+                       void *user) {
+    struct pww_frontend *fe = &g_fe;
+    for (int i = 0; i < n; i++) {
+        fe->samples[fe->n_samples % SBUF] = x[i];
+        fe->n_samples++;
+        // frame f is ready when its centered window [f*HOP-200, f*HOP+200)
+        // is fully available
+        long long center = fe->next_frame * PWW_FE_HOP;
+        while (center + PWW_FE_WIN / 2 <= fe->n_samples) {
+            float mel[PWW_FE_NMELS];
+            long long start = center - PWW_FE_WIN / 2;
+            memset(fft_re, 0, sizeof(fft_re));
+            memset(fft_im, 0, sizeof(fft_im));
+            int off = (NFFT - PWW_FE_WIN) / 2;   // window centered in FFT
+            for (int w = 0; w < PWW_FE_WIN; w++) {
+                long long s = start + w;
+                // reflect padding at the stream start (center=True)
+                if (s < 0) s = -s;
+                float v = (s < fe->n_samples)
+                              ? fe->samples[s % SBUF] : 0.f;
+                fft_re[off + w] = v * PWW_FE_HANN[w];
+            }
+            fft512();
+            for (int m = 0; m < PWW_FE_NMELS; m++) mel[m] = 0.f;
+            for (int b = 0; b < PWW_FE_NFREQS; b++) {
+                float p = fft_re[b] * fft_re[b] + fft_im[b] * fft_im[b];
+                const float *fbrow = PWW_FE_MELFB + b;
+                for (int m = 0; m < PWW_FE_NMELS; m++)
+                    mel[m] += p * fbrow[(size_t)m * PWW_FE_NFREQS];
+            }
+            for (int m = 0; m < PWW_FE_NMELS; m++) {
+                float lm = logf(mel[m] + 1e-6f);
+                // causal EMA mean subtraction (features.py lfilter)
+                float e = fe->ema_init
+                              ? (1.f - PWW_FE_EMA_ALPHA) * fe->ema[m]
+                                    + PWW_FE_EMA_ALPHA * lm
+                              : PWW_FE_EMA_ALPHA * lm;
+                fe->ema[m] = e;
+                mel[m] = lm - e;
+            }
+            fe->ema_init = 1;
+            emit(mel, user);
+            fe->next_frame++;
+            center = fe->next_frame * PWW_FE_HOP;
+        }
+    }
+}

engine_c/pww_frontend.h

@@ -0,0 +1,15 @@
+// Streaming log-mel frontend matching the training features exactly.
+#pragma once
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+void pww_frontend_reset(void);
+void pww_frontend_push(const float *x, int n,
+                       void (*emit)(const float *mel, void *user),
+                       void *user);
+
+#ifdef __cplusplus
+}
+#endif

examples/neptuno_universal.json

@@ -0,0 +1,69 @@
+{
+  "phrase": "neptuno",
+  "spotters": [
+    {
+      "phones": [
+        "N",
+        "EH",
+        "P",
+        "T",
+        "UW",
+        "N",
+        "OW"
+      ],
+      "source": "dictionary",
+      "threshold": -2.5
+    },
+    {
+      "phones": [
+        "N",
+        "EH",
+        "T",
+        "UW",
+        "N",
+        "OW"
+      ],
+      "source": "population",
+      "threshold": -2.5
+    },
+    {
+      "phones": [
+        "N",
+        "AE",
+        "T",
+        "UW",
+        "N",
+        "OW"
+      ],
+      "source": "population",
+      "threshold": -2.5
+    },
+    {
+      "phones": [
+        "N",
+        "EH",
+        "T",
+        "T",
+        "UW",
+        "N",
+        "OW"
+      ],
+      "source": "population",
+      "threshold": -2.5
+    },
+    {
+      "phones": [
+        "N",
+        "EH",
+        "P",
+        "T",
+        "UW",
+        "N",
+        "OW"
+      ],
+      "source": "population",
+      "threshold": -2.5
+    }
+  ],
+  "mode": "universal"
+}

phoneme_engine/decoder.py

@@ -0,0 +1,236 @@
+"""Streaming keyword spotting over CTC phoneme posteriors.
+
+Keyword-filler decoding with two robustness measures beyond the textbook
+version (both port directly to C on the ESP32):
+
+  - minimum phone duration: every phone is expanded to two chained states,
+    so an alignment must spend >= 2 frames (40 ms) per phone. Kills
+    spurious single-frame matches.
+  - duration-normalized scoring: each Viterbi token carries the frame at
+    which its path entered the keyword, and the detection statistic is
+    (keyword_score - filler_score) / path_duration -- an average
+    per-frame deficit, comparable across phrase lengths and speaking
+    rates. Alignments faster than 2 frames/phone or slower than
+    MAX_FRAMES_PER_PHONE are rejected outright.
+
+Enrollment of a new wake word is just: phones = text_to_phones(phrase).
+State chain per phone i: A_i -> B_i (same label), optional blank after.
+"""
+
+import numpy as np
+
+from .phones import BLANK, PHONE_TO_ID, text_to_phones
+
+NEG_INF = -1e30
+# 200 ms per phone upper bound: fluent speech runs 40-120 ms/phone, and
+# looser caps let garbage alignments crawl across continuous speech
+# (observed 1.5-2.3 s "matches" on background TV at 400 ms/phone)
+MAX_FRAMES_PER_PHONE = 10
+
+# Highly confusable phone pairs: an alignment may match either member.
+# Text pronunciations (CMUdict/G2P) use canonical vowels, but real and TTS
+# speech often reduces them -- e.g. "orbit" is listed as AO R B AH T yet
+# actually said as AO R B IH T. Without this, a sharper acoustic model
+# *punishes* the mismatch harder.
+CONFUSABLE = {
+    "AH": ("AH", "IH", "ER"),   # schwa reduces/r-colors: orbit -> orbERt
+    "IH": ("IH", "AH"),
+    "ER": ("ER", "AH"),
+    "AO": ("AO", "AA"),         # cot-caught merger and accent variation
+    "AE": ("AE", "AA"),         # trap-father variation (sakura, tanaka)
+    "UH": ("UH", "UW"),         # lax/tense u (book/boot neighbors)
+    "K": ("K", "G"),            # stops voice between vowels: nakuma->naguma
+    "G": ("G", "K"),
+    "T": ("T", "D"),            # also covers tapped/rolled R heard as D
+    "D": ("D", "T"),
+}
+
+
+class KeywordSpotter:
+    def __init__(self, phrase, threshold=-4.4, refractory_frames=50,
+                 strong_margin=1.5, strong_ratio=0.5, phones=None,
+                 wildcards=None, wildcard_penalty=2.2):
+        """threshold: average per-frame deficit tolerated along the keyword
+        path (log-prob units, <= 0; closer to 0 = stricter).
+        refractory_frames: minimum output frames (20 ms) between fires.
+        strong_margin/strong_ratio: evidence requirement -- at least
+        strong_ratio of the path's phone frames must have their phone
+        within strong_margin log-prob of the frame's best class. Prevents
+        noisy/mushy audio (where nothing is clearly heard) from firing on
+        average-score alone.
+        phones: explicit phoneme sequence (e.g. from voice enrollment);
+        overrides the dictionary pronunciation of `phrase`."""
+        if phones is None:
+            phones = [p for p in text_to_phones(phrase) if p in PHONE_TO_ID]
+        else:
+            phones = [p for p in phones if p in PHONE_TO_ID]
+        if len(phones) < 2:
+            raise ValueError(f"phrase too short to spot: {phrase!r}")
+        self.phrase = phrase
+        self.phones = phones
+        self.threshold = threshold
+        self.refractory_frames = refractory_frames
+        self.strong_margin = strong_margin
+        self.strong_ratio = strong_ratio
+        # Mismatch tolerance (default OFF): measured on universal clips
+        # vs negatives, wildcarding lifted impostor scores as much as
+        # genuine ones and REDUCED recall at zero-FA operating points
+        # (vucano negmax -4.14 -> -2.89). Kept only for future
+        # per-count-lattice experiments.
+        if wildcards is None:
+            wildcards = 0
+        self.max_wild = wildcards * 2   # budget in frames
+        self.wild_pen = wildcard_penalty
+
+        # Build states: leading blank, then per phone A,B (+ trailing blank)
+        labels = [BLANK]
+        self.preds = [[0]]              # predecessor state ids (self-loop
+        entry = [True]                  # implied for every state)
+        for i, p in enumerate(phones):
+            pid = PHONE_TO_ID[p]
+            a = len(labels)
+            if i == 0:
+                labels.append(pid); self.preds.append([0]); entry.append(True)
+            else:
+                pre = [a - 1]           # blank between phones
+                if labels[a - 2] != pid:
+                    pre.append(a - 2)   # skip blank (different phones only)
+                labels.append(pid); self.preds.append(pre); entry.append(False)
+            labels.append(pid)          # B state: only from A
+            self.preds.append([a]); entry.append(False)
+            labels.append(BLANK)        # blank after phone
+            self.preds.append([a + 1]); entry.append(False)
+        self.labels = labels
+        # allowed emission ids per state (confusable vowels match either)
+        self.allowed = []
+        id_to_phone = {v: k for k, v in PHONE_TO_ID.items()}
+        for lab in labels:
+            if lab == BLANK:
+                self.allowed.append((BLANK,))
+            else:
+                ph = id_to_phone[lab]
+                self.allowed.append(tuple(
+                    PHONE_TO_ID[p] for p in CONFUSABLE.get(ph, (ph,))))
+        self.entry = entry
+        self.n_states = len(labels)
+        self.finals = [self.n_states - 1, self.n_states - 2]
+        # min: ~60 ms per phone on average (individual phones may be
+        # shorter); max: 400 ms per phone. Anything outside is not a
+        # human saying the phrase.
+        self.min_dur = 3 * len(phones)
+        self.max_dur = MAX_FRAMES_PER_PHONE * len(phones)
+        self.reset()
+
+    def reset(self):
+        self.rel = np.full(self.n_states, NEG_INF)
+        self.start = np.zeros(self.n_states, dtype=np.int64)
+        self.pframes = np.zeros(self.n_states, dtype=np.int64)
+        self.strong = np.zeros(self.n_states, dtype=np.int64)
+        self.wild = np.zeros(self.n_states, dtype=np.int64)
+        self.cooldown = 0
+        self.t = 0
+
+    def _update(self, log_probs_frame):
+        """One Viterbi DP step. Returns the best duration-valid normalized
+        score at a final state this frame (or None). No side effects on
+        detection state."""
+        lp = log_probs_frame
+        filler = float(lp.max())
+        prev_rel, prev_start = self.rel, self.start
+        prev_pf, prev_strong = self.pframes, self.strong
+        prev_wild = self.wild
+        cur_rel = np.full(self.n_states, NEG_INF)
+        cur_start = np.zeros(self.n_states, dtype=np.int64)
+        cur_pf = np.zeros(self.n_states, dtype=np.int64)
+        cur_strong = np.zeros(self.n_states, dtype=np.int64)
+        cur_wild = np.zeros(self.n_states, dtype=np.int64)
+        for s in range(self.n_states):
+            best = prev_rel[s]
+            best_start, best_pf, best_sf, best_w = (
+                prev_start[s], prev_pf[s], prev_strong[s], prev_wild[s])
+            for q in self.preds[s]:
+                if prev_rel[q] > best:
+                    best = prev_rel[q]
+                    best_start, best_pf, best_sf, best_w = (
+                        prev_start[q], prev_pf[q], prev_strong[q],
+                        prev_wild[q])
+            if self.entry[s] and 0.0 >= best:
+                # (re)start the keyword here; >= keeps the start time fresh
+                # while idling in silence, so duration stays meaningful
+                best, best_start, best_pf, best_sf, best_w = \
+                    0.0, self.t, 0, 0, 0
+            emit = max(float(lp[i]) for i in self.allowed[s])
+            is_phone = self.labels[s] != BLANK
+            used_wild = 0
+            if is_phone and best_w < self.max_wild:
+                # mismatch tolerance: accept the frame's best class at a
+                # penalty when the expected phone isn't there (bounded
+                # budget converts exact matching into similarity matching)
+                soft = filler - self.wild_pen
+                if soft > emit:
+                    emit = soft
+                    used_wild = 1
+            cur_rel[s] = best + emit - filler
+            cur_start[s] = best_start
+            cur_pf[s] = best_pf + (1 if is_phone else 0)
+            cur_strong[s] = best_sf + (
+                1 if is_phone and emit >= filler - self.strong_margin else 0)
+            cur_wild[s] = best_w + used_wild
+        self.rel, self.start = cur_rel, cur_start
+        self.pframes, self.strong = cur_pf, cur_strong
+        self.wild = cur_wild
+        self.t += 1
+        norm_best = None
+        for s in self.finals:
+            dur = self.t - cur_start[s]
+            pf = cur_pf[s]
+            if not (self.min_dur <= dur <= self.max_dur):
+                continue
+            if pf < 2 * len(self.phones):
+                continue
+            # evidence requirement: the phrase's phones must actually have
+            # been the near-top hypothesis for enough of the match, not
+            # merely "not too costly on average" (mushy noisy audio)
+            if cur_strong[s] < self.strong_ratio * pf:
+                continue
+            # normalize by phone frames only: time spent in blank states is
+            # free in silence, so counting it would let blank-padded paths
+            # dilute their deficit below any threshold
+            norm = cur_rel[s] / pf
+            if norm_best is None or norm > norm_best[0]:
+                norm_best = (float(norm), int(dur))
+        return norm_best
+
+    def step(self, log_probs_frame):
+        """Consume one frame of log-probs (C,). Returns (score, duration)
+        if the keyword fired this frame, else None. duration is in output
+        frames (20 ms each), for energy-gating by the caller."""
+        norm = self._update(log_probs_frame)
+        if self.cooldown > 0:
+            self.cooldown -= 1
+            return None
+        if norm is not None and norm[0] > self.threshold:
+            self.cooldown = self.refractory_frames
+            self.rel = np.full(self.n_states, NEG_INF)
+            return norm
+        return None
+
+    def run(self, log_probs):
+        """Offline helper: (T, C) log-probs -> list of (frame, score)."""
+        hits = []
+        for t in range(log_probs.shape[0]):
+            s = self.step(log_probs[t])
+            if s is not None:
+                hits.append((t, s[0]))
+        return hits
+
+    def best_score(self, log_probs):
+        """Max normalized final-state score over a clip (pure Viterbi, no
+        firing/reset side effects) -- used for calibration."""
+        self.reset()
+        best = -np.inf
+        for t in range(log_probs.shape[0]):
+            s = self._update(log_probs[t])
+            if s is not None and s[0] > best:
+                best = s[0]
+        return best

phoneme_engine/enroll.py

@@ -0,0 +1,254 @@
+"""Voice enrollment: learn YOUR pronunciation of a wake word in seconds.
+
+Records the phrase three times, extracts the phoneme sequence the model
+actually hears for your voice/accent/mic, and saves the variants alongside
+the dictionary pronunciation:
+
+  python -m phoneme_engine.enroll "hey orbit"
+
+Then use it live:
+
+  python -m phoneme_engine.live_demo "hey orbit" --enrolled
+
+No training happens -- enrollment is just phoneme sequences (a few bytes),
+same as typed enrollment, so it stays instant and fully on-device later.
+"""
+
+import argparse
+import json
+import time
+from pathlib import Path
+
+import numpy as np
+import sounddevice as sd
+import soundfile as sf
+import torch
+
+from .decoder import KeywordSpotter
+from .features import SAMPLE_RATE, LogMel
+from .model import PhonemeTCN
+from .phones import BLANK, ids_to_phones
+
+ROOT = Path(__file__).resolve().parent.parent
+ENROLL_DIR = ROOT / "enrollments"
+
+
+def slug(phrase):
+    return "".join(c if c.isalnum() else "_" for c in phrase.lower())
+
+
+def min_variant_len(n_dict_phones):
+    """Variants much shorter than the real phrase false-trigger constantly
+    (a 3-phone variant matches half of English)."""
+    return max(4, round(0.6 * n_dict_phones))
+
+
+def phone_segment(lp, merge_gap=15, margin=3):
+    """(t0, t1) output-frame span of the densest contiguous run of frames
+    where the model hears an actual phoneme (blank prob < 0.5). Isolates
+    the spoken phrase from background sounds elsewhere in the take."""
+    p_blank = np.exp(lp[:, BLANK])
+    active = p_blank < 0.5
+    runs = []
+    cur = None
+    n = len(active)
+    for i in range(n + 1):
+        a = active[i] if i < n else False
+        if a and cur is None:
+            cur = i
+        elif not a and cur is not None:
+            if i + merge_gap <= n and active[i:i + merge_gap].any():
+                continue
+            runs.append((cur, i))
+            cur = None
+    if not runs:
+        return 0, lp.shape[0]
+    # densest = most active frames inside the run
+    t0, t1 = max(runs, key=lambda r: active[r[0]:r[1]].sum())
+    return max(0, t0 - margin), min(lp.shape[0], t1 + margin)
+
+
+def extract_phones(lp, segment=True):
+    """Greedy phoneme sequence from (T, C) log-probs, ignoring blanks.
+    With segment=True only the densest phoneme burst is transcribed
+    (background-proof)."""
+    t0, t1 = phone_segment(lp) if segment else (0, lp.shape[0])
+    ids = lp[t0:t1].argmax(axis=1).tolist()
+    seq = []
+    prev = None
+    for i in ids:
+        if i != prev and i != BLANK:
+            seq.append(i)
+        prev = i
+    return ids_to_phones(seq)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("phrase")
+    ap.add_argument("--takes", type=int, default=3)
+    ap.add_argument("--seconds", type=float, default=4.0)
+    ap.add_argument("--ckpt", default=str(ROOT / "checkpoints" / "best.pt"))
+    args = ap.parse_args()
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    frontend = LogMel().to(device).eval()
+    model = PhonemeTCN().to(device).eval()
+    state = torch.load(args.ckpt, map_location=device, weights_only=True)
+    model.load_state_dict(state["model"])
+
+    dict_spotter = KeywordSpotter(args.phrase)
+    print(f"dictionary pronunciation: {' '.join(dict_spotter.phones)}")
+
+    variants = []
+    take_lps = []
+    ENROLL_DIR.mkdir(exist_ok=True)
+    for take in range(1, args.takes + 1):
+        # capture starts before the countdown so the stream is already
+        # open when the user speaks (stream startup used to clip speech)
+        total = 3 * 0.7 + args.seconds
+        audio = sd.rec(int(total * SAMPLE_RATE),
+                       samplerate=SAMPLE_RATE, channels=1, dtype="float32")
+        print(f"\ntake {take}/{args.takes}: say {args.phrase!r} once, "
+              "normally")
+        for i in (3, 2, 1):
+            print(f"  {i}...", flush=True)
+            time.sleep(0.7)
+        print("  SPEAK NOW", flush=True)
+        sd.wait()
+        audio = audio[:, 0]
+        peak_db = 20 * np.log10(np.abs(audio).max() + 1e-12)
+        if peak_db < -30:
+            print(f"  !! too quiet (peak {peak_db:.1f} dBFS) - retake")
+            continue
+        sf.write(str(ENROLL_DIR / f"{slug(args.phrase)}_take{take}.wav"),
+                 audio, SAMPLE_RATE)
+        wav = torch.from_numpy(audio).unsqueeze(0).to(device)
+        with torch.no_grad():
+            lp = torch.log_softmax(model(frontend(wav)).float(),
+                                   dim=2)[0].cpu().numpy()
+        phones = extract_phones(lp)
+        print(f"  heard: {' '.join(phones) or '(nothing)'}")
+        n_dict = len(dict_spotter.phones)
+        if len(phones) < min_variant_len(n_dict):
+            print("  !! too few phonemes heard (short variants false-"
+                  "trigger constantly) - retake ignored")
+            continue
+        if len(phones) > 2 * n_dict + 4:
+            print("  !! heard too much (background speech?) - retake ignored")
+            continue
+        variants.append(phones)
+        take_lps.append(lp)
+
+    if not variants:
+        print("\nno usable takes - nothing saved")
+        return
+
+    # dedupe identical variants
+    uniq = []
+    for v in variants:
+        if v not in uniq:
+            uniq.append(v)
+
+    spotters = calibrate_spotters(args.phrase, uniq, take_lps, model,
+                                  frontend, device)
+    out = {
+        "phrase": args.phrase,
+        "spotters": spotters,
+    }
+    path = ENROLL_DIR / f"{slug(args.phrase)}.json"
+    path.write_text(json.dumps(out, indent=2), encoding="utf-8")
+    print(f"\nsaved {len(spotters)} calibrated spotter(s) -> {path}")
+    print("test live with:\n  python -m phoneme_engine.live_demo "
+          f"\"{args.phrase}\" --enrolled")
+
+
+def calibrate_spotters(phrase, variants, take_lps, model, frontend, device,
+                       n_negatives=60):
+    """Per-variant thresholds: each pronunciation (dictionary + voice
+    variants) is calibrated independently against the user's takes and a
+    sample of negative speech. Variants that cannot separate the user's
+    voice from negatives are dropped -- a loose variant with a shared
+    threshold poisons the whole enrollment."""
+    candidates = [("dictionary", None)] + [("voice", v) for v in variants]
+
+    neg_lps = []
+    neg_dir = ROOT / "data" / "LibriSpeech" / "dev-clean"
+    if neg_dir.exists():
+        from .spot_file import load_wav
+        files = sorted(neg_dir.glob("**/*.flac"))
+        rng = np.random.default_rng(0)
+        idx = rng.choice(len(files), min(n_negatives, len(files)),
+                         replace=False)
+        with torch.no_grad():
+            for i in sorted(idx):
+                wav = load_wav(files[i], device)
+                neg_lps.append(torch.log_softmax(
+                    model(frontend(wav)).float(), dim=2)[0].cpu().numpy())
+
+    out = []
+    fallback = None
+    print()
+    for source, phones in candidates:
+        sp = KeywordSpotter(phrase, phones=phones)
+        scores = [sp.best_score(lp) for lp in take_lps]
+        pos_best, pos_worst = max(scores), min(scores)
+        neg = max((sp.best_score(lp) for lp in neg_lps), default=-np.inf)
+        label = f"{source}: {' '.join(sp.phones)}"
+        if pos_best - neg < 0.75:
+            print(f"  dropped {label} (your best {pos_best:.2f} vs "
+                  f"negative {neg:.2f} - too false-alarm prone)")
+            if pos_best != -np.inf:
+                strict = round(float(min(-2.5, max(-6.0, neg + 0.3))), 2)
+                cand = {"phones": list(sp.phones), "source": source,
+                        "threshold": strict}
+                if fallback is None or pos_best - neg > fallback[0]:
+                    fallback = (pos_best - neg, cand)
+            continue
+        # anchor on the WEAKEST take (live speech varies more than
+        # enrollment takes), floored above the hardest negative
+        thr = max(pos_worst - 0.5, neg + 0.3) if neg_lps else pos_worst - 0.5
+        thr = round(float(min(-2.5, max(-6.0, thr))), 2)
+        print(f"  kept {label}  threshold {thr:.2f} "
+              f"(your takes {pos_worst:.2f}..{pos_best:.2f}, "
+              f"hardest negative {neg:.2f})")
+        entry = {"phones": list(sp.phones), "source": source,
+                 "threshold": thr}
+        out.append(entry)
+        if fallback is None or pos_best - neg > fallback[0]:
+            fallback = (pos_best - neg, entry)
+
+    if not out and fallback is not None:
+        print("  (all candidates marginal - keeping the best one strictly)")
+        out.append(fallback[1])
+    return out
+
+
+def load_spotter_configs(phrase, default_threshold=-4.4, profile=None):
+    """Returns [(phones, threshold), ...] for a phrase: the calibrated
+    enrollment if present, else just the dictionary pronunciation.
+    profile selects an alternate enrollment file, e.g. 'universal'."""
+    name = slug(phrase) + (f"_{profile}" if profile else "")
+    path = ENROLL_DIR / f"{name}.json"
+    dict_phones = list(KeywordSpotter(phrase).phones)
+    if path.exists():
+        data = json.loads(path.read_text(encoding="utf-8"))
+        if "spotters" in data:
+            configs = []
+            min_len = min_variant_len(len(dict_phones))
+            for s in data["spotters"]:
+                if s["source"] == "dictionary" or len(s["phones"]) >= min_len:
+                    configs.append((s["phones"], s["threshold"]))
+            if configs:
+                return configs
+        # legacy schema: variants without per-variant thresholds
+        configs = [(dict_phones, default_threshold)]
+        for v in data.get("variants", []):
+            if len(v) >= min_variant_len(len(dict_phones)):
+                configs.append((v, default_threshold))
+        return configs
+    return [(dict_phones, default_threshold)]
+
+
+if __name__ == "__main__":
+    main()

phoneme_engine/enroll_universal.py

@@ -0,0 +1,183 @@
+"""Universal (speaker-independent) enrollment: build a wake word config
+from the *population's* pronunciations instead of one person's.
+
+Synthesizes the phrase with many diverse TTS voices, keeps only
+Whisper-verified clips, extracts each clip's observed phoneme variant,
+keeps variants that recur across different voices, and calibrates every
+spotter against negative speech. Zero training; output is the same
+~50-byte-per-spotter config schema as personal enrollment.
+
+  python -m phoneme_engine.enroll_universal "sakura" --say "sah koo rah"
+
+Then:  python -m phoneme_engine.live_demo "sakura" --enrolled --profile universal
+"""
+
+import argparse
+import json
+import subprocess
+import sys
+from collections import Counter
+from pathlib import Path
+
+import numpy as np
+import soundfile as sf
+import torch
+
+from .decoder import KeywordSpotter
+from .enroll import ENROLL_DIR, extract_phones, min_variant_len, slug
+from .features import LogMel
+from .model import PhonemeTCN
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+def _norm(s):
+    return "".join(c for c in s.lower() if c.isalpha())
+
+
+def _lev(a, b):
+    prev = list(range(len(b) + 1))
+    for i, x in enumerate(a, 1):
+        cur = [i]
+        for j, y in enumerate(b, 1):
+            cur.append(min(prev[j] + 1, cur[j - 1] + 1,
+                           prev[j - 1] + (x != y)))
+        prev = cur
+    return prev[-1]
+
+
+_DIGITS = {"0": "zero", "1": "one", "2": "two", "3": "three", "4": "four",
+           "5": "five", "6": "six", "7": "seven", "8": "eight",
+           "9": "nine"}
+
+
+def transcript_matches(text, phrase, say, target_phones):
+    """Letter-level OR phoneme-level match. Whisper spells invented words
+    unpredictably ('Neptune-O', 'NEP 2 NO'), but its spellings are
+    usually phonetically faithful - so G2P the transcript and compare
+    phone sequences."""
+    from .phones import text_to_phones
+    t_letters = _norm(text)
+    for tgt in (_norm(phrase), _norm(say)):
+        if _lev(t_letters, tgt) <= max(1, len(tgt) // 4):
+            return True
+    spoken = "".join(_DIGITS.get(c, c) if c.isdigit() else c
+                     for c in text.lower())
+    t_phones = text_to_phones(spoken)
+    tol = max(1, (len(target_phones) + 2) // 3)
+    return _lev(t_phones, list(target_phones)) <= tol
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("phrase")
+    ap.add_argument("--say", default=None,
+                    help="phonetic respelling for TTS (exotic words get "
+                         "anglicized otherwise, e.g. sakura -> secure-ah)")
+    ap.add_argument("--n", type=int, default=36)
+    ap.add_argument("--min-votes", type=int, default=3,
+                    help="a variant must appear in this many different "
+                         "voices to count as population-level")
+    ap.add_argument("--ckpt", default=str(ROOT / "checkpoints" / "best.pt"))
+    args = ap.parse_args()
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    frontend = LogMel().to(device).eval()
+    model = PhonemeTCN().to(device).eval()
+    state = torch.load(args.ckpt, map_location=device, weights_only=True)
+    model.load_state_dict(state["model"])
+
+    # 1. synthesize across diverse voices
+    say = args.say or args.phrase
+    clips = ROOT / "enrollments" / f"_univ_{slug(args.phrase)}"
+    if not clips.exists() or len(list(clips.glob("*.wav"))) < args.n:
+        subprocess.run([sys.executable, "-m",
+                        "phoneme_engine.make_piper_clips", say,
+                        "--n", str(args.n), "--out", str(clips),
+                        "--seed", "17"], check=True)
+
+    # 2. Whisper-verify
+    from transformers import pipeline
+    asr = pipeline("automatic-speech-recognition",
+                   model="openai/whisper-small", device=0,
+                   dtype=torch.float16)
+    from .phones import text_to_phones
+    target_phones = text_to_phones(say)
+    verified = []
+    for f in sorted(clips.glob("*.wav")):
+        audio, sr = sf.read(f, dtype="float32")
+        text = asr({"raw": audio, "sampling_rate": sr})["text"]
+        if transcript_matches(text, args.phrase, say, target_phones):
+            verified.append(f)
+    print(f"verified {len(verified)}/{args.n} clips")
+    if len(verified) < 10:
+        print("too few verified clips - check the respelling")
+        return
+
+    # 3. extract variants + vote across voices
+    lps = []
+    votes = Counter()
+    with torch.no_grad():
+        for f in verified:
+            audio, _ = sf.read(f, dtype="float32")
+            wav = torch.from_numpy(audio).float().unsqueeze(0).to(device)
+            lp = torch.log_softmax(model(frontend(wav)).float(),
+                                   dim=2)[0].cpu().numpy()
+            lps.append(lp)
+            v = tuple(extract_phones(lp))
+            n_dict = len(KeywordSpotter(args.phrase).phones)
+            if min_variant_len(n_dict) <= len(v) <= 2 * n_dict + 4:
+                votes[v] += 1
+    pop_variants = [list(v) for v, c in votes.most_common()
+                    if c >= args.min_votes]
+    print("population variants:",
+          [(" ".join(v), votes[tuple(v)]) for v in pop_variants])
+
+    # 4. calibrate dictionary + population variants against negatives,
+    #    with the verified clips as positives
+    z = np.load(ROOT / "bench_results" / "lp_neg_ls.npz",
+                allow_pickle=True)
+    neg_lps = list(z["lps"])[:200]
+
+    spotters = []
+    for source, phones in ([("dictionary", None)]
+                           + [("population", v) for v in pop_variants]):
+        sp = KeywordSpotter(args.phrase, phones=phones)
+        pos = sorted(sp.best_score(lp) for lp in lps)
+        neg = max(sp.best_score(lp) for lp in neg_lps)
+        pos_med = pos[len(pos) // 2]
+        if pos_med - neg < 0.5:
+            print(f"  dropped {source}: {' '.join(sp.phones)} "
+                  f"(median {pos_med:.2f} vs neg {neg:.2f})")
+            continue
+        # universal threshold: catch the median speaker, stay above the
+        # hardest negative
+        thr = float(min(-2.5, max(-6.0, max(pos_med - 0.4, neg + 0.3))))
+        kept = sum(1 for s in pos if s > thr) / len(pos)
+        print(f"  kept {source}: {' '.join(sp.phones)}  thr {thr:.2f} "
+              f"(covers {kept:.0%} of voices; hardest neg {neg:.2f})")
+        spotters.append({"phones": list(sp.phones), "source": source,
+                         "threshold": thr})
+
+    if not spotters:
+        print("phrase refused: no spotter separable from negatives")
+        return
+
+    # coverage of the union
+    def hit(lp):
+        return any(KeywordSpotter(args.phrase, phones=s["phones"],
+                                  threshold=s["threshold"])
+                   .best_score(lp) > s["threshold"] for s in spotters)
+    union = sum(1 for lp in lps if hit(lp)) / len(lps)
+    print(f"union coverage across {len(lps)} voices: {union:.0%}")
+
+    out = {"phrase": args.phrase, "spotters": spotters,
+           "mode": "universal"}
+    ENROLL_DIR.mkdir(exist_ok=True)
+    path = ENROLL_DIR / f"{slug(args.phrase)}_universal.json"
+    path.write_text(json.dumps(out, indent=2), encoding="utf-8")
+    print(f"saved -> {path}")
+
+
+if __name__ == "__main__":
+    main()

phoneme_engine/features.py

@@ -0,0 +1,55 @@
+"""Log-mel feature frontend.
+
+40 log-mel bands, 16 kHz, 25 ms window / 10 ms hop. Deliberately matches
+what is cheap to compute on the ESP32-S3 later (esp-dsp / TFLite-Micro
+audio frontend), so the on-device features can be made to line up with
+training.
+"""
+
+import torch
+import torchaudio
+
+SAMPLE_RATE = 16000
+N_MELS = 40
+WIN_LENGTH = 400   # 25 ms
+HOP_LENGTH = 160   # 10 ms
+N_FFT = 512
+
+
+class LogMel(torch.nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.mel = torchaudio.transforms.MelSpectrogram(
+            sample_rate=SAMPLE_RATE,
+            n_fft=N_FFT,
+            win_length=WIN_LENGTH,
+            hop_length=HOP_LENGTH,
+            n_mels=N_MELS,
+            center=True,
+            power=2.0,
+        )
+
+    EMA_ALPHA = 0.02  # ~0.5 s time constant at 10 ms hop
+
+    def forward(self, waveform):
+        """waveform (B, samples) -> features (B, T, N_MELS).
+
+        Normalization is a causal per-channel EMA mean subtraction so the
+        exact same computation can run frame-by-frame on the device.
+        """
+        mel = self.mel(waveform)                    # (B, n_mels, T)
+        logmel = torch.log(mel + 1e-6)
+        a = self.EMA_ALPHA
+        ema = torchaudio.functional.lfilter(
+            logmel,
+            a_coeffs=torch.tensor([1.0, -(1.0 - a)], device=logmel.device),
+            b_coeffs=torch.tensor([a, 0.0], device=logmel.device),
+            clamp=False,
+        )
+        logmel = logmel - ema
+        return logmel.transpose(1, 2)               # (B, T, n_mels)
+
+
+def num_frames(num_samples):
+    """Frame count produced for a waveform length (center=True)."""
+    return num_samples // HOP_LENGTH + 1

phoneme_engine/live_demo.py

@@ -0,0 +1,174 @@
+"""Live microphone wake word demo.
+
+Type any phrase, it is enrolled instantly (no training) and spotted in the
+live mic stream:
+
+  python -m phoneme_engine.live_demo "hey orbit"
+
+Implementation notes:
+  - the causal model runs on a sliding 3 s window every 0.24 s; 0.24 s is
+    exactly 24 input frames = 12 output frames, so the emitted log-prob
+    stream stays frame-aligned (0.25 s = 12.5 frames caused stitching
+    artifacts and phantom detections)
+  - detections are energy-gated: the matched time span must contain real
+    audio well above the rolling noise floor, so silence/room noise cannot
+    fire no matter what the model hallucinates
+  - a mic level line is printed every few seconds for diagnosing input
+    device problems
+"""
+
+import argparse
+import queue
+import sys
+import time
+from collections import deque
+from pathlib import Path
+
+import numpy as np
+import sounddevice as sd
+import torch
+
+from .decoder import KeywordSpotter
+from .features import HOP_LENGTH, SAMPLE_RATE, LogMel
+from .model import PhonemeTCN
+from .phones import ids_to_phones
+
+ROOT = Path(__file__).resolve().parent.parent
+WINDOW_S = 3.0
+STEP_SAMPLES = 24 * HOP_LENGTH          # 0.24 s = exactly 12 output frames
+OUT_FRAMES_PER_STEP = 12
+SAMPLES_PER_OUT_FRAME = 2 * HOP_LENGTH  # 20 ms
+
+
+def dbfs(x):
+    rms = float(np.sqrt(np.mean(x ** 2)) + 1e-12)
+    return 20 * np.log10(rms)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("phrase")
+    ap.add_argument("--threshold", type=float, default=None,
+                    help="avg per-frame deficit tolerance (more negative "
+                         "= more lenient). Default: the enrollment's "
+                         "auto-calibrated threshold, else -4.4")
+    ap.add_argument("--ckpt", default=str(ROOT / "checkpoints" / "best.pt"))
+    ap.add_argument("--show-phones", action="store_true",
+                    help="print the greedy phoneme stream (debug)")
+    ap.add_argument("--min-speech-db", type=float, default=-50.0,
+                    help="absolute floor: matched span must be louder")
+    ap.add_argument("--enrolled", action="store_true",
+                    help="also match voice-enrolled pronunciation variants "
+                         "(see phoneme_engine.enroll)")
+    ap.add_argument("--profile", default=None,
+                    help="enrollment profile suffix, e.g. 'universal' "
+                         "loads enrollments/<phrase>_universal.json")
+    args = ap.parse_args()
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    frontend = LogMel().to(device).eval()
+    model = PhonemeTCN().to(device).eval()
+    state = torch.load(args.ckpt, map_location=device, weights_only=True)
+    model.load_state_dict(state["model"])
+    per = state.get("best_per")
+    print(f"loaded {args.ckpt}" + (f" (dev PER {per:.3f})" if per else ""))
+
+    if args.enrolled:
+        from .enroll import load_spotter_configs
+        configs = load_spotter_configs(args.phrase, profile=args.profile)
+    else:
+        thr = args.threshold if args.threshold is not None else -4.4
+        configs = [(None, thr)]
+    spotters = []
+    for phones, thr in configs:
+        if args.threshold is not None:
+            thr = args.threshold
+        sp = KeywordSpotter(args.phrase, threshold=thr, phones=phones)
+        spotters.append(sp)
+        print(f"enrolled {args.phrase!r} -> {' '.join(sp.phones)} "
+              f"(threshold {thr:.2f})")
+    print("listening... Ctrl+C to stop")
+
+    q = queue.Queue()
+
+    def callback(indata, frames, t, status):
+        if status:
+            print(status, file=sys.stderr)
+        q.put(indata[:, 0].copy())
+
+    window = np.zeros(int(WINDOW_S * SAMPLE_RATE), dtype=np.float32)
+    pending = np.zeros(0, dtype=np.float32)
+    # per-output-frame energy history, aligned with spotter.t
+    energy_db = deque(maxlen=2000)
+    noise_floor = deque(maxlen=400)     # ~8 s of frame energies
+    last_level_print = time.time()
+
+    with sd.InputStream(samplerate=SAMPLE_RATE, channels=1,
+                        dtype="float32", blocksize=STEP_SAMPLES // 2,
+                        callback=callback):
+        with torch.no_grad():
+            while True:
+                pending = np.concatenate([pending, q.get()])
+                while pending.shape[0] >= STEP_SAMPLES:
+                    chunk = pending[:STEP_SAMPLES]
+                    pending = pending[STEP_SAMPLES:]
+                    window = np.concatenate([window[STEP_SAMPLES:], chunk])
+
+                    for k in range(OUT_FRAMES_PER_STEP):
+                        seg = chunk[k * SAMPLES_PER_OUT_FRAME:
+                                    (k + 1) * SAMPLES_PER_OUT_FRAME]
+                        e = dbfs(seg)
+                        energy_db.append(e)
+                        noise_floor.append(e)
+
+                    if time.time() - last_level_print > 5.0:
+                        floor = np.percentile(noise_floor, 20)
+                        print(f"[mic level {energy_db[-1]:6.1f} dBFS, "
+                              f"noise floor {floor:6.1f}]", flush=True)
+                        last_level_print = time.time()
+
+                    wav = torch.from_numpy(window).unsqueeze(0).to(device)
+                    logits = model(frontend(wav))
+                    lp = torch.log_softmax(logits.float(), dim=2)[0]
+                    lp_new = lp[-OUT_FRAMES_PER_STEP:].cpu().numpy()
+                    if args.show_phones:
+                        ids = lp_new.argmax(axis=1).tolist()
+                        s = " ".join(ids_to_phones(ids))
+                        if s.strip():
+                            print(f"[{s}]", flush=True)
+                    for t_idx in range(lp_new.shape[0]):
+                        hit = None
+                        for sp in spotters:
+                            h = sp.step(lp_new[t_idx])
+                            if h is not None and (hit is None
+                                                  or h[0] > hit[0]):
+                                hit = h
+                        if hit is None:
+                            continue
+                        score, dur = hit
+                        # one utterance shouldn't fire multiple variants
+                        for sp in spotters:
+                            sp.reset()
+                            sp.cooldown = sp.refractory_frames
+                        span = list(energy_db)[-(dur + (OUT_FRAMES_PER_STEP
+                                                        - 1 - t_idx)):]
+                        span = span[:dur] if dur <= len(span) else span
+                        speech_db = float(np.mean(sorted(span)[len(span)//2:]))
+                        floor = float(np.percentile(noise_floor, 20))
+                        if speech_db < args.min_speech_db or \
+                                speech_db < floor + 6.0:
+                            print(f"(suppressed: score={score:.2f} but "
+                                  f"span energy {speech_db:.1f} dBFS ~ "
+                                  f"floor {floor:.1f})", flush=True)
+                            continue
+                        print(f"*** DETECTED {args.phrase!r} "
+                              f"score={score:.2f} "
+                              f"at {time.strftime('%H:%M:%S')} ***",
+                              flush=True)
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        print("\nstopped")

phoneme_engine/model.py

@@ -0,0 +1,72 @@
+"""Streaming phoneme recognizer: a small causal TCN with CTC output.
+
+Design constraints (for later ESP32-S3 deployment via TFLite-Micro):
+  - only Conv1d / BatchNorm / ReLU / residual add (all INT8-friendly and
+    supported by esp-nn optimized kernels once folded/exported)
+  - strictly causal (left padding only) so it can run on a live stream
+  - ~350k parameters -> ~400 KB at INT8, fits the S3 easily
+
+Input:  (B, T, 40) log-mel frames at 10 ms
+Output: (B, T', NUM_CLASSES) logits at 20 ms (stem has stride 2)
+"""
+
+import torch
+import torch.nn as nn
+
+from .phones import NUM_CLASSES
+
+
+class CausalConv1d(nn.Module):
+    """Conv1d with left-only padding (streaming safe)."""
+
+    def __init__(self, in_ch, out_ch, kernel, stride=1, dilation=1, groups=1):
+        super().__init__()
+        self.left_pad = dilation * (kernel - 1)
+        self.conv = nn.Conv1d(in_ch, out_ch, kernel, stride=stride,
+                              dilation=dilation, groups=groups)
+
+    def forward(self, x):
+        x = nn.functional.pad(x, (self.left_pad, 0))
+        return self.conv(x)
+
+
+class TCNBlock(nn.Module):
+    """Depthwise-separable causal conv block with residual."""
+
+    def __init__(self, channels, kernel, dilation):
+        super().__init__()
+        self.dw = CausalConv1d(channels, channels, kernel,
+                               dilation=dilation, groups=channels)
+        self.bn1 = nn.BatchNorm1d(channels)
+        self.pw = nn.Conv1d(channels, channels, 1)
+        self.bn2 = nn.BatchNorm1d(channels)
+        self.act = nn.ReLU()
+
+    def forward(self, x):
+        y = self.act(self.bn1(self.dw(x)))
+        y = self.bn2(self.pw(y))
+        return self.act(x + y)
+
+
+class PhonemeTCN(nn.Module):
+    def __init__(self, n_mels=40, channels=192, kernel=5,
+                 dilations=(1, 2, 4, 8, 1, 2, 4, 8)):
+        super().__init__()
+        self.stem = CausalConv1d(n_mels, channels, kernel, stride=2)
+        self.stem_bn = nn.BatchNorm1d(channels)
+        self.act = nn.ReLU()
+        self.blocks = nn.Sequential(
+            *[TCNBlock(channels, kernel, d) for d in dilations])
+        self.head = nn.Conv1d(channels, NUM_CLASSES, 1)
+
+    def forward(self, feats):
+        """feats (B, T, n_mels) -> logits (B, T//2, NUM_CLASSES)."""
+        x = feats.transpose(1, 2)              # (B, n_mels, T)
+        x = self.act(self.stem_bn(self.stem(x)))
+        x = self.blocks(x)
+        return self.head(x).transpose(1, 2)    # (B, T', classes)
+
+    @staticmethod
+    def out_lengths(in_lengths):
+        """Output frame count for input frame counts (stride-2 stem)."""
+        return torch.div(in_lengths - 1, 2, rounding_mode="floor") + 1

phoneme_engine/phones.py

@@ -0,0 +1,83 @@
+"""Phoneme inventory and lexicon for the phoneme engine.
+
+Uses the 39-phone stress-less ARPAbet set (CMUdict) plus CTC blank at
+index 0. Word pronunciations come from CMUdict, with grapheme-to-phoneme
+fallback for out-of-vocabulary words (names, made-up wake words).
+"""
+
+import functools
+import os
+import re
+from pathlib import Path
+
+import pronouncing
+
+# Pin NLTK data to a project-local folder. AppData can be virtualized
+# differently per shell (app-container redirects), which made resources
+# downloaded in one terminal invisible in another and tripped NLTK's
+# path-security checks.
+_NLTK_DIR = Path(__file__).resolve().parent.parent / "nltk_data"
+os.environ["NLTK_DATA"] = str(_NLTK_DIR)
+
+# 39 ARPAbet phones, stress stripped. Index 0 is reserved for CTC blank.
+PHONES = [
+    "AA", "AE", "AH", "AO", "AW", "AY", "B", "CH", "D", "DH",
+    "EH", "ER", "EY", "F", "G", "HH", "IH", "IY", "JH", "K",
+    "L", "M", "N", "NG", "OW", "OY", "P", "R", "S", "SH",
+    "T", "TH", "UH", "UW", "V", "W", "Y", "Z", "ZH",
+]
+BLANK = 0
+PHONE_TO_ID = {p: i + 1 for i, p in enumerate(PHONES)}
+ID_TO_PHONE = {i + 1: p for i, p in enumerate(PHONES)}
+NUM_CLASSES = len(PHONES) + 1  # 40 including blank
+
+_g2p = None
+
+
+def _ensure_nltk_data():
+    import nltk
+    _NLTK_DIR.mkdir(exist_ok=True)
+    if str(_NLTK_DIR) not in nltk.data.path:
+        nltk.data.path.insert(0, str(_NLTK_DIR))
+    for res, sub in (("averaged_perceptron_tagger_eng", "taggers"),
+                     ("averaged_perceptron_tagger", "taggers"),
+                     ("cmudict", "corpora")):
+        try:
+            nltk.data.find(f"{sub}/{res}")
+        except LookupError:
+            nltk.download(res, download_dir=str(_NLTK_DIR), quiet=True)
+
+
+def _g2p_phones(word):
+    global _g2p
+    if _g2p is None:
+        _ensure_nltk_data()
+        from g2p_en import G2p
+        _g2p = G2p()
+    return [re.sub(r"\d", "", p) for p in _g2p(word)
+            if re.match(r"^[A-Z]+$", re.sub(r"\d", "", p))]
+
+
+@functools.lru_cache(maxsize=200000)
+def word_phones(word):
+    """Phonemes for a single word (lowercase in, ARPAbet out)."""
+    prons = pronouncing.phones_for_word(word)
+    if prons:
+        return tuple(re.sub(r"\d", "", p) for p in prons[0].split())
+    return tuple(_g2p_phones(word))
+
+
+def text_to_phones(text):
+    """Transcript -> flat phoneme list. Words it can't phonemize are skipped."""
+    phones = []
+    for word in re.findall(r"[a-z']+", text.lower()):
+        phones.extend(word_phones(word))
+    return phones
+
+
+def text_to_ids(text):
+    return [PHONE_TO_ID[p] for p in text_to_phones(text) if p in PHONE_TO_ID]
+
+
+def ids_to_phones(ids):
+    return [ID_TO_PHONE[i] for i in ids if i in ID_TO_PHONE]

phoneme_engine/quantize.py

@@ -0,0 +1,368 @@
+"""Post-training INT8 quantization for the PhonemeTCN, targeting a small
+custom streaming engine on the ESP32-S3.
+
+Scheme (classic TFLite-style, but explicit and portable):
+  - weights: symmetric per-output-channel int8, BN folded into conv
+  - activations: per-tensor asymmetric-free symmetric int8 (ReLU outputs
+    use unsigned range via zero offset 0..127 semantics kept simple:
+    symmetric [-127,127] everywhere)
+  - accumulators: int32; requantization by fixed-point multiplier per layer
+
+Steps:
+  python -m phoneme_engine.quantize calibrate   # activation ranges
+  python -m phoneme_engine.quantize verify      # int8 sim vs float parity
+  python -m phoneme_engine.quantize export      # C header + weights bin
+"""
+
+import json
+import sys
+from pathlib import Path
+
+import numpy as np
+import torch
+
+from .data import LibriPhonemes, collate
+from .features import LogMel
+from .model import PhonemeTCN
+
+ROOT = Path(__file__).resolve().parent.parent
+QDIR = ROOT / "export"
+
+
+def fold_bn(conv_w, conv_b, bn):
+    """Fold BatchNorm into conv weights/bias (eval-mode running stats)."""
+    gamma = bn.weight.detach().numpy()
+    beta = bn.bias.detach().numpy()
+    mean = bn.running_mean.detach().numpy()
+    var = bn.running_var.detach().numpy()
+    scale = gamma / np.sqrt(var + bn.eps)
+    w = conv_w * scale[:, None, None]
+    b = (conv_b if conv_b is not None else 0.0) * scale + beta - mean * scale
+    return w, b
+
+
+def extract_layers(model):
+    """Flatten the model into a list of layer dicts with folded BN.
+
+    Layer kinds: conv (dense conv1d), dw (depthwise), each with
+    weight (out, in, k) / (ch, 1, k), bias, stride, dilation, plus
+    residual bookkeeping: blocks add their input to the pw output.
+    """
+    layers = []
+    w, b = fold_bn(model.stem.conv.weight.detach().numpy(),
+                   None if model.stem.conv.bias is None
+                   else model.stem.conv.bias.detach().numpy(),
+                   model.stem_bn)
+    layers.append(dict(kind="conv", w=w, b=b, stride=2, dilation=1,
+                       relu=True, residual=False))
+    for blk in model.blocks:
+        w, b = fold_bn(blk.dw.conv.weight.detach().numpy(),
+                       None if blk.dw.conv.bias is None
+                       else blk.dw.conv.bias.detach().numpy(),
+                       blk.bn1)
+        layers.append(dict(kind="dw", w=w, b=b, stride=1,
+                           dilation=blk.dw.conv.dilation[0], relu=True,
+                           residual=False))
+        w, b = fold_bn(blk.pw.weight.detach().numpy(),
+                       None if blk.pw.bias is None
+                       else blk.pw.bias.detach().numpy(),
+                       blk.bn2)
+        # pw output adds the block input, then ReLU
+        layers.append(dict(kind="conv", w=w, b=b, stride=1, dilation=1,
+                           relu=True, residual=True))
+    layers.append(dict(kind="conv",
+                       w=model.head.weight.detach().numpy(),
+                       b=model.head.bias.detach().numpy(),
+                       stride=1, dilation=1, relu=False, residual=False))
+    return layers
+
+
+def float_forward(layers, feats):
+    """Reference float forward pass on (T, 40) features using the flat
+    layer list. Must match the PyTorch model exactly (verified)."""
+    x = feats.T  # (C, T)
+    block_input = None
+    for lay in layers:
+        if lay["kind"] == "dw":
+            block_input = x  # residual adds the block's input, saved here
+        w, b = lay["w"], lay["b"]
+        k = w.shape[2]
+        d = lay["dilation"]
+        pad = d * (k - 1)
+        xin = np.pad(x, ((0, 0), (pad, 0)))
+        T = x.shape[1]
+        out_T = (T - 1) // lay["stride"] + 1
+        out_C = w.shape[0]
+        y = np.zeros((out_C, out_T), dtype=np.float64)
+        for t in range(out_T):
+            base = t * lay["stride"] + pad
+            taps = xin[:, [base - d * (k - 1 - i) for i in range(k)]]
+            if lay["kind"] == "dw":
+                y[:, t] = (taps * w[:, 0, :]).sum(axis=1) + b
+            else:
+                y[:, t] = np.tensordot(w, taps, axes=([1, 2], [0, 1])) + b
+        if lay["residual"]:
+            y = y + block_input[:, :out_T]
+        if lay["relu"]:
+            y = np.maximum(y, 0.0)
+        x = y
+    return x.T  # (T, classes)
+
+
+def collect_calibration_feats(n_utts=64):
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    frontend = LogMel().to(device).eval()
+    ds = LibriPhonemes(str(ROOT / "data"), "dev-clean")
+    feats = []
+    with torch.no_grad():
+        for i in range(0, n_utts * 40, 40):
+            wav, _ = ds[i % len(ds)]
+            f = frontend(wav.unsqueeze(0).to(device))[0].cpu().numpy()
+            feats.append(f[:400])  # up to 4 s per utterance
+            if len(feats) >= n_utts:
+                break
+    return feats
+
+
+def calibrate_scales(layers, feats_list, pctl=99.95):
+    """Per-layer activation scales from representative audio: runs the
+    float replay and records robust max-abs at every layer boundary."""
+    n_layers = len(layers)
+    maxima = [[] for _ in range(n_layers + 1)]  # +1 for the input feats
+    for feats in feats_list:
+        maxima[0].append(np.percentile(np.abs(feats), pctl))
+        x = feats.T
+        block_input = None
+        for li, lay in enumerate(layers):
+            if lay["kind"] == "dw":
+                block_input = x
+            w, b = lay["w"], lay["b"]
+            k = w.shape[2]
+            d = lay["dilation"]
+            pad = d * (k - 1)
+            xin = np.pad(x, ((0, 0), (pad, 0)))
+            out_T = (x.shape[1] - 1) // lay["stride"] + 1
+            y = np.zeros((w.shape[0], out_T))
+            for t in range(out_T):
+                base = t * lay["stride"] + pad
+                taps = xin[:, [base - d * (k - 1 - i) for i in range(k)]]
+                if lay["kind"] == "dw":
+                    y[:, t] = (taps * w[:, 0, :]).sum(axis=1) + b
+                else:
+                    y[:, t] = np.tensordot(w, taps,
+                                           axes=([1, 2], [0, 1])) + b
+            if lay["residual"]:
+                y = y + block_input[:, :out_T]
+            if lay["relu"]:
+                y = np.maximum(y, 0.0)
+            maxima[li + 1].append(np.percentile(np.abs(y), pctl))
+            x = y
+    scales = [max(float(np.max(m)), 1e-3) / 127.0 for m in maxima]
+    return scales
+
+
+def quantize_weights(layers):
+    """Symmetric per-output-channel int8 weights; returns quantized copies
+    (float values on the int8 grid) plus the raw int8 arrays and scales."""
+    qlayers = []
+    for lay in layers:
+        w = lay["w"]
+        s_w = np.abs(w).reshape(w.shape[0], -1).max(axis=1) / 127.0
+        s_w = np.maximum(s_w, 1e-8)
+        w_int = np.clip(np.round(w / s_w[:, None, None]), -127, 127)
+        q = dict(lay)
+        q["w"] = w_int * s_w[:, None, None]
+        q["w_int"] = w_int.astype(np.int8)
+        q["s_w"] = s_w
+        qlayers.append(q)
+    return qlayers
+
+
+def fake_quant_forward(qlayers, scales, feats):
+    """Float replay with activations snapped to the int8 grid at every
+    layer boundary -- numerically equivalent to the integer engine."""
+    def snap(x, s):
+        return np.clip(np.round(x / s), -127, 127) * s
+
+    x = snap(feats.T, scales[0])
+    block_input = None
+    block_input_scale = None
+    for li, lay in enumerate(qlayers):
+        if lay["kind"] == "dw":
+            block_input = x
+            block_input_scale = scales[li]
+        w, b = lay["w"], lay["b"]
+        k = w.shape[2]
+        d = lay["dilation"]
+        pad = d * (k - 1)
+        xin = np.pad(x, ((0, 0), (pad, 0)))
+        out_T = (x.shape[1] - 1) // lay["stride"] + 1
+        y = np.zeros((w.shape[0], out_T))
+        for t in range(out_T):
+            base = t * lay["stride"] + pad
+            taps = xin[:, [base - d * (k - 1 - i) for i in range(k)]]
+            if lay["kind"] == "dw":
+                y[:, t] = (taps * w[:, 0, :]).sum(axis=1) + b
+            else:
+                y[:, t] = np.tensordot(w, taps, axes=([1, 2], [0, 1])) + b
+        if lay["residual"]:
+            y = y + snap(block_input[:, :out_T], block_input_scale)
+        if lay["relu"]:
+            y = np.maximum(y, 0.0)
+        x = snap(y, scales[li + 1])
+    return x.T
+
+
+def main():
+    cmd = sys.argv[1] if len(sys.argv) > 1 else "verify"
+    QDIR.mkdir(exist_ok=True)
+    device = "cpu"
+    model = PhonemeTCN().eval()
+    state = torch.load(ROOT / "checkpoints" / "best.pt",
+                       map_location=device, weights_only=True)
+    model.load_state_dict(state["model"])
+    layers = extract_layers(model)
+    print(f"{len(layers)} layers extracted")
+
+    if cmd == "sanity":
+        feats = collect_calibration_feats(2)
+        x = torch.from_numpy(feats[0]).float().unsqueeze(0)
+        with torch.no_grad():
+            ref = model(x)[0].numpy()
+        ours = float_forward(layers, feats[0])
+        err = np.abs(ref - ours).max()
+        print(f"float reference vs pytorch max abs err: {err:.2e}")
+        assert err < 1e-3, "layer extraction is wrong"
+        print("sanity OK")
+
+    elif cmd == "calibrate":
+        feats_list = collect_calibration_feats(48)
+        print(f"calibrating on {len(feats_list)} utterances...")
+        scales = calibrate_scales(layers, feats_list)
+        (QDIR / "act_scales.json").write_text(json.dumps(scales))
+        print("activation scales:", [round(s, 5) for s in scales])
+        print(f"saved -> {QDIR / 'act_scales.json'}")
+
+    elif cmd == "verify":
+        scales = json.loads((QDIR / "act_scales.json").read_text())
+        qlayers = quantize_weights(layers)
+        from .decoder import KeywordSpotter
+        from .spot_file import load_wav
+        from .features import LogMel
+        device = "cuda" if torch.cuda.is_available() else "cpu"
+        frontend = LogMel().to(device).eval()
+
+        agree = tot = 0
+        deltas = []
+        clips = [(f, "sakura") for f in
+                 sorted((ROOT / "test_clips" / "sakura_piper").glob("*.wav"))[:8]]
+        clips += [(f, "hey orbit") for f in
+                  sorted((ROOT / "test_clips" / "hey_orbit_piper").glob("*.wav"))[:8]]
+        clips.append((ROOT / "diag_last.wav", "hey orbit"))
+        for f, phrase in clips:
+            wav = load_wav(str(f), device)
+            with torch.no_grad():
+                feats = frontend(wav)[0].cpu().numpy()
+            lf = float_forward(layers, feats)
+            lq = fake_quant_forward(qlayers, scales, feats)
+            agree += (lf.argmax(1) == lq.argmax(1)).sum()
+            tot += lf.shape[0]
+            # spotting parity: decoder consumes logit differences directly
+            sp = KeywordSpotter(phrase)
+            sf_ = sp.best_score(lf - lf.max(axis=1, keepdims=True))
+            sq_ = sp.best_score(lq - lq.max(axis=1, keepdims=True))
+            if np.isfinite(sf_) or np.isfinite(sq_):
+                deltas.append(abs(sf_ - sq_))
+            print(f"  {f.name} [{phrase}]: float {sf_:7.2f}  int8 {sq_:7.2f}")
+        print(f"frame argmax agreement: {agree/tot:.4f}")
+        finite = [d for d in deltas if np.isfinite(d)]
+        print(f"spot score delta (finite pairs): max {max(finite):.3f} "
+              f"mean {np.mean(finite):.3f}; gate flips: "
+              f"{len(deltas) - len(finite)}")
+
+    elif cmd == "export":
+        scales = json.loads((QDIR / "act_scales.json").read_text())
+        qlayers = quantize_weights(layers)
+        from .phones import PHONES
+        lines = ["// Auto-generated by phoneme_engine.quantize export",
+                 "// PhonemeTCN int8 weights + scales for the streaming",
+                 "// wake word engine. Do not edit by hand.",
+                 "#pragma once", "#include <stdint.h>", ""]
+        lines.append(f"#define PWW_NUM_LAYERS {len(qlayers)}")
+        lines.append(f"#define PWW_NUM_CLASSES {len(PHONES) + 1}")
+        lines.append(f"#define PWW_INPUT_SCALE {scales[0]:.8f}f")
+        lines.append(f"#define PWW_LOGIT_SCALE {scales[-1]:.8f}f")
+        lines.append("")
+        phones_str = ", ".join(f'"{p}"' for p in PHONES)
+        lines.append(f"static const char *PWW_PHONES[] = {{{phones_str}}};")
+        lines.append("")
+        meta_rows = []
+        total_bytes = 0
+        for li, lay in enumerate(qlayers):
+            w_int = lay["w_int"]
+            out_c, in_c, k = w_int.shape
+            flat = w_int.flatten()
+            total_bytes += flat.size
+            arr = ", ".join(str(int(v)) for v in flat)
+            lines.append(f"static const int8_t PWW_W{li}[] = {{{arr}}};")
+            # combined scale per channel: s_in * s_w[c]  (float requant)
+            comb = scales[li] * lay["s_w"]
+            arr = ", ".join(f"{v:.8e}f" for v in comb)
+            lines.append(f"static const float PWW_S{li}[] = {{{arr}}};")
+            arr = ", ".join(f"{v:.8e}f" for v in lay["b"])
+            lines.append(f"static const float PWW_B{li}[] = {{{arr}}};")
+            # depthwise weights are stored (ch, 1, k): the layer's true
+            # input width is out_c, not the stored dim
+            eff_in = out_c if lay["kind"] == "dw" else in_c
+            meta_rows.append(
+                f"  {{{1 if lay['kind'] == 'dw' else 0}, {eff_in}, {out_c}, "
+                f"{k}, {lay['stride']}, {lay['dilation']}, "
+                f"{1 if lay['relu'] else 0}, {1 if lay['residual'] else 0}, "
+                f"PWW_W{li}, PWW_S{li}, PWW_B{li}, "
+                f"{scales[li + 1]:.8f}f}}")
+            lines.append("")
+        lines.append(
+            "typedef struct { uint8_t is_dw; uint16_t in_c, out_c; "
+            "uint8_t k, stride, dilation, relu, residual; "
+            "const int8_t *w; const float *s; const float *b; "
+            "float out_scale; } pww_layer_t;")
+        lines.append("")
+        lines.append("static const pww_layer_t PWW_LAYERS[] = {")
+        lines.append(",\n".join(meta_rows))
+        lines.append("};")
+        path = QDIR / "model_int8.h"
+        path.write_text("\n".join(lines), encoding="utf-8")
+        print(f"exported {total_bytes/1024:.0f} KB of int8 weights "
+              f"-> {path}")
+
+    elif cmd == "export-frontend":
+        # exact DSP constants from the training frontend, so the C mel
+        # frontend is identical by construction
+        from .features import (HOP_LENGTH, N_FFT, N_MELS, SAMPLE_RATE,
+                               WIN_LENGTH, LogMel)
+        fe = LogMel()
+        fb = fe.mel.mel_scale.fb.numpy()          # (n_freqs, n_mels)
+        win = torch.hann_window(WIN_LENGTH, periodic=True).numpy()
+        lines = ["// Auto-generated: mel frontend constants (exact copy of",
+                 "// the training features). Do not edit.",
+                 "#pragma once", ""]
+        lines.append(f"#define PWW_FE_SR {SAMPLE_RATE}")
+        lines.append(f"#define PWW_FE_NFFT {N_FFT}")
+        lines.append(f"#define PWW_FE_WIN {WIN_LENGTH}")
+        lines.append(f"#define PWW_FE_HOP {HOP_LENGTH}")
+        lines.append(f"#define PWW_FE_NMELS {N_MELS}")
+        lines.append(f"#define PWW_FE_NFREQS {fb.shape[0]}")
+        lines.append("#define PWW_FE_EMA_ALPHA 0.02f")
+        lines.append("")
+        arr = ", ".join(f"{v:.8e}f" for v in win)
+        lines.append(f"static const float PWW_FE_HANN[] = {{{arr}}};")
+        arr = ", ".join(f"{v:.8e}f" for v in fb.T.flatten())
+        lines.append("// mel filterbank, row-major (n_mels, n_freqs)")
+        lines.append(f"static const float PWW_FE_MELFB[] = {{{arr}}};")
+        path = QDIR / "frontend_data.h"
+        path.write_text("\n".join(lines), encoding="utf-8")
+        print(f"exported frontend constants -> {path}")
+
+
+if __name__ == "__main__":
+    main()

phoneme_engine/score_phrase.py

@@ -0,0 +1,340 @@
+"""Wake word phrase scorer.
+
+Scores a candidate wake word phrase on how resistant it will be to false
+activations, using phoneme statistics of spoken English:
+
+  1. rarity      - mean surprisal (-log2 P) of the phrase's phoneme bigrams,
+                   estimated from CMUdict pronunciations weighted by real
+                   word-usage frequency (wordfreq). Rare sound sequences are
+                   less likely to appear in everyday speech.
+  2. confusability - minimum weighted phonetic edit distance (normalized)
+                   between the phrase and common words / common two-word
+                   concatenations. Low distance = easy to false-trigger.
+  3. syllables   - counts vowel nuclei; 3-5 syllables is the sweet spot for
+                   small streaming models like microWakeWord.
+  4. diversity   - fraction of distinct phonemes; repeated sounds carry less
+                   discriminative information.
+
+Usage:
+  python score_phrase.py "hey jarvis"            # score one phrase
+  python score_phrase.py --rank candidates.txt   # rank a file of phrases
+  python score_phrase.py --suggest 8             # auto-generate + rank candidates
+"""
+
+import argparse
+import functools
+import json
+import math
+import random
+import re
+import sys
+
+import pronouncing
+from wordfreq import top_n_list, word_frequency
+
+VOWELS = {
+    "AA", "AE", "AH", "AO", "AW", "AY", "EH", "ER", "EY",
+    "IH", "IY", "OW", "OY", "UH", "UW",
+}
+
+# Broad phonetic classes used to soften substitution costs: confusing two
+# sounds from the same class is cheaper (more likely for a model) than
+# crossing classes.
+PHONE_CLASS = {}
+for _class, _phones in {
+    "vowel": VOWELS,
+    "stop": {"B", "D", "G", "K", "P", "T"},
+    "affricate": {"CH", "JH"},
+    "fricative": {"DH", "F", "HH", "S", "SH", "TH", "V", "Z", "ZH"},
+    "nasal": {"M", "N", "NG"},
+    "liquid": {"L", "R"},
+    "glide": {"W", "Y"},
+}.items():
+    for _p in _phones:
+        PHONE_CLASS[_p] = _class
+
+# Acoustic reliability of each phone for a small on-device model,
+# validated empirically with live tests: V/F/TH-type weak fricatives get
+# swallowed or confused; nasals, stops and sibilants are heard stably.
+RELIABILITY = {}
+for _score, _phones in [
+    (1.0, ["N", "M", "K", "T", "D", "P", "B", "G", "S", "SH", "CH", "JH",
+           "NG", "Z"]),
+    (0.8, ["AA", "AE", "AH", "AO", "AW", "AY", "EH", "ER", "EY", "IH",
+           "IY", "OW", "OY", "UH", "UW", "L", "R"]),
+    (0.5, ["F", "V", "W", "Y", "HH", "TH", "DH", "ZH"]),
+]:
+    for _p in _phones:
+        RELIABILITY[_p] = _score
+
+_g2p = None
+
+
+def g2p_word(word):
+    """Phonemes for an out-of-vocabulary word via letter-to-sound rules."""
+    global _g2p
+    if _g2p is None:
+        from g2p_en import G2p
+        _g2p = G2p()
+    return [re.sub(r"\d", "", p) for p in _g2p(word) if re.match(r"[A-Z]", p)]
+
+
+def phones_for_word(word):
+    """CMUdict pronunciation if available, else grapheme-to-phoneme."""
+    prons = pronouncing.phones_for_word(word.lower())
+    if prons:
+        return re.sub(r"\d", "", prons[0]).split()
+    return g2p_word(word)
+
+
+def phones_for_phrase(phrase):
+    phones = []
+    for word in re.findall(r"[a-zA-Z']+", phrase):
+        phones.extend(phones_for_word(word))
+    return phones
+
+
+@functools.lru_cache(maxsize=1)
+def bigram_model(vocab_size=30000):
+    """Frequency-weighted phoneme bigram probabilities for spoken English.
+
+    '#' marks a word boundary so cross-word statistics are approximated.
+    Returns (probabilities, common_word_phones) where common_word_phones is
+    the pronunciation cache used by the confusability search.
+    """
+    counts = {}
+    total = 0.0
+    word_phones = {}
+    for word in top_n_list("en", vocab_size):
+        prons = pronouncing.phones_for_word(word)
+        if not prons:
+            continue
+        phones = re.sub(r"\d", "", prons[0]).split()
+        if not phones:
+            continue
+        word_phones[word] = phones
+        weight = word_frequency(word, "en")
+        seq = ["#"] + phones + ["#"]
+        for a, b in zip(seq, seq[1:]):
+            counts[(a, b)] = counts.get((a, b), 0.0) + weight
+            total += weight
+    n_types = max(len(counts), 1)
+    probs = {k: v / total for k, v in counts.items()}
+    floor = 0.01 / (total * n_types)  # smoothing for unseen bigrams
+    return probs, floor, word_phones
+
+
+def rarity(phones):
+    """Mean surprisal in bits of the phrase's phoneme bigrams."""
+    probs, floor, _ = bigram_model()
+    seq = ["#"] + phones + ["#"]
+    bits = [-math.log2(probs.get((a, b), floor)) for a, b in zip(seq, seq[1:])]
+    return sum(bits) / len(bits)
+
+
+def sub_cost(a, b):
+    if a == b:
+        return 0.0
+    if PHONE_CLASS.get(a) == PHONE_CLASS.get(b):
+        return 0.5
+    return 1.0
+
+
+def edit_distance(a, b, bail_above=None):
+    """Weighted phonetic Levenshtein distance with optional early exit."""
+    prev = [i * 1.0 for i in range(len(b) + 1)]
+    for i, pa in enumerate(a, 1):
+        cur = [i * 1.0]
+        for j, pb in enumerate(b, 1):
+            cur.append(min(prev[j] + 1.0, cur[j - 1] + 1.0,
+                           prev[j - 1] + sub_cost(pa, pb)))
+        if bail_above is not None and min(cur) > bail_above:
+            return bail_above + 1.0
+        prev = cur
+    return prev[-1]
+
+
+def confusability(phones, n_single=3000, n_pairs=400):
+    """Min normalized phonetic distance to common words and word pairs.
+
+    Higher = safer. Distances are normalized by the longer sequence length so
+    short and long phrases are comparable.
+    """
+    _, _, word_phones = bigram_model()
+    singles = [w for w in top_n_list("en", n_single) if w in word_phones]
+    pair_words = [w for w in singles[:n_pairs]]
+
+    best = float("inf")
+    best_match = None
+    for w in singles:
+        wp = word_phones[w]
+        norm = max(len(phones), len(wp))
+        d = edit_distance(phones, wp, bail_above=best * norm) / norm
+        if d < best:
+            best, best_match = d, w
+
+    # Two-word concatenations approximate the phrase appearing inside
+    # continuous speech. Prune by length: only pairs within +/-4 phones.
+    target_len = len(phones)
+    for w1 in pair_words:
+        p1 = word_phones[w1]
+        if len(p1) >= target_len + 4:
+            continue
+        for w2 in pair_words:
+            cat = p1 + word_phones[w2]
+            if abs(len(cat) - target_len) > 4:
+                continue
+            norm = max(target_len, len(cat))
+            d = edit_distance(phones, cat, bail_above=best * norm) / norm
+            if d < best:
+                best, best_match = d, f"{w1} {w2}"
+    return best, best_match
+
+
+def syllable_count(phones):
+    return sum(1 for p in phones if p in VOWELS)
+
+
+def score_phrase(phrase, fast=False):
+    phones = phones_for_phrase(phrase)
+    if not phones:
+        raise ValueError(f"could not phonemize: {phrase!r}")
+    syl = syllable_count(phones)
+    div = len(set(phones)) / len(phones)
+    rar = rarity(phones)
+    conf, nearest = (None, None)
+    if not fast:
+        conf, nearest = confusability(phones)
+
+    # Syllable sweet spot: full credit for 3-5, penalized outside.
+    syl_score = {0: 0.0, 1: 0.2, 2: 0.6, 3: 1.0, 4: 1.0, 5: 1.0}.get(syl, 0.8)
+
+    # Acoustic reliability: mean over phones, with the onset (first phone)
+    # triple-weighted -- a swallowed first sound loses the whole match.
+    rel_vals = [RELIABILITY.get(p, 0.7) for p in phones]
+    reliability = (2 * rel_vals[0] + sum(rel_vals)) / (2 + len(rel_vals))
+
+    # Composite on a 0-100 scale. Weights favor confusability (the direct
+    # false-trigger proxy), rarity, and acoustic reliability.
+    parts = [
+        ("rarity", min(rar / 14.0, 1.0), 0.25),
+        ("confusability", min((conf or 0) / 0.45, 1.0), 0.30),
+        ("reliability", reliability, 0.25),
+        ("syllables", syl_score, 0.15),
+        ("diversity", div, 0.05),
+    ]
+    total = sum(v * w for _, v, w in parts) * 100
+
+    return {
+        "phrase": phrase,
+        "phones": " ".join(phones),
+        "syllables": syl,
+        "rarity_bits": round(rar, 2),
+        "min_distance": round(conf, 3) if conf is not None else None,
+        "nearest_common": nearest,
+        "reliability": round(reliability, 2),
+        "diversity": round(div, 2),
+        "score": round(total, 1),
+    }
+
+
+# --- candidate generation ---------------------------------------------------
+
+RARE_ONSETS = ["Z", "ZH", "V", "TH", "SH", "CH", "JH", "G", "K", "DR", "GR",
+               "KW", "SK", "SN", "PL", "KL", "TR", "FL"]
+NUCLEI = ["AY", "OY", "AW", "EY", "OW", "IY", "UW", "AA", "ER"]
+CODAS = ["", "K", "S", "KS", "N", "M", "SH", "Z", "NT", "RD", "L"]
+
+# Respellings so TTS engines pronounce generated names predictably.
+SPELL = {
+    "Z": "z", "ZH": "zh", "V": "v", "TH": "th", "SH": "sh", "CH": "ch",
+    "JH": "j", "G": "g", "K": "k", "DR": "dr", "GR": "gr", "KW": "qu",
+    "SK": "sk", "SN": "sn", "PL": "pl", "KL": "cl", "TR": "tr", "FL": "fl",
+    "AY": "y", "OY": "oy", "AW": "ow", "EY": "ay", "OW": "o", "IY": "ee",
+    "UW": "oo", "AA": "a", "ER": "er",
+    "S": "s", "KS": "x", "N": "n", "M": "m", "NT": "nt", "RD": "rd",
+    "L": "l", "": "",
+}
+
+
+def generate_candidates(n=200, seed=7):
+    rng = random.Random(seed)
+    out = set()
+    while len(out) < n:
+        syls = []
+        for _ in range(rng.choice([2, 3, 3])):
+            syls.append(rng.choice(RARE_ONSETS) + "|" + rng.choice(NUCLEI)
+                        + "|" + rng.choice(CODAS))
+        name = "".join(SPELL[p] for s in syls for p in s.split("|"))
+        if len(name) < 4:
+            continue
+        prefix = rng.choice(["hey ", "okay ", ""])
+        out.add(prefix + name)
+    return sorted(out)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("phrase", nargs="?", help="phrase to score")
+    ap.add_argument("--rank", help="file with one phrase per line")
+    ap.add_argument("--suggest", type=int, metavar="N",
+                    help="generate candidates and print top N")
+    ap.add_argument("--json", action="store_true")
+    args = ap.parse_args()
+
+    if args.phrase:
+        result = score_phrase(args.phrase)
+        print(json.dumps(result, indent=2) if args.json else format_row(result, header=True))
+        return
+
+    phrases = []
+    if args.rank:
+        with open(args.rank, encoding="utf-8") as f:
+            phrases = [l.strip() for l in f if l.strip()]
+    elif args.suggest:
+        print("generating and pre-screening candidates...", file=sys.stderr)
+        cands = generate_candidates()
+        # cheap pass on rarity/syllables first, full scoring on survivors
+        pre = []
+        for c in cands:
+            try:
+                ph = phones_for_phrase(c)
+            except Exception:
+                continue
+            if 3 <= syllable_count(ph) <= 5:
+                pre.append((rarity(ph), c))
+        pre.sort(reverse=True)
+        phrases = [c for _, c in pre[: args.suggest * 3]]
+    else:
+        ap.print_help()
+        return
+
+    results = []
+    for p in phrases:
+        try:
+            results.append(score_phrase(p))
+        except ValueError as e:
+            print(f"skipped: {e}", file=sys.stderr)
+    results.sort(key=lambda r: -r["score"])
+    if args.suggest:
+        results = results[: args.suggest]
+    if args.json:
+        print(json.dumps(results, indent=2))
+    else:
+        for i, r in enumerate(results):
+            print(format_row(r, header=(i == 0)))
+
+
+def format_row(r, header=False):
+    row = (f"{r['score']:6.1f}  {r['phrase']:<20} syl={r['syllables']} "
+           f"rel={r['reliability']:.2f} rarity={r['rarity_bits']:5.2f}b "
+           f"dist={r['min_distance']} (nearest: {r['nearest_common']})  "
+           f"[{r['phones']}]")
+    if header:
+        return (" score  phrase               details\n"
+                " -----  ------               -------\n" + row)
+    return row
+
+
+if __name__ == "__main__":
+    main()

phoneme_engine/spot_file.py

@@ -0,0 +1,76 @@
+"""Offline spotting: run a wake word over a wav file (or a directory).
+
+  python -m phoneme_engine.spot_file "hey orbit" clip.wav --threshold -2.5
+
+Prints detections with timestamps, plus the greedy phoneme transcription
+with --show-phones. Used for automated testing (TTS-generated positives,
+LibriSpeech negatives) without a microphone.
+"""
+
+import argparse
+from pathlib import Path
+
+import soundfile as sf
+import torch
+import torchaudio
+
+from .decoder import KeywordSpotter
+from .features import SAMPLE_RATE, LogMel
+from .model import PhonemeTCN
+from .phones import ids_to_phones
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+def load_wav(path, device):
+    audio, sr = sf.read(str(path), dtype="float32")
+    wav = torch.from_numpy(audio)
+    if wav.ndim == 2:
+        wav = wav.mean(dim=1)
+    wav = wav.unsqueeze(0)
+    if sr != SAMPLE_RATE:
+        wav = torchaudio.functional.resample(wav, sr, SAMPLE_RATE)
+    return wav.to(device)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("phrase")
+    ap.add_argument("path")
+    ap.add_argument("--threshold", type=float, default=-4.4)
+    ap.add_argument("--ckpt", default=str(ROOT / "checkpoints" / "best.pt"))
+    ap.add_argument("--show-phones", action="store_true")
+    args = ap.parse_args()
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    frontend = LogMel().to(device).eval()
+    model = PhonemeTCN().to(device).eval()
+    state = torch.load(args.ckpt, map_location=device, weights_only=True)
+    model.load_state_dict(state["model"])
+
+    p = Path(args.path)
+    files = sorted(p.glob("**/*.wav")) + sorted(p.glob("**/*.flac")) \
+        if p.is_dir() else [p]
+
+    total_hits = 0
+    with torch.no_grad():
+        for f in files:
+            wav = load_wav(f, device)
+            logits = model(frontend(wav))
+            lp = torch.log_softmax(logits.float(), dim=2)[0].cpu().numpy()
+            if args.show_phones:
+                ids = lp.argmax(axis=1).tolist()
+                print(f"{f.name} phones: {' '.join(ids_to_phones(ids))}")
+            spotter = KeywordSpotter(args.phrase, threshold=args.threshold)
+            hits = spotter.run(lp)
+            total_hits += len(hits)
+            for frame, score in hits:
+                print(f"{f.name}: DETECT at {frame * 0.02:.2f}s "
+                      f"score={score:.2f}")
+            if not hits:
+                print(f"{f.name}: no detection")
+    print(f"total detections: {total_hits} across {len(files)} file(s)")
+
+
+if __name__ == "__main__":
+    main()

phoneme_tcn_student.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78477bd4175f161df058b48ff084cfd01ab05c1f980902bc9c8f62d32cba03f5
+size 4410358

phoneme_tcn_teacher.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dae5cdd457d97d4297c0392389622ca348202c1a451f5c04e79bc007d81b6f10
+size 40277054