loudink-v1.1 / STATUS.md
nsalerni's picture
loudink-v1.1 1.1.0: skills PR-A–E canary (record/replay, slots, expanded daily-mac)
d506628 verified
|
Raw
History Blame
2.71 kB
# loudink-v1.1 status — **PR-A→E complete**
Last updated: 2026-07-08
## Hugging Face
| Line | Repo | Role |
|------|------|------|
| **Stable GA** | [nsalerni/loudink-v1](https://huggingface.co/nsalerni/loudink-v1) | **0.4.0** freeze |
| **Canary** | [nsalerni/loudink-v1.1](https://huggingface.co/nsalerni/loudink-v1.1) | **1.1.0** skills + expanded daily-mac |
Rebuild canary:
```bash
python -m gemmaflow_tune.cli.build_loudink_v1_1_publish --force
python -m gemmaflow_tune.cli.publish_sota --model loudink-v1.1 \
--commit-message "loudink-v1.1 1.1.0: skills PR-A–E canary"
```
## PR board
| PR | Deliverable | Status |
|----|-------------|--------|
| **PR-A** | Skill store + recorder schema + compiler + null-safe IR | **done** |
| **PR-B** | Paraphrase invoke + skill bench + IR labels + runner wiring | **done** (19/19) |
| **PR-C** | Slot fill (`for` / `about` / named) | **done** |
| **PR-D** | Daily-mac expand (Focus/DND, Reminders, reply, tabs, Notion/Linear, writing) | **done** (69/69) |
| **PR-E** | Vision fallback scaffold (disabled by default) | **done** |
## Headline metrics (canary)
| Metric | Bar | Result |
|--------|-----|--------|
| Skill invoke (n=19) | ≥95% named / ≥85% paraphrase | **100%** shipped |
| Daily-Mac overall (n=69) | no drop vs 0.4 | **100%** shipped |
| Writing neural | hold ≥80% | **92%** |
| Writing p50 | ≤250 ms | **171 ms** |
| Computer / multi-step | hold 100% | **100%** |
| IR honesty (n=112) | ≥45% neural | **46.4%** (v0.4 lineage) |
### Daily-Mac scorecard (v1.1 expanded)
| Slice | Shipped | Neural | p50 |
|-------|---------|--------|-----|
| Writing (n=25) | **100%** | **92%** | **171 ms** |
| Computer (n=44) | **100%** | — | ~0 ms FP |
| Multi-step (n=25) | **100%** | — | ~0 ms FP |
| **Overall (n=69)** | **100%** | — | mean ~60 ms |
Scorecard: `artifacts/loudink_v1/product_scorecard.json`
Skill bench: `artifacts/benchmarks/loudink_v1_1/skill_invoke/`
## Production stack (v1.1 canary)
```
writer: artifacts/sft_loudink_v1_rft_daily/adapters
polish: structural (+ daily rules)
ir: artifacts/sft_loudink_v1_1_skill_invoke_ir/adapters
ir_fallback: artifacts/sft_loudink_v1_ir_honesty/adapters
skills: SkillStore (example seed in publish bundle)
runner: loudink_v1 + skill_store_path
bench: configs/benchmark_loudink_v1_daily_mac.yaml
configs/benchmark_loudink_v1_1_skill_invoke.yaml
```
## Out of scope (later product)
- Client recorder UI (Flowcast owns capture UX)
- Optional real vision weights/resolver
- AirDrop / Bluetooth / long-horizon 1:1 prep suites (roadmap backlog)
- Training cloud models on private user recordings (forbidden by default)