docs/HOW_IT_WORKS.md

# How Fugu-Lite Works

## Purpose

Fugu-Lite is a compact, Fugu-inspired learned router. A frozen local language model reads a
task and a small trainable head chooses one external worker. The chosen worker then answers
through OpenRouter or an OpenAI-compatible endpoint.

It is not Sakana AI's proprietary Fugu implementation. V1 makes one routing decision per
prompt; it does not yet perform recursive delegation, multi-turn planning, or answer
synthesis.

## Runtime path

1. The task is normalized into a prompt plus an optional domain label.
2. `Qwen/Qwen3-0.6B` encodes the routing prompt locally.
3. A LayerNorm and linear classification head produce one probability per worker.
4. The highest-probability worker is selected.
5. The provider client calls that worker and records the served model, response, latency,
   token usage, cost when reported, and fallback errors.
6. The CLI can return only the route, call the selected worker, or expose the router through
   an OpenAI-compatible HTTP endpoint.

## Learning path

```mermaid
flowchart LR
    A[Benchmark tasks] --> B[Call every worker]
    B --> C[Grade every answer]
    C --> D[Reward matrix JSONL]
    D --> E[SFT routing head]
    E --> F[Optional contextual-bandit RL]
    F --> G[Optional Sep-CMA-ES]
    G --> H[Held-out evaluation]
```

The backbone stays frozen. Only the small routing head is trained, which keeps local GPU
requirements modest and makes offline experimentation inexpensive after reward collection.

- SFT converts each reward vector into a soft target distribution and trains the head with
  cross-entropy.
- Contextual-bandit RL can optimize expected or sampled reward when the reward matrix has
  useful differences between workers.
- Sep-CMA-ES caches backbone features and evolves the final linear layer. This supports
  objectives that are awkward to differentiate.

## Reward data

Each task is sent to every configured worker. With repetitions enabled, worker rewards are
averaged before the row is saved. The row preserves the individual responses and operational
measurements for auditability.

The default utility is quality minus configurable cost and latency penalties. The V1 real
run set both penalties to zero, so its recorded routing reward is answer quality only.

## Normal workflow

```bash
# Validate the machine and configuration
fugu-lite doctor
pytest -q

# Collect or resume reward data
fugu-lite generate \
  --workers configs/workers.real-cheap.yaml \
  --tasks data/tasks.real-300.jsonl \
  --output data/rewards.real-300.jsonl \
  --resume

# Train and evaluate a routing head
fugu-lite train-sft \
  --config configs/train_sft.yaml \
  --data data/rewards.real-300.jsonl \
  --output artifacts/router-sft-real-v1

fugu-lite evaluate \
  --checkpoint artifacts/router-sft-real-v1 \
  --data data/rewards.real-300.jsonl \
  --split test \
  --output artifacts/eval-sft-real-v1.json
```

Use `route` to inspect the selected worker without an API call, `ask` to route and call a
worker, and `serve` to expose one OpenAI-compatible endpoint.

## Safety and reproducibility

- Python 3.10-3.13 is supported; Python 3.14 is intentionally excluded by the package.
- `.env` and virtual environments are ignored. Only `.env.example` is stored.
- The real reward snapshot contains benchmark prompts, worker responses, generation IDs,
  cost, latency, and token counts. Treat it as experiment data, not as a secret store.
- Do not publish the repository publicly without reviewing the redistribution terms for the
  benchmark rows and tokenizer files.
- A router is useful only if it beats the best fixed worker on held-out utility. High training
  accuracy or validation utility alone is not sufficient.