USAGE.md

# Universal usage recipe

The same 4 steps work on every platform (transformers.js, Python, Android, iOS).
Only the ONNX runtime and tokenizer *binding* differ; the algorithm is identical.

## Artifacts

- `onnx/encoder_model.onnx` — encoder (int8)
- `onnx/decoder_model_merged.onnx` — decoder with KV cache, handles first + cached steps (int8)
- `tokenizer.json` — HuggingFace fast tokenizer (validated; loads in `tokenizers`, transformers.js, DJL, swift-transformers)
- `lang_tokens.json` — `{ "lang_to_id": {"en":128022,...}, "eos":2, "pad":1, "unk":3, "decoder_start":2 }`

## Algorithm

Given `text` and target language `tgt` (e.g. `"en"`):

1. **Encode + prepend target-language token** (SMaLL-100 puts the target token on
   the *source*; the tokenizer already appends `</s>`=2):

   ```
   ids = [ lang_to_id[tgt] ] + tokenizer.encode(text).ids      # ends with 2 (</s>)
   attention_mask = [1] * len(ids)
   ```

2. **Run the encoder** once:

   ```
   encoder_hidden_states = encoder(input_ids=ids, attention_mask)
   ```

3. **Greedy decode** with the merged decoder (KV cache). Start from
   `decoder_start_token_id = 2`. The merged decoder takes a `use_cache_branch`
   flag: `false` on the first step (empty past), `true` afterwards.

   ```
   cur = 2 ; past = empty KV (3 layers × {decoder,encoder} × {key,value}, shape [1,16,0,64])
   use_cache = false ; out = []
   loop (max 128):
       logits, present = decoder(
           input_ids=[cur], encoder_hidden_states, encoder_attention_mask=attention_mask,
           past_key_values=past, use_cache_branch=use_cache)
       next = argmax(logits[0, -1])
       if next == 2 (</s>): break
       out.append(next)
       # roll KV: decoder.key/value always update; encoder.key/value only on step 1 then frozen
       past = present (decoder KV) ; keep encoder KV from step 1
       cur = next ; use_cache = true
   ```

   Model dims: 3 decoder layers, 16 heads, head_dim 64, d_model 1024.
   Decoder ONNX inputs: `encoder_attention_mask`, `input_ids`,
   `encoder_hidden_states`, `past_key_values.{0..2}.{decoder,encoder}.{key,value}`,
   `use_cache_branch`. Outputs: `logits`, `present.{0..2}...`.

4. **Decode** ids → text (skip special tokens):

   ```
   result = tokenizer.decode(out, skip_special_tokens=true)
   ```

`optimum` / transformers.js `.generate()` do steps 2–3 for you; on Android/iOS
you run the loop yourself with onnxruntime (see `examples/`).

## Language codes

`lang_tokens.json` maps 100 ISO codes → token ids (e.g. `en`, `zh`, `ja`, `ko`,
`fr`, `de`, `es`, `ru`, `ar`, `hi`, `th`, `vi`, …). Pass the **target** code.