Upload USAGE.md with huggingface_hub
Browse files
USAGE.md
CHANGED
|
@@ -229,7 +229,9 @@ Two independent decode-bandwidth levers:
|
|
| 229 |
- **Block-selector** (`--sparse-attn on`): per-block min/max key
|
| 230 |
bounds give an upper bound on each block's attention mass; decode
|
| 231 |
visits only the top-K blocks (+ sinks + recent). Flags:
|
| 232 |
-
`--sparse-attn-block-size` (128), `--sparse-attn-topk` (0 =
|
|
|
|
|
|
|
| 233 |
`--sparse-attn-recent`, `--sparse-attn-sink` (1),
|
| 234 |
`--sparse-attn-refresh` (8 — re-select every N decode steps),
|
| 235 |
`--sparse-attn-mode` (0). Default **off**.
|
|
|
|
| 229 |
- **Block-selector** (`--sparse-attn on`): per-block min/max key
|
| 230 |
bounds give an upper bound on each block's attention mass; decode
|
| 231 |
visits only the top-K blocks (+ sinks + recent). Flags:
|
| 232 |
+
`--sparse-attn-block-size` (128), `--sparse-attn-topk` (default 0 =
|
| 233 |
+
visit **all** blocks, i.e. no skipping; pass `auto` for adaptive
|
| 234 |
+
max(64, n_blocks/4), or an explicit block count),
|
| 235 |
`--sparse-attn-recent`, `--sparse-attn-sink` (1),
|
| 236 |
`--sparse-attn-refresh` (8 — re-select every N decode steps),
|
| 237 |
`--sparse-attn-mode` (0). Default **off**.
|