ManniX-ITA commited on
Commit
b2f2b76
·
verified ·
1 Parent(s): 60eaeb9

Upload USAGE.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. USAGE.md +3 -1
USAGE.md CHANGED
@@ -229,7 +229,9 @@ Two independent decode-bandwidth levers:
229
  - **Block-selector** (`--sparse-attn on`): per-block min/max key
230
  bounds give an upper bound on each block's attention mass; decode
231
  visits only the top-K blocks (+ sinks + recent). Flags:
232
- `--sparse-attn-block-size` (128), `--sparse-attn-topk` (0 = auto),
 
 
233
  `--sparse-attn-recent`, `--sparse-attn-sink` (1),
234
  `--sparse-attn-refresh` (8 — re-select every N decode steps),
235
  `--sparse-attn-mode` (0). Default **off**.
 
229
  - **Block-selector** (`--sparse-attn on`): per-block min/max key
230
  bounds give an upper bound on each block's attention mass; decode
231
  visits only the top-K blocks (+ sinks + recent). Flags:
232
+ `--sparse-attn-block-size` (128), `--sparse-attn-topk` (default 0 =
233
+ visit **all** blocks, i.e. no skipping; pass `auto` for adaptive
234
+ max(64, n_blocks/4), or an explicit block count),
235
  `--sparse-attn-recent`, `--sparse-attn-sink` (1),
236
  `--sparse-attn-refresh` (8 — re-select every N decode steps),
237
  `--sparse-attn-mode` (0). Default **off**.