| [probe] loading Qwen/Qwen3.6-35B-A3B (torch.bfloat16, sdpa-attn, chunked-score) |
| [transformers] The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention |
|
Loading weights: 0%| | 0/1026 [00:00<?, ?it/s]
Loading weights: 1%|▏ | 13/1026 [00:00<00:09, 110.42it/s]
Loading weights: 3%|▎ | 31/1026 [00:00<00:06, 145.24it/s]
Loading weights: 5%|▍ | 49/1026 [00:00<00:06, 157.32it/s]
Loading weights: 6%|▋ | 65/1026 [00:00<00:07, 125.59it/s]
Loading weights: 8%|▊ | 83/1026 [00:00<00:06, 140.19it/s]
Loading weights: 10%|▉ | 100/1026 [00:00<00:06, 147.79it/s]
Loading weights: 12%|█▏ | 119/1026 [00:00<00:05, 157.88it/s]
Loading weights: 13%|█▎ | 136/1026 [00:00<00:05, 161.14it/s]
Loading weights: 15%|█▍ | 153/1026 [00:01<00:05, 160.90it/s]
Loading weights: 17%|█▋ | 170/1026 [00:01<00:05, 161.48it/s]
Loading weights: 18%|█▊ | 188/1026 [00:01<00:05, 165.30it/s]
Loading weights: 20%|█▉ | 205/1026 [00:01<00:04, 165.96it/s]
Loading weights: 22%|██▏ | 222/1026 [00:01<00:04, 164.02it/s]
Loading weights: 23%|██▎ | 239/1026 [00:01<00:04, 163.35it/s]
Loading weights: 25%|██▌ | 257/1026 [00:01<00:04, 166.82it/s]
Loading weights: 27%|██▋ | 274/1026 [00:01<00:04, 166.77it/s]
Loading weights: 28%|██▊ | 291/1026 [00:01<00:04, 164.43it/s]
Loading weights: 30%|███ | 308/1026 [00:01<00:04, 164.71it/s]
Loading weights: 32%|███▏ | 326/1026 [00:02<00:04, 167.00it/s]
Loading weights: 33%|███▎ | 343/1026 [00:02<00:04, 167.34it/s]
Loading weights: 35%|███▌ | 360/1026 [00:02<00:04, 164.86it/s]
Loading weights: 37%|███▋ | 377/1026 [00:02<00:03, 165.11it/s]
Loading weights: 38%|███▊ | 394/1026 [00:02<00:03, 165.95it/s]
Loading weights: 40%|████ | 415/1026 [00:02<00:03, 178.09it/s]
Loading weights: 42%|████▏ | 433/1026 [00:02<00:03, 176.37it/s]
Loading weights: 44%|████▍ | 451/1026 [00:02<00:03, 176.08it/s]
Loading weights: 46%|████▌ | 469/1026 [00:02<00:03, 176.18it/s]
Loading weights: 48%|████▊ | 496/1026 [00:03<00:02, 187.42it/s]
Loading weights: 50%|█████ | 515/1026 [00:03<00:03, 158.14it/s]
Loading weights: 52%|█████▏ | 533/1026 [00:03<00:03, 162.30it/s]
Loading weights: 54%|█████▎ | 550/1026 [00:03<00:02, 163.18it/s]
Loading weights: 57%|█████▋ | 582/1026 [00:03<00:02, 205.62it/s]
Loading weights: 59%|█████▉ | 604/1026 [00:03<00:02, 164.66it/s]
Loading weights: 61%|██████ | 623/1026 [00:03<00:02, 169.42it/s]
Loading weights: 63%|██████▎ | 647/1026 [00:03<00:02, 186.82it/s]
Loading weights: 65%|██████▌ | 667/1026 [00:03<00:01, 188.80it/s]
Loading weights: 67%|██████▋ | 687/1026 [00:04<00:02, 151.40it/s]
Loading weights: 100%|██████████| 1026/1026 [00:04<00:00, 241.32it/s] |
| [probe] replaced lm_head with Identity (probe discards logits) |
| [probe] GA layers=10 num_h=16 num_kv=2 head_dim=256 |
| [probe] streaming up to 384 long fineweb docs (min_chars=1310720, scan cap=5000000)... |
| [probe] scanned 50000 rows, kept 46 (building doc, 975354/1310720 chars) |
| [probe] scanned 100000 rows, kept 95 (building doc, 169829/1310720 chars) |
| [probe] scanned 150000 rows, kept 142 (building doc, 636461/1310720 chars) |
| [probe] scanned 200000 rows, kept 191 (building doc, 146570/1310720 chars) |
| [probe] scanned 250000 rows, kept 239 (building doc, 1033589/1310720 chars) |
| [probe] scanned 300000 rows, kept 289 (building doc, 49218/1310720 chars) |
| [probe] scanned 350000 rows, kept 338 (building doc, 340642/1310720 chars) |
| [probe] got 384 candidates |
| [transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (298196 > 262144). Running this sequence through the model will result in indexing errors |
| [probe] 1/128 samples scored |
| [probe] 2/128 samples scored |
| [probe] 4/128 samples scored |
| [probe] 6/128 samples scored |
| [probe] 8/128 samples scored |
| [probe] 10/128 samples scored |
| [probe] 12/128 samples scored |
| [probe] 14/128 samples scored |
| [probe] 16/128 samples scored |
| [probe] 18/128 samples scored |
| [probe] 20/128 samples scored |
| [probe] 22/128 samples scored |
| [probe] 24/128 samples scored |
| [probe] 26/128 samples scored |
| [probe] 28/128 samples scored |
| [probe] 30/128 samples scored |
| [probe] 32/128 samples scored |
| [probe] 34/128 samples scored |
| [probe] 36/128 samples scored |
| [probe] 38/128 samples scored |
| [probe] 40/128 samples scored |
| [probe] 42/128 samples scored |
| [probe] 44/128 samples scored |
| [probe] 46/128 samples scored |
| [probe] 48/128 samples scored |
| [probe] 50/128 samples scored |
| [probe] 52/128 samples scored |
| [probe] 54/128 samples scored |
| [probe] 56/128 samples scored |
| [probe] 58/128 samples scored |
| [probe] 60/128 samples scored |
| [probe] 62/128 samples scored |
| [probe] 64/128 samples scored |
| [probe] 66/128 samples scored |
| [probe] 68/128 samples scored |
| [probe] 70/128 samples scored |
| [probe] 72/128 samples scored |
| [probe] 74/128 samples scored |
| [probe] 76/128 samples scored |
| [probe] 78/128 samples scored |
| [probe] 80/128 samples scored |
| [probe] 82/128 samples scored |
| [probe] 84/128 samples scored |
| [probe] 86/128 samples scored |
| [probe] 88/128 samples scored |
| [probe] 90/128 samples scored |
| [probe] 92/128 samples scored |
| [probe] 94/128 samples scored |
| [probe] 96/128 samples scored |
| [probe] 98/128 samples scored |
| [probe] 100/128 samples scored |
| [probe] 102/128 samples scored |
| [probe] 104/128 samples scored |
| [probe] 106/128 samples scored |
| [probe] 108/128 samples scored |
| [probe] 110/128 samples scored |
| [probe] 112/128 samples scored |
| [probe] 114/128 samples scored |
| [probe] 116/128 samples scored |
| [probe] 118/128 samples scored |
| [probe] 120/128 samples scored |
| [probe] 122/128 samples scored |
| [probe] 124/128 samples scored |
| [probe] 126/128 samples scored |
| [probe] 128/128 samples scored |
| [probe] wrote probe_results/qwen3_6_35b_a3b_seq262k.json retrieval=24/160 = 0.150 |
|
|
