0.00.175.697 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.175.700 I device_info:
0.00.267.725 I   - CUDA0   : NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (97244 MiB, 96677 MiB free)
0.00.267.739 I   - CPU     : AMD Ryzen Threadripper 9960X 24-Cores (257066 MiB, 257066 MiB free)
0.00.267.827 I system_info: n_threads = 24 (n_threads_batch = 24) / 48 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.267.912 I srv          init: using 47 threads for HTTP server
0.00.268.116 I srv         start: binding port with default address family
0.00.269.287 I srv  llama_server: loading model
0.00.269.335 I srv    load_model: loading model '/home/ripper/ornith-35b-lab/artifacts/quant/ornith-1.0-35b-IQ4_XS.gguf'
0.00.567.175 I srv    load_model: [spec] estimated memory usage of draft model is 2885.59 MiB
0.00.567.206 I common_init_result: fitting params to device memory ...
0.00.567.208 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.589.396 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.604.947 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.636.863 I srv    load_model: loading draft model '/home/ripper/ornith-35b-lab/artifacts/mtp/ornith-1.0-35b-mtp-chaincorr-h2r9fw125-h1r2-h3r6ce375-lr5e7-Q5_K_M.gguf'
0.02.969.700 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.978.682 I srv    load_model: initializing slots, n_slots = 16
0.02.987.063 I common_context_can_seq_rm: the context supports bounded partial sequence removal
0.02.993.751 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
0.02.993.754 I common_speculative_impl_draft_mtp: - n_max=2, n_min=0, p_min=0.00, n_embd=2048, backend_sampling=1
0.02.993.755 I common_speculative_impl_draft_mtp: - gpu_layers=99, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[default]
0.02.993.897 I common_speculative_impl_draft_mtp: - draft sampler top_k=1 top_p=1.000 temp=1.000
0.02.993.924 I common_speculative_impl_draft_mtp: - fast backend MTP sampling enabled
0.02.993.933 I common_speculative_impl_draft_mtp: - backend draft sampler top_k=1 top_p=1.000 temp=1.000 contexts=1
0.03.013.214 I srv    load_model: speculative decoding context initialized
0.03.013.217 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 512
0.03.013.227 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 512
0.03.013.227 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 512
0.03.013.227 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 512
0.03.013.228 I slot   load_model: id  4 | task -1 | new slot, n_ctx = 512
0.03.013.228 I slot   load_model: id  5 | task -1 | new slot, n_ctx = 512
0.03.013.228 I slot   load_model: id  6 | task -1 | new slot, n_ctx = 512
0.03.013.228 I slot   load_model: id  7 | task -1 | new slot, n_ctx = 512
0.03.013.228 I slot   load_model: id  8 | task -1 | new slot, n_ctx = 512
0.03.013.229 I slot   load_model: id  9 | task -1 | new slot, n_ctx = 512
0.03.013.229 I slot   load_model: id 10 | task -1 | new slot, n_ctx = 512
0.03.013.229 I slot   load_model: id 11 | task -1 | new slot, n_ctx = 512
0.03.013.229 I slot   load_model: id 12 | task -1 | new slot, n_ctx = 512
0.03.013.229 I slot   load_model: id 13 | task -1 | new slot, n_ctx = 512
0.03.013.230 I slot   load_model: id 14 | task -1 | new slot, n_ctx = 512
0.03.013.230 I slot   load_model: id 15 | task -1 | new slot, n_ctx = 512
0.03.013.277 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.03.013.278 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.03.013.278 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.03.013.279 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.03.013.295 I srv          init: idle slots will be saved to prompt cache upon starting a new task
0.03.022.012 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think>

</think>

Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>

</think>

'
0.03.028.333 I srv          init: init: chat template, thinking = 0
0.03.028.379 I srv  llama_server: model loaded
0.03.028.384 I srv  llama_server: server is listening on http://127.0.0.1:18239
0.03.028.391 I srv  update_slots: all slots are idle
0.38.931.796 I srv    operator(): Chat format: peg-native
0.38.931.803 I srv    operator(): Chat format: peg-native
0.38.931.811 I srv    operator(): Chat format: peg-native
0.38.931.925 I srv    operator(): Chat format: peg-native
0.38.931.933 I srv    operator(): Chat format: peg-native
0.38.931.960 I slot get_availabl: id 15 | task -1 | selected slot by LRU, t_last = -1
0.38.931.962 I srv  get_availabl: updating prompt cache
0.38.931.968 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.931.974 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.931.977 I srv  get_availabl: prompt cache update took 0.01 ms
0.38.931.986 I srv    operator(): Chat format: peg-native
0.38.932.021 I slot launch_slot_: id 15 | task 1 | processing task, is_child = 0
0.38.932.024 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.024 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.026 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.026 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.027 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.027 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.028 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.028 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.028 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.029 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.029 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.932.030 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.932.031 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.932.031 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.932.032 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.38.932.033 I slot get_availabl: id 14 | task -1 | selected slot by LRU, t_last = -1
0.38.932.033 I srv  get_availabl: updating prompt cache
0.38.932.034 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.932.035 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.932.036 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.932.049 I slot launch_slot_: id 14 | task 2 | processing task, is_child = 0
0.38.932.051 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.051 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.052 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.052 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.052 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.053 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.053 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.054 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.054 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.054 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.054 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.932.055 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.932.055 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.932.055 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.932.056 I slot get_availabl: id 13 | task -1 | selected slot by LRU, t_last = -1
0.38.932.057 I srv  get_availabl: updating prompt cache
0.38.932.057 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.932.058 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.932.058 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.932.072 I slot launch_slot_: id 13 | task 0 | processing task, is_child = 0
0.38.932.073 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.074 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.074 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.075 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.075 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.077 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.077 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.077 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.077 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.078 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.078 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.932.078 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.932.079 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.932.080 I slot get_availabl: id 12 | task -1 | selected slot by LRU, t_last = -1
0.38.932.080 I srv  get_availabl: updating prompt cache
0.38.932.081 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.932.081 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.932.082 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.932.095 I slot launch_slot_: id 12 | task 4 | processing task, is_child = 0
0.38.932.097 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.097 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.097 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.098 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.099 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.099 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.099 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.100 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.100 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.100 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.101 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.932.101 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.932.102 I slot get_availabl: id 11 | task -1 | selected slot by LRU, t_last = -1
0.38.932.102 I srv  get_availabl: updating prompt cache
0.38.932.103 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.932.103 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.932.104 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.932.116 I slot launch_slot_: id 11 | task 3 | processing task, is_child = 0
0.38.932.118 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.118 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.118 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.119 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.119 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.119 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.120 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.121 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.121 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.121 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.121 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.932.122 I slot get_availabl: id 10 | task -1 | selected slot by LRU, t_last = -1
0.38.932.123 I srv  get_availabl: updating prompt cache
0.38.932.123 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.932.124 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.932.124 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.932.137 I slot launch_slot_: id 10 | task 5 | processing task, is_child = 0
0.38.932.138 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.932.139 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.932.139 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.932.139 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.932.139 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.932.140 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.932.140 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.932.142 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.932.142 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.932.142 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.932.178 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.190 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.205 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.211 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.256 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.260 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.932.663 I srv    operator(): Chat format: peg-native
0.38.936.519 I srv    operator(): Chat format: peg-native
0.38.936.755 I srv    operator(): Chat format: peg-native
0.38.937.337 I srv    operator(): Chat format: peg-native
0.38.937.453 I srv    operator(): Chat format: peg-native
0.38.938.001 I srv    operator(): Chat format: peg-native
0.38.938.538 I srv    operator(): Chat format: peg-native
0.38.941.037 I srv    operator(): Chat format: peg-native
0.38.943.192 I srv    operator(): Chat format: peg-native
0.38.949.106 I srv    operator(): Chat format: peg-native
0.39.138.375 I slot get_availabl: id  9 | task -1 | selected slot by LRU, t_last = -1
0.39.138.380 I srv  get_availabl: updating prompt cache
0.39.138.384 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.388 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.388 I srv  get_availabl: prompt cache update took 0.01 ms
0.39.138.456 I slot launch_slot_: id  9 | task 7 | processing task, is_child = 0
0.39.138.461 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.461 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.461 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.461 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.461 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.138.462 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.138.462 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.39.138.462 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.39.138.462 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.39.138.464 I slot get_availabl: id  8 | task -1 | selected slot by LRU, t_last = -1
0.39.138.464 I srv  get_availabl: updating prompt cache
0.39.138.464 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.465 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.465 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.479 I slot launch_slot_: id  8 | task 8 | processing task, is_child = 0
0.39.138.480 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.480 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.480 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.480 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.481 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.138.481 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.138.481 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.39.138.481 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.39.138.482 I slot get_availabl: id  7 | task -1 | selected slot by LRU, t_last = -1
0.39.138.482 I srv  get_availabl: updating prompt cache
0.39.138.483 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.483 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.483 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.497 I slot launch_slot_: id  7 | task 9 | processing task, is_child = 0
0.39.138.498 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.499 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.499 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.499 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.499 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.138.499 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.138.500 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.39.138.501 I slot get_availabl: id  6 | task -1 | selected slot by LRU, t_last = -1
0.39.138.501 I srv  get_availabl: updating prompt cache
0.39.138.502 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.502 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.502 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.514 I slot launch_slot_: id  6 | task 10 | processing task, is_child = 0
0.39.138.515 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.516 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.516 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.516 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.516 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.138.517 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.138.518 I slot get_availabl: id  5 | task -1 | selected slot by LRU, t_last = -1
0.39.138.518 I srv  get_availabl: updating prompt cache
0.39.138.518 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.519 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.520 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.532 I slot launch_slot_: id  5 | task 11 | processing task, is_child = 0
0.39.138.533 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.533 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.534 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.534 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.534 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.138.535 I slot get_availabl: id  4 | task -1 | selected slot by LRU, t_last = -1
0.39.138.536 I srv  get_availabl: updating prompt cache
0.39.138.536 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.538 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.538 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.549 I slot launch_slot_: id  4 | task 12 | processing task, is_child = 0
0.39.138.550 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.550 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.551 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.551 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.138.552 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.39.138.552 I srv  get_availabl: updating prompt cache
0.39.138.552 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.553 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.553 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.564 I slot launch_slot_: id  3 | task 13 | processing task, is_child = 0
0.39.138.565 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.566 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.566 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.138.567 I slot get_availabl: id  2 | task -1 | selected slot by LRU, t_last = -1
0.39.138.567 I srv  get_availabl: updating prompt cache
0.39.138.568 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.570 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.570 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.580 I slot launch_slot_: id  2 | task 14 | processing task, is_child = 0
0.39.138.581 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.581 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.138.582 I slot get_availabl: id  1 | task -1 | selected slot by LRU, t_last = -1
0.39.138.583 I srv  get_availabl: updating prompt cache
0.39.138.583 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.583 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.583 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.594 I slot launch_slot_: id  1 | task 15 | processing task, is_child = 0
0.39.138.596 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.138.597 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
0.39.138.598 I srv  get_availabl: updating prompt cache
0.39.138.599 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.138.599 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.138.599 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.138.609 I slot launch_slot_: id  0 | task 16 | processing task, is_child = 0
0.39.138.666 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.794 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.801 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.819 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.824 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.836 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.854 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.869 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.872 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.138.873 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.153.117 I slot create_check: id 10 | task 5 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.167.056 I slot create_check: id 11 | task 3 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.180.551 I slot create_check: id 12 | task 4 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.194.250 I slot create_check: id 13 | task 0 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.207.912 I slot create_check: id 14 | task 2 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.221.591 I slot create_check: id 15 | task 1 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.472.372 I slot create_check: id  0 | task 16 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.485.844 I slot create_check: id  1 | task 15 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.499.541 I slot create_check: id  2 | task 14 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.512.944 I slot create_check: id  3 | task 13 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.526.485 I slot create_check: id  4 | task 12 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.540.006 I slot create_check: id  5 | task 11 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.553.492 I slot create_check: id  6 | task 10 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.566.939 I slot create_check: id  7 | task 9 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.580.492 I slot create_check: id  8 | task 8 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.593.975 I slot create_check: id  9 | task 7 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.866.011 I slot print_timing: id 10 | task 5 | n_decoded =    102, tg =  23.10 t/s, tg_3s =  23.10 t/s
0.43.981.599 I slot print_timing: id  1 | task 15 | n_decoded =    102, tg =  23.97 t/s, tg_3s =  23.97 t/s
0.43.982.190 I slot print_timing: id  2 | task 14 | n_decoded =    102, tg =  23.98 t/s, tg_3s =  23.98 t/s
0.43.982.809 I slot print_timing: id  3 | task 13 | n_decoded =    102, tg =  23.98 t/s, tg_3s =  23.98 t/s
0.43.983.406 I slot print_timing: id  4 | task 12 | n_decoded =    102, tg =  23.98 t/s, tg_3s =  23.98 t/s
0.43.985.698 I slot print_timing: id  9 | task 7 | n_decoded =    102, tg =  23.98 t/s, tg_3s =  23.98 t/s
0.44.351.256 I slot print_timing: id 12 | task 4 | n_decoded =    101, tg =  20.62 t/s, tg_3s =  20.62 t/s
0.44.353.052 I slot print_timing: id 15 | task 1 | n_decoded =    101, tg =  20.62 t/s, tg_3s =  20.62 t/s
0.44.466.061 I slot print_timing: id  0 | task 16 | n_decoded =    101, tg =  21.31 t/s, tg_3s =  21.31 t/s
0.44.468.963 I slot print_timing: id  6 | task 10 | n_decoded =    101, tg =  21.31 t/s, tg_3s =  21.31 t/s
0.44.469.542 I slot print_timing: id  7 | task 9 | n_decoded =    101, tg =  21.31 t/s, tg_3s =  21.31 t/s
0.44.471.147 I slot print_timing: id 10 | task 5 | prompt eval time =     519.02 ms /    25 tokens (   20.76 ms per token,    48.17 tokens per second)
0.44.471.149 I slot print_timing: id 10 | task 5 |        eval time =    5019.95 ms /   114 tokens (   44.03 ms per token,    22.71 tokens per second)
0.44.471.149 I slot print_timing: id 10 | task 5 |       total time =    5538.97 ms /   139 tokens
0.44.471.155 I slot print_timing: id 10 | task 5 |    graphs reused =          1
0.44.471.158 I slot print_timing: id 10 | task 5 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.471.179 I statistics        draft-mtp: #calls(b,g,a) =   16     40    625, #gen drafts =    630, #acc drafts =   490, #gen tokens =   1260, #acc tokens =   915, #mean acc len = 2.46, #acc rate/pos = (0.784, 0.680), dur(b,g,a) = 0.010, 109.811, 0.765 ms
0.44.471.215 I slot      release: id 10 | task 5 | stop processing: n_tokens = 140, truncated = 0
0.44.480.389 I srv    operator(): Chat format: peg-native
0.44.586.099 I slot print_timing: id  1 | task 15 | prompt eval time =     588.46 ms /    25 tokens (   23.54 ms per token,    42.48 tokens per second)
0.44.586.102 I slot print_timing: id  1 | task 15 |        eval time =    4858.96 ms /   114 tokens (   42.62 ms per token,    23.46 tokens per second)
0.44.586.102 I slot print_timing: id  1 | task 15 |       total time =    5447.43 ms /   139 tokens
0.44.586.103 I slot print_timing: id  1 | task 15 |    graphs reused =          1
0.44.586.106 I slot print_timing: id  1 | task 15 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.586.116 I statistics        draft-mtp: #calls(b,g,a) =   16     41    632, #gen drafts =    645, #acc drafts =   494, #gen tokens =   1290, #acc tokens =   923, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.679), dur(b,g,a) = 0.010, 118.044, 0.774 ms
0.44.586.138 I slot      release: id  1 | task 15 | stop processing: n_tokens = 140, truncated = 0
0.44.586.732 I slot print_timing: id  2 | task 14 | prompt eval time =     589.09 ms /    25 tokens (   23.56 ms per token,    42.44 tokens per second)
0.44.586.734 I slot print_timing: id  2 | task 14 |        eval time =    4858.95 ms /   114 tokens (   42.62 ms per token,    23.46 tokens per second)
0.44.586.734 I slot print_timing: id  2 | task 14 |       total time =    5448.04 ms /   139 tokens
0.44.586.734 I slot print_timing: id  2 | task 14 |    graphs reused =          1
0.44.586.735 I slot print_timing: id  2 | task 14 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.586.738 I statistics        draft-mtp: #calls(b,g,a) =   16     41    633, #gen drafts =    645, #acc drafts =   495, #gen tokens =   1290, #acc tokens =   925, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.679), dur(b,g,a) = 0.010, 118.044, 0.775 ms
0.44.586.750 I slot      release: id  2 | task 14 | stop processing: n_tokens = 140, truncated = 0
0.44.587.334 I slot print_timing: id  3 | task 13 | prompt eval time =     589.60 ms /    25 tokens (   23.58 ms per token,    42.40 tokens per second)
0.44.587.336 I slot print_timing: id  3 | task 13 |        eval time =    4858.93 ms /   114 tokens (   42.62 ms per token,    23.46 tokens per second)
0.44.587.337 I slot print_timing: id  3 | task 13 |       total time =    5448.53 ms /   139 tokens
0.44.587.337 I slot print_timing: id  3 | task 13 |    graphs reused =          1
0.44.587.339 I slot print_timing: id  3 | task 13 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.587.344 I statistics        draft-mtp: #calls(b,g,a) =   16     41    634, #gen drafts =    645, #acc drafts =   496, #gen tokens =   1290, #acc tokens =   927, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.680), dur(b,g,a) = 0.010, 118.044, 0.777 ms
0.44.587.361 I slot      release: id  3 | task 13 | stop processing: n_tokens = 140, truncated = 0
0.44.587.935 I slot print_timing: id  4 | task 12 | prompt eval time =     590.25 ms /    25 tokens (   23.61 ms per token,    42.36 tokens per second)
0.44.587.936 I slot print_timing: id  4 | task 12 |        eval time =    4858.87 ms /   114 tokens (   42.62 ms per token,    23.46 tokens per second)
0.44.587.937 I slot print_timing: id  4 | task 12 |       total time =    5449.11 ms /   139 tokens
0.44.587.937 I slot print_timing: id  4 | task 12 |    graphs reused =          1
0.44.587.938 I slot print_timing: id  4 | task 12 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.587.940 I statistics        draft-mtp: #calls(b,g,a) =   16     41    635, #gen drafts =    645, #acc drafts =   497, #gen tokens =   1290, #acc tokens =   929, #mean acc len = 2.46, #acc rate/pos = (0.783, 0.680), dur(b,g,a) = 0.010, 118.044, 0.778 ms
0.44.587.952 I slot      release: id  4 | task 12 | stop processing: n_tokens = 140, truncated = 0
0.44.590.243 I slot print_timing: id  9 | task 7 | prompt eval time =     593.17 ms /    25 tokens (   23.73 ms per token,    42.15 tokens per second)
0.44.590.246 I slot print_timing: id  9 | task 7 |        eval time =    4858.20 ms /   114 tokens (   42.62 ms per token,    23.47 tokens per second)
0.44.590.246 I slot print_timing: id  9 | task 7 |       total time =    5451.36 ms /   139 tokens
0.44.590.247 I slot print_timing: id  9 | task 7 |    graphs reused =          1
0.44.590.250 I slot print_timing: id  9 | task 7 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.590.256 I statistics        draft-mtp: #calls(b,g,a) =   16     41    640, #gen drafts =    645, #acc drafts =   500, #gen tokens =   1290, #acc tokens =   935, #mean acc len = 2.46, #acc rate/pos = (0.781, 0.680), dur(b,g,a) = 0.010, 118.044, 0.783 ms
0.44.590.272 I slot      release: id  9 | task 7 | stop processing: n_tokens = 140, truncated = 0
0.44.594.367 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.594.369 I srv  get_availabl: updating prompt cache
0.44.594.418 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.596.357 I srv    operator(): Chat format: peg-native
0.44.597.009 I srv    operator(): Chat format: peg-native
0.44.597.210 I srv    operator(): Chat format: peg-native
0.44.598.581 I srv    operator(): Chat format: peg-native
0.44.601.060 I srv    operator(): Chat format: peg-native
0.44.622.552 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.622.557 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.558 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.559 I srv  get_availabl: prompt cache update took 28.19 ms
0.44.622.603 I slot launch_slot_: id  1 | task 59 | processing task, is_child = 0
0.44.622.605 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.44.622.628 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.630 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.630 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.44.622.646 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.647 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.647 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.44.622.663 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.664 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.665 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.44.622.681 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.682 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.682 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.622.698 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.699 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.703 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.622.704 I srv  get_availabl: updating prompt cache
0.44.622.718 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.719 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.720 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.622.720 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.721 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.721 I srv  get_availabl: prompt cache update took 0.02 ms
0.44.622.737 I slot launch_slot_: id  2 | task 60 | processing task, is_child = 0
0.44.622.738 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.44.622.752 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.753 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.753 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.44.622.766 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.767 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.767 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.44.622.781 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.782 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.782 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.622.795 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.796 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.798 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.622.798 I srv  get_availabl: updating prompt cache
0.44.622.813 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.814 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.814 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.622.814 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.815 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.815 I srv  get_availabl: prompt cache update took 0.02 ms
0.44.622.829 I slot launch_slot_: id  3 | task 61 | processing task, is_child = 0
0.44.622.830 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.44.622.843 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.844 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.845 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.44.622.858 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.859 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.859 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.622.873 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.874 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.875 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.622.875 I srv  get_availabl: updating prompt cache
0.44.622.889 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.890 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.890 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.622.890 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.890 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.891 I srv  get_availabl: prompt cache update took 0.01 ms
0.44.622.904 I slot launch_slot_: id  4 | task 62 | processing task, is_child = 0
0.44.622.905 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.44.622.918 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.919 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.919 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.622.933 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.934 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.935 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.622.935 I srv  get_availabl: updating prompt cache
0.44.622.948 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.949 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.950 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.622.950 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.950 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.950 I srv  get_availabl: prompt cache update took 0.01 ms
0.44.622.965 I slot launch_slot_: id  9 | task 63 | processing task, is_child = 0
0.44.622.966 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.622.979 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.980 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.981 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.622.981 I srv  get_availabl: updating prompt cache
0.44.622.995 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.622.996 I srv         alloc:  - prompt is already in the cache, skipping
0.44.622.996 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.622.996 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.622.996 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.622.997 I srv  get_availabl: prompt cache update took 0.01 ms
0.44.623.010 I slot launch_slot_: id 10 | task 64 | processing task, is_child = 0
0.44.628.763 I slot   operator(): id  1 | task 59 | Checking checkpoint with [20, 20] against 3...
0.44.628.766 W slot   operator(): id  1 | task 59 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.628.768 W slot   operator(): id  1 | task 59 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.631.012 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.631.038 I slot   operator(): id  2 | task 60 | Checking checkpoint with [20, 20] against 3...
0.44.631.039 W slot   operator(): id  2 | task 60 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.631.041 W slot   operator(): id  2 | task 60 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.633.087 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.633.094 I slot   operator(): id  3 | task 61 | Checking checkpoint with [20, 20] against 24...
0.44.636.782 W slot   operator(): id  3 | task 61 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.636.807 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.636.830 I slot   operator(): id  4 | task 62 | Checking checkpoint with [20, 20] against 3...
0.44.636.831 W slot   operator(): id  4 | task 62 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.636.832 W slot   operator(): id  4 | task 62 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.638.869 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.638.886 I slot   operator(): id  9 | task 63 | Checking checkpoint with [20, 20] against 24...
0.44.642.407 W slot   operator(): id  9 | task 63 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.642.446 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.642.450 I slot   operator(): id 10 | task 64 | Checking checkpoint with [20, 20] against 3...
0.44.642.450 W slot   operator(): id 10 | task 64 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.642.451 W slot   operator(): id 10 | task 64 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.645.359 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.830.271 I slot create_check: id  1 | task 59 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.843.986 I slot create_check: id  2 | task 60 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.44.857.616 I slot create_check: id  4 | task 62 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.871.158 I slot create_check: id 10 | task 64 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.45.598.519 I slot print_timing: id 12 | task 4 | prompt eval time =     520.33 ms /    28 tokens (   18.58 ms per token,    53.81 tokens per second)
0.45.598.522 I slot print_timing: id 12 | task 4 |        eval time =    6145.96 ms /   125 tokens (   49.17 ms per token,    20.34 tokens per second)
0.45.598.522 I slot print_timing: id 12 | task 4 |       total time =    6666.29 ms /   153 tokens
0.45.598.523 I slot print_timing: id 12 | task 4 |    graphs reused =          1
0.45.598.526 I slot print_timing: id 12 | task 4 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.598.536 I statistics        draft-mtp: #calls(b,g,a) =   22     48    744, #gen drafts =    747, #acc drafts =   569, #gen tokens =   1494, #acc tokens =  1064, #mean acc len = 2.43, #acc rate/pos = (0.765, 0.665), dur(b,g,a) = 0.011, 142.463, 0.917 ms
0.45.598.558 I slot      release: id 12 | task 4 | stop processing: n_tokens = 152, truncated = 0
0.45.599.763 I slot print_timing: id 15 | task 1 | prompt eval time =     522.16 ms /    28 tokens (   18.65 ms per token,    53.62 tokens per second)
0.45.599.765 I slot print_timing: id 15 | task 1 |        eval time =    6145.37 ms /   125 tokens (   49.16 ms per token,    20.34 tokens per second)
0.45.599.765 I slot print_timing: id 15 | task 1 |       total time =    6667.53 ms /   153 tokens
0.45.599.766 I slot print_timing: id 15 | task 1 |    graphs reused =          1
0.45.599.768 I slot print_timing: id 15 | task 1 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.599.775 I statistics        draft-mtp: #calls(b,g,a) =   22     48    747, #gen drafts =    747, #acc drafts =   570, #gen tokens =   1494, #acc tokens =  1066, #mean acc len = 2.43, #acc rate/pos = (0.763, 0.664), dur(b,g,a) = 0.011, 142.463, 0.920 ms
0.45.599.790 I slot      release: id 15 | task 1 | stop processing: n_tokens = 152, truncated = 0
0.45.607.920 I srv    operator(): Chat format: peg-native
0.45.607.943 I srv    operator(): Chat format: peg-native
0.45.700.719 I slot print_timing: id  0 | task 16 | prompt eval time =     587.81 ms /    28 tokens (   20.99 ms per token,    47.63 tokens per second)
0.45.700.722 I slot print_timing: id  0 | task 16 |        eval time =    5974.23 ms /   125 tokens (   47.79 ms per token,    20.92 tokens per second)
0.45.700.722 I slot print_timing: id  0 | task 16 |       total time =    6562.03 ms /   153 tokens
0.45.700.723 I slot print_timing: id  0 | task 16 |    graphs reused =          1
0.45.700.726 I slot print_timing: id  0 | task 16 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.700.737 I statistics        draft-mtp: #calls(b,g,a) =   22     49    748, #gen drafts =    761, #acc drafts =   571, #gen tokens =   1522, #acc tokens =  1068, #mean acc len = 2.43, #acc rate/pos = (0.763, 0.664), dur(b,g,a) = 0.011, 145.991, 0.922 ms
0.45.700.756 I slot      release: id  0 | task 16 | stop processing: n_tokens = 152, truncated = 0
0.45.703.101 I slot print_timing: id  6 | task 10 | prompt eval time =     591.39 ms /    28 tokens (   21.12 ms per token,    47.35 tokens per second)
0.45.703.103 I slot print_timing: id  6 | task 10 |        eval time =    5972.83 ms /   125 tokens (   47.78 ms per token,    20.93 tokens per second)
0.45.703.104 I slot print_timing: id  6 | task 10 |       total time =    6564.22 ms /   153 tokens
0.45.703.104 I slot print_timing: id  6 | task 10 |    graphs reused =          1
0.45.703.107 I slot print_timing: id  6 | task 10 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.703.114 I statistics        draft-mtp: #calls(b,g,a) =   22     49    754, #gen drafts =    761, #acc drafts =   573, #gen tokens =   1522, #acc tokens =  1072, #mean acc len = 2.42, #acc rate/pos = (0.760, 0.662), dur(b,g,a) = 0.011, 145.991, 0.928 ms
0.45.703.133 I slot      release: id  6 | task 10 | stop processing: n_tokens = 152, truncated = 0
0.45.703.720 I slot print_timing: id  7 | task 9 | prompt eval time =     592.02 ms /    28 tokens (   21.14 ms per token,    47.30 tokens per second)
0.45.703.722 I slot print_timing: id  7 | task 9 |        eval time =    5972.83 ms /   125 tokens (   47.78 ms per token,    20.93 tokens per second)
0.45.703.722 I slot print_timing: id  7 | task 9 |       total time =    6564.85 ms /   153 tokens
0.45.703.722 I slot print_timing: id  7 | task 9 |    graphs reused =          1
0.45.703.723 I slot print_timing: id  7 | task 9 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.703.726 I statistics        draft-mtp: #calls(b,g,a) =   22     49    755, #gen drafts =    761, #acc drafts =   574, #gen tokens =   1522, #acc tokens =  1074, #mean acc len = 2.42, #acc rate/pos = (0.760, 0.662), dur(b,g,a) = 0.011, 145.991, 0.929 ms
0.45.703.735 I slot      release: id  7 | task 9 | stop processing: n_tokens = 152, truncated = 0
0.45.706.657 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.45.706.659 I srv  get_availabl: updating prompt cache
0.45.706.701 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.709.896 I srv    operator(): Chat format: peg-native
0.45.711.470 I srv    operator(): Chat format: peg-native
0.45.711.967 I srv    operator(): Chat format: peg-native
0.45.734.930 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.45.734.935 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.734.936 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.734.936 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.734.937 I srv  get_availabl: prompt cache update took 28.28 ms
0.45.734.984 I slot launch_slot_: id  0 | task 73 | processing task, is_child = 0
0.45.734.986 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.45.735.010 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.012 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.013 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.45.735.028 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.030 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.030 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.45.735.047 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.048 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.048 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.45.735.063 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.065 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.068 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.45.735.070 I srv  get_availabl: updating prompt cache
0.45.735.083 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.085 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.085 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.45.735.086 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.735.086 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.735.086 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.735.086 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.735.106 I slot launch_slot_: id  6 | task 74 | processing task, is_child = 0
0.45.735.107 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.45.735.121 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.122 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.122 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.45.735.136 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.137 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.137 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.45.735.151 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.152 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.153 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
0.45.735.153 I srv  get_availabl: updating prompt cache
0.45.735.168 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.169 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.169 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.120
0.45.735.169 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.735.170 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.735.170 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.735.170 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.735.185 I slot launch_slot_: id  7 | task 75 | processing task, is_child = 0
0.45.735.186 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.45.735.200 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.201 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.202 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.45.735.215 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.216 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.217 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.45.735.217 I srv  get_availabl: updating prompt cache
0.45.735.231 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.232 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.232 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.45.735.233 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.735.233 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.735.233 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.735.233 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.735.246 I slot launch_slot_: id 12 | task 76 | processing task, is_child = 0
0.45.735.248 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.45.735.261 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.262 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.263 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.45.735.263 I srv  get_availabl: updating prompt cache
0.45.735.277 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.735.278 I srv         alloc:  - prompt is already in the cache, skipping
0.45.735.279 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.45.735.279 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.735.280 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.735.280 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.735.280 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.735.294 I slot launch_slot_: id 15 | task 77 | processing task, is_child = 0
0.45.740.013 I slot   operator(): id  0 | task 73 | Checking checkpoint with [23, 23] against 3...
0.45.740.016 W slot   operator(): id  0 | task 73 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.740.018 W slot   operator(): id  0 | task 73 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.741.603 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.741.629 I slot   operator(): id  6 | task 74 | Checking checkpoint with [23, 23] against 27...
0.45.745.236 W slot   operator(): id  6 | task 74 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.45.745.267 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.745.283 I slot   operator(): id  7 | task 75 | Checking checkpoint with [23, 23] against 3...
0.45.745.285 W slot   operator(): id  7 | task 75 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.745.286 W slot   operator(): id  7 | task 75 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.746.846 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.746.860 I slot   operator(): id 12 | task 76 | Checking checkpoint with [23, 23] against 3...
0.45.746.862 W slot   operator(): id 12 | task 76 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.746.864 W slot   operator(): id 12 | task 76 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.748.229 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.748.255 I slot   operator(): id 15 | task 77 | Checking checkpoint with [23, 23] against 27...
0.45.751.793 W slot   operator(): id 15 | task 77 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.45.751.825 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.926.129 I slot create_check: id  0 | task 73 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.45.939.366 I slot create_check: id  7 | task 75 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.45.952.962 I slot create_check: id 12 | task 76 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.46.434.970 I slot print_timing: id 11 | task 3 | n_decoded =    100, tg =  14.32 t/s, tg_3s =  14.32 t/s
0.46.435.864 I slot print_timing: id 13 | task 0 | n_decoded =    100, tg =  14.32 t/s, tg_3s =  14.32 t/s
0.46.436.467 I slot print_timing: id 14 | task 2 | n_decoded =    100, tg =  14.32 t/s, tg_3s =  14.32 t/s
0.46.553.316 I slot print_timing: id  5 | task 11 | n_decoded =    100, tg =  14.65 t/s, tg_3s =  14.65 t/s
0.46.555.111 I slot print_timing: id  8 | task 8 | n_decoded =    100, tg =  14.66 t/s, tg_3s =  14.65 t/s
0.48.231.968 I slot print_timing: id 11 | task 3 | prompt eval time =     519.66 ms /    29 tokens (   17.92 ms per token,    55.81 tokens per second)
0.48.231.972 I slot print_timing: id 11 | task 3 |        eval time =    8780.09 ms /   128 tokens (   68.59 ms per token,    14.58 tokens per second)
0.48.231.972 I slot print_timing: id 11 | task 3 |       total time =    9299.75 ms /   157 tokens
0.48.231.974 I slot print_timing: id 11 | task 3 |    graphs reused =          1
0.48.231.977 I slot print_timing: id 11 | task 3 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.231.992 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1057, #gen drafts =   1070, #acc drafts =   783, #gen tokens =   2135, #acc tokens =  1464, #mean acc len = 2.39, #acc rate/pos = (0.741, 0.644), dur(b,g,a) = 0.013, 206.652, 1.321 ms
0.48.232.013 I slot      release: id 11 | task 3 | stop processing: n_tokens = 156, truncated = 0
0.48.232.227 I slot print_timing: id 13 | task 0 | prompt eval time =     520.91 ms /    29 tokens (   17.96 ms per token,    55.67 tokens per second)
0.48.232.229 I slot print_timing: id 13 | task 0 |        eval time =    8779.10 ms /   128 tokens (   68.59 ms per token,    14.58 tokens per second)
0.48.232.229 I slot print_timing: id 13 | task 0 |       total time =    9300.02 ms /   157 tokens
0.48.232.230 I slot print_timing: id 13 | task 0 |    graphs reused =          1
0.48.232.231 I slot print_timing: id 13 | task 0 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.232.234 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1057, #gen drafts =   1070, #acc drafts =   783, #gen tokens =   2135, #acc tokens =  1464, #mean acc len = 2.39, #acc rate/pos = (0.741, 0.644), dur(b,g,a) = 0.013, 206.652, 1.321 ms
0.48.232.259 I slot      release: id 13 | task 0 | stop processing: n_tokens = 156, truncated = 0
0.48.232.469 I slot print_timing: id 14 | task 2 | prompt eval time =     521.57 ms /    29 tokens (   17.99 ms per token,    55.60 tokens per second)
0.48.232.472 I slot print_timing: id 14 | task 2 |        eval time =    8778.68 ms /   128 tokens (   68.58 ms per token,    14.58 tokens per second)
0.48.232.472 I slot print_timing: id 14 | task 2 |       total time =    9300.25 ms /   157 tokens
0.48.232.472 I slot print_timing: id 14 | task 2 |    graphs reused =          1
0.48.232.474 I slot print_timing: id 14 | task 2 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.232.476 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1057, #gen drafts =   1070, #acc drafts =   783, #gen tokens =   2135, #acc tokens =  1464, #mean acc len = 2.39, #acc rate/pos = (0.741, 0.644), dur(b,g,a) = 0.013, 206.652, 1.321 ms
0.48.232.495 I slot      release: id 14 | task 2 | stop processing: n_tokens = 156, truncated = 0
0.48.243.142 I srv    operator(): Chat format: peg-native
0.48.243.148 I srv    operator(): Chat format: peg-native
0.48.243.205 I srv    operator(): Chat format: peg-native
0.48.330.631 I slot print_timing: id  5 | task 11 | prompt eval time =     590.83 ms /    29 tokens (   20.37 ms per token,    49.08 tokens per second)
0.48.330.635 I slot print_timing: id  5 | task 11 |        eval time =    8600.95 ms /   128 tokens (   67.19 ms per token,    14.88 tokens per second)
0.48.330.636 I slot print_timing: id  5 | task 11 |       total time =    9191.78 ms /   157 tokens
0.48.330.637 I slot print_timing: id  5 | task 11 |    graphs reused =          1
0.48.330.641 I slot print_timing: id  5 | task 11 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.330.658 I statistics        draft-mtp: #calls(b,g,a) =   27     70   1070, #gen drafts =   1081, #acc drafts =   790, #gen tokens =   2157, #acc tokens =  1476, #mean acc len = 2.38, #acc rate/pos = (0.738, 0.641), dur(b,g,a) = 0.013, 212.597, 1.338 ms
0.48.330.683 I slot      release: id  5 | task 11 | stop processing: n_tokens = 156, truncated = 0
0.48.330.884 I slot print_timing: id  8 | task 8 | prompt eval time =     592.60 ms /    29 tokens (   20.43 ms per token,    48.94 tokens per second)
0.48.330.886 I slot print_timing: id  8 | task 8 |        eval time =    8599.42 ms /   128 tokens (   67.18 ms per token,    14.88 tokens per second)
0.48.330.886 I slot print_timing: id  8 | task 8 |       total time =    9192.02 ms /   157 tokens
0.48.330.887 I slot print_timing: id  8 | task 8 |    graphs reused =          1
0.48.330.888 I slot print_timing: id  8 | task 8 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.330.891 I statistics        draft-mtp: #calls(b,g,a) =   27     70   1070, #gen drafts =   1081, #acc drafts =   790, #gen tokens =   2157, #acc tokens =  1476, #mean acc len = 2.38, #acc rate/pos = (0.738, 0.641), dur(b,g,a) = 0.013, 212.597, 1.338 ms
0.48.330.905 I slot      release: id  8 | task 8 | stop processing: n_tokens = 156, truncated = 0
0.48.336.589 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.48.336.591 I srv  get_availabl: updating prompt cache
0.48.336.644 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.340.889 I srv    operator(): Chat format: peg-native
0.48.340.899 I srv    operator(): Chat format: peg-native
0.48.364.897 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.48.364.904 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.364.905 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.364.905 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.364.906 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.364.906 I srv  get_availabl: prompt cache update took 28.31 ms
0.48.364.992 I slot launch_slot_: id  5 | task 100 | processing task, is_child = 0
0.48.364.994 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.48.365.022 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.026 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.026 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.48.365.043 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.045 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.046 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.48.365.061 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.062 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.062 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.48.365.078 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.080 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.085 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.48.365.085 I srv  get_availabl: updating prompt cache
0.48.365.100 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.101 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.102 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.48.365.103 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.365.103 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.365.104 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.365.104 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.365.104 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.365.123 I slot launch_slot_: id  8 | task 99 | processing task, is_child = 0
0.48.365.125 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.48.365.139 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.142 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.142 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.48.365.156 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.157 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.158 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.48.365.172 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.173 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.174 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.48.365.174 I srv  get_availabl: updating prompt cache
0.48.365.188 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.190 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.190 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.48.365.191 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.365.191 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.365.191 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.365.192 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.365.193 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.365.209 I slot launch_slot_: id 11 | task 101 | processing task, is_child = 0
0.48.365.210 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.48.365.224 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.225 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.225 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.48.365.240 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.241 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.243 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.48.365.244 I srv  get_availabl: updating prompt cache
0.48.365.258 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.259 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.259 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.48.365.260 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.365.260 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.365.262 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.365.262 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.365.263 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.365.278 I slot launch_slot_: id 13 | task 102 | processing task, is_child = 0
0.48.365.279 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.48.365.293 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.294 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.296 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.48.365.297 I srv  get_availabl: updating prompt cache
0.48.365.311 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.365.312 I srv         alloc:  - prompt is already in the cache, skipping
0.48.365.312 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.48.365.313 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.365.313 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.365.313 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.365.314 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.365.315 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.365.336 I slot launch_slot_: id 14 | task 103 | processing task, is_child = 0
0.48.371.351 I slot   operator(): id  5 | task 100 | Checking checkpoint with [24, 24] against 28...
0.48.374.894 W slot   operator(): id  5 | task 100 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.48.374.948 I slot   operator(): id  8 | task 99 | Checking checkpoint with [24, 24] against 3...
0.48.374.950 W slot   operator(): id  8 | task 99 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.374.950 W slot   operator(): id  8 | task 99 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.374.955 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.376.646 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.376.665 I slot   operator(): id 11 | task 101 | Checking checkpoint with [24, 24] against 3...
0.48.376.665 W slot   operator(): id 11 | task 101 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.376.668 W slot   operator(): id 11 | task 101 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.378.539 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.378.580 I slot   operator(): id 13 | task 102 | Checking checkpoint with [24, 24] against 3...
0.48.378.581 W slot   operator(): id 13 | task 102 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.378.583 W slot   operator(): id 13 | task 102 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.380.656 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.380.669 I slot   operator(): id 14 | task 103 | Checking checkpoint with [24, 24] against 28...
0.48.384.202 W slot   operator(): id 14 | task 103 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.48.384.222 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.557.257 I slot create_check: id  8 | task 99 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.48.571.126 I slot create_check: id 11 | task 101 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.48.584.803 I slot create_check: id 13 | task 102 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.49.306.438 I slot print_timing: id  3 | task 61 | n_decoded =    102, tg =  22.66 t/s, tg_3s =  22.66 t/s
0.49.309.444 I slot print_timing: id  9 | task 63 | n_decoded =    102, tg =  22.65 t/s, tg_3s =  22.65 t/s
0.49.908.986 I slot print_timing: id  3 | task 61 | prompt eval time =     172.84 ms /     4 tokens (   43.21 ms per token,    23.14 tokens per second)
0.49.908.989 I slot print_timing: id  3 | task 61 |        eval time =    5103.03 ms /   114 tokens (   44.76 ms per token,    22.34 tokens per second)
0.49.908.989 I slot print_timing: id  3 | task 61 |       total time =    5275.87 ms /   118 tokens
0.49.908.990 I slot print_timing: id  3 | task 61 |    graphs reused =          1
0.49.908.993 I slot print_timing: id  3 | task 61 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.49.909.006 I statistics        draft-mtp: #calls(b,g,a) =   32     82   1253, #gen drafts =   1265, #acc drafts =   924, #gen tokens =   2525, #acc tokens =  1719, #mean acc len = 2.37, #acc rate/pos = (0.737, 0.634), dur(b,g,a) = 0.015, 251.725, 1.559 ms
0.49.909.029 I slot      release: id  3 | task 61 | stop processing: n_tokens = 140, truncated = 0
0.49.912.607 I slot print_timing: id  9 | task 63 | prompt eval time =     167.34 ms /     4 tokens (   41.83 ms per token,    23.90 tokens per second)
0.49.912.609 I slot print_timing: id  9 | task 63 |        eval time =    5106.37 ms /   114 tokens (   44.79 ms per token,    22.33 tokens per second)
0.49.912.610 I slot print_timing: id  9 | task 63 |       total time =    5273.70 ms /   118 tokens
0.49.912.610 I slot print_timing: id  9 | task 63 |    graphs reused =          1
0.49.912.613 I slot print_timing: id  9 | task 63 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.49.912.618 I statistics        draft-mtp: #calls(b,g,a) =   32     82   1259, #gen drafts =   1265, #acc drafts =   930, #gen tokens =   2525, #acc tokens =  1731, #mean acc len = 2.37, #acc rate/pos = (0.739, 0.636), dur(b,g,a) = 0.015, 251.725, 1.565 ms
0.49.912.638 I slot      release: id  9 | task 63 | stop processing: n_tokens = 140, truncated = 0
0.49.917.633 I srv    operator(): Chat format: peg-native
0.49.921.294 I srv    operator(): Chat format: peg-native
0.50.021.168 I slot print_timing: id  2 | task 60 | n_decoded =    101, tg =  20.08 t/s, tg_3s =  20.08 t/s
0.50.023.961 I slot print_timing: id 10 | task 64 | n_decoded =    101, tg =  20.07 t/s, tg_3s =  20.07 t/s
0.50.026.520 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.50.026.522 I srv  get_availabl: updating prompt cache
0.50.026.563 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.026.566 I srv         alloc:  - prompt is already in the cache, skipping
0.50.026.567 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.50.026.570 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.50.026.571 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.50.026.572 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.50.026.572 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.50.026.572 I srv  get_availabl: prompt cache update took 0.05 ms
0.50.026.639 I slot launch_slot_: id  3 | task 117 | processing task, is_child = 0
0.50.026.641 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.50.026.656 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.026.658 I srv         alloc:  - prompt is already in the cache, skipping
0.50.026.659 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.50.026.660 I srv  get_availabl: updating prompt cache
0.50.026.673 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.026.674 I srv         alloc:  - prompt is already in the cache, skipping
0.50.026.674 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.50.026.675 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.50.026.676 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.50.026.676 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.50.026.677 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.50.026.677 I srv  get_availabl: prompt cache update took 0.02 ms
0.50.026.694 I slot launch_slot_: id  9 | task 118 | processing task, is_child = 0
0.50.031.935 I slot   operator(): id  3 | task 117 | Checking checkpoint with [20, 20] against 3...
0.50.031.937 W slot   operator(): id  3 | task 117 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.50.031.940 W slot   operator(): id  3 | task 117 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.50.033.407 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.50.033.432 I slot   operator(): id  9 | task 118 | Checking checkpoint with [20, 20] against 24...
0.50.037.241 W slot   operator(): id  9 | task 118 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.50.037.274 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.50.185.449 I slot create_check: id  3 | task 117 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.50.420.187 I slot print_timing: id  7 | task 75 | n_decoded =    102, tg =  23.44 t/s, tg_3s =  23.44 t/s
0.50.783.056 I slot print_timing: id  6 | task 74 | n_decoded =    101, tg =  20.69 t/s, tg_3s =  20.69 t/s
0.50.788.129 I slot print_timing: id 15 | task 77 | n_decoded =    101, tg =  20.67 t/s, tg_3s =  20.67 t/s
0.51.026.169 I slot print_timing: id  7 | task 75 | prompt eval time =     322.90 ms /    25 tokens (   12.92 ms per token,    77.42 tokens per second)
0.51.026.172 I slot print_timing: id  7 | task 75 |        eval time =    4957.96 ms /   114 tokens (   43.49 ms per token,    22.99 tokens per second)
0.51.026.172 I slot print_timing: id  7 | task 75 |       total time =    5280.86 ms /   139 tokens
0.51.026.173 I slot print_timing: id  7 | task 75 |    graphs reused =          1
0.51.026.176 I slot print_timing: id  7 | task 75 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.51.026.188 I statistics        draft-mtp: #calls(b,g,a) =   34     91   1396, #gen drafts =   1404, #acc drafts =  1028, #gen tokens =   2803, #acc tokens =  1914, #mean acc len = 2.37, #acc rate/pos = (0.736, 0.635), dur(b,g,a) = 0.017, 282.473, 1.738 ms
0.51.026.214 I slot      release: id  7 | task 75 | stop processing: n_tokens = 140, truncated = 0
0.51.034.661 I srv    operator(): Chat format: peg-native
0.51.144.423 I slot print_timing: id  2 | task 60 | prompt eval time =     359.38 ms /    28 tokens (   12.83 ms per token,    77.91 tokens per second)
0.51.144.426 I slot print_timing: id  2 | task 60 |        eval time =    6153.97 ms /   125 tokens (   49.23 ms per token,    20.31 tokens per second)
0.51.144.426 I slot print_timing: id  2 | task 60 |       total time =    6513.34 ms /   153 tokens
0.51.144.427 I slot print_timing: id  2 | task 60 |    graphs reused =          1
0.51.144.430 I slot print_timing: id  2 | task 60 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
0.51.144.442 I statistics        draft-mtp: #calls(b,g,a) =   34     92   1407, #gen drafts =   1419, #acc drafts =  1036, #gen tokens =   2833, #acc tokens =  1930, #mean acc len = 2.37, #acc rate/pos = (0.736, 0.635), dur(b,g,a) = 0.017, 290.838, 1.750 ms
0.51.144.465 I slot      release: id  2 | task 60 | stop processing: n_tokens = 152, truncated = 0
0.51.148.197 I slot print_timing: id 10 | task 64 | prompt eval time =     349.18 ms /    28 tokens (   12.47 ms per token,    80.19 tokens per second)
0.51.148.199 I slot print_timing: id 10 | task 64 |        eval time =    6156.52 ms /   125 tokens (   49.25 ms per token,    20.30 tokens per second)
0.51.148.200 I slot print_timing: id 10 | task 64 |       total time =    6505.70 ms /   153 tokens
0.51.148.201 I slot print_timing: id 10 | task 64 |    graphs reused =          1
0.51.148.203 I slot print_timing: id 10 | task 64 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
0.51.148.210 I statistics        draft-mtp: #calls(b,g,a) =   34     92   1414, #gen drafts =   1419, #acc drafts =  1042, #gen tokens =   2833, #acc tokens =  1941, #mean acc len = 2.37, #acc rate/pos = (0.737, 0.636), dur(b,g,a) = 0.017, 290.838, 1.757 ms
0.51.148.229 I slot      release: id 10 | task 64 | stop processing: n_tokens = 152, truncated = 0
0.51.150.624 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.51.150.626 I srv  get_availabl: updating prompt cache
0.51.150.668 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.150.670 I srv         alloc:  - prompt is already in the cache, skipping
0.51.150.671 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.51.150.674 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.150.676 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.150.676 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.150.676 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.150.677 I srv  get_availabl: prompt cache update took 0.05 ms
0.51.150.739 I slot launch_slot_: id  2 | task 128 | processing task, is_child = 0
0.51.150.740 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.51.150.756 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.150.758 I srv         alloc:  - prompt is already in the cache, skipping
0.51.150.759 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.51.150.774 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.150.776 I srv         alloc:  - prompt is already in the cache, skipping
0.51.153.418 I srv    operator(): Chat format: peg-native
0.51.156.563 I slot   operator(): id  2 | task 128 | Checking checkpoint with [23, 23] against 3...
0.51.156.565 W slot   operator(): id  2 | task 128 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.51.156.569 W slot   operator(): id  2 | task 128 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.51.156.759 I srv    operator(): Chat format: peg-native
0.51.158.152 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.274.922 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.51.274.925 I srv  get_availabl: updating prompt cache
0.51.274.965 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.274.968 I srv         alloc:  - prompt is already in the cache, skipping
0.51.274.969 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.51.274.971 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.274.973 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.274.973 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.274.974 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.274.975 I srv  get_availabl: prompt cache update took 0.05 ms
0.51.275.042 I slot launch_slot_: id 10 | task 130 | processing task, is_child = 0
0.51.275.043 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.51.275.059 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.275.060 I srv         alloc:  - prompt is already in the cache, skipping
0.51.275.063 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.51.275.064 I srv  get_availabl: updating prompt cache
0.51.275.077 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.275.078 I srv         alloc:  - prompt is already in the cache, skipping
0.51.275.079 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.51.275.079 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.275.080 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.275.080 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.275.080 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.275.081 I srv  get_availabl: prompt cache update took 0.02 ms
0.51.275.099 I slot launch_slot_: id  7 | task 131 | processing task, is_child = 0
0.51.294.601 I slot create_check: id  2 | task 128 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.51.294.609 I slot   operator(): id  7 | task 131 | Checking checkpoint with [20, 20] against 24...
0.51.298.070 W slot   operator(): id  7 | task 131 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.51.298.098 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.298.119 I slot   operator(): id 10 | task 130 | Checking checkpoint with [23, 23] against 27...
0.51.301.601 W slot   operator(): id 10 | task 130 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.51.301.630 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.780.878 I slot print_timing: id  1 | task 59 | n_decoded =    100, tg =  14.73 t/s, tg_3s =  14.73 t/s
0.51.782.361 I slot print_timing: id  4 | task 62 | n_decoded =    100, tg =  14.72 t/s, tg_3s =  14.72 t/s
0.51.903.467 I slot print_timing: id  6 | task 74 | prompt eval time =     160.01 ms /     4 tokens (   40.00 ms per token,    25.00 tokens per second)
0.51.903.470 I slot print_timing: id  6 | task 74 |        eval time =    6001.79 ms /   125 tokens (   48.01 ms per token,    20.83 tokens per second)
0.51.903.471 I slot print_timing: id  6 | task 74 |       total time =    6161.80 ms /   129 tokens
0.51.903.472 I slot print_timing: id  6 | task 74 |    graphs reused =          1
0.51.903.475 I slot print_timing: id  6 | task 74 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.51.903.489 I statistics        draft-mtp: #calls(b,g,a) =   37     98   1500, #gen drafts =   1509, #acc drafts =  1105, #gen tokens =   3013, #acc tokens =  2056, #mean acc len = 2.37, #acc rate/pos = (0.737, 0.634), dur(b,g,a) = 0.018, 313.430, 1.866 ms
0.51.903.511 I slot      release: id  6 | task 74 | stop processing: n_tokens = 152, truncated = 0
0.51.908.236 I slot print_timing: id 15 | task 77 | prompt eval time =     153.67 ms /     4 tokens (   38.42 ms per token,    26.03 tokens per second)
0.51.908.239 I slot print_timing: id 15 | task 77 |        eval time =    6006.28 ms /   125 tokens (   48.05 ms per token,    20.81 tokens per second)
0.51.908.239 I slot print_timing: id 15 | task 77 |       total time =    6159.95 ms /   129 tokens
0.51.908.239 I slot print_timing: id 15 | task 77 |    graphs reused =          1
0.51.908.242 I slot print_timing: id 15 | task 77 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.51.908.249 I statistics        draft-mtp: #calls(b,g,a) =   37     98   1509, #gen drafts =   1509, #acc drafts =  1113, #gen tokens =   3013, #acc tokens =  2070, #mean acc len = 2.37, #acc rate/pos = (0.738, 0.634), dur(b,g,a) = 0.018, 313.430, 1.877 ms
0.51.908.269 I slot      release: id 15 | task 77 | stop processing: n_tokens = 152, truncated = 0
0.51.912.797 I srv    operator(): Chat format: peg-native
0.51.916.704 I srv    operator(): Chat format: peg-native
0.52.015.878 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.52.015.881 I srv  get_availabl: updating prompt cache
0.52.015.922 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.015.924 I srv         alloc:  - prompt is already in the cache, skipping
0.52.015.925 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.52.015.927 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.52.015.928 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.52.015.929 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.52.015.929 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.52.015.930 I srv  get_availabl: prompt cache update took 0.05 ms
0.52.015.998 I slot launch_slot_: id  6 | task 138 | processing task, is_child = 0
0.52.016.000 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.52.016.016 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.016.016 I srv         alloc:  - prompt is already in the cache, skipping
0.52.016.018 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.52.016.019 I srv  get_availabl: updating prompt cache
0.52.016.032 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.016.033 I srv         alloc:  - prompt is already in the cache, skipping
0.52.016.034 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.52.016.034 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.52.016.035 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.52.016.035 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.52.016.035 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.52.016.036 I srv  get_availabl: prompt cache update took 0.02 ms
0.52.016.054 I slot launch_slot_: id 15 | task 139 | processing task, is_child = 0
0.52.020.580 I slot   operator(): id  6 | task 138 | Checking checkpoint with [23, 23] against 3...
0.52.020.582 W slot   operator(): id  6 | task 138 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.52.020.583 W slot   operator(): id  6 | task 138 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.52.022.075 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.022.104 I slot   operator(): id 15 | task 139 | Checking checkpoint with [23, 23] against 27...
0.52.026.059 W slot   operator(): id 15 | task 139 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.52.026.090 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.174.655 I slot create_check: id  6 | task 138 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.52.771.888 I slot print_timing: id  0 | task 73 | n_decoded =    100, tg =  14.92 t/s, tg_3s =  14.92 t/s
0.52.778.243 I slot print_timing: id 12 | task 76 | n_decoded =    100, tg =  14.90 t/s, tg_3s =  14.90 t/s
0.53.016.562 I slot print_timing: id  8 | task 99 | n_decoded =    102, tg =  23.63 t/s, tg_3s =  23.63 t/s
0.53.019.150 I slot print_timing: id 13 | task 102 | n_decoded =    102, tg =  23.62 t/s, tg_3s =  23.62 t/s
0.53.612.327 I slot print_timing: id  1 | task 59 | prompt eval time =     361.06 ms /    29 tokens (   12.45 ms per token,    80.32 tokens per second)
0.53.612.334 I slot print_timing: id  1 | task 59 |        eval time =    8622.48 ms /   128 tokens (   67.36 ms per token,    14.84 tokens per second)
0.53.612.335 I slot print_timing: id  1 | task 59 |       total time =    8983.54 ms /   157 tokens
0.53.612.336 I slot print_timing: id  1 | task 59 |    graphs reused =          1
0.53.612.338 I slot print_timing: id  1 | task 59 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.53.612.352 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1712, #gen drafts =   1726, #acc drafts =  1252, #gen tokens =   3445, #acc tokens =  2330, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.630), dur(b,g,a) = 0.020, 358.631, 2.132 ms
0.53.612.377 I slot      release: id  1 | task 59 | stop processing: n_tokens = 156, truncated = 0
0.53.612.575 I slot print_timing: id  4 | task 62 | prompt eval time =     354.23 ms /    29 tokens (   12.21 ms per token,    81.87 tokens per second)
0.53.612.577 I slot print_timing: id  4 | task 62 |        eval time =    8621.50 ms /   128 tokens (   67.36 ms per token,    14.85 tokens per second)
0.53.612.577 I slot print_timing: id  4 | task 62 |       total time =    8975.73 ms /   157 tokens
0.53.612.577 I slot print_timing: id  4 | task 62 |    graphs reused =          1
0.53.612.578 I slot print_timing: id  4 | task 62 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.53.612.580 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1712, #gen drafts =   1726, #acc drafts =  1252, #gen tokens =   3445, #acc tokens =  2330, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.630), dur(b,g,a) = 0.020, 358.631, 2.132 ms
0.53.612.612 I slot      release: id  4 | task 62 | stop processing: n_tokens = 156, truncated = 0
0.53.616.053 I slot print_timing: id  8 | task 99 | prompt eval time =     324.74 ms /    25 tokens (   12.99 ms per token,    76.99 tokens per second)
0.53.616.055 I slot print_timing: id  8 | task 99 |        eval time =    4916.35 ms /   114 tokens (   43.13 ms per token,    23.19 tokens per second)
0.53.616.055 I slot print_timing: id  8 | task 99 |       total time =    5241.09 ms /   139 tokens
0.53.616.056 I slot print_timing: id  8 | task 99 |    graphs reused =          1
0.53.616.058 I slot print_timing: id  8 | task 99 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.53.616.065 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1719, #gen drafts =   1726, #acc drafts =  1257, #gen tokens =   3445, #acc tokens =  2339, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.629), dur(b,g,a) = 0.020, 358.631, 2.143 ms
0.53.616.085 I slot      release: id  8 | task 99 | stop processing: n_tokens = 140, truncated = 0
0.53.617.720 I slot print_timing: id 11 | task 101 | n_decoded =    101, tg =  20.54 t/s, tg_3s =  20.54 t/s
0.53.618.899 I slot print_timing: id 13 | task 102 | prompt eval time =     321.81 ms /    25 tokens (   12.87 ms per token,    77.69 tokens per second)
0.53.618.901 I slot print_timing: id 13 | task 102 |        eval time =    4918.49 ms /   114 tokens (   43.14 ms per token,    23.18 tokens per second)
0.53.618.902 I slot print_timing: id 13 | task 102 |       total time =    5240.30 ms /   139 tokens
0.53.618.902 I slot print_timing: id 13 | task 102 |    graphs reused =          1
0.53.618.905 I slot print_timing: id 13 | task 102 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.53.618.912 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1724, #gen drafts =   1726, #acc drafts =  1262, #gen tokens =   3445, #acc tokens =  2348, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.630), dur(b,g,a) = 0.020, 358.631, 2.149 ms
0.53.618.933 I slot      release: id 13 | task 102 | stop processing: n_tokens = 140, truncated = 0
0.53.622.041 I srv    operator(): Chat format: peg-native
0.53.622.050 I srv    operator(): Chat format: peg-native
0.53.623.961 I srv    operator(): Chat format: peg-native
0.53.627.184 I srv    operator(): Chat format: peg-native
0.53.716.807 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.53.716.809 I srv  get_availabl: updating prompt cache
0.53.716.853 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.716.855 I srv         alloc:  - prompt is already in the cache, skipping
0.53.716.856 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.53.716.858 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.716.859 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.716.860 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.716.861 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.716.863 I srv  get_availabl: prompt cache update took 0.05 ms
0.53.716.929 I slot launch_slot_: id  8 | task 154 | processing task, is_child = 0
0.53.716.931 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.53.716.948 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.716.950 I srv         alloc:  - prompt is already in the cache, skipping
0.53.716.950 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.53.716.966 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.716.967 I srv         alloc:  - prompt is already in the cache, skipping
0.53.716.967 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.53.716.983 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.716.984 I srv         alloc:  - prompt is already in the cache, skipping
0.53.716.986 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.53.716.986 I srv  get_availabl: updating prompt cache
0.53.717.000 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.717.003 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.003 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.53.717.004 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.717.004 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.717.005 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.717.005 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.717.005 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.717.023 I slot launch_slot_: id  1 | task 155 | processing task, is_child = 0
0.53.717.024 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.53.717.037 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.717.039 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.039 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.53.717.052 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.717.053 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.055 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.53.717.055 I srv  get_availabl: updating prompt cache
0.53.717.068 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.717.070 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.070 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.53.717.070 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.717.071 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.717.071 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.717.071 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.717.071 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.717.086 I slot launch_slot_: id  4 | task 156 | processing task, is_child = 0
0.53.717.090 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.53.717.103 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.717.104 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.106 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.53.717.106 I srv  get_availabl: updating prompt cache
0.53.717.120 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.717.121 I srv         alloc:  - prompt is already in the cache, skipping
0.53.717.121 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.53.717.122 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.717.122 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.717.122 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.717.122 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.717.122 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.717.137 I slot launch_slot_: id 13 | task 157 | processing task, is_child = 0
0.53.723.532 I slot   operator(): id  1 | task 155 | Checking checkpoint with [24, 24] against 28...
0.53.727.475 W slot   operator(): id  1 | task 155 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.53.727.508 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.727.526 I slot   operator(): id  4 | task 156 | Checking checkpoint with [24, 24] against 3...
0.53.727.527 W slot   operator(): id  4 | task 156 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.53.727.528 W slot   operator(): id  4 | task 156 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.53.729.092 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.729.117 I slot   operator(): id  8 | task 154 | Checking checkpoint with [20, 20] against 24...
0.53.732.786 W slot   operator(): id  8 | task 154 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.53.732.806 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.732.827 I slot   operator(): id 13 | task 157 | Checking checkpoint with [20, 20] against 24...
0.53.736.347 W slot   operator(): id 13 | task 157 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.53.736.387 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.888.003 I slot create_check: id  4 | task 156 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.54.485.358 I slot print_timing: id  9 | task 118 | n_decoded =    102, tg =  23.59 t/s, tg_3s =  23.59 t/s
0.54.598.523 I slot print_timing: id  0 | task 73 | prompt eval time =     327.88 ms /    29 tokens (   11.31 ms per token,    88.45 tokens per second)
0.54.598.525 I slot print_timing: id  0 | task 73 |        eval time =    8530.60 ms /   128 tokens (   66.65 ms per token,    15.00 tokens per second)
0.54.598.526 I slot print_timing: id  0 | task 73 |       total time =    8858.48 ms /   157 tokens
0.54.598.526 I slot print_timing: id  0 | task 73 |    graphs reused =          1
0.54.598.529 I slot print_timing: id  0 | task 73 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.54.598.545 I statistics        draft-mtp: #calls(b,g,a) =   43    120   1829, #gen drafts =   1843, #acc drafts =  1333, #gen tokens =   3677, #acc tokens =  2483, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.629), dur(b,g,a) = 0.022, 390.205, 2.279 ms
0.54.598.570 I slot      release: id  0 | task 73 | stop processing: n_tokens = 156, truncated = 0
0.54.598.767 I slot print_timing: id 12 | task 76 | prompt eval time =     321.68 ms /    29 tokens (   11.09 ms per token,    90.15 tokens per second)
0.54.598.769 I slot print_timing: id 12 | task 76 |        eval time =    8530.22 ms /   128 tokens (   66.64 ms per token,    15.01 tokens per second)
0.54.598.769 I slot print_timing: id 12 | task 76 |       total time =    8851.90 ms /   157 tokens
0.54.598.769 I slot print_timing: id 12 | task 76 |    graphs reused =          1
0.54.598.770 I slot print_timing: id 12 | task 76 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.54.598.772 I statistics        draft-mtp: #calls(b,g,a) =   43    120   1829, #gen drafts =   1843, #acc drafts =  1333, #gen tokens =   3677, #acc tokens =  2483, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.629), dur(b,g,a) = 0.022, 390.205, 2.279 ms
0.54.598.788 I slot      release: id 12 | task 76 | stop processing: n_tokens = 156, truncated = 0
0.54.607.707 I srv    operator(): Chat format: peg-native
0.54.607.784 I srv    operator(): Chat format: peg-native
0.54.712.490 I slot print_timing: id 11 | task 101 | prompt eval time =     323.30 ms /    28 tokens (   11.55 ms per token,    86.61 tokens per second)
0.54.712.493 I slot print_timing: id 11 | task 101 |        eval time =    6012.49 ms /   125 tokens (   48.10 ms per token,    20.79 tokens per second)
0.54.712.493 I slot print_timing: id 11 | task 101 |       total time =    6335.79 ms /   153 tokens
0.54.712.494 I slot print_timing: id 11 | task 101 |    graphs reused =          1
0.54.712.497 I slot print_timing: id 11 | task 101 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
0.54.712.508 I statistics        draft-mtp: #calls(b,g,a) =   43    121   1854, #gen drafts =   1857, #acc drafts =  1352, #gen tokens =   3705, #acc tokens =  2518, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.629), dur(b,g,a) = 0.022, 393.902, 2.313 ms
0.54.712.532 I slot      release: id 11 | task 101 | stop processing: n_tokens = 152, truncated = 0
0.54.713.868 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.54.713.870 I srv  get_availabl: updating prompt cache
0.54.713.913 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.713.916 I srv         alloc:  - prompt is already in the cache, skipping
0.54.713.916 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.54.713.919 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.713.920 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.713.926 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.713.928 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.713.928 I srv  get_availabl: prompt cache update took 0.06 ms
0.54.713.991 I slot launch_slot_: id  0 | task 166 | processing task, is_child = 0
0.54.713.993 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.54.714.010 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.54.714.011 I srv         alloc:  - prompt is already in the cache, skipping
0.54.714.011 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.54.714.027 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.714.028 I srv         alloc:  - prompt is already in the cache, skipping
0.54.714.030 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.54.714.030 I srv  get_availabl: updating prompt cache
0.54.714.045 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.54.714.046 I srv         alloc:  - prompt is already in the cache, skipping
0.54.714.047 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.54.714.047 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.714.048 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.714.049 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.714.050 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.714.050 I srv  get_availabl: prompt cache update took 0.02 ms
0.54.714.066 I slot launch_slot_: id 11 | task 167 | processing task, is_child = 0
0.54.714.067 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.54.714.080 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.714.081 I srv         alloc:  - prompt is already in the cache, skipping
0.54.717.742 I slot   operator(): id  0 | task 166 | Checking checkpoint with [24, 24] against 28...
0.54.721.071 I srv    operator(): Chat format: peg-native
0.54.721.612 W slot   operator(): id  0 | task 166 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.54.721.646 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.721.659 I slot   operator(): id 11 | task 167 | Checking checkpoint with [23, 23] against 27...
0.54.725.233 W slot   operator(): id 11 | task 167 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.54.725.271 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.838.914 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.54.838.917 I srv  get_availabl: updating prompt cache
0.54.838.955 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.838.958 I srv         alloc:  - prompt is already in the cache, skipping
0.54.838.959 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.54.838.961 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.838.962 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.838.963 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.838.963 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.838.964 I srv  get_availabl: prompt cache update took 0.05 ms
0.54.839.032 I slot launch_slot_: id 12 | task 169 | processing task, is_child = 0
0.54.842.853 I slot   operator(): id 12 | task 169 | Checking checkpoint with [24, 24] against 3...
0.54.842.856 W slot   operator(): id 12 | task 169 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.54.842.859 W slot   operator(): id 12 | task 169 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.54.844.410 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.990.845 I slot create_check: id 12 | task 169 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.55.106.489 I slot print_timing: id  5 | task 100 | n_decoded =    100, tg =  15.21 t/s, tg_3s =  15.21 t/s
0.55.108.535 I slot print_timing: id  9 | task 118 | prompt eval time =     128.29 ms /     4 tokens (   32.07 ms per token,    31.18 tokens per second)
0.55.108.537 I slot print_timing: id  9 | task 118 |        eval time =    4946.80 ms /   114 tokens (   43.39 ms per token,    23.05 tokens per second)
0.55.108.538 I slot print_timing: id  9 | task 118 |       total time =    5075.08 ms /   118 tokens
0.55.108.539 I slot print_timing: id  9 | task 118 |    graphs reused =          1
0.55.108.542 I slot print_timing: id  9 | task 118 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.55.108.555 I statistics        draft-mtp: #calls(b,g,a) =   46    124   1895, #gen drafts =   1900, #acc drafts =  1382, #gen tokens =   3791, #acc tokens =  2573, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.628), dur(b,g,a) = 0.023, 405.103, 2.370 ms
0.55.108.587 I slot      release: id  9 | task 118 | stop processing: n_tokens = 140, truncated = 0
0.55.110.710 I slot print_timing: id 14 | task 103 | n_decoded =    100, tg =  15.20 t/s, tg_3s =  15.20 t/s
0.55.116.925 I srv    operator(): Chat format: peg-native
0.55.222.172 I slot print_timing: id  3 | task 117 | n_decoded =    101, tg =  20.51 t/s, tg_3s =  20.51 t/s
0.55.227.996 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.55.227.998 I srv  get_availabl: updating prompt cache
0.55.228.049 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.55.228.051 I srv         alloc:  - prompt is already in the cache, skipping
0.55.228.052 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.55.228.056 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.55.228.057 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.55.228.058 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.55.228.059 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.55.228.059 I srv  get_availabl: prompt cache update took 0.06 ms
0.55.228.132 I slot launch_slot_: id  9 | task 173 | processing task, is_child = 0
0.55.232.650 I slot   operator(): id  9 | task 173 | Checking checkpoint with [20, 20] against 3...
0.55.232.652 W slot   operator(): id  9 | task 173 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.55.232.654 W slot   operator(): id  9 | task 173 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.55.234.618 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.385.648 I slot create_check: id  9 | task 173 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.55.741.422 I slot print_timing: id  7 | task 131 | n_decoded =    102, tg =  23.59 t/s, tg_3s =  23.59 t/s
0.56.346.906 I slot print_timing: id  3 | task 117 | prompt eval time =     265.68 ms /    28 tokens (    9.49 ms per token,   105.39 tokens per second)
0.56.346.910 I slot print_timing: id  3 | task 117 |        eval time =    6049.25 ms /   125 tokens (   48.39 ms per token,    20.66 tokens per second)
0.56.346.910 I slot print_timing: id  3 | task 117 |       total time =    6314.93 ms /   153 tokens
0.56.346.911 I slot print_timing: id  3 | task 117 |    graphs reused =          1
0.56.346.914 I slot print_timing: id  3 | task 117 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
0.56.346.927 I statistics        draft-mtp: #calls(b,g,a) =   47    134   2045, #gen drafts =   2057, #acc drafts =  1495, #gen tokens =   4105, #acc tokens =  2782, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.629), dur(b,g,a) = 0.023, 437.496, 2.570 ms
0.56.346.957 I slot      release: id  3 | task 117 | stop processing: n_tokens = 152, truncated = 0
0.56.348.755 I slot print_timing: id  7 | task 131 | prompt eval time =     122.93 ms /     4 tokens (   30.73 ms per token,    32.54 tokens per second)
0.56.348.758 I slot print_timing: id  7 | task 131 |        eval time =    4931.20 ms /   114 tokens (   43.26 ms per token,    23.12 tokens per second)
0.56.348.759 I slot print_timing: id  7 | task 131 |       total time =    5054.13 ms /   118 tokens
0.56.348.759 I slot print_timing: id  7 | task 131 |    graphs reused =          1
0.56.348.761 I slot print_timing: id  7 | task 131 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.56.348.766 I statistics        draft-mtp: #calls(b,g,a) =   47    134   2049, #gen drafts =   2057, #acc drafts =  1497, #gen tokens =   4105, #acc tokens =  2786, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.629), dur(b,g,a) = 0.023, 437.496, 2.574 ms
0.56.348.787 I slot      release: id  7 | task 131 | stop processing: n_tokens = 140, truncated = 0
0.56.350.611 I slot print_timing: id 10 | task 130 | n_decoded =    101, tg =  20.48 t/s, tg_3s =  20.48 t/s
0.56.356.119 I srv    operator(): Chat format: peg-native
0.56.357.183 I srv    operator(): Chat format: peg-native
0.56.464.110 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.56.464.113 I srv  get_availabl: updating prompt cache
0.56.464.155 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.56.464.159 I srv         alloc:  - prompt is already in the cache, skipping
0.56.464.160 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.56.464.163 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.56.464.163 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.56.464.164 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.56.464.164 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.56.464.165 I srv  get_availabl: prompt cache update took 0.05 ms
0.56.464.231 I slot launch_slot_: id  3 | task 184 | processing task, is_child = 0
0.56.464.233 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.56.464.248 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.56.464.249 I srv         alloc:  - prompt is already in the cache, skipping
0.56.464.251 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.56.464.251 I srv  get_availabl: updating prompt cache
0.56.464.264 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.56.464.265 I srv         alloc:  - prompt is already in the cache, skipping
0.56.464.266 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.56.464.266 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.56.464.267 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.56.464.267 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.56.464.267 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.56.464.269 I srv  get_availabl: prompt cache update took 0.02 ms
0.56.464.286 I slot launch_slot_: id  7 | task 185 | processing task, is_child = 0
0.56.469.508 I slot   operator(): id  3 | task 184 | Checking checkpoint with [23, 23] against 27...
0.56.473.326 W slot   operator(): id  3 | task 184 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.56.473.354 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.473.386 I slot   operator(): id  7 | task 185 | Checking checkpoint with [20, 20] against 24...
0.56.476.854 W slot   operator(): id  7 | task 185 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.56.476.902 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.949.911 I slot print_timing: id  5 | task 100 | prompt eval time =     161.42 ms /     4 tokens (   40.35 ms per token,    24.78 tokens per second)
0.56.949.915 I slot print_timing: id  5 | task 100 |        eval time =    8417.11 ms /   128 tokens (   65.76 ms per token,    15.21 tokens per second)
0.56.949.915 I slot print_timing: id  5 | task 100 |       total time =    8578.53 ms /   132 tokens
0.56.949.916 I slot print_timing: id  5 | task 100 |    graphs reused =          1
0.56.949.920 I slot print_timing: id  5 | task 100 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
0.56.949.935 I statistics        draft-mtp: #calls(b,g,a) =   49    139   2117, #gen drafts =   2131, #acc drafts =  1542, #gen tokens =   4251, #acc tokens =  2872, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.628), dur(b,g,a) = 0.024, 458.865, 2.661 ms
0.56.949.960 I slot      release: id  5 | task 100 | stop processing: n_tokens = 156, truncated = 0
0.56.950.167 I slot print_timing: id 14 | task 103 | prompt eval time =     152.38 ms /     4 tokens (   38.10 ms per token,    26.25 tokens per second)
0.56.950.168 I slot print_timing: id 14 | task 103 |        eval time =    8417.10 ms /   128 tokens (   65.76 ms per token,    15.21 tokens per second)
0.56.950.169 I slot print_timing: id 14 | task 103 |       total time =    8569.49 ms /   132 tokens
0.56.950.169 I slot print_timing: id 14 | task 103 |    graphs reused =          1
0.56.950.170 I slot print_timing: id 14 | task 103 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
0.56.950.172 I statistics        draft-mtp: #calls(b,g,a) =   49    139   2117, #gen drafts =   2131, #acc drafts =  1542, #gen tokens =   4251, #acc tokens =  2872, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.628), dur(b,g,a) = 0.024, 458.865, 2.661 ms
0.56.950.188 I slot      release: id 14 | task 103 | stop processing: n_tokens = 156, truncated = 0
0.56.957.145 I slot print_timing: id 15 | task 139 | n_decoded =    101, tg =  21.01 t/s, tg_3s =  21.01 t/s
0.56.959.282 I srv    operator(): Chat format: peg-native
0.56.959.287 I srv    operator(): Chat format: peg-native
0.57.067.741 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.57.067.744 I srv  get_availabl: updating prompt cache
0.57.067.785 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.067.786 I srv         alloc:  - prompt is already in the cache, skipping
0.57.067.787 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.57.067.790 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.067.791 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.067.791 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.067.792 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.067.792 I srv  get_availabl: prompt cache update took 0.05 ms
0.57.067.859 I slot launch_slot_: id  5 | task 191 | processing task, is_child = 0
0.57.067.860 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.57.067.876 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.067.877 I srv         alloc:  - prompt is already in the cache, skipping
0.57.067.878 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.57.067.879 I srv  get_availabl: updating prompt cache
0.57.067.892 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.067.892 I srv         alloc:  - prompt is already in the cache, skipping
0.57.067.893 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.57.067.893 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.067.894 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.067.894 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.067.894 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.067.894 I srv  get_availabl: prompt cache update took 0.02 ms
0.57.067.912 I slot launch_slot_: id 14 | task 192 | processing task, is_child = 0
0.57.073.283 I slot   operator(): id  5 | task 191 | Checking checkpoint with [24, 24] against 3...
0.57.073.285 W slot   operator(): id  5 | task 191 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.57.073.289 W slot   operator(): id  5 | task 191 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.57.074.829 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.074.860 I slot   operator(): id 14 | task 192 | Checking checkpoint with [24, 24] against 28...
0.57.078.697 W slot   operator(): id 14 | task 192 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.57.078.725 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.227.499 I slot create_check: id  5 | task 191 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.57.466.435 I slot print_timing: id 10 | task 130 | prompt eval time =     119.67 ms /     4 tokens (   29.92 ms per token,    33.43 tokens per second)
0.57.466.438 I slot print_timing: id 10 | task 130 |        eval time =    6048.59 ms /   125 tokens (   48.39 ms per token,    20.67 tokens per second)
0.57.466.439 I slot print_timing: id 10 | task 130 |       total time =    6168.26 ms /   129 tokens
0.57.466.440 I slot print_timing: id 10 | task 130 |    graphs reused =          1
0.57.466.443 I slot print_timing: id 10 | task 130 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
0.57.466.456 I statistics        draft-mtp: #calls(b,g,a) =   51    143   2185, #gen drafts =   2190, #acc drafts =  1591, #gen tokens =   4369, #acc tokens =  2961, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.627), dur(b,g,a) = 0.026, 476.518, 2.751 ms
0.57.466.484 I slot      release: id 10 | task 130 | stop processing: n_tokens = 152, truncated = 0
0.57.475.064 I srv    operator(): Chat format: peg-native
0.57.584.371 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
0.57.584.373 I srv  get_availabl: updating prompt cache
0.57.584.417 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.57.584.419 I srv         alloc:  - prompt is already in the cache, skipping
0.57.584.420 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.120
0.57.584.423 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.584.423 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.584.424 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.584.424 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.584.425 I srv  get_availabl: prompt cache update took 0.05 ms
0.57.584.492 I slot launch_slot_: id 10 | task 197 | processing task, is_child = 0
0.57.588.945 I slot   operator(): id 10 | task 197 | Checking checkpoint with [23, 23] against 3...
0.57.588.946 W slot   operator(): id 10 | task 197 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.57.588.948 W slot   operator(): id 10 | task 197 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.57.590.469 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.737.249 I slot create_check: id 10 | task 197 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.58.093.660 I slot print_timing: id  2 | task 128 | n_decoded =    100, tg =  14.98 t/s, tg_3s =  14.98 t/s
0.58.100.039 I slot print_timing: id 15 | task 139 | prompt eval time =     128.41 ms /     4 tokens (   32.10 ms per token,    31.15 tokens per second)
0.58.100.041 I slot print_timing: id 15 | task 139 |        eval time =    5949.48 ms /   125 tokens (   47.60 ms per token,    21.01 tokens per second)
0.58.100.042 I slot print_timing: id 15 | task 139 |       total time =    6077.89 ms /   129 tokens
0.58.100.042 I slot print_timing: id 15 | task 139 |    graphs reused =          1
0.58.100.045 I slot print_timing: id 15 | task 139 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.58.100.058 I statistics        draft-mtp: #calls(b,g,a) =   52    148   2267, #gen drafts =   2267, #acc drafts =  1654, #gen tokens =   4523, #acc tokens =  3077, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.628), dur(b,g,a) = 0.027, 494.617, 2.861 ms
0.58.100.086 I slot      release: id 15 | task 139 | stop processing: n_tokens = 152, truncated = 0
0.58.108.741 I srv    operator(): Chat format: peg-native
0.58.209.849 I slot print_timing: id  8 | task 154 | n_decoded =    102, tg =  23.47 t/s, tg_3s =  23.47 t/s
0.58.212.717 I slot print_timing: id 13 | task 157 | n_decoded =    102, tg =  23.46 t/s, tg_3s =  23.46 t/s
0.58.213.007 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.58.213.008 I srv  get_availabl: updating prompt cache
0.58.213.048 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.58.213.050 I srv         alloc:  - prompt is already in the cache, skipping
0.58.213.051 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.58.213.054 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.58.213.055 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.58.213.055 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.58.213.055 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.58.213.056 I srv  get_availabl: prompt cache update took 0.05 ms
0.58.213.123 I slot launch_slot_: id 15 | task 203 | processing task, is_child = 0
0.58.215.886 I slot   operator(): id 15 | task 203 | Checking checkpoint with [23, 23] against 3...
0.58.215.888 W slot   operator(): id 15 | task 203 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.215.891 W slot   operator(): id 15 | task 203 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.58.217.481 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.363.283 I slot create_check: id 15 | task 203 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.58.842.242 I slot print_timing: id  8 | task 154 | prompt eval time =     134.63 ms /     4 tokens (   33.66 ms per token,    29.71 tokens per second)
0.58.842.245 I slot print_timing: id  8 | task 154 |        eval time =    4978.48 ms /   114 tokens (   43.67 ms per token,    22.90 tokens per second)
0.58.842.245 I slot print_timing: id  8 | task 154 |       total time =    5113.10 ms /   118 tokens
0.58.842.246 I slot print_timing: id  8 | task 154 |    graphs reused =          1
0.58.842.249 I slot print_timing: id  8 | task 154 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.58.842.263 I statistics        draft-mtp: #calls(b,g,a) =   53    154   2353, #gen drafts =   2360, #acc drafts =  1717, #gen tokens =   4709, #acc tokens =  3193, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.627), dur(b,g,a) = 0.028, 511.125, 2.973 ms
0.58.842.288 I slot      release: id  8 | task 154 | stop processing: n_tokens = 140, truncated = 0
0.58.844.975 I slot print_timing: id 13 | task 157 | prompt eval time =     131.18 ms /     4 tokens (   32.79 ms per token,    30.49 tokens per second)
0.58.844.977 I slot print_timing: id 13 | task 157 |        eval time =    4980.95 ms /   114 tokens (   43.69 ms per token,    22.89 tokens per second)
0.58.844.977 I slot print_timing: id 13 | task 157 |       total time =    5112.13 ms /   118 tokens
0.58.844.977 I slot print_timing: id 13 | task 157 |    graphs reused =          1
0.58.844.980 I slot print_timing: id 13 | task 157 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.58.844.988 I statistics        draft-mtp: #calls(b,g,a) =   53    154   2358, #gen drafts =   2360, #acc drafts =  1721, #gen tokens =   4709, #acc tokens =  3201, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.628), dur(b,g,a) = 0.028, 511.125, 2.979 ms
0.58.845.012 I slot      release: id 13 | task 157 | stop processing: n_tokens = 140, truncated = 0
0.58.850.912 I srv    operator(): Chat format: peg-native
0.58.853.249 I srv    operator(): Chat format: peg-native
0.58.951.250 I slot print_timing: id  4 | task 156 | n_decoded =    101, tg =  20.40 t/s, tg_3s =  20.40 t/s
0.58.952.293 I slot print_timing: id  6 | task 138 | n_decoded =    100, tg =  15.00 t/s, tg_3s =  15.00 t/s
0.58.955.679 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.58.955.681 I srv  get_availabl: updating prompt cache
0.58.955.721 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.58.955.723 I srv         alloc:  - prompt is already in the cache, skipping
0.58.955.724 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.58.955.728 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.58.955.729 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.58.955.729 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.58.955.730 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.58.955.730 I srv  get_availabl: prompt cache update took 0.05 ms
0.58.955.796 I slot launch_slot_: id  8 | task 210 | processing task, is_child = 0
0.58.955.797 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.58.955.812 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.58.955.812 I srv         alloc:  - prompt is already in the cache, skipping
0.58.955.813 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.58.955.814 I srv  get_availabl: updating prompt cache
0.58.955.826 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.58.955.827 I srv         alloc:  - prompt is already in the cache, skipping
0.58.955.827 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.58.955.827 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.58.955.828 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
0.58.955.828 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
0.58.955.828 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
0.58.955.829 I srv  get_availabl: prompt cache update took 0.02 ms
0.58.955.846 I slot launch_slot_: id 13 | task 211 | processing task, is_child = 0
0.58.961.162 I slot   operator(): id  8 | task 210 | Checking checkpoint with [20, 20] against 3...
0.58.961.164 W slot   operator(): id  8 | task 210 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.961.167 W slot   operator(): id  8 | task 210 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.58.962.700 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.962.733 I slot   operator(): id 13 | task 211 | Checking checkpoint with [20, 20] against 24...
0.58.966.573 W slot   operator(): id 13 | task 211 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.58.966.602 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.116.173 I slot create_check: id  8 | task 210 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.59.475.210 I slot print_timing: id 12 | task 169 | n_decoded =    102, tg =  23.33 t/s, tg_3s =  23.33 t/s
0.59.840.109 I slot print_timing: id 11 | task 167 | n_decoded =    101, tg =  20.17 t/s, tg_3s =  20.17 t/s
0.59.954.039 I slot print_timing: id  2 | task 128 | prompt eval time =     260.69 ms /    29 tokens (    8.99 ms per token,   111.24 tokens per second)
0.59.954.041 I slot print_timing: id  2 | task 128 |        eval time =    8536.75 ms /   128 tokens (   66.69 ms per token,    14.99 tokens per second)
0.59.954.042 I slot print_timing: id  2 | task 128 |       total time =    8797.45 ms /   157 tokens
0.59.954.043 I slot print_timing: id  2 | task 128 |    graphs reused =          1
0.59.954.045 I slot print_timing: id  2 | task 128 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.59.954.058 I statistics        draft-mtp: #calls(b,g,a) =   55    163   2483, #gen drafts =   2498, #acc drafts =  1809, #gen tokens =   4984, #acc tokens =  3364, #mean acc len = 2.35, #acc rate/pos = (0.729, 0.626), dur(b,g,a) = 0.029, 544.161, 3.134 ms
0.59.954.083 I slot      release: id  2 | task 128 | stop processing: n_tokens = 156, truncated = 0
0.59.962.700 I srv    operator(): Chat format: peg-native
1.00.070.639 I slot print_timing: id  4 | task 156 | prompt eval time =     272.73 ms /    28 tokens (    9.74 ms per token,   102.67 tokens per second)
1.00.070.642 I slot print_timing: id  4 | task 156 |        eval time =    6070.33 ms /   125 tokens (   48.56 ms per token,    20.59 tokens per second)
1.00.070.642 I slot print_timing: id  4 | task 156 |       total time =    6343.06 ms /   153 tokens
1.00.070.643 I slot print_timing: id  4 | task 156 |    graphs reused =          1
1.00.070.646 I slot print_timing: id  4 | task 156 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.00.070.657 I statistics        draft-mtp: #calls(b,g,a) =   55    164   2502, #gen drafts =   2513, #acc drafts =  1822, #gen tokens =   5014, #acc tokens =  3389, #mean acc len = 2.35, #acc rate/pos = (0.728, 0.626), dur(b,g,a) = 0.029, 547.847, 3.161 ms
1.00.070.682 I slot      release: id  4 | task 156 | stop processing: n_tokens = 152, truncated = 0
1.00.074.160 I slot print_timing: id 12 | task 169 | prompt eval time =     260.20 ms /    25 tokens (   10.41 ms per token,    96.08 tokens per second)
1.00.074.162 I slot print_timing: id 12 | task 169 |        eval time =    4971.10 ms /   114 tokens (   43.61 ms per token,    22.93 tokens per second)
1.00.074.162 I slot print_timing: id 12 | task 169 |       total time =    5231.29 ms /   139 tokens
1.00.074.163 I slot print_timing: id 12 | task 169 |    graphs reused =          1
1.00.074.165 I slot print_timing: id 12 | task 169 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.00.074.171 I statistics        draft-mtp: #calls(b,g,a) =   55    164   2510, #gen drafts =   2513, #acc drafts =  1827, #gen tokens =   5014, #acc tokens =  3397, #mean acc len = 2.35, #acc rate/pos = (0.728, 0.625), dur(b,g,a) = 0.029, 547.847, 3.170 ms
1.00.074.192 I slot      release: id 12 | task 169 | stop processing: n_tokens = 140, truncated = 0
1.00.075.383 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.00.075.384 I srv  get_availabl: updating prompt cache
1.00.075.428 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.00.075.430 I srv         alloc:  - prompt is already in the cache, skipping
1.00.075.431 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.00.075.434 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.075.436 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.075.436 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.075.437 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.075.438 I srv  get_availabl: prompt cache update took 0.05 ms
1.00.075.502 I slot launch_slot_: id  2 | task 221 | processing task, is_child = 0
1.00.075.503 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
1.00.075.521 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.00.075.522 I srv         alloc:  - prompt is already in the cache, skipping
1.00.075.522 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.00.075.540 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.075.541 I srv         alloc:  - prompt is already in the cache, skipping
1.00.079.359 I srv    operator(): Chat format: peg-native
1.00.081.293 I slot   operator(): id  2 | task 221 | Checking checkpoint with [24, 24] against 28...
1.00.083.060 I srv    operator(): Chat format: peg-native
1.00.085.132 W slot   operator(): id  2 | task 221 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.00.085.172 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.191.471 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.00.191.474 I srv  get_availabl: updating prompt cache
1.00.191.517 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.00.191.519 I srv         alloc:  - prompt is already in the cache, skipping
1.00.191.520 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.00.191.522 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.191.524 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.191.524 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.191.525 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.191.526 I srv  get_availabl: prompt cache update took 0.05 ms
1.00.191.594 I slot launch_slot_: id  4 | task 223 | processing task, is_child = 0
1.00.191.595 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.00.191.611 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.191.612 I srv         alloc:  - prompt is already in the cache, skipping
1.00.191.614 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.00.191.614 I srv  get_availabl: updating prompt cache
1.00.191.629 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.191.629 I srv         alloc:  - prompt is already in the cache, skipping
1.00.191.629 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.00.191.630 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.191.630 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.191.631 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.191.632 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.191.632 I srv  get_availabl: prompt cache update took 0.02 ms
1.00.191.652 I slot launch_slot_: id 12 | task 224 | processing task, is_child = 0
1.00.197.015 I slot   operator(): id  4 | task 223 | Checking checkpoint with [23, 23] against 27...
1.00.200.891 W slot   operator(): id  4 | task 223 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.00.200.934 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.200.948 I slot   operator(): id 12 | task 224 | Checking checkpoint with [20, 20] against 24...
1.00.204.503 W slot   operator(): id 12 | task 224 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.00.204.537 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.561.074 I slot print_timing: id  1 | task 155 | n_decoded =    100, tg =  14.93 t/s, tg_3s =  14.93 t/s
1.00.801.514 I slot print_timing: id  6 | task 138 | prompt eval time =     266.55 ms /    29 tokens (    9.19 ms per token,   108.80 tokens per second)
1.00.801.517 I slot print_timing: id  6 | task 138 |        eval time =    8514.35 ms /   128 tokens (   66.52 ms per token,    15.03 tokens per second)
1.00.801.518 I slot print_timing: id  6 | task 138 |       total time =    8780.90 ms /   157 tokens
1.00.801.519 I slot print_timing: id  6 | task 138 |    graphs reused =          1
1.00.801.522 I slot print_timing: id  6 | task 138 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.00.801.540 I statistics        draft-mtp: #calls(b,g,a) =   58    170   2588, #gen drafts =   2603, #acc drafts =  1880, #gen tokens =   5193, #acc tokens =  3495, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.624), dur(b,g,a) = 0.032, 571.655, 3.276 ms
1.00.801.567 I slot      release: id  6 | task 138 | stop processing: n_tokens = 156, truncated = 0
1.00.810.448 I srv    operator(): Chat format: peg-native
1.00.920.684 I slot print_timing: id  7 | task 185 | n_decoded =    102, tg =  23.56 t/s, tg_3s =  23.56 t/s
1.00.922.774 I slot print_timing: id 11 | task 167 | prompt eval time =     110.42 ms /     4 tokens (   27.61 ms per token,    36.23 tokens per second)
1.00.922.775 I slot print_timing: id 11 | task 167 |        eval time =    6090.65 ms /   125 tokens (   48.73 ms per token,    20.52 tokens per second)
1.00.922.776 I slot print_timing: id 11 | task 167 |       total time =    6201.07 ms /   129 tokens
1.00.922.777 I slot print_timing: id 11 | task 167 |    graphs reused =          1
1.00.922.779 I slot print_timing: id 11 | task 167 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.00.922.790 I statistics        draft-mtp: #calls(b,g,a) =   58    171   2614, #gen drafts =   2618, #acc drafts =  1898, #gen tokens =   5223, #acc tokens =  3529, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.624), dur(b,g,a) = 0.032, 576.256, 3.312 ms
1.00.922.816 I slot      release: id 11 | task 167 | stop processing: n_tokens = 152, truncated = 0
1.00.925.103 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.00.925.104 I srv  get_availabl: updating prompt cache
1.00.925.145 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.00.925.147 I srv         alloc:  - prompt is already in the cache, skipping
1.00.925.148 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.00.925.150 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.925.151 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.925.152 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.925.152 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.925.153 I srv  get_availabl: prompt cache update took 0.05 ms
1.00.925.217 I slot launch_slot_: id  6 | task 231 | processing task, is_child = 0
1.00.925.218 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
1.00.925.234 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.00.925.235 I srv         alloc:  - prompt is already in the cache, skipping
1.00.930.675 I slot   operator(): id  6 | task 231 | Checking checkpoint with [24, 24] against 28...
1.00.931.045 I srv    operator(): Chat format: peg-native
1.00.934.516 W slot   operator(): id  6 | task 231 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.00.934.545 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.047.484 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.01.047.487 I srv  get_availabl: updating prompt cache
1.01.047.529 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.01.047.531 I srv         alloc:  - prompt is already in the cache, skipping
1.01.047.532 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.01.047.534 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.01.047.535 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.01.047.536 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.01.047.536 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.01.047.536 I srv  get_availabl: prompt cache update took 0.05 ms
1.01.047.602 I slot launch_slot_: id 11 | task 233 | processing task, is_child = 0
1.01.051.768 I slot   operator(): id 11 | task 233 | Checking checkpoint with [23, 23] against 27...
1.01.055.752 W slot   operator(): id 11 | task 233 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.01.055.789 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.532.308 I slot print_timing: id  0 | task 166 | n_decoded =    100, tg =  14.92 t/s, tg_3s =  14.92 t/s
1.01.533.991 I slot print_timing: id  3 | task 184 | n_decoded =    101, tg =  20.43 t/s, tg_3s =  20.43 t/s
1.01.536.075 I slot print_timing: id  7 | task 185 | prompt eval time =     117.72 ms /     4 tokens (   29.43 ms per token,    33.98 tokens per second)
1.01.536.076 I slot print_timing: id  7 | task 185 |        eval time =    4944.94 ms /   114 tokens (   43.38 ms per token,    23.05 tokens per second)
1.01.536.077 I slot print_timing: id  7 | task 185 |       total time =    5062.67 ms /   118 tokens
1.01.536.078 I slot print_timing: id  7 | task 185 |    graphs reused =          1
1.01.536.081 I slot print_timing: id  7 | task 185 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.01.536.095 I statistics        draft-mtp: #calls(b,g,a) =   60    176   2687, #gen drafts =   2695, #acc drafts =  1953, #gen tokens =   5377, #acc tokens =  3632, #mean acc len = 2.35, #acc rate/pos = (0.727, 0.625), dur(b,g,a) = 0.033, 594.147, 3.426 ms
1.01.536.122 I slot      release: id  7 | task 185 | stop processing: n_tokens = 140, truncated = 0
1.01.544.942 I srv    operator(): Chat format: peg-native
1.01.656.701 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.01.656.704 I srv  get_availabl: updating prompt cache
1.01.656.755 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.01.656.757 I srv         alloc:  - prompt is already in the cache, skipping
1.01.656.758 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.01.656.762 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.01.656.763 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.01.656.764 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.01.656.765 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.01.656.766 I srv  get_availabl: prompt cache update took 0.06 ms
1.01.656.831 I slot launch_slot_: id  7 | task 239 | processing task, is_child = 0
1.01.661.635 I slot   operator(): id  7 | task 239 | Checking checkpoint with [20, 20] against 24...
1.01.665.629 W slot   operator(): id  7 | task 239 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.01.665.668 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.143.943 I slot print_timing: id  9 | task 173 | n_decoded =    100, tg =  15.05 t/s, tg_3s =  15.05 t/s
1.02.144.552 I slot print_timing: id 10 | task 197 | n_decoded =    102, tg =  23.75 t/s, tg_3s =  23.75 t/s
1.02.262.710 I slot print_timing: id  5 | task 191 | n_decoded =    101, tg =  20.52 t/s, tg_3s =  20.52 t/s
1.02.380.235 I slot print_timing: id  1 | task 155 | prompt eval time =     139.94 ms /     4 tokens (   34.99 ms per token,    28.58 tokens per second)
1.02.380.238 I slot print_timing: id  1 | task 155 |        eval time =    8516.73 ms /   128 tokens (   66.54 ms per token,    15.03 tokens per second)
1.02.380.239 I slot print_timing: id  1 | task 155 |       total time =    8656.67 ms /   132 tokens
1.02.380.240 I slot print_timing: id  1 | task 155 |    graphs reused =          1
1.02.380.243 I slot print_timing: id  1 | task 155 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.02.380.260 I statistics        draft-mtp: #calls(b,g,a) =   61    183   2789, #gen drafts =   2804, #acc drafts =  2025, #gen tokens =   5594, #acc tokens =  3765, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.624), dur(b,g,a) = 0.034, 618.484, 3.557 ms
1.02.380.286 I slot      release: id  1 | task 155 | stop processing: n_tokens = 156, truncated = 0
1.02.389.555 I srv    operator(): Chat format: peg-native
1.02.502.974 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.02.502.976 I srv  get_availabl: updating prompt cache
1.02.503.026 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.02.503.028 I srv         alloc:  - prompt is already in the cache, skipping
1.02.503.029 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.02.503.032 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.503.033 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.503.034 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.503.034 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.503.035 I srv  get_availabl: prompt cache update took 0.06 ms
1.02.503.102 I slot launch_slot_: id  1 | task 247 | processing task, is_child = 0
1.02.506.784 I slot   operator(): id  1 | task 247 | Checking checkpoint with [24, 24] against 28...
1.02.510.658 W slot   operator(): id  1 | task 247 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.02.510.680 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.624.854 I slot print_timing: id  3 | task 184 | prompt eval time =     121.33 ms /     4 tokens (   30.33 ms per token,    32.97 tokens per second)
1.02.624.857 I slot print_timing: id  3 | task 184 |        eval time =    6033.98 ms /   125 tokens (   48.27 ms per token,    20.72 tokens per second)
1.02.624.857 I slot print_timing: id  3 | task 184 |       total time =    6155.31 ms /   129 tokens
1.02.624.858 I slot print_timing: id  3 | task 184 |    graphs reused =          1
1.02.624.861 I slot print_timing: id  3 | task 184 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.02.624.874 I statistics        draft-mtp: #calls(b,g,a) =   62    185   2822, #gen drafts =   2834, #acc drafts =  2052, #gen tokens =   5654, #acc tokens =  3815, #mean acc len = 2.35, #acc rate/pos = (0.727, 0.625), dur(b,g,a) = 0.035, 625.773, 3.602 ms
1.02.624.903 I slot      release: id  3 | task 184 | stop processing: n_tokens = 152, truncated = 0
1.02.634.001 I srv    operator(): Chat format: peg-native
1.02.743.037 I slot print_timing: id 10 | task 197 | prompt eval time =     260.69 ms /    25 tokens (   10.43 ms per token,    95.90 tokens per second)
1.02.743.039 I slot print_timing: id 10 | task 197 |        eval time =    4893.38 ms /   114 tokens (   42.92 ms per token,    23.30 tokens per second)
1.02.743.040 I slot print_timing: id 10 | task 197 |       total time =    5154.07 ms /   139 tokens
1.02.743.040 I slot print_timing: id 10 | task 197 |    graphs reused =          1
1.02.743.043 I slot print_timing: id 10 | task 197 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.02.743.058 I statistics        draft-mtp: #calls(b,g,a) =   62    186   2844, #gen drafts =   2849, #acc drafts =  2065, #gen tokens =   5684, #acc tokens =  3839, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.624), dur(b,g,a) = 0.035, 629.584, 3.629 ms
1.02.743.086 I slot      release: id 10 | task 197 | stop processing: n_tokens = 140, truncated = 0
1.02.745.608 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.02.745.610 I srv  get_availabl: updating prompt cache
1.02.745.661 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.02.745.662 I srv         alloc:  - prompt is already in the cache, skipping
1.02.745.663 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.02.745.667 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.745.668 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.745.669 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.745.669 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.745.670 I srv  get_availabl: prompt cache update took 0.06 ms
1.02.745.738 I slot launch_slot_: id  3 | task 250 | processing task, is_child = 0
1.02.745.739 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.02.745.757 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.745.758 I srv         alloc:  - prompt is already in the cache, skipping
1.02.751.070 I slot   operator(): id  3 | task 250 | Checking checkpoint with [23, 23] against 27...
1.02.751.718 I srv    operator(): Chat format: peg-native
1.02.754.939 W slot   operator(): id  3 | task 250 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.02.754.969 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.868.284 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.02.868.287 I srv  get_availabl: updating prompt cache
1.02.868.338 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.868.340 I srv         alloc:  - prompt is already in the cache, skipping
1.02.868.341 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.02.868.345 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.868.346 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.868.346 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.868.347 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.868.347 I srv  get_availabl: prompt cache update took 0.06 ms
1.02.868.413 I slot launch_slot_: id 10 | task 252 | processing task, is_child = 0
1.02.872.503 I slot   operator(): id 10 | task 252 | Checking checkpoint with [20, 20] against 24...
1.02.876.363 W slot   operator(): id 10 | task 252 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.02.876.406 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.348.897 I slot print_timing: id  0 | task 166 | prompt eval time =     114.07 ms /     4 tokens (   28.52 ms per token,    35.07 tokens per second)
1.03.348.901 I slot print_timing: id  0 | task 166 |        eval time =    8517.05 ms /   128 tokens (   66.54 ms per token,    15.03 tokens per second)
1.03.348.901 I slot print_timing: id  0 | task 166 |       total time =    8631.12 ms /   132 tokens
1.03.348.902 I slot print_timing: id  0 | task 166 |    graphs reused =          1
1.03.348.907 I slot print_timing: id  0 | task 166 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.03.348.922 I statistics        draft-mtp: #calls(b,g,a) =   64    191   2910, #gen drafts =   2925, #acc drafts =  2111, #gen tokens =   5835, #acc tokens =  3926, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.624), dur(b,g,a) = 0.037, 647.263, 3.716 ms
1.03.348.945 I slot      release: id  0 | task 166 | stop processing: n_tokens = 156, truncated = 0
1.03.351.411 I slot print_timing: id  5 | task 191 | prompt eval time =     266.73 ms /    28 tokens (    9.53 ms per token,   104.98 tokens per second)
1.03.351.413 I slot print_timing: id  5 | task 191 |        eval time =    6011.36 ms /   125 tokens (   48.09 ms per token,    20.79 tokens per second)
1.03.351.414 I slot print_timing: id  5 | task 191 |       total time =    6278.09 ms /   153 tokens
1.03.351.414 I slot print_timing: id  5 | task 191 |    graphs reused =          1
1.03.351.416 I slot print_timing: id  5 | task 191 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.03.351.424 I statistics        draft-mtp: #calls(b,g,a) =   64    191   2915, #gen drafts =   2925, #acc drafts =  2114, #gen tokens =   5835, #acc tokens =  3932, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.624), dur(b,g,a) = 0.037, 647.263, 3.723 ms
1.03.351.447 I slot      release: id  5 | task 191 | stop processing: n_tokens = 152, truncated = 0
1.03.355.782 I slot print_timing: id 13 | task 211 | n_decoded =    102, tg =  23.91 t/s, tg_3s =  23.91 t/s
1.03.357.654 I srv    operator(): Chat format: peg-native
1.03.359.305 I srv    operator(): Chat format: peg-native
1.03.465.044 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.03.465.046 I srv  get_availabl: updating prompt cache
1.03.465.089 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.03.465.091 I srv         alloc:  - prompt is already in the cache, skipping
1.03.465.092 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.03.465.094 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.03.465.095 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.03.465.096 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.03.465.096 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.03.465.096 I srv  get_availabl: prompt cache update took 0.05 ms
1.03.465.162 I slot launch_slot_: id  0 | task 258 | processing task, is_child = 0
1.03.465.163 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.03.465.179 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.03.465.180 I srv         alloc:  - prompt is already in the cache, skipping
1.03.465.181 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.03.465.181 I srv  get_availabl: updating prompt cache
1.03.465.195 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.03.465.195 I srv         alloc:  - prompt is already in the cache, skipping
1.03.465.195 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.03.465.196 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.03.465.197 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.03.465.197 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.03.465.197 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.03.465.198 I srv  get_availabl: prompt cache update took 0.01 ms
1.03.465.215 I slot launch_slot_: id  5 | task 259 | processing task, is_child = 0
1.03.469.274 I slot   operator(): id  0 | task 258 | Checking checkpoint with [24, 24] against 28...
1.03.473.121 W slot   operator(): id  0 | task 258 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.03.473.152 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.473.171 I slot   operator(): id  5 | task 259 | Checking checkpoint with [23, 23] against 27...
1.03.476.671 W slot   operator(): id  5 | task 259 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.03.476.703 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.717.229 I slot print_timing: id 14 | task 192 | n_decoded =    100, tg =  15.35 t/s, tg_3s =  15.35 t/s
1.03.951.835 I slot print_timing: id  9 | task 173 | prompt eval time =     265.70 ms /    29 tokens (    9.16 ms per token,   109.15 tokens per second)
1.03.951.838 I slot print_timing: id  9 | task 173 |        eval time =    8453.45 ms /   128 tokens (   66.04 ms per token,    15.14 tokens per second)
1.03.951.838 I slot print_timing: id  9 | task 173 |       total time =    8719.15 ms /   157 tokens
1.03.951.839 I slot print_timing: id  9 | task 173 |    graphs reused =          1
1.03.951.841 I slot print_timing: id  9 | task 173 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.03.951.857 I statistics        draft-mtp: #calls(b,g,a) =   66    196   2985, #gen drafts =   3000, #acc drafts =  2165, #gen tokens =   5984, #acc tokens =  4025, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.623), dur(b,g,a) = 0.038, 665.130, 3.819 ms
1.03.951.883 I slot      release: id  9 | task 173 | stop processing: n_tokens = 156, truncated = 0
1.03.958.435 I slot print_timing: id 13 | task 211 | prompt eval time =     127.79 ms /     4 tokens (   31.95 ms per token,    31.30 tokens per second)
1.03.958.438 I slot print_timing: id 13 | task 211 |        eval time =    4867.90 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
1.03.958.438 I slot print_timing: id 13 | task 211 |       total time =    4995.69 ms /   118 tokens
1.03.958.439 I slot print_timing: id 13 | task 211 |    graphs reused =          1
1.03.958.441 I slot print_timing: id 13 | task 211 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.03.958.448 I statistics        draft-mtp: #calls(b,g,a) =   66    196   2998, #gen drafts =   3000, #acc drafts =  2175, #gen tokens =   5984, #acc tokens =  4043, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.623), dur(b,g,a) = 0.038, 665.130, 3.838 ms
1.03.958.468 I slot      release: id 13 | task 211 | stop processing: n_tokens = 140, truncated = 0
1.03.960.530 I srv    operator(): Chat format: peg-native
1.03.967.213 I srv    operator(): Chat format: peg-native
1.04.066.549 I slot print_timing: id  8 | task 210 | n_decoded =    101, tg =  20.88 t/s, tg_3s =  20.88 t/s
1.04.069.239 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.04.069.240 I srv  get_availabl: updating prompt cache
1.04.069.280 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.04.069.281 I srv         alloc:  - prompt is already in the cache, skipping
1.04.069.282 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.04.069.285 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.04.069.286 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.04.069.287 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.04.069.287 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.04.069.288 I srv  get_availabl: prompt cache update took 0.05 ms
1.04.069.358 I slot launch_slot_: id 13 | task 265 | processing task, is_child = 0
1.04.069.359 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
1.04.069.375 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.069.376 I srv         alloc:  - prompt is already in the cache, skipping
1.04.069.378 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.04.069.378 I srv  get_availabl: updating prompt cache
1.04.069.391 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.069.392 I srv         alloc:  - prompt is already in the cache, skipping
1.04.069.392 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.04.069.392 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.04.069.393 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.04.069.393 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.04.069.393 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.04.069.394 I srv  get_availabl: prompt cache update took 0.01 ms
1.04.069.410 I slot launch_slot_: id  9 | task 266 | processing task, is_child = 0
1.04.074.033 I slot   operator(): id  9 | task 266 | Checking checkpoint with [24, 24] against 28...
1.04.077.852 W slot   operator(): id  9 | task 266 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.04.077.893 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.077.898 I slot   operator(): id 13 | task 265 | Checking checkpoint with [20, 20] against 24...
1.04.081.374 W slot   operator(): id 13 | task 265 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.04.081.408 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.562.097 I slot print_timing: id 12 | task 224 | n_decoded =    102, tg =  24.04 t/s, tg_3s =  24.04 t/s
1.05.048.303 I slot print_timing: id 15 | task 203 | n_decoded =    100, tg =  15.21 t/s, tg_3s =  15.21 t/s
1.05.163.668 I slot print_timing: id  4 | task 223 | n_decoded =    101, tg =  20.85 t/s, tg_3s =  20.85 t/s
1.05.165.601 I slot print_timing: id  8 | task 210 | prompt eval time =     267.07 ms /    28 tokens (    9.54 ms per token,   104.84 tokens per second)
1.05.165.603 I slot print_timing: id  8 | task 210 |        eval time =    5937.35 ms /   125 tokens (   47.50 ms per token,    21.05 tokens per second)
1.05.165.604 I slot print_timing: id  8 | task 210 |       total time =    6204.41 ms /   153 tokens
1.05.165.605 I slot print_timing: id  8 | task 210 |    graphs reused =          1
1.05.165.607 I slot print_timing: id  8 | task 210 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.05.165.619 I statistics        draft-mtp: #calls(b,g,a) =   68    206   3149, #gen drafts =   3156, #acc drafts =  2285, #gen tokens =   6296, #acc tokens =  4247, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.623), dur(b,g,a) = 0.039, 696.606, 4.039 ms
1.05.165.642 I slot      release: id  8 | task 210 | stop processing: n_tokens = 152, truncated = 0
1.05.167.712 I slot print_timing: id 12 | task 224 | prompt eval time =     117.70 ms /     4 tokens (   29.43 ms per token,    33.98 tokens per second)
1.05.167.716 I slot print_timing: id 12 | task 224 |        eval time =    4849.05 ms /   114 tokens (   42.54 ms per token,    23.51 tokens per second)
1.05.167.716 I slot print_timing: id 12 | task 224 |       total time =    4966.75 ms /   118 tokens
1.05.167.717 I slot print_timing: id 12 | task 224 |    graphs reused =          1
1.05.167.720 I slot print_timing: id 12 | task 224 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.05.167.727 I statistics        draft-mtp: #calls(b,g,a) =   68    206   3153, #gen drafts =   3156, #acc drafts =  2288, #gen tokens =   6296, #acc tokens =  4253, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.623), dur(b,g,a) = 0.039, 696.606, 4.044 ms
1.05.167.749 I slot      release: id 12 | task 224 | stop processing: n_tokens = 140, truncated = 0
1.05.176.837 I srv    operator(): Chat format: peg-native
1.05.177.462 I srv    operator(): Chat format: peg-native
1.05.278.598 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.05.278.601 I srv  get_availabl: updating prompt cache
1.05.278.642 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.05.278.643 I srv         alloc:  - prompt is already in the cache, skipping
1.05.278.645 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.05.278.647 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.278.648 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.278.649 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.278.649 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.278.649 I srv  get_availabl: prompt cache update took 0.05 ms
1.05.278.713 I slot launch_slot_: id  8 | task 277 | processing task, is_child = 0
1.05.278.714 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.05.278.730 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.05.278.731 I srv         alloc:  - prompt is already in the cache, skipping
1.05.278.732 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.05.278.733 I srv  get_availabl: updating prompt cache
1.05.278.745 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.05.278.746 I srv         alloc:  - prompt is already in the cache, skipping
1.05.278.746 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.05.278.747 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.278.747 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.278.748 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.278.748 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.278.748 I srv  get_availabl: prompt cache update took 0.02 ms
1.05.278.766 I slot launch_slot_: id 12 | task 278 | processing task, is_child = 0
1.05.283.976 I slot   operator(): id  8 | task 277 | Checking checkpoint with [23, 23] against 27...
1.05.287.807 W slot   operator(): id  8 | task 277 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.05.287.841 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.287.863 I slot   operator(): id 12 | task 278 | Checking checkpoint with [20, 20] against 24...
1.05.291.411 W slot   operator(): id 12 | task 278 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.05.291.443 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.523.335 I slot print_timing: id 14 | task 192 | prompt eval time =     128.16 ms /     4 tokens (   32.04 ms per token,    31.21 tokens per second)
1.05.523.342 I slot print_timing: id 14 | task 192 |        eval time =    8320.28 ms /   128 tokens (   65.00 ms per token,    15.38 tokens per second)
1.05.523.342 I slot print_timing: id 14 | task 192 |       total time =    8448.44 ms /   132 tokens
1.05.523.343 I slot print_timing: id 14 | task 192 |    graphs reused =          1
1.05.523.346 I slot print_timing: id 14 | task 192 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
1.05.523.360 I statistics        draft-mtp: #calls(b,g,a) =   70    209   3184, #gen drafts =   3199, #acc drafts =  2305, #gen tokens =   6381, #acc tokens =  4286, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.622), dur(b,g,a) = 0.039, 710.580, 4.083 ms
1.05.523.383 I slot      release: id 14 | task 192 | stop processing: n_tokens = 156, truncated = 0
1.05.532.232 I srv    operator(): Chat format: peg-native
1.05.646.745 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.05.646.748 I srv  get_availabl: updating prompt cache
1.05.646.789 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.05.646.791 I srv         alloc:  - prompt is already in the cache, skipping
1.05.646.792 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.05.646.794 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.646.796 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.646.796 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.646.796 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.646.797 I srv  get_availabl: prompt cache update took 0.05 ms
1.05.646.860 I slot launch_slot_: id 14 | task 282 | processing task, is_child = 0
1.05.650.518 I slot   operator(): id 14 | task 282 | Checking checkpoint with [24, 24] against 28...
1.05.654.356 W slot   operator(): id 14 | task 282 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.05.654.385 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.011.855 I slot print_timing: id  7 | task 239 | n_decoded =    102, tg =  24.09 t/s, tg_3s =  24.09 t/s
1.06.013.964 I slot print_timing: id 11 | task 233 | n_decoded =    101, tg =  20.84 t/s, tg_3s =  20.84 t/s
1.06.252.195 I slot print_timing: id  4 | task 223 | prompt eval time =     121.36 ms /     4 tokens (   30.34 ms per token,    32.96 tokens per second)
1.06.252.199 I slot print_timing: id  4 | task 223 |        eval time =    5933.76 ms /   125 tokens (   47.47 ms per token,    21.07 tokens per second)
1.06.252.199 I slot print_timing: id  4 | task 223 |       total time =    6055.13 ms /   129 tokens
1.06.252.200 I slot print_timing: id  4 | task 223 |    graphs reused =          1
1.06.252.205 I slot print_timing: id  4 | task 223 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.06.252.217 I statistics        draft-mtp: #calls(b,g,a) =   71    215   3282, #gen drafts =   3293, #acc drafts =  2377, #gen tokens =   6569, #acc tokens =  4421, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.623), dur(b,g,a) = 0.041, 729.118, 4.215 ms
1.06.252.241 I slot      release: id  4 | task 223 | stop processing: n_tokens = 152, truncated = 0
1.06.261.574 I srv    operator(): Chat format: peg-native
1.06.373.046 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.06.373.048 I srv  get_availabl: updating prompt cache
1.06.373.089 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.06.373.091 I srv         alloc:  - prompt is already in the cache, skipping
1.06.373.092 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.06.373.095 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.06.373.096 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.06.373.097 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.06.373.097 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.06.373.098 I srv  get_availabl: prompt cache update took 0.05 ms
1.06.373.164 I slot launch_slot_: id  4 | task 289 | processing task, is_child = 0
1.06.377.319 I slot   operator(): id  4 | task 289 | Checking checkpoint with [23, 23] against 27...
1.06.381.141 W slot   operator(): id  4 | task 289 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.06.381.176 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.618.230 I slot print_timing: id  7 | task 239 | prompt eval time =     116.23 ms /     4 tokens (   29.06 ms per token,    34.41 tokens per second)
1.06.618.233 I slot print_timing: id  7 | task 239 |        eval time =    4840.34 ms /   114 tokens (   42.46 ms per token,    23.55 tokens per second)
1.06.618.233 I slot print_timing: id  7 | task 239 |       total time =    4956.57 ms /   118 tokens
1.06.618.234 I slot print_timing: id  7 | task 239 |    graphs reused =          1
1.06.618.237 I slot print_timing: id  7 | task 239 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.06.618.249 I statistics        draft-mtp: #calls(b,g,a) =   72    218   3331, #gen drafts =   3339, #acc drafts =  2411, #gen tokens =   6661, #acc tokens =  4484, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.622), dur(b,g,a) = 0.041, 740.159, 4.275 ms
1.06.618.277 I slot      release: id  7 | task 239 | stop processing: n_tokens = 140, truncated = 0
1.06.627.116 I srv    operator(): Chat format: peg-native
1.06.731.755 I slot print_timing: id  2 | task 221 | n_decoded =    100, tg =  15.27 t/s, tg_3s =  15.27 t/s
1.06.737.760 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.06.737.762 I srv  get_availabl: updating prompt cache
1.06.737.803 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.06.737.804 I srv         alloc:  - prompt is already in the cache, skipping
1.06.737.805 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.06.737.808 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.06.737.809 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.06.737.810 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.06.737.810 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.06.737.810 I srv  get_availabl: prompt cache update took 0.05 ms
1.06.737.874 I slot launch_slot_: id  7 | task 293 | processing task, is_child = 0
1.06.742.396 I slot   operator(): id  7 | task 293 | Checking checkpoint with [20, 20] against 24...
1.06.746.265 W slot   operator(): id  7 | task 293 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.06.746.299 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.856.493 I slot print_timing: id 15 | task 203 | prompt eval time =     259.04 ms /    29 tokens (    8.93 ms per token,   111.95 tokens per second)
1.06.856.497 I slot print_timing: id 15 | task 203 |        eval time =    8381.55 ms /   128 tokens (   65.48 ms per token,    15.27 tokens per second)
1.06.856.497 I slot print_timing: id 15 | task 203 |       total time =    8640.59 ms /   157 tokens
1.06.856.498 I slot print_timing: id 15 | task 203 |    graphs reused =          1
1.06.856.500 I slot print_timing: id 15 | task 203 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.06.856.514 I statistics        draft-mtp: #calls(b,g,a) =   73    220   3354, #gen drafts =   3368, #acc drafts =  2428, #gen tokens =   6718, #acc tokens =  4513, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.622), dur(b,g,a) = 0.041, 749.436, 4.306 ms
1.06.856.542 I slot      release: id 15 | task 203 | stop processing: n_tokens = 156, truncated = 0
1.06.865.653 I srv    operator(): Chat format: peg-native
1.06.977.891 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.06.977.894 I srv  get_availabl: updating prompt cache
1.06.977.935 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.06.977.938 I srv         alloc:  - prompt is already in the cache, skipping
1.06.977.939 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.06.977.942 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.06.977.943 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.06.977.943 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.06.977.944 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.06.977.944 I srv  get_availabl: prompt cache update took 0.05 ms
1.06.978.011 I slot launch_slot_: id 15 | task 296 | processing task, is_child = 0
1.06.980.808 I slot   operator(): id 15 | task 296 | Checking checkpoint with [24, 24] against 28...
1.06.984.688 W slot   operator(): id 15 | task 296 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.06.984.716 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.102.092 I slot print_timing: id 11 | task 233 | prompt eval time =     116.49 ms /     4 tokens (   29.12 ms per token,    34.34 tokens per second)
1.07.102.096 I slot print_timing: id 11 | task 233 |        eval time =    5933.78 ms /   125 tokens (   47.47 ms per token,    21.07 tokens per second)
1.07.102.097 I slot print_timing: id 11 | task 233 |       total time =    6050.27 ms /   129 tokens
1.07.102.098 I slot print_timing: id 11 | task 233 |    graphs reused =          1
1.07.102.101 I slot print_timing: id 11 | task 233 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.07.102.113 I statistics        draft-mtp: #calls(b,g,a) =   74    222   3395, #gen drafts =   3398, #acc drafts =  2456, #gen tokens =   6778, #acc tokens =  4563, #mean acc len = 2.34, #acc rate/pos = (0.723, 0.621), dur(b,g,a) = 0.042, 754.925, 4.358 ms
1.07.102.146 I slot      release: id 11 | task 233 | stop processing: n_tokens = 152, truncated = 0
1.07.110.751 I srv    operator(): Chat format: peg-native
1.07.217.194 I slot print_timing: id 10 | task 252 | n_decoded =    102, tg =  24.12 t/s, tg_3s =  24.12 t/s
1.07.219.320 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.07.219.322 I srv  get_availabl: updating prompt cache
1.07.219.368 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.07.219.370 I srv         alloc:  - prompt is already in the cache, skipping
1.07.219.371 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.07.219.374 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.07.219.375 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.07.219.375 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.07.219.375 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.07.219.376 I srv  get_availabl: prompt cache update took 0.05 ms
1.07.219.443 I slot launch_slot_: id 11 | task 299 | processing task, is_child = 0
1.07.223.537 I slot   operator(): id 11 | task 299 | Checking checkpoint with [23, 23] against 27...
1.07.227.398 W slot   operator(): id 11 | task 299 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.07.227.435 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.584.814 I slot print_timing: id  6 | task 231 | n_decoded =    100, tg =  15.28 t/s, tg_3s =  15.28 t/s
1.07.704.500 I slot print_timing: id  3 | task 250 | n_decoded =    101, tg =  20.85 t/s, tg_3s =  20.85 t/s
1.07.828.556 I slot print_timing: id 10 | task 252 | prompt eval time =     116.07 ms /     4 tokens (   29.02 ms per token,    34.46 tokens per second)
1.07.828.559 I slot print_timing: id 10 | task 252 |        eval time =    4839.97 ms /   114 tokens (   42.46 ms per token,    23.55 tokens per second)
1.07.828.560 I slot print_timing: id 10 | task 252 |       total time =    4956.03 ms /   118 tokens
1.07.828.560 I slot print_timing: id 10 | task 252 |    graphs reused =          1
1.07.828.563 I slot print_timing: id 10 | task 252 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.07.828.575 I statistics        draft-mtp: #calls(b,g,a) =   75    228   3487, #gen drafts =   3492, #acc drafts =  2524, #gen tokens =   6966, #acc tokens =  4690, #mean acc len = 2.34, #acc rate/pos = (0.724, 0.621), dur(b,g,a) = 0.042, 774.232, 4.476 ms
1.07.828.602 I slot      release: id 10 | task 252 | stop processing: n_tokens = 140, truncated = 0
1.07.836.973 I srv    operator(): Chat format: peg-native
1.07.946.697 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.07.946.699 I srv  get_availabl: updating prompt cache
1.07.946.741 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.07.946.743 I srv         alloc:  - prompt is already in the cache, skipping
1.07.946.744 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.07.946.746 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.07.946.747 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.07.946.748 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.07.946.748 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.07.946.749 I srv  get_availabl: prompt cache update took 0.05 ms
1.07.946.814 I slot launch_slot_: id 10 | task 306 | processing task, is_child = 0
1.07.951.294 I slot   operator(): id 10 | task 306 | Checking checkpoint with [20, 20] against 24...
1.07.955.145 W slot   operator(): id 10 | task 306 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.07.955.183 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.431.828 I slot print_timing: id  5 | task 259 | n_decoded =    101, tg =  20.86 t/s, tg_3s =  20.86 t/s
1.08.436.193 I slot print_timing: id 13 | task 265 | n_decoded =    102, tg =  24.05 t/s, tg_3s =  24.05 t/s
1.08.549.328 I slot print_timing: id  2 | task 221 | prompt eval time =     103.60 ms /     4 tokens (   25.90 ms per token,    38.61 tokens per second)
1.08.549.338 I slot print_timing: id  2 | task 221 |        eval time =    8364.41 ms /   128 tokens (   65.35 ms per token,    15.30 tokens per second)
1.08.549.338 I slot print_timing: id  2 | task 221 |       total time =    8468.01 ms /   132 tokens
1.08.549.339 I slot print_timing: id  2 | task 221 |    graphs reused =          1
1.08.549.342 I slot print_timing: id  2 | task 221 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.08.549.357 I statistics        draft-mtp: #calls(b,g,a) =   76    234   3570, #gen drafts =   3585, #acc drafts =  2583, #gen tokens =   7151, #acc tokens =  4799, #mean acc len = 2.34, #acc rate/pos = (0.724, 0.621), dur(b,g,a) = 0.042, 794.652, 4.575 ms
1.08.549.383 I slot      release: id  2 | task 221 | stop processing: n_tokens = 156, truncated = 0
1.08.558.216 I srv    operator(): Chat format: peg-native
1.08.671.559 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.08.671.561 I srv  get_availabl: updating prompt cache
1.08.671.603 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.08.671.606 I srv         alloc:  - prompt is already in the cache, skipping
1.08.671.607 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.08.671.609 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.08.671.610 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.08.671.611 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.08.671.611 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.08.671.611 I srv  get_availabl: prompt cache update took 0.05 ms
1.08.671.675 I slot launch_slot_: id  2 | task 313 | processing task, is_child = 0
1.08.675.380 I slot   operator(): id  2 | task 313 | Checking checkpoint with [24, 24] against 28...
1.08.679.254 W slot   operator(): id  2 | task 313 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.08.679.286 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.793.176 I slot print_timing: id  3 | task 250 | prompt eval time =     110.33 ms /     4 tokens (   27.58 ms per token,    36.25 tokens per second)
1.08.793.179 I slot print_timing: id  3 | task 250 |        eval time =    5931.73 ms /   125 tokens (   47.45 ms per token,    21.07 tokens per second)
1.08.793.179 I slot print_timing: id  3 | task 250 |       total time =    6042.07 ms /   129 tokens
1.08.793.180 I slot print_timing: id  3 | task 250 |    graphs reused =          1
1.08.793.183 I slot print_timing: id  3 | task 250 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.08.793.194 I statistics        draft-mtp: #calls(b,g,a) =   77    236   3603, #gen drafts =   3615, #acc drafts =  2606, #gen tokens =   7211, #acc tokens =  4843, #mean acc len = 2.34, #acc rate/pos = (0.723, 0.621), dur(b,g,a) = 0.043, 802.004, 4.613 ms
1.08.793.219 I slot      release: id  3 | task 250 | stop processing: n_tokens = 152, truncated = 0
1.08.801.821 I srv    operator(): Chat format: peg-native
1.08.915.276 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.08.915.279 I srv  get_availabl: updating prompt cache
1.08.915.323 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.08.915.325 I srv         alloc:  - prompt is already in the cache, skipping
1.08.915.326 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.08.915.333 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.08.915.334 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.08.915.335 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.08.915.335 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.08.915.336 I srv  get_availabl: prompt cache update took 0.06 ms
1.08.915.407 I slot launch_slot_: id  3 | task 316 | processing task, is_child = 0
1.08.919.375 I slot   operator(): id  3 | task 316 | Checking checkpoint with [23, 23] against 27...
1.08.924.070 W slot   operator(): id  3 | task 316 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.08.924.112 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.046.031 I slot print_timing: id 13 | task 265 | prompt eval time =     117.19 ms /     4 tokens (   29.30 ms per token,    34.13 tokens per second)
1.09.046.034 I slot print_timing: id 13 | task 265 |        eval time =    4850.92 ms /   114 tokens (   42.55 ms per token,    23.50 tokens per second)
1.09.046.035 I slot print_timing: id 13 | task 265 |       total time =    4968.11 ms /   118 tokens
1.09.046.036 I slot print_timing: id 13 | task 265 |    graphs reused =          1
1.09.046.039 I slot print_timing: id 13 | task 265 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.09.046.052 I statistics        draft-mtp: #calls(b,g,a) =   78    238   3643, #gen drafts =   3645, #acc drafts =  2633, #gen tokens =   7271, #acc tokens =  4893, #mean acc len = 2.34, #acc rate/pos = (0.723, 0.620), dur(b,g,a) = 0.044, 809.729, 4.669 ms
1.09.046.081 I slot      release: id 13 | task 265 | stop processing: n_tokens = 140, truncated = 0
1.09.055.256 I srv    operator(): Chat format: peg-native
1.09.158.999 I slot print_timing: id  1 | task 247 | n_decoded =    100, tg =  15.30 t/s, tg_3s =  15.30 t/s
1.09.164.817 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.09.164.820 I srv  get_availabl: updating prompt cache
1.09.164.860 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.09.164.862 I srv         alloc:  - prompt is already in the cache, skipping
1.09.164.863 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.09.164.866 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.164.867 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.164.867 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.164.868 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.164.868 I srv  get_availabl: prompt cache update took 0.05 ms
1.09.164.936 I slot launch_slot_: id 13 | task 319 | processing task, is_child = 0
1.09.168.697 I slot   operator(): id 13 | task 319 | Checking checkpoint with [20, 20] against 24...
1.09.172.608 W slot   operator(): id 13 | task 319 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.09.172.650 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.406.387 I slot print_timing: id  6 | task 231 | prompt eval time =     109.80 ms /     4 tokens (   27.45 ms per token,    36.43 tokens per second)
1.09.406.392 I slot print_timing: id  6 | task 231 |        eval time =    8365.89 ms /   128 tokens (   65.36 ms per token,    15.30 tokens per second)
1.09.406.392 I slot print_timing: id  6 | task 231 |       total time =    8475.68 ms /   132 tokens
1.09.406.393 I slot print_timing: id  6 | task 231 |    graphs reused =          1
1.09.406.397 I slot print_timing: id  6 | task 231 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.09.406.411 I statistics        draft-mtp: #calls(b,g,a) =   79    241   3675, #gen drafts =   3690, #acc drafts =  2653, #gen tokens =   7360, #acc tokens =  4931, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.044, 821.648, 4.712 ms
1.09.406.433 I slot      release: id  6 | task 231 | stop processing: n_tokens = 156, truncated = 0
1.09.415.798 I srv    operator(): Chat format: peg-native
1.09.525.903 I slot print_timing: id  5 | task 259 | prompt eval time =     117.22 ms /     4 tokens (   29.31 ms per token,    34.12 tokens per second)
1.09.525.907 I slot print_timing: id  5 | task 259 |        eval time =    5935.46 ms /   125 tokens (   47.48 ms per token,    21.06 tokens per second)
1.09.525.907 I slot print_timing: id  5 | task 259 |       total time =    6052.68 ms /   129 tokens
1.09.525.908 I slot print_timing: id  5 | task 259 |    graphs reused =          1
1.09.525.911 I slot print_timing: id  5 | task 259 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.09.525.923 I statistics        draft-mtp: #calls(b,g,a) =   79    242   3696, #gen drafts =   3705, #acc drafts =  2668, #gen tokens =   7390, #acc tokens =  4959, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.044, 826.324, 4.744 ms
1.09.525.951 I slot      release: id  5 | task 259 | stop processing: n_tokens = 152, truncated = 0
1.09.531.066 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.09.531.067 I srv  get_availabl: updating prompt cache
1.09.531.110 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.09.531.112 I srv         alloc:  - prompt is already in the cache, skipping
1.09.531.113 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.09.531.116 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.531.117 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.531.117 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.531.117 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.531.118 I srv  get_availabl: prompt cache update took 0.05 ms
1.09.531.182 I slot launch_slot_: id  6 | task 323 | processing task, is_child = 0
1.09.531.183 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.09.531.199 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.09.531.200 I srv         alloc:  - prompt is already in the cache, skipping
1.09.534.648 I srv    operator(): Chat format: peg-native
1.09.535.308 I slot   operator(): id  6 | task 323 | Checking checkpoint with [24, 24] against 28...
1.09.539.176 W slot   operator(): id  6 | task 323 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.09.539.210 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.652.353 I slot print_timing: id 12 | task 278 | n_decoded =    102, tg =  24.01 t/s, tg_3s =  24.01 t/s
1.09.653.686 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.09.653.688 I srv  get_availabl: updating prompt cache
1.09.653.730 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.09.653.731 I srv         alloc:  - prompt is already in the cache, skipping
1.09.653.732 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.09.653.735 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.653.736 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.653.736 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.653.737 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.653.738 I srv  get_availabl: prompt cache update took 0.05 ms
1.09.653.802 I slot launch_slot_: id  5 | task 325 | processing task, is_child = 0
1.09.657.979 I slot   operator(): id  5 | task 325 | Checking checkpoint with [23, 23] against 27...
1.09.661.819 W slot   operator(): id  5 | task 325 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.09.661.859 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.137.226 I slot print_timing: id  0 | task 258 | n_decoded =    100, tg =  15.27 t/s, tg_3s =  15.27 t/s
1.10.263.000 I slot print_timing: id  8 | task 277 | n_decoded =    101, tg =  20.79 t/s, tg_3s =  20.79 t/s
1.10.264.752 I slot print_timing: id 12 | task 278 | prompt eval time =     116.53 ms /     4 tokens (   29.13 ms per token,    34.33 tokens per second)
1.10.264.753 I slot print_timing: id 12 | task 278 |        eval time =    4860.34 ms /   114 tokens (   42.63 ms per token,    23.46 tokens per second)
1.10.264.754 I slot print_timing: id 12 | task 278 |       total time =    4976.87 ms /   118 tokens
1.10.264.755 I slot print_timing: id 12 | task 278 |    graphs reused =          1
1.10.264.758 I slot print_timing: id 12 | task 278 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.10.264.772 I statistics        draft-mtp: #calls(b,g,a) =   81    248   3795, #gen drafts =   3798, #acc drafts =  2743, #gen tokens =   7576, #acc tokens =  5097, #mean acc len = 2.34, #acc rate/pos = (0.723, 0.620), dur(b,g,a) = 0.045, 845.697, 4.868 ms
1.10.264.800 I slot      release: id 12 | task 278 | stop processing: n_tokens = 140, truncated = 0
1.10.273.157 I srv    operator(): Chat format: peg-native
1.10.381.602 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.10.381.605 I srv  get_availabl: updating prompt cache
1.10.381.645 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.10.381.647 I srv         alloc:  - prompt is already in the cache, skipping
1.10.381.648 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.10.381.650 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.10.381.651 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.10.381.652 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.10.381.652 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.10.381.653 I srv  get_availabl: prompt cache update took 0.05 ms
1.10.381.716 I slot launch_slot_: id 12 | task 332 | processing task, is_child = 0
1.10.385.546 I slot   operator(): id 12 | task 332 | Checking checkpoint with [20, 20] against 24...
1.10.389.417 W slot   operator(): id 12 | task 332 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.10.389.453 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.747.872 I slot print_timing: id  9 | task 266 | n_decoded =    100, tg =  15.26 t/s, tg_3s =  15.26 t/s
1.10.983.932 I slot print_timing: id  1 | task 247 | prompt eval time =     116.30 ms /     4 tokens (   29.07 ms per token,    34.39 tokens per second)
1.10.983.935 I slot print_timing: id  1 | task 247 |        eval time =    8360.82 ms /   128 tokens (   65.32 ms per token,    15.31 tokens per second)
1.10.983.935 I slot print_timing: id  1 | task 247 |       total time =    8477.12 ms /   132 tokens
1.10.983.936 I slot print_timing: id  1 | task 247 |    graphs reused =          1
1.10.983.939 I slot print_timing: id  1 | task 247 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.10.983.953 I statistics        draft-mtp: #calls(b,g,a) =   82    254   3876, #gen drafts =   3891, #acc drafts =  2796, #gen tokens =   7761, #acc tokens =  5194, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.619), dur(b,g,a) = 0.046, 865.350, 4.974 ms
1.10.983.978 I slot      release: id  1 | task 247 | stop processing: n_tokens = 156, truncated = 0
1.10.993.092 I srv    operator(): Chat format: peg-native
1.11.102.810 I slot print_timing: id  7 | task 293 | n_decoded =    102, tg =  24.02 t/s, tg_3s =  24.02 t/s
1.11.107.101 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.11.107.103 I srv  get_availabl: updating prompt cache
1.11.107.147 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.11.107.149 I srv         alloc:  - prompt is already in the cache, skipping
1.11.107.150 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.11.107.153 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.107.153 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.107.154 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.107.155 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.107.155 I srv  get_availabl: prompt cache update took 0.05 ms
1.11.107.219 I slot launch_slot_: id  1 | task 339 | processing task, is_child = 0
1.11.110.893 I slot   operator(): id  1 | task 339 | Checking checkpoint with [24, 24] against 28...
1.11.114.944 W slot   operator(): id  1 | task 339 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.11.114.983 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.351.865 I slot print_timing: id  4 | task 289 | n_decoded =    101, tg =  20.79 t/s, tg_3s =  20.79 t/s
1.11.353.793 I slot print_timing: id  8 | task 277 | prompt eval time =     120.13 ms /     4 tokens (   30.03 ms per token,    33.30 tokens per second)
1.11.353.795 I slot print_timing: id  8 | task 277 |        eval time =    5949.65 ms /   125 tokens (   47.60 ms per token,    21.01 tokens per second)
1.11.353.796 I slot print_timing: id  8 | task 277 |       total time =    6069.78 ms /   129 tokens
1.11.353.797 I slot print_timing: id  8 | task 277 |    graphs reused =          1
1.11.353.799 I slot print_timing: id  8 | task 277 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.11.353.810 I statistics        draft-mtp: #calls(b,g,a) =   83    257   3930, #gen drafts =   3937, #acc drafts =  2839, #gen tokens =   7853, #acc tokens =  5275, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.047, 875.453, 5.046 ms
1.11.353.835 I slot      release: id  8 | task 277 | stop processing: n_tokens = 152, truncated = 0
1.11.362.523 I srv    operator(): Chat format: peg-native
1.11.474.305 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.11.474.307 I srv  get_availabl: updating prompt cache
1.11.474.355 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.11.474.357 I srv         alloc:  - prompt is already in the cache, skipping
1.11.474.358 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.11.474.360 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.474.361 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.474.362 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.474.363 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.474.363 I srv  get_availabl: prompt cache update took 0.06 ms
1.11.474.427 I slot launch_slot_: id  8 | task 343 | processing task, is_child = 0
1.11.479.207 I slot   operator(): id  8 | task 343 | Checking checkpoint with [23, 23] against 27...
1.11.483.308 W slot   operator(): id  8 | task 343 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.11.483.355 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.720.411 I slot print_timing: id  7 | task 293 | prompt eval time =     113.87 ms /     4 tokens (   28.47 ms per token,    35.13 tokens per second)
1.11.720.415 I slot print_timing: id  7 | task 293 |        eval time =    4864.13 ms /   114 tokens (   42.67 ms per token,    23.44 tokens per second)
1.11.720.416 I slot print_timing: id  7 | task 293 |       total time =    4978.00 ms /   118 tokens
1.11.720.417 I slot print_timing: id  7 | task 293 |    graphs reused =          1
1.11.720.419 I slot print_timing: id  7 | task 293 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.11.720.431 I statistics        draft-mtp: #calls(b,g,a) =   84    260   3975, #gen drafts =   3983, #acc drafts =  2868, #gen tokens =   7945, #acc tokens =  5331, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.048, 887.844, 5.102 ms
1.11.720.457 I slot      release: id  7 | task 293 | stop processing: n_tokens = 140, truncated = 0
1.11.728.977 I srv    operator(): Chat format: peg-native
1.11.841.076 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.11.841.078 I srv  get_availabl: updating prompt cache
1.11.841.118 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.11.841.119 I srv         alloc:  - prompt is already in the cache, skipping
1.11.841.120 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.11.841.124 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.841.125 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.841.126 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.841.126 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.841.127 I srv  get_availabl: prompt cache update took 0.05 ms
1.11.841.192 I slot launch_slot_: id  7 | task 347 | processing task, is_child = 0
1.11.845.782 I slot   operator(): id  7 | task 347 | Checking checkpoint with [20, 20] against 24...
1.11.849.582 W slot   operator(): id  7 | task 347 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.11.849.619 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.960.027 I slot print_timing: id  0 | task 258 | prompt eval time =     120.84 ms /     4 tokens (   30.21 ms per token,    33.10 tokens per second)
1.11.960.030 I slot print_timing: id  0 | task 258 |        eval time =    8369.88 ms /   128 tokens (   65.39 ms per token,    15.29 tokens per second)
1.11.960.030 I slot print_timing: id  0 | task 258 |       total time =    8490.72 ms /   132 tokens
1.11.960.031 I slot print_timing: id  0 | task 258 |    graphs reused =          1
1.11.960.034 I slot print_timing: id  0 | task 258 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.11.960.049 I statistics        draft-mtp: #calls(b,g,a) =   84    262   3998, #gen drafts =   4012, #acc drafts =  2887, #gen tokens =   8002, #acc tokens =  5367, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.048, 897.187, 5.133 ms
1.11.960.074 I slot      release: id  0 | task 258 | stop processing: n_tokens = 156, truncated = 0
1.11.969.013 I srv    operator(): Chat format: peg-native
1.12.081.639 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.12.081.642 I srv  get_availabl: updating prompt cache
1.12.081.683 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.081.685 I srv         alloc:  - prompt is already in the cache, skipping
1.12.081.686 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.12.081.689 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.081.690 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.081.690 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.081.691 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.081.691 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.081.757 I slot launch_slot_: id  0 | task 350 | processing task, is_child = 0
1.12.084.551 I slot   operator(): id  0 | task 350 | Checking checkpoint with [24, 24] against 28...
1.12.088.557 W slot   operator(): id  0 | task 350 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.12.088.589 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.206.614 I slot print_timing: id 11 | task 299 | n_decoded =    101, tg =  20.75 t/s, tg_3s =  20.75 t/s
1.12.208.458 I slot print_timing: id 14 | task 282 | n_decoded =    100, tg =  15.52 t/s, tg_3s =  15.52 t/s
1.12.326.660 I slot print_timing: id 10 | task 306 | n_decoded =    102, tg =  23.95 t/s, tg_3s =  23.95 t/s
1.12.444.639 I slot print_timing: id  4 | task 289 | prompt eval time =     116.44 ms /     4 tokens (   29.11 ms per token,    34.35 tokens per second)
1.12.444.642 I slot print_timing: id  4 | task 289 |        eval time =    5950.84 ms /   125 tokens (   47.61 ms per token,    21.01 tokens per second)
1.12.444.642 I slot print_timing: id  4 | task 289 |       total time =    6067.29 ms /   129 tokens
1.12.444.643 I slot print_timing: id  4 | task 289 |    graphs reused =          1
1.12.444.646 I slot print_timing: id  4 | task 289 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.12.444.659 I statistics        draft-mtp: #calls(b,g,a) =   86    266   4063, #gen drafts =   4074, #acc drafts =  2934, #gen tokens =   8125, #acc tokens =  5453, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.050, 908.285, 5.214 ms
1.12.444.685 I slot      release: id  4 | task 289 | stop processing: n_tokens = 152, truncated = 0
1.12.453.277 I srv    operator(): Chat format: peg-native
1.12.557.934 I slot print_timing: id  9 | task 266 | prompt eval time =     120.78 ms /     4 tokens (   30.20 ms per token,    33.12 tokens per second)
1.12.557.937 I slot print_timing: id  9 | task 266 |        eval time =    8363.09 ms /   128 tokens (   65.34 ms per token,    15.31 tokens per second)
1.12.557.937 I slot print_timing: id  9 | task 266 |       total time =    8483.88 ms /   132 tokens
1.12.557.938 I slot print_timing: id  9 | task 266 |    graphs reused =          1
1.12.557.941 I slot print_timing: id  9 | task 266 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.12.557.953 I statistics        draft-mtp: #calls(b,g,a) =   86    267   4074, #gen drafts =   4088, #acc drafts =  2940, #gen tokens =   8153, #acc tokens =  5464, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.050, 913.720, 5.226 ms
1.12.557.977 I slot      release: id  9 | task 266 | stop processing: n_tokens = 156, truncated = 0
1.12.565.009 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.12.565.011 I srv  get_availabl: updating prompt cache
1.12.565.050 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.12.565.051 I srv         alloc:  - prompt is already in the cache, skipping
1.12.565.052 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.12.565.055 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.565.056 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.565.056 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.565.057 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.565.057 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.565.121 I slot launch_slot_: id  4 | task 355 | processing task, is_child = 0
1.12.565.122 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
1.12.565.137 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.565.138 I srv         alloc:  - prompt is already in the cache, skipping
1.12.566.818 I srv    operator(): Chat format: peg-native
1.12.570.609 I slot   operator(): id  4 | task 355 | Checking checkpoint with [23, 23] against 27...
1.12.574.460 W slot   operator(): id  4 | task 355 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.12.574.499 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.688.120 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.12.688.122 I srv  get_availabl: updating prompt cache
1.12.688.164 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.688.165 I srv         alloc:  - prompt is already in the cache, skipping
1.12.688.166 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.12.688.169 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.688.169 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.688.170 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.688.170 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.688.171 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.688.236 I slot launch_slot_: id  9 | task 357 | processing task, is_child = 0
1.12.692.434 I slot   operator(): id  9 | task 357 | Checking checkpoint with [24, 24] against 3...
1.12.692.436 W slot   operator(): id  9 | task 357 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.12.692.438 W slot   operator(): id  9 | task 357 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.12.694.510 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.843.405 I slot create_check: id  9 | task 357 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.12.961.485 I slot print_timing: id 10 | task 306 | prompt eval time =     116.53 ms /     4 tokens (   29.13 ms per token,    34.32 tokens per second)
1.12.961.488 I slot print_timing: id 10 | task 306 |        eval time =    4893.64 ms /   114 tokens (   42.93 ms per token,    23.30 tokens per second)
1.12.961.489 I slot print_timing: id 10 | task 306 |       total time =    5010.17 ms /   118 tokens
1.12.961.490 I slot print_timing: id 10 | task 306 |    graphs reused =          1
1.12.961.493 I slot print_timing: id 10 | task 306 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.12.961.505 I statistics        draft-mtp: #calls(b,g,a) =   88    270   4127, #gen drafts =   4132, #acc drafts =  2977, #gen tokens =   8241, #acc tokens =  5532, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.619), dur(b,g,a) = 0.052, 927.706, 5.293 ms
1.12.961.533 I slot      release: id 10 | task 306 | stop processing: n_tokens = 140, truncated = 0
1.12.970.109 I srv    operator(): Chat format: peg-native
1.13.081.882 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.13.081.884 I srv  get_availabl: updating prompt cache
1.13.081.925 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.13.081.927 I srv         alloc:  - prompt is already in the cache, skipping
1.13.081.928 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.13.081.931 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.13.081.932 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.13.081.933 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.13.081.933 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.13.081.934 I srv  get_availabl: prompt cache update took 0.05 ms
1.13.081.999 I slot launch_slot_: id 10 | task 361 | processing task, is_child = 0
1.13.086.508 I slot   operator(): id 10 | task 361 | Checking checkpoint with [20, 20] against 3...
1.13.086.510 W slot   operator(): id 10 | task 361 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.13.086.513 W slot   operator(): id 10 | task 361 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.13.087.981 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.235.977 I slot create_check: id 10 | task 361 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.13.353.135 I slot print_timing: id 11 | task 299 | prompt eval time =     116.11 ms /     4 tokens (   29.03 ms per token,    34.45 tokens per second)
1.13.353.138 I slot print_timing: id 11 | task 299 |        eval time =    6013.42 ms /   125 tokens (   48.11 ms per token,    20.79 tokens per second)
1.13.353.138 I slot print_timing: id 11 | task 299 |       total time =    6129.53 ms /   129 tokens
1.13.353.139 I slot print_timing: id 11 | task 299 |    graphs reused =          1
1.13.353.142 I slot print_timing: id 11 | task 299 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.13.353.155 I statistics        draft-mtp: #calls(b,g,a) =   89    273   4173, #gen drafts =   4177, #acc drafts =  3008, #gen tokens =   8331, #acc tokens =  5588, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.053, 940.325, 5.351 ms
1.13.353.182 I slot      release: id 11 | task 299 | stop processing: n_tokens = 152, truncated = 0
1.13.361.303 I srv    operator(): Chat format: peg-native
1.13.471.587 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.13.471.589 I srv  get_availabl: updating prompt cache
1.13.471.632 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.13.471.634 I srv         alloc:  - prompt is already in the cache, skipping
1.13.471.635 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.13.471.638 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.13.471.639 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.13.471.640 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.13.471.640 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.13.471.641 I srv  get_availabl: prompt cache update took 0.05 ms
1.13.471.708 I slot launch_slot_: id 11 | task 365 | processing task, is_child = 0
1.13.475.802 I slot   operator(): id 11 | task 365 | Checking checkpoint with [23, 23] against 27...
1.13.479.734 W slot   operator(): id 11 | task 365 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.13.479.770 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.600.407 I slot print_timing: id 13 | task 319 | n_decoded =    102, tg =  23.64 t/s, tg_3s =  23.64 t/s
1.13.722.357 I slot print_timing: id 15 | task 296 | n_decoded =    100, tg =  15.09 t/s, tg_3s =  15.09 t/s
1.13.958.837 I slot print_timing: id  3 | task 316 | n_decoded =    101, tg =  20.53 t/s, tg_3s =  20.53 t/s
1.14.198.750 I slot print_timing: id 14 | task 282 | prompt eval time =     116.26 ms /     4 tokens (   29.07 ms per token,    34.41 tokens per second)
1.14.198.753 I slot print_timing: id 14 | task 282 |        eval time =    8431.94 ms /   128 tokens (   65.87 ms per token,    15.18 tokens per second)
1.14.198.754 I slot print_timing: id 14 | task 282 |       total time =    8548.20 ms /   132 tokens
1.14.198.755 I slot print_timing: id 14 | task 282 |    graphs reused =          1
1.14.198.758 I slot print_timing: id 14 | task 282 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.485, 0.368)
1.14.198.772 I statistics        draft-mtp: #calls(b,g,a) =   90    280   4271, #gen drafts =   4286, #acc drafts =  3082, #gen tokens =   8548, #acc tokens =  5726, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.619), dur(b,g,a) = 0.054, 963.340, 5.487 ms
1.14.198.798 I slot      release: id 14 | task 282 | stop processing: n_tokens = 156, truncated = 0
1.14.206.378 I slot print_timing: id 13 | task 319 | prompt eval time =     116.70 ms /     4 tokens (   29.17 ms per token,    34.28 tokens per second)
1.14.206.380 I slot print_timing: id 13 | task 319 |        eval time =    4920.97 ms /   114 tokens (   43.17 ms per token,    23.17 tokens per second)
1.14.206.380 I slot print_timing: id 13 | task 319 |       total time =    5037.66 ms /   118 tokens
1.14.206.381 I slot print_timing: id 13 | task 319 |    graphs reused =          1
1.14.206.383 I slot print_timing: id 13 | task 319 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.14.206.391 I statistics        draft-mtp: #calls(b,g,a) =   90    280   4285, #gen drafts =   4286, #acc drafts =  3093, #gen tokens =   8548, #acc tokens =  5748, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.620), dur(b,g,a) = 0.054, 963.340, 5.506 ms
1.14.206.415 I slot      release: id 13 | task 319 | stop processing: n_tokens = 140, truncated = 0
1.14.207.604 I srv    operator(): Chat format: peg-native
1.14.214.964 I srv    operator(): Chat format: peg-native
1.14.314.473 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.14.314.476 I srv  get_availabl: updating prompt cache
1.14.314.518 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.14.314.520 I srv         alloc:  - prompt is already in the cache, skipping
1.14.314.520 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.14.314.524 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.14.314.524 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.14.314.525 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.14.314.527 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.14.314.527 I srv  get_availabl: prompt cache update took 0.05 ms
1.14.314.594 I slot launch_slot_: id 13 | task 373 | processing task, is_child = 0
1.14.314.595 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
1.14.314.610 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.14.314.611 I srv         alloc:  - prompt is already in the cache, skipping
1.14.314.613 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.14.314.613 I srv  get_availabl: updating prompt cache
1.14.314.626 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.14.314.626 I srv         alloc:  - prompt is already in the cache, skipping
1.14.314.626 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.14.314.627 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.14.314.627 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.14.314.628 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.14.314.628 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.14.314.628 I srv  get_availabl: prompt cache update took 0.01 ms
1.14.314.645 I slot launch_slot_: id 14 | task 374 | processing task, is_child = 0
1.14.318.255 I slot   operator(): id 13 | task 373 | Checking checkpoint with [20, 20] against 24...
1.14.322.100 W slot   operator(): id 13 | task 373 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.14.322.131 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.322.154 I slot   operator(): id 14 | task 374 | Checking checkpoint with [24, 24] against 28...
1.14.325.675 W slot   operator(): id 14 | task 374 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.14.325.710 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.684.368 I slot print_timing: id  5 | task 325 | n_decoded =    101, tg =  20.57 t/s, tg_3s =  20.57 t/s
1.14.809.325 I slot print_timing: id 12 | task 332 | n_decoded =    102, tg =  23.68 t/s, tg_3s =  23.68 t/s
1.15.046.262 I slot print_timing: id  3 | task 316 | prompt eval time =     120.15 ms /     4 tokens (   30.04 ms per token,    33.29 tokens per second)
1.15.046.265 I slot print_timing: id  3 | task 316 |        eval time =    6006.70 ms /   125 tokens (   48.05 ms per token,    20.81 tokens per second)
1.15.046.266 I slot print_timing: id  3 | task 316 |       total time =    6126.85 ms /   129 tokens
1.15.046.267 I slot print_timing: id  3 | task 316 |    graphs reused =          1
1.15.046.269 I slot print_timing: id  3 | task 316 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.15.046.282 I statistics        draft-mtp: #calls(b,g,a) =   92    287   4382, #gen drafts =   4394, #acc drafts =  3162, #gen tokens =   8764, #acc tokens =  5875, #mean acc len = 2.34, #acc rate/pos = (0.722, 0.619), dur(b,g,a) = 0.055, 984.691, 5.637 ms
1.15.046.309 I slot      release: id  3 | task 316 | stop processing: n_tokens = 152, truncated = 0
1.15.054.867 I srv    operator(): Chat format: peg-native
1.15.167.869 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.15.167.871 I srv  get_availabl: updating prompt cache
1.15.167.914 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.15.167.917 I srv         alloc:  - prompt is already in the cache, skipping
1.15.167.918 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.15.167.921 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.167.922 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.167.923 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.167.924 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.167.924 I srv  get_availabl: prompt cache update took 0.05 ms
1.15.167.992 I slot launch_slot_: id  3 | task 382 | processing task, is_child = 0
1.15.171.843 I slot   operator(): id  3 | task 382 | Checking checkpoint with [23, 23] against 27...
1.15.175.710 W slot   operator(): id  3 | task 382 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.15.175.738 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.409.612 I slot print_timing: id  2 | task 313 | n_decoded =    100, tg =  15.11 t/s, tg_3s =  15.11 t/s
1.15.414.503 I slot print_timing: id 12 | task 332 | prompt eval time =     116.67 ms /     4 tokens (   29.17 ms per token,    34.28 tokens per second)
1.15.414.506 I slot print_timing: id 12 | task 332 |        eval time =    4912.27 ms /   114 tokens (   43.09 ms per token,    23.21 tokens per second)
1.15.414.506 I slot print_timing: id 12 | task 332 |       total time =    5028.94 ms /   118 tokens
1.15.414.507 I slot print_timing: id 12 | task 332 |    graphs reused =          1
1.15.414.510 I slot print_timing: id 12 | task 332 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.15.414.522 I statistics        draft-mtp: #calls(b,g,a) =   93    290   4437, #gen drafts =   4440, #acc drafts =  3200, #gen tokens =   8855, #acc tokens =  5945, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.619), dur(b,g,a) = 0.056, 995.117, 5.707 ms
1.15.414.549 I slot      release: id 12 | task 332 | stop processing: n_tokens = 140, truncated = 0
1.15.423.257 I srv    operator(): Chat format: peg-native
1.15.521.646 I slot print_timing: id 15 | task 296 | prompt eval time =     115.96 ms /     4 tokens (   28.99 ms per token,    34.50 tokens per second)
1.15.521.649 I slot print_timing: id 15 | task 296 |        eval time =    8424.85 ms /   128 tokens (   65.82 ms per token,    15.19 tokens per second)
1.15.521.649 I slot print_timing: id 15 | task 296 |       total time =    8540.81 ms /   132 tokens
1.15.521.650 I slot print_timing: id 15 | task 296 |    graphs reused =          1
1.15.521.653 I slot print_timing: id 15 | task 296 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.15.521.666 I statistics        draft-mtp: #calls(b,g,a) =   93    291   4440, #gen drafts =   4454, #acc drafts =  3201, #gen tokens =   8883, #acc tokens =  5947, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.056, 998.768, 5.710 ms
1.15.521.690 I slot      release: id 15 | task 296 | stop processing: n_tokens = 156, truncated = 0
1.15.528.750 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.15.528.752 I srv  get_availabl: updating prompt cache
1.15.528.794 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.15.528.796 I srv         alloc:  - prompt is already in the cache, skipping
1.15.528.796 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.15.528.799 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.528.800 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.528.801 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.528.801 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.528.802 I srv  get_availabl: prompt cache update took 0.05 ms
1.15.528.865 I slot launch_slot_: id 12 | task 386 | processing task, is_child = 0
1.15.528.866 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
1.15.528.882 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.15.528.883 I srv         alloc:  - prompt is already in the cache, skipping
1.15.529.775 I srv    operator(): Chat format: peg-native
1.15.532.545 I slot   operator(): id 12 | task 386 | Checking checkpoint with [20, 20] against 24...
1.15.536.345 W slot   operator(): id 12 | task 386 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.15.536.383 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.649.231 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.15.649.233 I srv  get_availabl: updating prompt cache
1.15.649.276 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.15.649.278 I srv         alloc:  - prompt is already in the cache, skipping
1.15.649.279 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.15.649.282 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.649.283 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.649.284 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.649.285 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.649.285 I srv  get_availabl: prompt cache update took 0.05 ms
1.15.649.367 I slot launch_slot_: id 15 | task 388 | processing task, is_child = 0
1.15.652.161 I slot   operator(): id 15 | task 388 | Checking checkpoint with [24, 24] against 28...
1.15.656.002 W slot   operator(): id 15 | task 388 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.15.656.032 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.771.358 I slot print_timing: id  5 | task 325 | prompt eval time =     116.23 ms /     4 tokens (   29.06 ms per token,    34.42 tokens per second)
1.15.771.361 I slot print_timing: id  5 | task 325 |        eval time =    5997.11 ms /   125 tokens (   47.98 ms per token,    20.84 tokens per second)
1.15.771.361 I slot print_timing: id  5 | task 325 |       total time =    6113.34 ms /   129 tokens
1.15.771.362 I slot print_timing: id  5 | task 325 |    graphs reused =          1
1.15.771.365 I slot print_timing: id  5 | task 325 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.15.771.376 I statistics        draft-mtp: #calls(b,g,a) =   95    293   4474, #gen drafts =   4483, #acc drafts =  3225, #gen tokens =   8941, #acc tokens =  5991, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.057, 1005.145, 5.751 ms
1.15.771.402 I slot      release: id  5 | task 325 | stop processing: n_tokens = 152, truncated = 0
1.15.779.844 I srv    operator(): Chat format: peg-native
1.15.892.553 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.15.892.555 I srv  get_availabl: updating prompt cache
1.15.892.607 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.15.892.610 I srv         alloc:  - prompt is already in the cache, skipping
1.15.892.611 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.15.892.615 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.892.616 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.892.617 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.892.617 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.892.618 I srv  get_availabl: prompt cache update took 0.06 ms
1.15.892.685 I slot launch_slot_: id  5 | task 391 | processing task, is_child = 0
1.15.896.998 I slot   operator(): id  5 | task 391 | Checking checkpoint with [23, 23] against 27...
1.15.900.809 W slot   operator(): id  5 | task 391 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.15.900.859 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.259.334 I slot print_timing: id  6 | task 323 | n_decoded =    100, tg =  15.12 t/s, tg_3s =  15.12 t/s
1.16.259.934 I slot print_timing: id  7 | task 347 | n_decoded =    102, tg =  23.72 t/s, tg_3s =  23.72 t/s
1.16.502.466 I slot print_timing: id  8 | task 343 | n_decoded =    101, tg =  20.59 t/s, tg_3s =  20.59 t/s
1.16.865.866 I slot print_timing: id  7 | task 347 | prompt eval time =     114.55 ms /     4 tokens (   28.64 ms per token,    34.92 tokens per second)
1.16.865.869 I slot print_timing: id  7 | task 347 |        eval time =    4905.52 ms /   114 tokens (   43.03 ms per token,    23.24 tokens per second)
1.16.865.870 I slot print_timing: id  7 | task 347 |       total time =    5020.06 ms /   118 tokens
1.16.865.871 I slot print_timing: id  7 | task 347 |    graphs reused =          1
1.16.865.874 I slot print_timing: id  7 | task 347 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.16.865.886 I statistics        draft-mtp: #calls(b,g,a) =   96    302   4617, #gen drafts =   4625, #acc drafts =  3328, #gen tokens =   9225, #acc tokens =  6180, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.058, 1033.441, 5.927 ms
1.16.865.913 I slot      release: id  7 | task 347 | stop processing: n_tokens = 140, truncated = 0
1.16.874.870 I srv    operator(): Chat format: peg-native
1.16.986.575 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.16.986.577 I srv  get_availabl: updating prompt cache
1.16.986.627 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.16.986.629 I srv         alloc:  - prompt is already in the cache, skipping
1.16.986.630 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.16.986.634 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.16.986.635 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.16.986.636 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.16.986.636 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.16.986.637 I srv  get_availabl: prompt cache update took 0.06 ms
1.16.986.706 I slot launch_slot_: id  7 | task 401 | processing task, is_child = 0
1.16.991.570 I slot   operator(): id  7 | task 401 | Checking checkpoint with [20, 20] against 24...
1.16.995.424 W slot   operator(): id  7 | task 401 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.16.995.467 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.227.713 I slot print_timing: id  2 | task 313 | prompt eval time =     116.34 ms /     4 tokens (   29.08 ms per token,    34.38 tokens per second)
1.17.227.719 I slot print_timing: id  2 | task 313 |        eval time =    8435.96 ms /   128 tokens (   65.91 ms per token,    15.17 tokens per second)
1.17.227.719 I slot print_timing: id  2 | task 313 |       total time =    8552.30 ms /   132 tokens
1.17.227.720 I slot print_timing: id  2 | task 313 |    graphs reused =          1
1.17.227.724 I slot print_timing: id  2 | task 313 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.17.227.741 I statistics        draft-mtp: #calls(b,g,a) =   97    305   4655, #gen drafts =   4670, #acc drafts =  3354, #gen tokens =   9314, #acc tokens =  6228, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.617), dur(b,g,a) = 0.059, 1046.882, 5.974 ms
1.17.227.765 I slot      release: id  2 | task 313 | stop processing: n_tokens = 156, truncated = 0
1.17.233.115 I slot print_timing: id  9 | task 357 | n_decoded =    102, tg =  23.85 t/s, tg_3s =  23.85 t/s
1.17.236.897 I srv    operator(): Chat format: peg-native
1.17.350.880 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.17.350.883 I srv  get_availabl: updating prompt cache
1.17.350.936 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.17.350.938 I srv         alloc:  - prompt is already in the cache, skipping
1.17.350.939 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.17.350.941 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.350.942 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.350.942 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.350.943 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.350.943 I srv  get_availabl: prompt cache update took 0.06 ms
1.17.351.009 I slot launch_slot_: id  2 | task 405 | processing task, is_child = 0
1.17.354.735 I slot   operator(): id  2 | task 405 | Checking checkpoint with [24, 24] against 28...
1.17.358.585 W slot   operator(): id  2 | task 405 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.17.358.624 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.594.766 I slot print_timing: id  4 | task 355 | n_decoded =    101, tg =  20.55 t/s, tg_3s =  20.55 t/s
1.17.597.049 I slot print_timing: id  8 | task 343 | prompt eval time =     117.00 ms /     4 tokens (   29.25 ms per token,    34.19 tokens per second)
1.17.597.052 I slot print_timing: id  8 | task 343 |        eval time =    6000.81 ms /   125 tokens (   48.01 ms per token,    20.83 tokens per second)
1.17.597.052 I slot print_timing: id  8 | task 343 |       total time =    6117.81 ms /   129 tokens
1.17.597.053 I slot print_timing: id  8 | task 343 |    graphs reused =          1
1.17.597.056 I slot print_timing: id  8 | task 343 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.17.597.068 I statistics        draft-mtp: #calls(b,g,a) =   98    308   4709, #gen drafts =   4716, #acc drafts =  3396, #gen tokens =   9406, #acc tokens =  6308, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.059, 1057.150, 6.049 ms
1.17.597.101 I slot      release: id  8 | task 343 | stop processing: n_tokens = 152, truncated = 0
1.17.605.576 I srv    operator(): Chat format: peg-native
1.17.716.758 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.17.716.760 I srv  get_availabl: updating prompt cache
1.17.716.814 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.17.716.816 I srv         alloc:  - prompt is already in the cache, skipping
1.17.716.818 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.17.716.821 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.716.822 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.716.823 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.716.824 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.716.825 I srv  get_availabl: prompt cache update took 0.06 ms
1.17.716.890 I slot launch_slot_: id  8 | task 409 | processing task, is_child = 0
1.17.721.752 I slot   operator(): id  8 | task 409 | Checking checkpoint with [23, 23] against 27...
1.17.725.660 W slot   operator(): id  8 | task 409 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.17.725.703 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.839.827 I slot print_timing: id  1 | task 339 | n_decoded =    100, tg =  15.12 t/s, tg_3s =  15.12 t/s
1.17.843.054 I slot print_timing: id  9 | task 357 | prompt eval time =     263.81 ms /    25 tokens (   10.55 ms per token,    94.76 tokens per second)
1.17.843.057 I slot print_timing: id  9 | task 357 |        eval time =    4886.78 ms /   114 tokens (   42.87 ms per token,    23.33 tokens per second)
1.17.843.057 I slot print_timing: id  9 | task 357 |       total time =    5150.60 ms /   139 tokens
1.17.843.058 I slot print_timing: id  9 | task 357 |    graphs reused =          1
1.17.843.062 I slot print_timing: id  9 | task 357 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.17.843.075 I statistics        draft-mtp: #calls(b,g,a) =   99    310   4740, #gen drafts =   4746, #acc drafts =  3416, #gen tokens =   9466, #acc tokens =  6346, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.060, 1066.838, 6.090 ms
1.17.843.102 I slot      release: id  9 | task 357 | stop processing: n_tokens = 140, truncated = 0
1.17.851.398 I srv    operator(): Chat format: peg-native
1.17.960.688 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.17.960.691 I srv  get_availabl: updating prompt cache
1.17.960.741 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.17.960.743 I srv         alloc:  - prompt is already in the cache, skipping
1.17.960.744 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.17.960.748 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.960.749 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.960.750 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.960.751 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.960.751 I srv  get_availabl: prompt cache update took 0.06 ms
1.17.960.818 I slot launch_slot_: id  9 | task 412 | processing task, is_child = 0
1.17.966.128 I slot   operator(): id  9 | task 412 | Checking checkpoint with [20, 20] against 24...
1.17.970.038 W slot   operator(): id  9 | task 412 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.17.970.082 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.080.911 I slot print_timing: id  6 | task 323 | prompt eval time =     111.50 ms /     4 tokens (   27.87 ms per token,    35.88 tokens per second)
1.18.080.914 I slot print_timing: id  6 | task 323 |        eval time =    8434.08 ms /   128 tokens (   65.89 ms per token,    15.18 tokens per second)
1.18.080.915 I slot print_timing: id  6 | task 323 |       total time =    8545.58 ms /   132 tokens
1.18.080.916 I slot print_timing: id  6 | task 323 |    graphs reused =          1
1.18.080.919 I slot print_timing: id  6 | task 323 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.18.080.934 I statistics        draft-mtp: #calls(b,g,a) =   99    312   4761, #gen drafts =   4775, #acc drafts =  3428, #gen tokens =   9523, #acc tokens =  6368, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.618), dur(b,g,a) = 0.060, 1076.299, 6.115 ms
1.18.080.958 I slot      release: id  6 | task 323 | stop processing: n_tokens = 156, truncated = 0
1.18.089.837 I srv    operator(): Chat format: peg-native
1.18.204.827 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.18.204.829 I srv  get_availabl: updating prompt cache
1.18.204.882 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.18.204.884 I srv         alloc:  - prompt is already in the cache, skipping
1.18.204.886 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.18.204.889 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.18.204.890 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.18.204.891 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.18.204.891 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.18.204.892 I srv  get_availabl: prompt cache update took 0.06 ms
1.18.204.958 I slot launch_slot_: id  6 | task 415 | processing task, is_child = 0
1.18.209.638 I slot   operator(): id  6 | task 415 | Checking checkpoint with [24, 24] against 28...
1.18.213.505 W slot   operator(): id  6 | task 415 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.18.213.547 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.453.479 I slot print_timing: id 11 | task 365 | n_decoded =    101, tg =  20.78 t/s, tg_3s =  20.78 t/s
1.18.693.481 I slot print_timing: id  4 | task 355 | prompt eval time =     110.29 ms /     4 tokens (   27.57 ms per token,    36.27 tokens per second)
1.18.693.486 I slot print_timing: id  4 | task 355 |        eval time =    6012.55 ms /   125 tokens (   48.10 ms per token,    20.79 tokens per second)
1.18.693.487 I slot print_timing: id  4 | task 355 |       total time =    6122.84 ms /   129 tokens
1.18.693.488 I slot print_timing: id  4 | task 355 |    graphs reused =          1
1.18.693.491 I slot print_timing: id  4 | task 355 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.18.693.504 I statistics        draft-mtp: #calls(b,g,a) =  101    317   4842, #gen drafts =   4853, #acc drafts =  3490, #gen tokens =   9679, #acc tokens =  6483, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.062, 1093.750, 6.229 ms
1.18.693.531 I slot      release: id  4 | task 355 | stop processing: n_tokens = 152, truncated = 0
1.18.698.030 I slot print_timing: id 13 | task 373 | n_decoded =    102, tg =  23.95 t/s, tg_3s =  23.95 t/s
1.18.702.428 I srv    operator(): Chat format: peg-native
1.18.807.201 I slot print_timing: id  0 | task 350 | n_decoded =    100, tg =  15.14 t/s, tg_3s =  15.14 t/s
1.18.814.273 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.18.814.274 I srv  get_availabl: updating prompt cache
1.18.814.326 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.18.814.333 I srv         alloc:  - prompt is already in the cache, skipping
1.18.814.334 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.18.814.337 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.18.814.338 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.18.814.339 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.18.814.340 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.18.814.340 I srv  get_availabl: prompt cache update took 0.07 ms
1.18.814.408 I slot launch_slot_: id  4 | task 421 | processing task, is_child = 0
1.18.818.541 I slot   operator(): id  4 | task 421 | Checking checkpoint with [23, 23] against 27...
1.18.822.386 W slot   operator(): id  4 | task 421 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.18.822.437 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.305.823 I slot print_timing: id 13 | task 373 | prompt eval time =     121.12 ms /     4 tokens (   30.28 ms per token,    33.03 tokens per second)
1.19.305.826 I slot print_timing: id 13 | task 373 |        eval time =    4866.43 ms /   114 tokens (   42.69 ms per token,    23.43 tokens per second)
1.19.305.827 I slot print_timing: id 13 | task 373 |       total time =    4987.55 ms /   118 tokens
1.19.305.828 I slot print_timing: id 13 | task 373 |    graphs reused =          1
1.19.305.831 I slot print_timing: id 13 | task 373 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.19.305.844 I statistics        draft-mtp: #calls(b,g,a) =  102    322   4929, #gen drafts =   4931, #acc drafts =  3552, #gen tokens =   9835, #acc tokens =  6598, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.618), dur(b,g,a) = 0.063, 1110.506, 6.331 ms
1.19.305.870 I slot      release: id 13 | task 373 | stop processing: n_tokens = 140, truncated = 0
1.19.314.314 I srv    operator(): Chat format: peg-native
1.19.421.072 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.19.421.075 I srv  get_availabl: updating prompt cache
1.19.421.125 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.19.421.127 I srv         alloc:  - prompt is already in the cache, skipping
1.19.421.129 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.19.421.132 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.421.134 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.421.134 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.421.135 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.421.136 I srv  get_availabl: prompt cache update took 0.06 ms
1.19.421.203 I slot launch_slot_: id 13 | task 427 | processing task, is_child = 0
1.19.424.981 I slot   operator(): id 13 | task 427 | Checking checkpoint with [20, 20] against 24...
1.19.428.858 W slot   operator(): id 13 | task 427 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.19.428.898 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.547.181 I slot print_timing: id 11 | task 365 | prompt eval time =     116.96 ms /     4 tokens (   29.24 ms per token,    34.20 tokens per second)
1.19.547.184 I slot print_timing: id 11 | task 365 |        eval time =    5954.38 ms /   125 tokens (   47.64 ms per token,    20.99 tokens per second)
1.19.547.185 I slot print_timing: id 11 | task 365 |       total time =    6071.34 ms /   129 tokens
1.19.547.186 I slot print_timing: id 11 | task 365 |    graphs reused =          1
1.19.547.189 I slot print_timing: id 11 | task 365 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.19.547.201 I statistics        draft-mtp: #calls(b,g,a) =  103    324   4958, #gen drafts =   4961, #acc drafts =  3568, #gen tokens =   9894, #acc tokens =  6628, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.064, 1117.998, 6.368 ms
1.19.547.231 I slot      release: id 11 | task 365 | stop processing: n_tokens = 152, truncated = 0
1.19.556.031 I srv    operator(): Chat format: peg-native
1.19.656.786 I slot print_timing: id  1 | task 339 | prompt eval time =     116.96 ms /     4 tokens (   29.24 ms per token,    34.20 tokens per second)
1.19.656.788 I slot print_timing: id  1 | task 339 |        eval time =    8428.90 ms /   128 tokens (   65.85 ms per token,    15.19 tokens per second)
1.19.656.788 I slot print_timing: id  1 | task 339 |       total time =    8545.86 ms /   132 tokens
1.19.656.789 I slot print_timing: id  1 | task 339 |    graphs reused =          1
1.19.656.793 I slot print_timing: id  1 | task 339 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.19.656.805 I statistics        draft-mtp: #calls(b,g,a) =  103    325   4961, #gen drafts =   4975, #acc drafts =  3570, #gen tokens =   9922, #acc tokens =  6632, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.064, 1122.947, 6.371 ms
1.19.656.835 I slot      release: id  1 | task 339 | stop processing: n_tokens = 156, truncated = 0
1.19.664.278 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.19.664.280 I srv  get_availabl: updating prompt cache
1.19.664.322 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.19.664.324 I srv         alloc:  - prompt is already in the cache, skipping
1.19.664.325 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.19.664.327 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.664.334 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.664.335 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.664.336 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.664.336 I srv  get_availabl: prompt cache update took 0.06 ms
1.19.664.402 I slot launch_slot_: id  1 | task 430 | processing task, is_child = 0
1.19.664.403 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
1.19.664.421 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.19.664.422 I srv         alloc:  - prompt is already in the cache, skipping
1.19.665.403 I srv    operator(): Chat format: peg-native
1.19.669.394 I slot   operator(): id  1 | task 430 | Checking checkpoint with [24, 24] against 28...
1.19.673.237 W slot   operator(): id  1 | task 430 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.19.673.275 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.786.432 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.19.786.434 I srv  get_availabl: updating prompt cache
1.19.786.475 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.19.786.478 I srv         alloc:  - prompt is already in the cache, skipping
1.19.786.479 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.19.786.481 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.786.482 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.786.483 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.786.483 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.786.484 I srv  get_availabl: prompt cache update took 0.05 ms
1.19.786.549 I slot launch_slot_: id 11 | task 432 | processing task, is_child = 0
1.19.790.646 I slot   operator(): id 11 | task 432 | Checking checkpoint with [23, 23] against 27...
1.19.794.652 W slot   operator(): id 11 | task 432 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.19.794.693 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.913.617 I slot print_timing: id 10 | task 361 | n_decoded =    100, tg =  15.23 t/s, tg_3s =  15.23 t/s
1.19.914.270 I slot print_timing: id 12 | task 386 | n_decoded =    102, tg =  23.88 t/s, tg_3s =  23.88 t/s
1.20.152.187 I slot print_timing: id  3 | task 382 | n_decoded =    101, tg =  20.77 t/s, tg_3s =  20.77 t/s
1.20.520.003 I slot print_timing: id 12 | task 386 | prompt eval time =     109.83 ms /     4 tokens (   27.46 ms per token,    36.42 tokens per second)
1.20.520.007 I slot print_timing: id 12 | task 386 |        eval time =    4877.61 ms /   114 tokens (   42.79 ms per token,    23.37 tokens per second)
1.20.520.007 I slot print_timing: id 12 | task 386 |       total time =    4987.44 ms /   118 tokens
1.20.520.008 I slot print_timing: id 12 | task 386 |    graphs reused =          1
1.20.520.011 I slot print_timing: id 12 | task 386 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.20.520.023 I statistics        draft-mtp: #calls(b,g,a) =  105    332   5081, #gen drafts =   5084, #acc drafts =  3658, #gen tokens =  10139, #acc tokens =  6797, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.618), dur(b,g,a) = 0.066, 1146.082, 6.528 ms
1.20.520.053 I slot      release: id 12 | task 386 | stop processing: n_tokens = 140, truncated = 0
1.20.529.038 I srv    operator(): Chat format: peg-native
1.20.628.565 I slot print_timing: id  0 | task 350 | prompt eval time =     116.39 ms /     4 tokens (   29.10 ms per token,    34.37 tokens per second)
1.20.628.568 I slot print_timing: id  0 | task 350 |        eval time =    8427.60 ms /   128 tokens (   65.84 ms per token,    15.19 tokens per second)
1.20.628.569 I slot print_timing: id  0 | task 350 |       total time =    8543.99 ms /   132 tokens
1.20.628.569 I slot print_timing: id  0 | task 350 |    graphs reused =          1
1.20.628.572 I slot print_timing: id  0 | task 350 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.20.628.585 I statistics        draft-mtp: #calls(b,g,a) =  105    333   5084, #gen drafts =   5098, #acc drafts =  3661, #gen tokens =  10167, #acc tokens =  6802, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.618), dur(b,g,a) = 0.066, 1149.861, 6.532 ms
1.20.628.609 I slot      release: id  0 | task 350 | stop processing: n_tokens = 156, truncated = 0
1.20.635.498 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.20.635.500 I srv  get_availabl: updating prompt cache
1.20.635.541 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.20.635.543 I srv         alloc:  - prompt is already in the cache, skipping
1.20.635.544 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.20.635.547 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.20.635.548 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.20.635.549 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.20.635.549 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.20.635.551 I srv  get_availabl: prompt cache update took 0.05 ms
1.20.635.617 I slot launch_slot_: id 12 | task 440 | processing task, is_child = 0
1.20.635.618 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.20.635.635 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.20.635.637 I srv         alloc:  - prompt is already in the cache, skipping
1.20.637.258 I srv    operator(): Chat format: peg-native
1.20.639.396 I slot   operator(): id 12 | task 440 | Checking checkpoint with [20, 20] against 24...
1.20.643.268 W slot   operator(): id 12 | task 440 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.20.643.295 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.756.174 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.20.756.177 I srv  get_availabl: updating prompt cache
1.20.756.218 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.20.756.220 I srv         alloc:  - prompt is already in the cache, skipping
1.20.756.221 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.20.756.224 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.20.756.225 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.20.756.226 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.20.756.227 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.20.756.228 I srv  get_availabl: prompt cache update took 0.05 ms
1.20.756.291 I slot launch_slot_: id  0 | task 442 | processing task, is_child = 0
1.20.759.025 I slot   operator(): id  0 | task 442 | Checking checkpoint with [24, 24] against 28...
1.20.762.930 W slot   operator(): id  0 | task 442 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.20.762.950 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.878.087 I slot print_timing: id  5 | task 391 | n_decoded =    101, tg =  20.76 t/s, tg_3s =  20.76 t/s
1.20.883.186 I slot print_timing: id 14 | task 374 | n_decoded =    100, tg =  15.52 t/s, tg_3s =  15.52 t/s
1.21.240.278 I slot print_timing: id  3 | task 382 | prompt eval time =     116.64 ms /     4 tokens (   29.16 ms per token,    34.30 tokens per second)
1.21.240.281 I slot print_timing: id  3 | task 382 |        eval time =    5951.76 ms /   125 tokens (   47.61 ms per token,    21.00 tokens per second)
1.21.240.282 I slot print_timing: id  3 | task 382 |       total time =    6068.39 ms /   129 tokens
1.21.240.283 I slot print_timing: id  3 | task 382 |    graphs reused =          1
1.21.240.286 I slot print_timing: id  3 | task 382 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.21.240.303 I statistics        draft-mtp: #calls(b,g,a) =  107    338   5163, #gen drafts =   5175, #acc drafts =  3719, #gen tokens =  10321, #acc tokens =  6909, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.618), dur(b,g,a) = 0.067, 1164.723, 6.630 ms
1.21.240.337 I slot      release: id  3 | task 382 | stop processing: n_tokens = 152, truncated = 0
1.21.248.979 I srv    operator(): Chat format: peg-native
1.21.358.342 I slot print_timing: id  7 | task 401 | n_decoded =    102, tg =  23.99 t/s, tg_3s =  23.99 t/s
1.21.362.225 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.21.362.227 I srv  get_availabl: updating prompt cache
1.21.362.269 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.21.362.271 I srv         alloc:  - prompt is already in the cache, skipping
1.21.362.272 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.21.362.275 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.21.362.276 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.21.362.276 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.21.362.277 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.21.362.277 I srv  get_availabl: prompt cache update took 0.05 ms
1.21.362.348 I slot launch_slot_: id  3 | task 448 | processing task, is_child = 0
1.21.366.212 I slot   operator(): id  3 | task 448 | Checking checkpoint with [23, 23] against 27...
1.21.370.036 W slot   operator(): id  3 | task 448 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.21.370.062 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.724.187 I slot print_timing: id 10 | task 361 | prompt eval time =     262.10 ms /    29 tokens (    9.04 ms per token,   110.64 tokens per second)
1.21.724.190 I slot print_timing: id 10 | task 361 |        eval time =    8375.55 ms /   128 tokens (   65.43 ms per token,    15.28 tokens per second)
1.21.724.190 I slot print_timing: id 10 | task 361 |       total time =    8637.65 ms /   157 tokens
1.21.724.191 I slot print_timing: id 10 | task 361 |    graphs reused =          1
1.21.724.193 I slot print_timing: id 10 | task 361 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.21.724.207 I statistics        draft-mtp: #calls(b,g,a) =  108    342   5221, #gen drafts =   5236, #acc drafts =  3756, #gen tokens =  10442, #acc tokens =  6978, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.068, 1179.296, 6.709 ms
1.21.724.232 I slot      release: id 10 | task 361 | stop processing: n_tokens = 156, truncated = 0
1.21.733.172 I srv    operator(): Chat format: peg-native
1.21.848.661 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.21.848.663 I srv  get_availabl: updating prompt cache
1.21.848.709 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.21.848.712 I srv         alloc:  - prompt is already in the cache, skipping
1.21.848.713 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.21.848.716 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.21.848.717 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.21.848.718 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.21.848.719 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.21.848.719 I srv  get_availabl: prompt cache update took 0.06 ms
1.21.848.787 I slot launch_slot_: id 10 | task 453 | processing task, is_child = 0
1.21.852.927 I slot   operator(): id 10 | task 453 | Checking checkpoint with [24, 24] against 3...
1.21.852.930 W slot   operator(): id 10 | task 453 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.21.852.933 W slot   operator(): id 10 | task 453 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.21.854.884 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.981.056 I slot print_timing: id  5 | task 391 | prompt eval time =     116.33 ms /     4 tokens (   29.08 ms per token,    34.38 tokens per second)
1.21.981.059 I slot print_timing: id  5 | task 391 |        eval time =    5967.69 ms /   125 tokens (   47.74 ms per token,    20.95 tokens per second)
1.21.981.059 I slot print_timing: id  5 | task 391 |       total time =    6084.02 ms /   129 tokens
1.21.981.060 I slot print_timing: id  5 | task 391 |    graphs reused =          1
1.21.981.063 I slot print_timing: id  5 | task 391 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.21.981.082 I statistics        draft-mtp: #calls(b,g,a) =  108    344   5257, #gen drafts =   5266, #acc drafts =  3782, #gen tokens =  10502, #acc tokens =  7027, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.068, 1187.862, 6.757 ms
1.21.981.113 I slot      release: id  5 | task 391 | stop processing: n_tokens = 152, truncated = 0
1.21.981.989 I slot print_timing: id  7 | task 401 | prompt eval time =     115.74 ms /     4 tokens (   28.94 ms per token,    34.56 tokens per second)
1.21.981.991 I slot print_timing: id  7 | task 401 |        eval time =    4874.66 ms /   114 tokens (   42.76 ms per token,    23.39 tokens per second)
1.21.981.991 I slot print_timing: id  7 | task 401 |       total time =    4990.40 ms /   118 tokens
1.21.981.992 I slot print_timing: id  7 | task 401 |    graphs reused =          1
1.21.981.994 I slot print_timing: id  7 | task 401 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.21.982.002 I statistics        draft-mtp: #calls(b,g,a) =  108    344   5259, #gen drafts =   5266, #acc drafts =  3783, #gen tokens =  10502, #acc tokens =  7029, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.068, 1187.862, 6.759 ms
1.21.982.024 I slot      release: id  7 | task 401 | stop processing: n_tokens = 140, truncated = 0
1.21.989.932 I srv    operator(): Chat format: peg-native
1.21.990.411 I srv    operator(): Chat format: peg-native
1.22.006.425 I slot create_check: id 10 | task 453 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.22.113.615 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
1.22.113.617 I srv  get_availabl: updating prompt cache
1.22.113.660 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.22.113.662 I srv         alloc:  - prompt is already in the cache, skipping
1.22.113.663 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
1.22.113.666 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.22.113.667 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.22.113.667 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.22.113.669 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.22.113.669 I srv  get_availabl: prompt cache update took 0.05 ms
1.22.113.735 I slot launch_slot_: id  5 | task 456 | processing task, is_child = 0
1.22.113.736 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
1.22.113.752 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.22.113.754 I srv         alloc:  - prompt is already in the cache, skipping
1.22.113.755 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.22.113.756 I srv  get_availabl: updating prompt cache
1.22.113.770 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.22.113.771 I srv         alloc:  - prompt is already in the cache, skipping
1.22.113.771 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.22.113.772 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.22.113.773 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.22.113.773 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.22.113.774 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.22.113.775 I srv  get_availabl: prompt cache update took 0.02 ms
1.22.113.792 I slot launch_slot_: id  7 | task 457 | processing task, is_child = 0
1.22.119.153 I slot   operator(): id  5 | task 456 | Checking checkpoint with [23, 23] against 3...
1.22.119.156 W slot   operator(): id  5 | task 456 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.22.119.159 W slot   operator(): id  5 | task 456 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.22.120.673 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.120.692 I slot   operator(): id  7 | task 457 | Checking checkpoint with [20, 20] against 3...
1.22.120.694 W slot   operator(): id  7 | task 457 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.22.120.696 W slot   operator(): id  7 | task 457 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.22.122.718 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.284.234 I slot create_check: id  5 | task 456 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.22.298.170 I slot create_check: id  7 | task 457 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.22.416.793 I slot print_timing: id  9 | task 412 | n_decoded =    102, tg =  23.53 t/s, tg_3s =  23.53 t/s
1.22.420.113 I slot print_timing: id 15 | task 388 | n_decoded =    100, tg =  15.03 t/s, tg_3s =  15.03 t/s
1.22.657.616 I slot print_timing: id  8 | task 409 | n_decoded =    101, tg =  20.96 t/s, tg_3s =  20.96 t/s
1.22.773.228 I slot print_timing: id 14 | task 374 | prompt eval time =     117.49 ms /     4 tokens (   29.37 ms per token,    34.05 tokens per second)
1.22.773.231 I slot print_timing: id 14 | task 374 |        eval time =    8333.56 ms /   128 tokens (   65.11 ms per token,    15.36 tokens per second)
1.22.773.232 I slot print_timing: id 14 | task 374 |       total time =    8451.05 ms /   132 tokens
1.22.773.232 I slot print_timing: id 14 | task 374 |    graphs reused =          1
1.22.773.236 I slot print_timing: id 14 | task 374 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
1.22.773.249 I statistics        draft-mtp: #calls(b,g,a) =  111    350   5339, #gen drafts =   5354, #acc drafts =  3843, #gen tokens =  10677, #acc tokens =  7139, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.069, 1213.562, 6.865 ms
1.22.773.274 I slot      release: id 14 | task 374 | stop processing: n_tokens = 156, truncated = 0
1.22.782.675 I srv    operator(): Chat format: peg-native
1.22.895.929 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.22.895.931 I srv  get_availabl: updating prompt cache
1.22.895.972 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.22.895.976 I srv         alloc:  - prompt is already in the cache, skipping
1.22.895.977 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.22.895.980 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.22.895.981 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.22.895.981 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.22.895.982 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.22.895.982 I srv  get_availabl: prompt cache update took 0.05 ms
1.22.896.046 I slot launch_slot_: id 14 | task 464 | processing task, is_child = 0
1.22.899.715 I slot   operator(): id 14 | task 464 | Checking checkpoint with [24, 24] against 3...
1.22.899.718 W slot   operator(): id 14 | task 464 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.22.899.721 W slot   operator(): id 14 | task 464 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.22.901.504 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.028.917 I slot print_timing: id  9 | task 412 | prompt eval time =     115.09 ms /     4 tokens (   28.77 ms per token,    34.75 tokens per second)
1.23.028.922 I slot print_timing: id  9 | task 412 |        eval time =    4947.68 ms /   114 tokens (   43.40 ms per token,    23.04 tokens per second)
1.23.028.923 I slot print_timing: id  9 | task 412 |       total time =    5062.77 ms /   118 tokens
1.23.028.924 I slot print_timing: id  9 | task 412 |    graphs reused =          1
1.23.028.927 I slot print_timing: id  9 | task 412 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.23.028.940 I statistics        draft-mtp: #calls(b,g,a) =  111    352   5379, #gen drafts =   5384, #acc drafts =  3870, #gen tokens =  10737, #acc tokens =  7191, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.069, 1220.851, 6.915 ms
1.23.028.970 I slot      release: id  9 | task 412 | stop processing: n_tokens = 140, truncated = 0
1.23.037.953 I srv    operator(): Chat format: peg-native
1.23.050.012 I slot create_check: id 14 | task 464 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.23.163.225 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.23.163.227 I srv  get_availabl: updating prompt cache
1.23.163.267 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.23.163.269 I srv         alloc:  - prompt is already in the cache, skipping
1.23.163.270 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.23.163.274 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.23.163.276 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.23.163.276 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.23.163.276 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.23.163.277 I srv  get_availabl: prompt cache update took 0.05 ms
1.23.163.347 I slot launch_slot_: id  9 | task 467 | processing task, is_child = 0
1.23.167.593 I slot   operator(): id  9 | task 467 | Checking checkpoint with [20, 20] against 3...
1.23.167.596 W slot   operator(): id  9 | task 467 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.23.167.599 W slot   operator(): id  9 | task 467 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.23.169.828 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.319.917 I slot create_check: id  9 | task 467 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.23.799.402 I slot print_timing: id  8 | task 409 | prompt eval time =     116.73 ms /     4 tokens (   29.18 ms per token,    34.27 tokens per second)
1.23.799.404 I slot print_timing: id  8 | task 409 |        eval time =    5960.88 ms /   125 tokens (   47.69 ms per token,    20.97 tokens per second)
1.23.799.404 I slot print_timing: id  8 | task 409 |       total time =    6077.61 ms /   129 tokens
1.23.799.405 I slot print_timing: id  8 | task 409 |    graphs reused =          1
1.23.799.408 I slot print_timing: id  8 | task 409 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.23.799.420 I statistics        draft-mtp: #calls(b,g,a) =  113    358   5469, #gen drafts =   5476, #acc drafts =  3936, #gen tokens =  10921, #acc tokens =  7312, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.070, 1242.911, 7.035 ms
1.23.799.443 I slot      release: id  8 | task 409 | stop processing: n_tokens = 152, truncated = 0
1.23.807.882 I srv    operator(): Chat format: peg-native
1.23.914.798 I slot print_timing: id  4 | task 421 | n_decoded =    101, tg =  20.28 t/s, tg_3s =  20.28 t/s
1.23.919.055 I slot print_timing: id 13 | task 427 | n_decoded =    102, tg =  23.30 t/s, tg_3s =  23.30 t/s
1.23.919.956 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.23.919.958 I srv  get_availabl: updating prompt cache
1.23.920.000 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.23.920.002 I srv         alloc:  - prompt is already in the cache, skipping
1.23.920.003 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.23.920.006 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.23.920.007 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.23.920.009 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.23.920.009 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.23.920.010 I srv  get_availabl: prompt cache update took 0.05 ms
1.23.920.072 I slot launch_slot_: id  8 | task 474 | processing task, is_child = 0
1.23.924.882 I slot   operator(): id  8 | task 474 | Checking checkpoint with [23, 23] against 27...
1.23.928.732 W slot   operator(): id  8 | task 474 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.23.928.755 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.163.109 I slot print_timing: id  2 | task 405 | n_decoded =    100, tg =  14.94 t/s, tg_3s =  14.94 t/s
1.24.280.134 I slot print_timing: id 15 | task 388 | prompt eval time =     116.04 ms /     4 tokens (   29.01 ms per token,    34.47 tokens per second)
1.24.280.137 I slot print_timing: id 15 | task 388 |        eval time =    8511.91 ms /   128 tokens (   66.50 ms per token,    15.04 tokens per second)
1.24.280.137 I slot print_timing: id 15 | task 388 |       total time =    8627.95 ms /   132 tokens
1.24.280.138 I slot print_timing: id 15 | task 388 |    graphs reused =          1
1.24.280.141 I slot print_timing: id 15 | task 388 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.24.280.154 I statistics        draft-mtp: #calls(b,g,a) =  114    362   5522, #gen drafts =   5537, #acc drafts =  3973, #gen tokens =  11042, #acc tokens =  7383, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.618), dur(b,g,a) = 0.071, 1257.982, 7.103 ms
1.24.280.179 I slot      release: id 15 | task 388 | stop processing: n_tokens = 156, truncated = 0
1.24.289.454 I srv    operator(): Chat format: peg-native
1.24.401.279 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.24.401.281 I srv  get_availabl: updating prompt cache
1.24.401.321 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.24.401.324 I srv         alloc:  - prompt is already in the cache, skipping
1.24.401.325 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.24.401.331 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.24.401.332 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.24.401.332 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.24.401.333 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.24.401.333 I srv  get_availabl: prompt cache update took 0.05 ms
1.24.401.397 I slot launch_slot_: id 15 | task 479 | processing task, is_child = 0
1.24.404.115 I slot   operator(): id 15 | task 479 | Checking checkpoint with [24, 24] against 3...
1.24.404.117 W slot   operator(): id 15 | task 479 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.24.404.120 W slot   operator(): id 15 | task 479 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.24.405.880 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.533.821 I slot print_timing: id 13 | task 427 | prompt eval time =     116.00 ms /     4 tokens (   29.00 ms per token,    34.48 tokens per second)
1.24.533.824 I slot print_timing: id 13 | task 427 |        eval time =    4992.82 ms /   114 tokens (   43.80 ms per token,    22.83 tokens per second)
1.24.533.824 I slot print_timing: id 13 | task 427 |       total time =    5108.82 ms /   118 tokens
1.24.533.825 I slot print_timing: id 13 | task 427 |    graphs reused =          1
1.24.533.828 I slot print_timing: id 13 | task 427 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.24.533.839 I statistics        draft-mtp: #calls(b,g,a) =  114    364   5566, #gen drafts =   5567, #acc drafts =  4003, #gen tokens =  11102, #acc tokens =  7437, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.071, 1263.344, 7.158 ms
1.24.533.867 I slot      release: id 13 | task 427 | stop processing: n_tokens = 140, truncated = 0
1.24.542.002 I srv    operator(): Chat format: peg-native
1.24.551.576 I slot create_check: id 15 | task 479 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.24.664.605 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.24.664.607 I srv  get_availabl: updating prompt cache
1.24.664.647 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.24.664.650 I srv         alloc:  - prompt is already in the cache, skipping
1.24.664.650 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.24.664.653 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.24.664.654 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.24.664.654 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.24.664.655 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.24.664.656 I srv  get_availabl: prompt cache update took 0.05 ms
1.24.664.720 I slot launch_slot_: id 13 | task 482 | processing task, is_child = 0
1.24.668.486 I slot   operator(): id 13 | task 482 | Checking checkpoint with [20, 20] against 3...
1.24.668.489 W slot   operator(): id 13 | task 482 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.24.668.492 W slot   operator(): id 13 | task 482 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.24.670.049 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.818.513 I slot create_check: id 13 | task 482 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.24.937.895 I slot print_timing: id 11 | task 432 | n_decoded =    101, tg =  20.08 t/s, tg_3s =  20.08 t/s
1.25.055.264 I slot print_timing: id  4 | task 421 | prompt eval time =     116.76 ms /     4 tokens (   29.19 ms per token,    34.26 tokens per second)
1.25.055.267 I slot print_timing: id  4 | task 421 |        eval time =    6119.92 ms /   125 tokens (   48.96 ms per token,    20.43 tokens per second)
1.25.055.267 I slot print_timing: id  4 | task 421 |       total time =    6236.68 ms /   129 tokens
1.25.055.268 I slot print_timing: id  4 | task 421 |    graphs reused =          1
1.25.055.271 I slot print_timing: id  4 | task 421 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.25.055.287 I statistics        draft-mtp: #calls(b,g,a) =  116    368   5616, #gen drafts =   5627, #acc drafts =  4042, #gen tokens =  11222, #acc tokens =  7509, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.072, 1277.134, 7.223 ms
1.25.055.317 I slot      release: id  4 | task 421 | stop processing: n_tokens = 152, truncated = 0
1.25.056.566 I slot print_timing: id  6 | task 415 | n_decoded =    100, tg =  14.86 t/s, tg_3s =  14.86 t/s
1.25.064.142 I srv    operator(): Chat format: peg-native
1.25.175.231 I slot print_timing: id 12 | task 440 | n_decoded =    102, tg =  23.05 t/s, tg_3s =  23.05 t/s
1.25.177.068 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.25.177.070 I srv  get_availabl: updating prompt cache
1.25.177.111 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.25.177.113 I srv         alloc:  - prompt is already in the cache, skipping
1.25.177.114 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.25.177.117 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.25.177.118 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.25.177.118 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.25.177.119 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.25.177.120 I srv  get_availabl: prompt cache update took 0.05 ms
1.25.177.185 I slot launch_slot_: id  4 | task 487 | processing task, is_child = 0
1.25.181.427 I slot   operator(): id  4 | task 487 | Checking checkpoint with [23, 23] against 27...
1.25.185.317 W slot   operator(): id  4 | task 487 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.25.185.364 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.788.917 I slot print_timing: id 12 | task 440 | prompt eval time =     109.92 ms /     4 tokens (   27.48 ms per token,    36.39 tokens per second)
1.25.788.921 I slot print_timing: id 12 | task 440 |        eval time =    5039.58 ms /   114 tokens (   44.21 ms per token,    22.62 tokens per second)
1.25.788.921 I slot print_timing: id 12 | task 440 |       total time =    5149.50 ms /   118 tokens
1.25.788.922 I slot print_timing: id 12 | task 440 |    graphs reused =          1
1.25.788.925 I slot print_timing: id 12 | task 440 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.25.788.936 I statistics        draft-mtp: #calls(b,g,a) =  117    374   5718, #gen drafts =   5721, #acc drafts =  4113, #gen tokens =  11410, #acc tokens =  7641, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.073, 1296.853, 7.359 ms
1.25.788.964 I slot      release: id 12 | task 440 | stop processing: n_tokens = 140, truncated = 0
1.25.798.007 I srv    operator(): Chat format: peg-native
1.25.906.092 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.25.906.094 I srv  get_availabl: updating prompt cache
1.25.906.136 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.25.906.138 I srv         alloc:  - prompt is already in the cache, skipping
1.25.906.138 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.25.906.141 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.25.906.142 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.25.906.143 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.25.906.144 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.25.906.145 I srv  get_availabl: prompt cache update took 0.05 ms
1.25.906.212 I slot launch_slot_: id 12 | task 494 | processing task, is_child = 0
1.25.910.948 I slot   operator(): id 12 | task 494 | Checking checkpoint with [20, 20] against 24...
1.25.914.820 W slot   operator(): id 12 | task 494 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.25.914.851 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.025.461 I slot print_timing: id  2 | task 405 | prompt eval time =     116.39 ms /     4 tokens (   29.10 ms per token,    34.37 tokens per second)
1.26.025.464 I slot print_timing: id  2 | task 405 |        eval time =    8554.31 ms /   128 tokens (   66.83 ms per token,    14.96 tokens per second)
1.26.025.465 I slot print_timing: id  2 | task 405 |       total time =    8670.70 ms /   132 tokens
1.26.025.466 I slot print_timing: id  2 | task 405 |    graphs reused =          1
1.26.025.469 I slot print_timing: id  2 | task 405 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.26.025.482 I statistics        draft-mtp: #calls(b,g,a) =  117    376   5736, #gen drafts =   5750, #acc drafts =  4127, #gen tokens =  11467, #acc tokens =  7667, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.073, 1305.406, 7.383 ms
1.26.025.507 I slot      release: id  2 | task 405 | stop processing: n_tokens = 156, truncated = 0
1.26.030.887 I slot print_timing: id 11 | task 432 | prompt eval time =     116.97 ms /     4 tokens (   29.24 ms per token,    34.20 tokens per second)
1.26.030.890 I slot print_timing: id 11 | task 432 |        eval time =    6123.23 ms /   125 tokens (   48.99 ms per token,    20.41 tokens per second)
1.26.030.890 I slot print_timing: id 11 | task 432 |       total time =    6240.19 ms /   129 tokens
1.26.030.891 I slot print_timing: id 11 | task 432 |    graphs reused =          1
1.26.030.894 I slot print_timing: id 11 | task 432 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.26.030.901 I statistics        draft-mtp: #calls(b,g,a) =  118    376   5747, #gen drafts =   5750, #acc drafts =  4133, #gen tokens =  11467, #acc tokens =  7679, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.074, 1305.406, 7.395 ms
1.26.030.925 I slot      release: id 11 | task 432 | stop processing: n_tokens = 152, truncated = 0
1.26.034.621 I srv    operator(): Chat format: peg-native
1.26.039.306 I srv    operator(): Chat format: peg-native
1.26.142.409 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.26.142.412 I srv  get_availabl: updating prompt cache
1.26.142.453 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.26.142.456 I srv         alloc:  - prompt is already in the cache, skipping
1.26.142.457 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.26.142.460 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.142.462 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.142.462 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.142.463 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.142.463 I srv  get_availabl: prompt cache update took 0.05 ms
1.26.142.529 I slot launch_slot_: id  2 | task 497 | processing task, is_child = 0
1.26.142.532 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
1.26.142.549 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.26.142.550 I srv         alloc:  - prompt is already in the cache, skipping
1.26.142.553 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.26.142.553 I srv  get_availabl: updating prompt cache
1.26.142.568 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.26.142.570 I srv         alloc:  - prompt is already in the cache, skipping
1.26.142.570 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.26.142.571 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.142.572 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.142.572 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.142.572 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.142.573 I srv  get_availabl: prompt cache update took 0.02 ms
1.26.142.590 I slot launch_slot_: id 11 | task 498 | processing task, is_child = 0
1.26.148.042 I slot   operator(): id  2 | task 497 | Checking checkpoint with [24, 24] against 28...
1.26.151.851 W slot   operator(): id  2 | task 497 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.26.151.893 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.151.912 I slot   operator(): id 11 | task 498 | Checking checkpoint with [23, 23] against 27...
1.26.155.524 W slot   operator(): id 11 | task 498 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.26.155.556 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.512.489 I slot print_timing: id  1 | task 430 | n_decoded =    100, tg =  14.85 t/s, tg_3s =  14.85 t/s
1.26.513.716 I slot print_timing: id  3 | task 448 | n_decoded =    101, tg =  20.08 t/s, tg_3s =  20.08 t/s
1.26.517.351 I slot print_timing: id 10 | task 453 | n_decoded =    102, tg =  23.12 t/s, tg_3s =  23.12 t/s
1.26.873.945 I slot print_timing: id  6 | task 415 | prompt eval time =     116.69 ms /     4 tokens (   29.17 ms per token,    34.28 tokens per second)
1.26.873.949 I slot print_timing: id  6 | task 415 |        eval time =    8547.58 ms /   128 tokens (   66.78 ms per token,    14.97 tokens per second)
1.26.873.949 I slot print_timing: id  6 | task 415 |       total time =    8664.27 ms /   132 tokens
1.26.873.951 I slot print_timing: id  6 | task 415 |    graphs reused =          1
1.26.873.954 I slot print_timing: id  6 | task 415 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.26.873.969 I statistics        draft-mtp: #calls(b,g,a) =  120    383   5842, #gen drafts =   5857, #acc drafts =  4201, #gen tokens =  11680, #acc tokens =  7805, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.074, 1331.670, 7.516 ms
1.26.873.991 I slot      release: id  6 | task 415 | stop processing: n_tokens = 156, truncated = 0
1.26.883.575 I srv    operator(): Chat format: peg-native
1.26.997.356 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.26.997.358 I srv  get_availabl: updating prompt cache
1.26.997.400 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.26.997.403 I srv         alloc:  - prompt is already in the cache, skipping
1.26.997.404 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.26.997.406 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.997.408 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.997.408 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.997.409 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.997.409 I srv  get_availabl: prompt cache update took 0.05 ms
1.26.997.477 I slot launch_slot_: id  6 | task 506 | processing task, is_child = 0
1.27.001.631 I slot   operator(): id  6 | task 506 | Checking checkpoint with [24, 24] against 3...
1.27.001.633 W slot   operator(): id  6 | task 506 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.27.001.636 W slot   operator(): id  6 | task 506 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.27.003.204 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.130.706 I slot print_timing: id 10 | task 453 | prompt eval time =     253.44 ms /    25 tokens (   10.14 ms per token,    98.64 tokens per second)
1.27.130.709 I slot print_timing: id 10 | task 453 |        eval time =    5024.32 ms /   114 tokens (   44.07 ms per token,    22.69 tokens per second)
1.27.130.710 I slot print_timing: id 10 | task 453 |       total time =    5277.76 ms /   139 tokens
1.27.130.711 I slot print_timing: id 10 | task 453 |    graphs reused =          1
1.27.130.713 I slot print_timing: id 10 | task 453 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.27.130.725 I statistics        draft-mtp: #calls(b,g,a) =  120    385   5882, #gen drafts =   5887, #acc drafts =  4228, #gen tokens =  11740, #acc tokens =  7855, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.074, 1340.246, 7.568 ms
1.27.130.752 I slot      release: id 10 | task 453 | stop processing: n_tokens = 140, truncated = 0
1.27.139.610 I srv    operator(): Chat format: peg-native
1.27.152.614 I slot create_check: id  6 | task 506 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.27.267.405 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.27.267.407 I srv  get_availabl: updating prompt cache
1.27.267.451 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.27.267.453 I srv         alloc:  - prompt is already in the cache, skipping
1.27.267.454 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.27.267.457 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.27.267.458 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.27.267.458 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.27.267.459 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.27.267.459 I srv  get_availabl: prompt cache update took 0.05 ms
1.27.267.528 I slot launch_slot_: id 10 | task 509 | processing task, is_child = 0
1.27.271.724 I slot   operator(): id 10 | task 509 | Checking checkpoint with [20, 20] against 3...
1.27.271.727 W slot   operator(): id 10 | task 509 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.27.271.730 W slot   operator(): id 10 | task 509 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.27.274.150 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.401.585 I slot print_timing: id  7 | task 457 | n_decoded =    101, tg =  20.25 t/s, tg_3s =  20.25 t/s
1.27.424.468 I slot create_check: id 10 | task 509 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.27.544.057 I slot print_timing: id 14 | task 464 | n_decoded =    102, tg =  23.25 t/s, tg_3s =  23.25 t/s
1.27.658.354 I slot print_timing: id  0 | task 442 | n_decoded =    100, tg =  14.74 t/s, tg_3s =  14.74 t/s
1.27.659.967 I slot print_timing: id  3 | task 448 | prompt eval time =     116.41 ms /     4 tokens (   29.10 ms per token,    34.36 tokens per second)
1.27.659.969 I slot print_timing: id  3 | task 448 |        eval time =    6177.32 ms /   125 tokens (   49.42 ms per token,    20.24 tokens per second)
1.27.659.970 I slot print_timing: id  3 | task 448 |       total time =    6293.73 ms /   129 tokens
1.27.659.971 I slot print_timing: id  3 | task 448 |    graphs reused =          1
1.27.659.974 I slot print_timing: id  3 | task 448 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.27.659.990 I statistics        draft-mtp: #calls(b,g,a) =  122    389   5935, #gen drafts =   5947, #acc drafts =  4272, #gen tokens =  11860, #acc tokens =  7936, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.617), dur(b,g,a) = 0.076, 1356.869, 7.638 ms
1.27.660.018 I slot      release: id  3 | task 448 | stop processing: n_tokens = 152, truncated = 0
1.27.669.190 I srv    operator(): Chat format: peg-native
1.27.780.662 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.27.780.665 I srv  get_availabl: updating prompt cache
1.27.780.707 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.27.780.709 I srv         alloc:  - prompt is already in the cache, skipping
1.27.780.710 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.27.780.713 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.27.780.713 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.27.780.714 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.27.780.714 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.27.780.715 I srv  get_availabl: prompt cache update took 0.05 ms
1.27.780.780 I slot launch_slot_: id  3 | task 514 | processing task, is_child = 0
1.27.784.620 I slot   operator(): id  3 | task 514 | Checking checkpoint with [23, 23] against 27...
1.27.788.481 W slot   operator(): id  3 | task 514 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.27.788.510 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.149.374 I slot print_timing: id 14 | task 464 | prompt eval time =     256.55 ms /    25 tokens (   10.26 ms per token,    97.45 tokens per second)
1.28.149.377 I slot print_timing: id 14 | task 464 |        eval time =    4993.10 ms /   114 tokens (   43.80 ms per token,    22.83 tokens per second)
1.28.149.377 I slot print_timing: id 14 | task 464 |       total time =    5249.64 ms /   139 tokens
1.28.149.378 I slot print_timing: id 14 | task 464 |    graphs reused =          1
1.28.149.380 I slot print_timing: id 14 | task 464 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.28.149.392 I statistics        draft-mtp: #calls(b,g,a) =  123    393   6008, #gen drafts =   6009, #acc drafts =  4321, #gen tokens =  11984, #acc tokens =  8028, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.077, 1370.043, 7.732 ms
1.28.149.416 I slot      release: id 14 | task 464 | stop processing: n_tokens = 140, truncated = 0
1.28.157.752 I srv    operator(): Chat format: peg-native
1.28.264.422 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.28.264.425 I srv  get_availabl: updating prompt cache
1.28.264.464 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.28.264.466 I srv         alloc:  - prompt is already in the cache, skipping
1.28.264.467 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.28.264.470 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.264.472 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.264.472 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.264.473 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.264.473 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.264.540 I slot launch_slot_: id 14 | task 519 | processing task, is_child = 0
1.28.268.941 I slot   operator(): id 14 | task 519 | Checking checkpoint with [20, 20] against 24...
1.28.272.753 W slot   operator(): id 14 | task 519 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.28.272.791 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.383.222 I slot print_timing: id  1 | task 430 | prompt eval time =     110.02 ms /     4 tokens (   27.51 ms per token,    36.36 tokens per second)
1.28.383.224 I slot print_timing: id  1 | task 430 |        eval time =    8603.78 ms /   128 tokens (   67.22 ms per token,    14.88 tokens per second)
1.28.383.225 I slot print_timing: id  1 | task 430 |       total time =    8713.80 ms /   132 tokens
1.28.383.226 I slot print_timing: id  1 | task 430 |    graphs reused =          1
1.28.383.228 I slot print_timing: id  1 | task 430 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.28.383.241 I statistics        draft-mtp: #calls(b,g,a) =  123    395   6024, #gen drafts =   6038, #acc drafts =  4331, #gen tokens =  12041, #acc tokens =  8046, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.077, 1378.021, 7.752 ms
1.28.383.264 I slot      release: id  1 | task 430 | stop processing: n_tokens = 156, truncated = 0
1.28.392.010 I srv    operator(): Chat format: peg-native
1.28.501.669 I slot print_timing: id  7 | task 457 | prompt eval time =     292.25 ms /    28 tokens (   10.44 ms per token,    95.81 tokens per second)
1.28.501.672 I slot print_timing: id  7 | task 457 |        eval time =    6088.69 ms /   125 tokens (   48.71 ms per token,    20.53 tokens per second)
1.28.501.672 I slot print_timing: id  7 | task 457 |       total time =    6380.94 ms /   153 tokens
1.28.501.674 I slot print_timing: id  7 | task 457 |    graphs reused =          1
1.28.501.676 I slot print_timing: id  7 | task 457 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.28.501.689 I statistics        draft-mtp: #calls(b,g,a) =  124    396   6045, #gen drafts =   6053, #acc drafts =  4347, #gen tokens =  12071, #acc tokens =  8076, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.077, 1381.627, 7.777 ms
1.28.501.713 I slot      release: id  7 | task 457 | stop processing: n_tokens = 152, truncated = 0
1.28.505.549 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.28.505.552 I srv  get_availabl: updating prompt cache
1.28.505.593 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.28.505.595 I srv         alloc:  - prompt is already in the cache, skipping
1.28.505.596 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.28.505.599 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.505.600 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.505.600 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.505.601 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.505.601 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.505.660 I slot launch_slot_: id  1 | task 522 | processing task, is_child = 0
1.28.505.664 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
1.28.505.680 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.28.505.681 I srv         alloc:  - prompt is already in the cache, skipping
1.28.510.448 I srv    operator(): Chat format: peg-native
1.28.511.052 I slot   operator(): id  1 | task 522 | Checking checkpoint with [24, 24] against 28...
1.28.514.916 W slot   operator(): id  1 | task 522 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.28.514.941 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.628.140 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.28.628.142 I srv  get_availabl: updating prompt cache
1.28.628.183 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.28.628.185 I srv         alloc:  - prompt is already in the cache, skipping
1.28.628.186 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.28.628.189 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.628.190 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.628.190 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.628.191 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.628.193 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.628.260 I slot launch_slot_: id  7 | task 524 | processing task, is_child = 0
1.28.633.120 I slot   operator(): id  7 | task 524 | Checking checkpoint with [23, 23] against 27...
1.28.637.007 W slot   operator(): id  7 | task 524 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.28.637.042 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.996.156 I slot print_timing: id  8 | task 474 | n_decoded =    101, tg =  20.38 t/s, tg_3s =  20.38 t/s
1.28.999.404 I slot print_timing: id 15 | task 479 | n_decoded =    102, tg =  23.49 t/s, tg_3s =  23.49 t/s
1.29.115.008 I slot print_timing: id  5 | task 456 | n_decoded =    100, tg =  14.92 t/s, tg_3s =  14.92 t/s
1.29.473.184 I slot print_timing: id  0 | task 442 | prompt eval time =     116.41 ms /     4 tokens (   29.10 ms per token,    34.36 tokens per second)
1.29.473.187 I slot print_timing: id  0 | task 442 |        eval time =    8597.72 ms /   128 tokens (   67.17 ms per token,    14.89 tokens per second)
1.29.473.187 I slot print_timing: id  0 | task 442 |       total time =    8714.13 ms /   132 tokens
1.29.473.188 I slot print_timing: id  0 | task 442 |    graphs reused =          1
1.29.473.191 I slot print_timing: id  0 | task 442 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.29.473.206 I statistics        draft-mtp: #calls(b,g,a) =  126    404   6162, #gen drafts =   6177, #acc drafts =  4429, #gen tokens =  12318, #acc tokens =  8230, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.077, 1408.400, 7.923 ms
1.29.473.230 I slot      release: id  0 | task 442 | stop processing: n_tokens = 156, truncated = 0
1.29.482.212 I srv    operator(): Chat format: peg-native
1.29.593.570 I slot print_timing: id 15 | task 479 | prompt eval time =     252.95 ms /    25 tokens (   10.12 ms per token,    98.83 tokens per second)
1.29.593.573 I slot print_timing: id 15 | task 479 |        eval time =    4936.48 ms /   114 tokens (   43.30 ms per token,    23.09 tokens per second)
1.29.593.573 I slot print_timing: id 15 | task 479 |       total time =    5189.44 ms /   139 tokens
1.29.593.574 I slot print_timing: id 15 | task 479 |    graphs reused =          1
1.29.593.576 I slot print_timing: id 15 | task 479 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.29.593.587 I statistics        draft-mtp: #calls(b,g,a) =  126    405   6192, #gen drafts =   6192, #acc drafts =  4450, #gen tokens =  12348, #acc tokens =  8266, #mean acc len = 2.33, #acc rate/pos = (0.719, 0.616), dur(b,g,a) = 0.077, 1411.049, 7.960 ms
1.29.593.612 I slot      release: id 15 | task 479 | stop processing: n_tokens = 140, truncated = 0
1.29.593.627 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.29.593.629 I srv  get_availabl: updating prompt cache
1.29.593.669 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.29.593.671 I srv         alloc:  - prompt is already in the cache, skipping
1.29.593.672 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.29.593.674 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.29.593.676 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.29.593.676 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.29.593.677 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.29.593.677 I srv  get_availabl: prompt cache update took 0.05 ms
1.29.593.740 I slot launch_slot_: id 15 | task 533 | processing task, is_child = 0
1.29.593.742 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.29.593.759 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.29.593.759 I srv         alloc:  - prompt is already in the cache, skipping
1.29.596.422 I slot   operator(): id 15 | task 533 | Checking checkpoint with [20, 20] against 24...
1.29.600.246 W slot   operator(): id 15 | task 533 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.29.600.275 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.602.138 I srv    operator(): Chat format: peg-native
1.29.712.303 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.29.712.306 I srv  get_availabl: updating prompt cache
1.29.712.350 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.29.712.352 I srv         alloc:  - prompt is already in the cache, skipping
1.29.712.353 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.29.712.356 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.29.712.357 I srv        update:    - prompt 0x64aa1eca4e90:     140 tokens, checkpoints:  1,   129.625 MiB
1.29.712.357 I srv        update:    - prompt 0x64aa1678b380:     152 tokens, checkpoints:  1,   129.977 MiB
1.29.712.358 I srv        update:    - prompt 0x64aa1ccb2110:     156 tokens, checkpoints:  1,   130.094 MiB
1.29.712.359 I srv  get_availabl: prompt cache update took 0.05 ms
1.29.712.423 I slot launch_slot_: id  0 | task 535 | processing task, is_child = 0
1.29.715.149 I slot   operator(): id  0 | task 535 | Checking checkpoint with [24, 24] against 28...
1.29.719.053 W slot   operator(): id  0 | task 535 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.29.719.080 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.077.774 I slot print_timing: id  8 | task 474 | prompt eval time =     116.48 ms /     4 tokens (   29.12 ms per token,    34.34 tokens per second)
1.30.077.777 I slot print_timing: id  8 | task 474 |        eval time =    6036.38 ms /   125 tokens (   48.29 ms per token,    20.71 tokens per second)
1.30.077.777 I slot print_timing: id  8 | task 474 |       total time =    6152.86 ms /   129 tokens
1.30.077.778 I slot print_timing: id  8 | task 474 |    graphs reused =          1
1.30.077.781 I slot print_timing: id  8 | task 474 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.30.077.793 I statistics        draft-mtp: #calls(b,g,a) =  128    409   6246, #gen drafts =   6253, #acc drafts =  4492, #gen tokens =  12470, #acc tokens =  8345, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.078, 1421.915, 8.029 ms
1.30.077.816 I slot      release: id  8 | task 474 | stop processing: n_tokens = 152, truncated = 0
1.30.078.421 I slot print_timing: id  9 | task 467 | n_decoded =    100, tg =  15.05 t/s, tg_3s =  15.05 t/s
1.30.192.647 I slot print_timing: id  4 | task 487 | n_decoded =    101, tg =  20.64 t/s, tg_3s =  20.64 t/s
1.30.311.923 I slot print_timing: id 12 | task 494 | n_decoded =    102, tg =  23.80 t/s, tg_3s =  23.80 t/s
1.30.884.568 I slot print_timing: id  5 | task 456 | prompt eval time =     293.51 ms /    29 tokens (   10.12 ms per token,    98.80 tokens per second)
1.30.884.572 I slot print_timing: id  5 | task 456 |        eval time =    8471.88 ms /   128 tokens (   66.19 ms per token,    15.11 tokens per second)
1.30.884.572 I slot print_timing: id  5 | task 456 |       total time =    8765.39 ms /   157 tokens
1.30.884.573 I slot print_timing: id  5 | task 456 |    graphs reused =          1
1.30.884.576 I slot print_timing: id  5 | task 456 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.30.884.590 I statistics        draft-mtp: #calls(b,g,a) =  128    416   6343, #gen drafts =   6357, #acc drafts =  4555, #gen tokens =  12677, #acc tokens =  8463, #mean acc len = 2.33, #acc rate/pos = (0.718, 0.616), dur(b,g,a) = 0.078, 1455.726, 8.149 ms
1.30.884.613 I slot      release: id  5 | task 456 | stop processing: n_tokens = 156, truncated = 0
1.30.890.742 I slot print_timing: id 12 | task 494 | prompt eval time =     114.82 ms /     4 tokens (   28.70 ms per token,    34.84 tokens per second)
1.30.890.745 I slot print_timing: id 12 | task 494 |        eval time =    4864.96 ms /   114 tokens (   42.68 ms per token,    23.43 tokens per second)
1.30.890.745 I slot print_timing: id 12 | task 494 |       total time =    4979.78 ms /   118 tokens
1.30.890.746 I slot print_timing: id 12 | task 494 |    graphs reused =          1
1.30.890.748 I slot print_timing: id 12 | task 494 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.30.890.756 I statistics        draft-mtp: #calls(b,g,a) =  128    416   6354, #gen drafts =   6357, #acc drafts =  4565, #gen tokens =  12677, #acc tokens =  8482, #mean acc len = 2.33, #acc rate/pos = (0.718, 0.616), dur(b,g,a) = 0.078, 1455.726, 8.162 ms
1.30.890.776 I slot      release: id 12 | task 494 | stop processing: n_tokens = 140, truncated = 0
1.31.098.739 I slot print_timing: id 11 | task 498 | n_decoded =    101, tg =  20.92 t/s, tg_3s =  20.92 t/s
1.31.199.834 I slot print_timing: id  4 | task 487 | prompt eval time =     116.75 ms /     4 tokens (   29.19 ms per token,    34.26 tokens per second)
1.31.199.837 I slot print_timing: id  4 | task 487 |        eval time =    5901.63 ms /   125 tokens (   47.21 ms per token,    21.18 tokens per second)
1.31.199.837 I slot print_timing: id  4 | task 487 |       total time =    6018.38 ms /   129 tokens
1.31.199.838 I slot print_timing: id  4 | task 487 |    graphs reused =          1
1.31.199.841 I slot print_timing: id  4 | task 487 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.31.199.851 I statistics        draft-mtp: #calls(b,g,a) =  128    419   6388, #gen drafts =   6396, #acc drafts =  4593, #gen tokens =  12755, #acc tokens =  8533, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.617), dur(b,g,a) = 0.078, 1472.509, 8.208 ms
1.31.199.874 I slot      release: id  4 | task 487 | stop processing: n_tokens = 152, truncated = 0
1.31.391.431 I slot print_timing: id  6 | task 506 | n_decoded =    102, tg =  24.69 t/s, tg_3s =  24.69 t/s
1.31.393.773 I slot print_timing: id 13 | task 482 | n_decoded =    100, tg =  15.47 t/s, tg_3s =  15.47 t/s
1.31.673.886 I slot print_timing: id  9 | task 467 | prompt eval time =     264.72 ms /    29 tokens (    9.13 ms per token,   109.55 tokens per second)
1.31.673.889 I slot print_timing: id  9 | task 467 |        eval time =    8241.55 ms /   128 tokens (   64.39 ms per token,    15.53 tokens per second)
1.31.673.889 I slot print_timing: id  9 | task 467 |       total time =    8506.27 ms /   157 tokens
1.31.673.890 I slot print_timing: id  9 | task 467 |    graphs reused =          1
1.31.673.893 I slot print_timing: id  9 | task 467 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.31.673.906 I statistics        draft-mtp: #calls(b,g,a) =  128    424   6444, #gen drafts =   6455, #acc drafts =  4627, #gen tokens =  12872, #acc tokens =  8596, #mean acc len = 2.33, #acc rate/pos = (0.718, 0.616), dur(b,g,a) = 0.078, 1499.872, 8.277 ms
1.31.673.931 I slot      release: id  9 | task 467 | stop processing: n_tokens = 156, truncated = 0
1.31.854.389 I slot print_timing: id  6 | task 506 | prompt eval time =     257.84 ms /    25 tokens (   10.31 ms per token,    96.96 tokens per second)
1.31.854.391 I slot print_timing: id  6 | task 506 |        eval time =    4594.90 ms /   114 tokens (   40.31 ms per token,    24.81 tokens per second)
1.31.854.392 I slot print_timing: id  6 | task 506 |       total time =    4852.74 ms /   139 tokens
1.31.854.393 I slot print_timing: id  6 | task 506 |    graphs reused =          1
1.31.854.395 I slot print_timing: id  6 | task 506 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.31.854.413 I statistics        draft-mtp: #calls(b,g,a) =  128    426   6471, #gen drafts =   6477, #acc drafts =  4645, #gen tokens =  12916, #acc tokens =  8629, #mean acc len = 2.33, #acc rate/pos = (0.718, 0.616), dur(b,g,a) = 0.078, 1510.633, 8.313 ms
1.31.854.436 I slot      release: id  6 | task 506 | stop processing: n_tokens = 140, truncated = 0
1.31.936.867 I slot print_timing: id 11 | task 498 | prompt eval time =     117.84 ms /     4 tokens (   29.46 ms per token,    33.94 tokens per second)
1.31.936.870 I slot print_timing: id 11 | task 498 |        eval time =    5667.08 ms /   125 tokens (   45.34 ms per token,    22.06 tokens per second)
1.31.936.871 I slot print_timing: id 11 | task 498 |       total time =    5784.93 ms /   129 tokens
1.31.936.872 I slot print_timing: id 11 | task 498 |    graphs reused =          1
1.31.936.875 I slot print_timing: id 11 | task 498 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.31.936.886 I statistics        draft-mtp: #calls(b,g,a) =  128    427   6484, #gen drafts =   6487, #acc drafts =  4654, #gen tokens =  12936, #acc tokens =  8646, #mean acc len = 2.33, #acc rate/pos = (0.718, 0.616), dur(b,g,a) = 0.078, 1515.877, 8.328 ms
1.31.936.910 I slot      release: id 11 | task 498 | stop processing: n_tokens = 152, truncated = 0
1.32.158.859 I slot print_timing: id 14 | task 519 | n_decoded =    102, tg =  27.02 t/s, tg_3s =  27.02 t/s
1.32.229.948 I slot print_timing: id  3 | task 514 | n_decoded =    101, tg =  23.33 t/s, tg_3s =  23.33 t/s
1.32.302.757 I slot print_timing: id  2 | task 497 | n_decoded =    100, tg =  16.57 t/s, tg_3s =  16.57 t/s
1.32.524.324 I slot print_timing: id 14 | task 519 | prompt eval time =     114.58 ms /     4 tokens (   28.64 ms per token,    34.91 tokens per second)
1.32.524.327 I slot print_timing: id 14 | task 519 |        eval time =    4140.79 ms /   114 tokens (   36.32 ms per token,    27.53 tokens per second)
1.32.524.333 I slot print_timing: id 14 | task 519 |       total time =    4255.37 ms /   118 tokens
1.32.524.334 I slot print_timing: id 14 | task 519 |    graphs reused =          1
1.32.524.337 I slot print_timing: id 14 | task 519 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.32.524.349 I statistics        draft-mtp: #calls(b,g,a) =  128    435   6558, #gen drafts =   6559, #acc drafts =  4705, #gen tokens =  13079, #acc tokens =  8741, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1556.101, 8.417 ms
1.32.524.369 I slot      release: id 14 | task 519 | stop processing: n_tokens = 140, truncated = 0
1.32.584.556 I slot print_timing: id 13 | task 482 | prompt eval time =     262.93 ms /    29 tokens (    9.07 ms per token,   110.30 tokens per second)
1.32.584.558 I slot print_timing: id 13 | task 482 |        eval time =    7653.12 ms /   128 tokens (   59.79 ms per token,    16.73 tokens per second)
1.32.584.559 I slot print_timing: id 13 | task 482 |       total time =    7916.05 ms /   157 tokens
1.32.584.560 I slot print_timing: id 13 | task 482 |    graphs reused =          1
1.32.584.563 I slot print_timing: id 13 | task 482 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.32.584.573 I statistics        draft-mtp: #calls(b,g,a) =  128    436   6559, #gen drafts =   6566, #acc drafts =  4706, #gen tokens =  13093, #acc tokens =  8743, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1560.741, 8.418 ms
1.32.584.591 I slot      release: id 13 | task 482 | stop processing: n_tokens = 156, truncated = 0
1.32.703.755 I slot print_timing: id  7 | task 524 | n_decoded =    101, tg =  25.55 t/s, tg_3s =  25.55 t/s
1.32.819.611 I slot print_timing: id  3 | task 514 | prompt eval time =     116.42 ms /     4 tokens (   29.10 ms per token,    34.36 tokens per second)
1.32.819.614 I slot print_timing: id  3 | task 514 |        eval time =    4918.55 ms /   125 tokens (   39.35 ms per token,    25.41 tokens per second)
1.32.819.615 I slot print_timing: id  3 | task 514 |       total time =    5034.97 ms /   129 tokens
1.32.819.616 I slot print_timing: id  3 | task 514 |    graphs reused =          1
1.32.819.619 I slot print_timing: id  3 | task 514 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.32.819.631 I statistics        draft-mtp: #calls(b,g,a) =  128    440   6591, #gen drafts =   6594, #acc drafts =  4727, #gen tokens =  13149, #acc tokens =  8783, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1579.234, 8.461 ms
1.32.819.650 I slot      release: id  3 | task 514 | stop processing: n_tokens = 152, truncated = 0
1.32.871.705 I slot print_timing: id 15 | task 533 | n_decoded =    102, tg =  32.22 t/s, tg_3s =  32.22 t/s
1.32.921.426 I slot print_timing: id 10 | task 509 | n_decoded =    100, tg =  18.57 t/s, tg_3s =  18.57 t/s
1.33.123.027 I slot print_timing: id 15 | task 533 | prompt eval time =     109.06 ms /     4 tokens (   27.27 ms per token,    36.68 tokens per second)
1.33.123.029 I slot print_timing: id 15 | task 533 |        eval time =    3417.53 ms /   114 tokens (   29.98 ms per token,    33.36 tokens per second)
1.33.123.030 I slot print_timing: id 15 | task 533 |       total time =    3526.59 ms /   118 tokens
1.33.123.031 I slot print_timing: id 15 | task 533 |    graphs reused =          1
1.33.123.034 I slot print_timing: id 15 | task 533 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.33.123.046 I statistics        draft-mtp: #calls(b,g,a) =  128    446   6630, #gen drafts =   6630, #acc drafts =  4752, #gen tokens =  13220, #acc tokens =  8828, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1604.618, 8.513 ms
1.33.123.064 I slot      release: id 15 | task 533 | stop processing: n_tokens = 140, truncated = 0
1.33.160.234 I slot print_timing: id  2 | task 497 | prompt eval time =     121.44 ms /     4 tokens (   30.36 ms per token,    32.94 tokens per second)
1.33.160.236 I slot print_timing: id  2 | task 497 |        eval time =    6890.74 ms /   128 tokens (   53.83 ms per token,    18.58 tokens per second)
1.33.160.237 I slot print_timing: id  2 | task 497 |       total time =    7012.18 ms /   132 tokens
1.33.160.237 I slot print_timing: id  2 | task 497 |    graphs reused =          1
1.33.160.240 I slot print_timing: id  2 | task 497 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.160.249 I statistics        draft-mtp: #calls(b,g,a) =  128    447   6630, #gen drafts =   6634, #acc drafts =  4752, #gen tokens =  13228, #acc tokens =  8828, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1607.756, 8.513 ms
1.33.160.269 I slot      release: id  2 | task 497 | stop processing: n_tokens = 156, truncated = 0
1.33.161.487 I slot print_timing: id  7 | task 524 | prompt eval time =     116.95 ms /     4 tokens (   29.24 ms per token,    34.20 tokens per second)
1.33.161.489 I slot print_timing: id  7 | task 524 |        eval time =    4411.40 ms /   125 tokens (   35.29 ms per token,    28.34 tokens per second)
1.33.161.490 I slot print_timing: id  7 | task 524 |       total time =    4528.36 ms /   129 tokens
1.33.161.490 I slot print_timing: id  7 | task 524 |    graphs reused =          1
1.33.161.492 I slot print_timing: id  7 | task 524 | draft acceptance = 0.76531 (   75 accepted /    98 generated), mean acceptance length =  2.53, acceptance rate per position = (0.816, 0.714)
1.33.161.498 I statistics        draft-mtp: #calls(b,g,a) =  128    447   6633, #gen drafts =   6634, #acc drafts =  4753, #gen tokens =  13228, #acc tokens =  8830, #mean acc len = 2.33, #acc rate/pos = (0.717, 0.615), dur(b,g,a) = 0.078, 1607.756, 8.517 ms
1.33.161.512 I slot      release: id  7 | task 524 | stop processing: n_tokens = 152, truncated = 0
1.33.263.480 I slot print_timing: id  1 | task 522 | n_decoded =    100, tg =  21.54 t/s, tg_3s =  21.54 t/s
1.33.410.118 I slot print_timing: id 10 | task 509 | prompt eval time =     265.03 ms /    29 tokens (    9.14 ms per token,   109.42 tokens per second)
1.33.410.121 I slot print_timing: id 10 | task 509 |        eval time =    5873.35 ms /   128 tokens (   45.89 ms per token,    21.79 tokens per second)
1.33.410.121 I slot print_timing: id 10 | task 509 |       total time =    6138.38 ms /   157 tokens
1.33.410.122 I slot print_timing: id 10 | task 509 |    graphs reused =          1
1.33.410.125 I slot print_timing: id 10 | task 509 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.410.139 I statistics        draft-mtp: #calls(b,g,a) =  128    457   6661, #gen drafts =   6663, #acc drafts =  4766, #gen tokens =  13285, #acc tokens =  8854, #mean acc len = 2.33, #acc rate/pos = (0.716, 0.614), dur(b,g,a) = 0.078, 1627.917, 8.554 ms
1.33.410.158 I slot      release: id 10 | task 509 | stop processing: n_tokens = 156, truncated = 0
1.33.478.537 I slot print_timing: id  0 | task 535 | n_decoded =    100, tg =  27.42 t/s, tg_3s =  27.42 t/s
1.33.557.943 I slot print_timing: id  1 | task 522 | prompt eval time =     110.39 ms /     4 tokens (   27.60 ms per token,    36.23 tokens per second)
1.33.557.946 I slot print_timing: id  1 | task 522 |        eval time =    4936.49 ms /   128 tokens (   38.57 ms per token,    25.93 tokens per second)
1.33.557.947 I slot print_timing: id  1 | task 522 |       total time =    5046.88 ms /   132 tokens
1.33.557.947 I slot print_timing: id  1 | task 522 |    graphs reused =          1
1.33.557.950 I slot print_timing: id  1 | task 522 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.557.964 I statistics        draft-mtp: #calls(b,g,a) =  128    466   6679, #gen drafts =   6680, #acc drafts =  4776, #gen tokens =  13318, #acc tokens =  8872, #mean acc len = 2.33, #acc rate/pos = (0.715, 0.613), dur(b,g,a) = 0.078, 1638.119, 8.576 ms
1.33.557.980 I slot      release: id  1 | task 522 | stop processing: n_tokens = 156, truncated = 0
1.33.641.688 I slot print_timing: id  0 | task 535 | prompt eval time =     115.77 ms /     4 tokens (   28.94 ms per token,    34.55 tokens per second)
1.33.641.690 I slot print_timing: id  0 | task 535 |        eval time =    3810.76 ms /   128 tokens (   29.77 ms per token,    33.59 tokens per second)
1.33.641.691 I slot print_timing: id  0 | task 535 |       total time =    3926.53 ms /   132 tokens
1.33.641.691 I slot print_timing: id  0 | task 535 |    graphs reused =          8
1.33.641.694 I slot print_timing: id  0 | task 535 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.641.707 I statistics        draft-mtp: #calls(b,g,a) =  128    475   6689, #gen drafts =   6689, #acc drafts =  4781, #gen tokens =  13335, #acc tokens =  8882, #mean acc len = 2.33, #acc rate/pos = (0.715, 0.613), dur(b,g,a) = 0.078, 1646.867, 8.585 ms
1.33.641.721 I slot      release: id  0 | task 535 | stop processing: n_tokens = 156, truncated = 0
1.33.641.731 I srv  update_slots: all slots are idle