0.00.139.954 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.139.957 I device_info:
0.00.218.632 I   - CUDA0   : NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (97244 MiB, 96677 MiB free)
0.00.218.646 I   - CPU     : AMD Ryzen Threadripper 9960X 24-Cores (257066 MiB, 257066 MiB free)
0.00.218.734 I system_info: n_threads = 24 (n_threads_batch = 24) / 48 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.218.816 I srv          init: using 47 threads for HTTP server
0.00.219.024 I srv         start: binding port with default address family
0.00.220.206 I srv  llama_server: loading model
0.00.220.243 I srv    load_model: loading model '/home/ripper/ornith-35b-lab/artifacts/quant/ornith-1.0-35b-IQ4_XS.gguf'
0.00.494.153 I srv    load_model: [spec] estimated memory usage of draft model is 3176.45 MiB
0.00.494.187 I common_init_result: fitting params to device memory ...
0.00.494.188 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.524.969 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.540.416 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.572.761 I srv    load_model: loading draft model '/home/ripper/ornith-35b-lab/artifacts/mtp/ornith-1.0-35b-mtp-chaincorr-h2r9fw125-h1r2-h3r6ce375-lr5e7-Q6_K.gguf'
0.02.918.176 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.927.045 I srv    load_model: initializing slots, n_slots = 16
0.02.935.360 I common_context_can_seq_rm: the context supports bounded partial sequence removal
0.02.942.047 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
0.02.942.051 I common_speculative_impl_draft_mtp: - n_max=2, n_min=0, p_min=0.00, n_embd=2048, backend_sampling=1
0.02.942.052 I common_speculative_impl_draft_mtp: - gpu_layers=99, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[default]
0.02.942.183 I common_speculative_impl_draft_mtp: - draft sampler top_k=1 top_p=1.000 temp=1.000
0.02.942.211 I common_speculative_impl_draft_mtp: - fast backend MTP sampling enabled
0.02.942.220 I common_speculative_impl_draft_mtp: - backend draft sampler top_k=1 top_p=1.000 temp=1.000 contexts=1
0.02.960.223 I srv    load_model: speculative decoding context initialized
0.02.960.225 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 512
0.02.960.235 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 512
0.02.960.235 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 512
0.02.960.235 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 512
0.02.960.236 I slot   load_model: id  4 | task -1 | new slot, n_ctx = 512
0.02.960.236 I slot   load_model: id  5 | task -1 | new slot, n_ctx = 512
0.02.960.236 I slot   load_model: id  6 | task -1 | new slot, n_ctx = 512
0.02.960.236 I slot   load_model: id  7 | task -1 | new slot, n_ctx = 512
0.02.960.237 I slot   load_model: id  8 | task -1 | new slot, n_ctx = 512
0.02.960.237 I slot   load_model: id  9 | task -1 | new slot, n_ctx = 512
0.02.960.237 I slot   load_model: id 10 | task -1 | new slot, n_ctx = 512
0.02.960.238 I slot   load_model: id 11 | task -1 | new slot, n_ctx = 512
0.02.960.238 I slot   load_model: id 12 | task -1 | new slot, n_ctx = 512
0.02.960.238 I slot   load_model: id 13 | task -1 | new slot, n_ctx = 512
0.02.960.238 I slot   load_model: id 14 | task -1 | new slot, n_ctx = 512
0.02.960.238 I slot   load_model: id 15 | task -1 | new slot, n_ctx = 512
0.02.960.285 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.02.960.286 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.02.960.286 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.02.960.287 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.02.960.303 I srv          init: idle slots will be saved to prompt cache upon starting a new task
0.02.968.969 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think>

</think>

Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>

</think>

'
0.02.975.308 I srv          init: init: chat template, thinking = 0
0.02.975.350 I srv  llama_server: model loaded
0.02.975.355 I srv  llama_server: server is listening on http://127.0.0.1:18237
0.02.975.361 I srv  update_slots: all slots are idle
0.38.980.864 I srv    operator(): Chat format: peg-native
0.38.980.872 I srv    operator(): Chat format: peg-native
0.38.980.877 I srv    operator(): Chat format: peg-native
0.38.980.884 I srv    operator(): Chat format: peg-native
0.38.980.888 I srv    operator(): Chat format: peg-native
0.38.980.891 I srv    operator(): Chat format: peg-native
0.38.980.986 I srv    operator(): Chat format: peg-native
0.38.981.023 I slot get_availabl: id 15 | task -1 | selected slot by LRU, t_last = -1
0.38.981.024 I srv  get_availabl: updating prompt cache
0.38.981.030 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.036 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.039 I srv  get_availabl: prompt cache update took 0.01 ms
0.38.981.084 I slot launch_slot_: id 15 | task 0 | processing task, is_child = 0
0.38.981.086 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.087 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.088 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.089 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.089 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.090 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.090 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.091 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.091 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.091 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.091 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.981.092 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.981.092 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.981.092 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.981.093 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.38.981.094 I slot get_availabl: id 14 | task -1 | selected slot by LRU, t_last = -1
0.38.981.095 I srv  get_availabl: updating prompt cache
0.38.981.095 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.096 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.097 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.112 I slot launch_slot_: id 14 | task 3 | processing task, is_child = 0
0.38.981.113 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.114 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.114 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.114 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.115 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.115 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.115 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.115 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.116 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.117 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.117 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.981.117 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.981.118 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.981.118 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.981.119 I slot get_availabl: id 13 | task -1 | selected slot by LRU, t_last = -1
0.38.981.119 I srv  get_availabl: updating prompt cache
0.38.981.120 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.121 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.121 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.141 I slot launch_slot_: id 13 | task 4 | processing task, is_child = 0
0.38.981.143 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.143 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.144 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.144 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.145 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.146 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.147 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.147 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.147 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.148 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.148 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.981.148 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.981.150 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.981.151 I slot get_availabl: id 12 | task -1 | selected slot by LRU, t_last = -1
0.38.981.151 I srv  get_availabl: updating prompt cache
0.38.981.152 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.153 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.153 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.167 I slot launch_slot_: id 12 | task 1 | processing task, is_child = 0
0.38.981.168 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.169 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.169 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.170 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.170 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.170 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.171 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.171 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.173 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.173 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.173 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.981.174 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.981.174 I slot get_availabl: id 11 | task -1 | selected slot by LRU, t_last = -1
0.38.981.174 I srv  get_availabl: updating prompt cache
0.38.981.175 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.175 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.175 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.188 I slot launch_slot_: id 11 | task 5 | processing task, is_child = 0
0.38.981.188 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.189 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.189 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.190 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.190 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.192 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.192 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.192 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.193 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.194 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.195 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.981.196 I slot get_availabl: id 10 | task -1 | selected slot by LRU, t_last = -1
0.38.981.196 I srv  get_availabl: updating prompt cache
0.38.981.196 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.197 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.197 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.210 I slot launch_slot_: id 10 | task 2 | processing task, is_child = 0
0.38.981.212 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.212 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.213 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.214 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.214 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.215 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.215 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.215 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.216 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.217 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.38.981.218 I slot get_availabl: id  9 | task -1 | selected slot by LRU, t_last = -1
0.38.981.218 I srv  get_availabl: updating prompt cache
0.38.981.218 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.981.219 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.981.219 I srv  get_availabl: prompt cache update took 0.00 ms
0.38.981.232 I slot launch_slot_: id  9 | task 6 | processing task, is_child = 0
0.38.981.234 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.38.981.234 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.38.981.235 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.38.981.235 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.38.981.237 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.38.981.237 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.38.981.237 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.38.981.238 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.38.981.238 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.38.981.277 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.982.150 I srv    operator(): Chat format: peg-native
0.38.983.637 I srv    operator(): Chat format: peg-native
0.38.984.176 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.178 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.185 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.188 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.191 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.192 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.984.515 I srv    operator(): Chat format: peg-native
0.38.984.918 I srv    operator(): Chat format: peg-native
0.38.986.480 I srv    operator(): Chat format: peg-native
0.38.986.856 I srv    operator(): Chat format: peg-native
0.38.987.009 I srv    operator(): Chat format: peg-native
0.38.987.595 I srv    operator(): Chat format: peg-native
0.38.988.784 I srv    operator(): Chat format: peg-native
0.39.203.999 I slot get_availabl: id  8 | task -1 | selected slot by LRU, t_last = -1
0.39.204.003 I srv  get_availabl: updating prompt cache
0.39.204.007 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.010 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.011 I srv  get_availabl: prompt cache update took 0.01 ms
0.39.204.077 I slot launch_slot_: id  8 | task 8 | processing task, is_child = 0
0.39.204.079 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.079 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.079 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.079 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.204.079 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.204.080 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.204.080 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.39.204.080 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.39.204.081 I slot get_availabl: id  7 | task -1 | selected slot by LRU, t_last = -1
0.39.204.081 I srv  get_availabl: updating prompt cache
0.39.204.082 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.082 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.082 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.096 I slot launch_slot_: id  7 | task 10 | processing task, is_child = 0
0.39.204.096 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.096 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.096 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.097 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.204.097 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.204.097 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.204.097 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.39.204.099 I slot get_availabl: id  6 | task -1 | selected slot by LRU, t_last = -1
0.39.204.099 I srv  get_availabl: updating prompt cache
0.39.204.099 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.100 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.100 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.112 I slot launch_slot_: id  6 | task 9 | processing task, is_child = 0
0.39.204.113 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.113 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.114 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.114 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.204.114 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.204.114 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.39.204.116 I slot get_availabl: id  5 | task -1 | selected slot by LRU, t_last = -1
0.39.204.116 I srv  get_availabl: updating prompt cache
0.39.204.116 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.117 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.117 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.127 I slot launch_slot_: id  5 | task 11 | processing task, is_child = 0
0.39.204.127 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.128 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.129 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.129 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.204.130 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.39.204.131 I slot get_availabl: id  4 | task -1 | selected slot by LRU, t_last = -1
0.39.204.131 I srv  get_availabl: updating prompt cache
0.39.204.132 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.132 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.133 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.148 I slot launch_slot_: id  4 | task 12 | processing task, is_child = 0
0.39.204.150 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.150 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.150 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.150 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.39.204.152 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.39.204.152 I srv  get_availabl: updating prompt cache
0.39.204.152 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.152 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.153 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.163 I slot launch_slot_: id  3 | task 13 | processing task, is_child = 0
0.39.204.164 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.165 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.165 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.39.204.166 I slot get_availabl: id  2 | task -1 | selected slot by LRU, t_last = -1
0.39.204.166 I srv  get_availabl: updating prompt cache
0.39.204.167 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.167 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.167 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.178 I slot launch_slot_: id  2 | task 14 | processing task, is_child = 0
0.39.204.178 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.179 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.39.204.180 I slot get_availabl: id  1 | task -1 | selected slot by LRU, t_last = -1
0.39.204.180 I srv  get_availabl: updating prompt cache
0.39.204.180 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.181 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.181 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.191 I slot launch_slot_: id  1 | task 15 | processing task, is_child = 0
0.39.204.192 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.39.204.193 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
0.39.204.194 I srv  get_availabl: updating prompt cache
0.39.204.194 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.39.204.194 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.39.204.194 I srv  get_availabl: prompt cache update took 0.00 ms
0.39.204.206 I slot launch_slot_: id  0 | task 16 | processing task, is_child = 0
0.39.204.326 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.391 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.426 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.432 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.433 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.438 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.445 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.446 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.204.454 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.39.218.499 I slot create_check: id  9 | task 6 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.232.074 I slot create_check: id 10 | task 2 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.245.860 I slot create_check: id 11 | task 5 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.259.464 I slot create_check: id 12 | task 1 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.272.980 I slot create_check: id 13 | task 4 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.286.519 I slot create_check: id 14 | task 3 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.299.911 I slot create_check: id 15 | task 0 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.549.793 I slot create_check: id  0 | task 16 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.564.355 I slot create_check: id  1 | task 15 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.579.092 I slot create_check: id  2 | task 14 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.593.062 I slot create_check: id  3 | task 13 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.607.132 I slot create_check: id  4 | task 12 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.621.074 I slot create_check: id  5 | task 11 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.635.269 I slot create_check: id  6 | task 9 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.649.581 I slot create_check: id  7 | task 10 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.663.947 I slot create_check: id  8 | task 8 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.953.582 I slot print_timing: id 10 | task 2 | n_decoded =    102, tg =  23.02 t/s, tg_3s =  23.02 t/s
0.43.954.222 I slot print_timing: id 11 | task 5 | n_decoded =    102, tg =  23.02 t/s, tg_3s =  23.02 t/s
0.43.955.116 I slot print_timing: id 13 | task 4 | n_decoded =    102, tg =  23.03 t/s, tg_3s =  23.03 t/s
0.44.069.788 I slot print_timing: id  0 | task 16 | n_decoded =    102, tg =  23.87 t/s, tg_3s =  23.87 t/s
0.44.071.948 I slot print_timing: id  4 | task 12 | n_decoded =    102, tg =  23.87 t/s, tg_3s =  23.87 t/s
0.44.073.142 I slot print_timing: id  7 | task 10 | n_decoded =    102, tg =  23.88 t/s, tg_3s =  23.88 t/s
0.44.441.502 I slot print_timing: id  9 | task 6 | n_decoded =    101, tg =  20.53 t/s, tg_3s =  20.53 t/s
0.44.444.054 I slot print_timing: id 14 | task 3 | n_decoded =    101, tg =  20.54 t/s, tg_3s =  20.54 t/s
0.44.559.178 I slot print_timing: id  1 | task 15 | n_decoded =    101, tg =  21.21 t/s, tg_3s =  21.21 t/s
0.44.560.060 I slot print_timing: id  3 | task 13 | n_decoded =    101, tg =  21.21 t/s, tg_3s =  21.21 t/s
0.44.562.792 I slot print_timing: id  8 | task 8 | n_decoded =    101, tg =  21.21 t/s, tg_3s =  21.21 t/s
0.44.563.985 I slot print_timing: id 10 | task 2 | prompt eval time =     542.25 ms /    25 tokens (   21.69 ms per token,    46.10 tokens per second)
0.44.563.987 I slot print_timing: id 10 | task 2 |        eval time =    5040.43 ms /   114 tokens (   44.21 ms per token,    22.62 tokens per second)
0.44.563.988 I slot print_timing: id 10 | task 2 |       total time =    5582.68 ms /   139 tokens
0.44.563.994 I slot print_timing: id 10 | task 2 |    graphs reused =          1
0.44.563.999 I slot print_timing: id 10 | task 2 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.564.020 I statistics        draft-mtp: #calls(b,g,a) =   16     40    626, #gen drafts =    631, #acc drafts =   492, #gen tokens =   1262, #acc tokens =   918, #mean acc len = 2.47, #acc rate/pos = (0.786, 0.681), dur(b,g,a) = 0.015, 117.298, 0.813 ms
0.44.564.059 I slot      release: id 10 | task 2 | stop processing: n_tokens = 140, truncated = 0
0.44.564.658 I slot print_timing: id 11 | task 5 | prompt eval time =     540.00 ms /    25 tokens (   21.60 ms per token,    46.30 tokens per second)
0.44.564.662 I slot print_timing: id 11 | task 5 |        eval time =    5040.49 ms /   114 tokens (   44.21 ms per token,    22.62 tokens per second)
0.44.564.662 I slot print_timing: id 11 | task 5 |       total time =    5580.48 ms /   139 tokens
0.44.564.663 I slot print_timing: id 11 | task 5 |    graphs reused =          1
0.44.564.666 I slot print_timing: id 11 | task 5 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.564.673 I statistics        draft-mtp: #calls(b,g,a) =   16     40    627, #gen drafts =    631, #acc drafts =   493, #gen tokens =   1262, #acc tokens =   920, #mean acc len = 2.47, #acc rate/pos = (0.786, 0.681), dur(b,g,a) = 0.015, 117.298, 0.814 ms
0.44.564.695 I slot      release: id 11 | task 5 | stop processing: n_tokens = 140, truncated = 0
0.44.565.585 I slot print_timing: id 13 | task 4 | prompt eval time =     541.47 ms /    25 tokens (   21.66 ms per token,    46.17 tokens per second)
0.44.565.588 I slot print_timing: id 13 | task 4 |        eval time =    5039.92 ms /   114 tokens (   44.21 ms per token,    22.62 tokens per second)
0.44.565.589 I slot print_timing: id 13 | task 4 |       total time =    5581.40 ms /   139 tokens
0.44.565.589 I slot print_timing: id 13 | task 4 |    graphs reused =          1
0.44.565.590 I slot print_timing: id 13 | task 4 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.565.596 I statistics        draft-mtp: #calls(b,g,a) =   16     40    629, #gen drafts =    631, #acc drafts =   494, #gen tokens =   1262, #acc tokens =   922, #mean acc len = 2.47, #acc rate/pos = (0.785, 0.680), dur(b,g,a) = 0.015, 117.298, 0.815 ms
0.44.565.616 I slot      release: id 13 | task 4 | stop processing: n_tokens = 140, truncated = 0
0.44.573.872 I srv    operator(): Chat format: peg-native
0.44.574.065 I srv    operator(): Chat format: peg-native
0.44.574.720 I srv    operator(): Chat format: peg-native
0.44.665.185 I slot print_timing: id  0 | task 16 | prompt eval time =     592.39 ms /    25 tokens (   23.70 ms per token,    42.20 tokens per second)
0.44.665.189 I slot print_timing: id  0 | task 16 |        eval time =    4868.54 ms /   114 tokens (   42.71 ms per token,    23.42 tokens per second)
0.44.665.190 I slot print_timing: id  0 | task 16 |       total time =    5460.93 ms /   139 tokens
0.44.665.192 I slot print_timing: id  0 | task 16 |    graphs reused =          1
0.44.665.196 I slot print_timing: id  0 | task 16 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.665.215 I statistics        draft-mtp: #calls(b,g,a) =   16     41    632, #gen drafts =    644, #acc drafts =   496, #gen tokens =   1288, #acc tokens =   926, #mean acc len = 2.47, #acc rate/pos = (0.785, 0.680), dur(b,g,a) = 0.015, 123.544, 0.821 ms
0.44.665.249 I slot      release: id  0 | task 16 | stop processing: n_tokens = 140, truncated = 0
0.44.667.317 I slot print_timing: id  4 | task 12 | prompt eval time =     594.67 ms /    25 tokens (   23.79 ms per token,    42.04 tokens per second)
0.44.667.321 I slot print_timing: id  4 | task 12 |        eval time =    4868.21 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
0.44.667.321 I slot print_timing: id  4 | task 12 |       total time =    5462.89 ms /   139 tokens
0.44.667.322 I slot print_timing: id  4 | task 12 |    graphs reused =          1
0.44.667.324 I slot print_timing: id  4 | task 12 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.667.333 I statistics        draft-mtp: #calls(b,g,a) =   16     41    636, #gen drafts =    644, #acc drafts =   499, #gen tokens =   1288, #acc tokens =   932, #mean acc len = 2.47, #acc rate/pos = (0.785, 0.681), dur(b,g,a) = 0.015, 123.544, 0.827 ms
0.44.667.358 I slot      release: id  4 | task 12 | stop processing: n_tokens = 140, truncated = 0
0.44.668.517 I slot print_timing: id  7 | task 10 | prompt eval time =     596.52 ms /    25 tokens (   23.86 ms per token,    41.91 tokens per second)
0.44.668.520 I slot print_timing: id  7 | task 10 |        eval time =    4867.55 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
0.44.668.521 I slot print_timing: id  7 | task 10 |       total time =    5464.07 ms /   139 tokens
0.44.668.521 I slot print_timing: id  7 | task 10 |    graphs reused =          1
0.44.668.523 I slot print_timing: id  7 | task 10 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.44.668.530 I statistics        draft-mtp: #calls(b,g,a) =   16     41    639, #gen drafts =    644, #acc drafts =   500, #gen tokens =   1288, #acc tokens =   934, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.679), dur(b,g,a) = 0.015, 123.544, 0.832 ms
0.44.668.553 I slot      release: id  7 | task 10 | stop processing: n_tokens = 140, truncated = 0
0.44.670.278 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.670.281 I srv  get_availabl: updating prompt cache
0.44.670.328 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.673.825 I srv    operator(): Chat format: peg-native
0.44.675.263 I srv    operator(): Chat format: peg-native
0.44.676.680 I srv    operator(): Chat format: peg-native
0.44.698.467 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.698.473 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.698.473 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.698.475 I srv  get_availabl: prompt cache update took 28.19 ms
0.44.698.559 I slot launch_slot_: id  0 | task 59 | processing task, is_child = 0
0.44.698.561 I slot process_sing: id  4 | task -1 | saving idle slot to prompt cache
0.44.698.589 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.592 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.592 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.44.698.609 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.611 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.611 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.698.627 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.629 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.630 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.698.645 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.646 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.647 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.44.698.662 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.664 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.668 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.698.668 I srv  get_availabl: updating prompt cache
0.44.698.682 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.684 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.684 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.698.685 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.698.686 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.698.686 I srv  get_availabl: prompt cache update took 0.02 ms
0.44.698.707 I slot launch_slot_: id  4 | task 60 | processing task, is_child = 0
0.44.698.708 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
0.44.698.722 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.723 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.723 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.698.737 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.738 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.738 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.698.761 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.762 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.763 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.44.698.784 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.785 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.787 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.698.787 I srv  get_availabl: updating prompt cache
0.44.698.805 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.806 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.807 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.698.807 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.698.807 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.698.809 I srv  get_availabl: prompt cache update took 0.02 ms
0.44.698.827 I slot launch_slot_: id  7 | task 61 | processing task, is_child = 0
0.44.698.829 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.44.698.852 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.853 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.853 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.698.872 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.874 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.874 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.44.698.895 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.896 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.898 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.698.899 I srv  get_availabl: updating prompt cache
0.44.698.919 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.921 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.921 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.698.922 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.698.924 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.698.925 I srv  get_availabl: prompt cache update took 0.03 ms
0.44.698.943 I slot launch_slot_: id 10 | task 62 | processing task, is_child = 0
0.44.698.945 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.698.960 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.961 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.961 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.44.698.977 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.979 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.981 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.698.981 I srv  get_availabl: updating prompt cache
0.44.698.995 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.698.996 I srv         alloc:  - prompt is already in the cache, skipping
0.44.698.997 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.698.997 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.698.998 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.698.999 I srv  get_availabl: prompt cache update took 0.02 ms
0.44.699.015 I slot launch_slot_: id 11 | task 63 | processing task, is_child = 0
0.44.699.017 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.44.699.031 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.699.032 I srv         alloc:  - prompt is already in the cache, skipping
0.44.699.034 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.699.034 I srv  get_availabl: updating prompt cache
0.44.699.056 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.699.057 I srv         alloc:  - prompt is already in the cache, skipping
0.44.699.057 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.699.058 I srv        update:  - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.699.061 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.44.699.061 I srv  get_availabl: prompt cache update took 0.03 ms
0.44.699.077 I slot launch_slot_: id 13 | task 64 | processing task, is_child = 0
0.44.705.085 I slot   operator(): id  0 | task 59 | Checking checkpoint with [20, 20] against 3...
0.44.705.088 W slot   operator(): id  0 | task 59 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.705.092 W slot   operator(): id  0 | task 59 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.706.716 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.706.736 I slot   operator(): id  4 | task 60 | Checking checkpoint with [20, 20] against 3...
0.44.706.738 W slot   operator(): id  4 | task 60 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.706.742 W slot   operator(): id  4 | task 60 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.708.210 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.708.235 I slot   operator(): id  7 | task 61 | Checking checkpoint with [20, 20] against 24...
0.44.711.929 W slot   operator(): id  7 | task 61 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.711.974 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.711.979 I slot   operator(): id 10 | task 62 | Checking checkpoint with [20, 20] against 3...
0.44.711.980 W slot   operator(): id 10 | task 62 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.711.980 W slot   operator(): id 10 | task 62 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.714.119 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.714.152 I slot   operator(): id 11 | task 63 | Checking checkpoint with [20, 20] against 3...
0.44.714.154 W slot   operator(): id 11 | task 63 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.714.158 W slot   operator(): id 11 | task 63 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.716.269 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.716.292 I slot   operator(): id 13 | task 64 | Checking checkpoint with [20, 20] against 24...
0.44.719.873 W slot   operator(): id 13 | task 64 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.719.913 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.908.496 I slot create_check: id  0 | task 59 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.922.117 I slot create_check: id  4 | task 60 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.44.936.446 I slot create_check: id 10 | task 62 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.949.957 I slot create_check: id 11 | task 63 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.45.678.275 I slot print_timing: id  9 | task 6 | prompt eval time =     541.56 ms /    28 tokens (   19.34 ms per token,    51.70 tokens per second)
0.45.678.279 I slot print_timing: id  9 | task 6 |        eval time =    6155.41 ms /   125 tokens (   49.24 ms per token,    20.31 tokens per second)
0.45.678.280 I slot print_timing: id  9 | task 6 |       total time =    6696.97 ms /   153 tokens
0.45.678.281 I slot print_timing: id  9 | task 6 |    graphs reused =          1
0.45.678.284 I slot print_timing: id  9 | task 6 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.678.297 I statistics        draft-mtp: #calls(b,g,a) =   22     48    740, #gen drafts =    746, #acc drafts =   567, #gen tokens =   1492, #acc tokens =  1059, #mean acc len = 2.43, #acc rate/pos = (0.766, 0.665), dur(b,g,a) = 0.017, 149.656, 0.971 ms
0.45.678.327 I slot      release: id  9 | task 6 | stop processing: n_tokens = 152, truncated = 0
0.45.681.050 I slot print_timing: id 14 | task 3 | prompt eval time =     542.13 ms /    28 tokens (   19.36 ms per token,    51.65 tokens per second)
0.45.681.053 I slot print_timing: id 14 | task 3 |        eval time =    6154.71 ms /   125 tokens (   49.24 ms per token,    20.31 tokens per second)
0.45.681.053 I slot print_timing: id 14 | task 3 |       total time =    6696.84 ms /   153 tokens
0.45.681.054 I slot print_timing: id 14 | task 3 |    graphs reused =          1
0.45.681.056 I slot print_timing: id 14 | task 3 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.681.063 I statistics        draft-mtp: #calls(b,g,a) =   22     48    745, #gen drafts =    746, #acc drafts =   571, #gen tokens =   1492, #acc tokens =  1067, #mean acc len = 2.43, #acc rate/pos = (0.766, 0.666), dur(b,g,a) = 0.017, 149.656, 0.976 ms
0.45.681.084 I slot      release: id 14 | task 3 | stop processing: n_tokens = 152, truncated = 0
0.45.686.465 I srv    operator(): Chat format: peg-native
0.45.690.233 I srv    operator(): Chat format: peg-native
0.45.785.569 I slot print_timing: id  1 | task 15 | prompt eval time =     592.99 ms /    28 tokens (   21.18 ms per token,    47.22 tokens per second)
0.45.785.572 I slot print_timing: id  1 | task 15 |        eval time =    5988.28 ms /   125 tokens (   47.91 ms per token,    20.87 tokens per second)
0.45.785.573 I slot print_timing: id  1 | task 15 |       total time =    6581.28 ms /   153 tokens
0.45.785.574 I slot print_timing: id  1 | task 15 |    graphs reused =          1
0.45.785.577 I slot print_timing: id  1 | task 15 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.785.593 I statistics        draft-mtp: #calls(b,g,a) =   22     49    748, #gen drafts =    760, #acc drafts =   572, #gen tokens =   1520, #acc tokens =  1069, #mean acc len = 2.43, #acc rate/pos = (0.765, 0.664), dur(b,g,a) = 0.017, 154.585, 0.981 ms
0.45.785.623 I slot      release: id  1 | task 15 | stop processing: n_tokens = 152, truncated = 0
0.45.786.853 I slot print_timing: id  3 | task 13 | prompt eval time =     594.07 ms /    28 tokens (   21.22 ms per token,    47.13 tokens per second)
0.45.786.856 I slot print_timing: id  3 | task 13 |        eval time =    5988.37 ms /   125 tokens (   47.91 ms per token,    20.87 tokens per second)
0.45.786.857 I slot print_timing: id  3 | task 13 |       total time =    6582.44 ms /   153 tokens
0.45.786.857 I slot print_timing: id  3 | task 13 |    graphs reused =          1
0.45.786.859 I slot print_timing: id  3 | task 13 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.786.866 I statistics        draft-mtp: #calls(b,g,a) =   22     49    750, #gen drafts =    760, #acc drafts =   574, #gen tokens =   1520, #acc tokens =  1073, #mean acc len = 2.43, #acc rate/pos = (0.765, 0.665), dur(b,g,a) = 0.017, 154.585, 0.985 ms
0.45.786.892 I slot      release: id  3 | task 13 | stop processing: n_tokens = 152, truncated = 0
0.45.789.291 I slot print_timing: id  8 | task 8 | prompt eval time =     597.12 ms /    28 tokens (   21.33 ms per token,    46.89 tokens per second)
0.45.789.295 I slot print_timing: id  8 | task 8 |        eval time =    5987.70 ms /   125 tokens (   47.90 ms per token,    20.88 tokens per second)
0.45.789.296 I slot print_timing: id  8 | task 8 |       total time =    6584.82 ms /   153 tokens
0.45.789.297 I slot print_timing: id  8 | task 8 |    graphs reused =          1
0.45.789.299 I slot print_timing: id  8 | task 8 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.45.789.305 I statistics        draft-mtp: #calls(b,g,a) =   22     49    755, #gen drafts =    760, #acc drafts =   577, #gen tokens =   1520, #acc tokens =  1079, #mean acc len = 2.43, #acc rate/pos = (0.764, 0.665), dur(b,g,a) = 0.017, 154.585, 0.992 ms
0.45.789.332 I slot      release: id  8 | task 8 | stop processing: n_tokens = 152, truncated = 0
0.45.792.001 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.45.792.003 I srv  get_availabl: updating prompt cache
0.45.792.044 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.793.968 I srv    operator(): Chat format: peg-native
0.45.794.814 I srv    operator(): Chat format: peg-native
0.45.797.661 I srv    operator(): Chat format: peg-native
0.45.819.841 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.45.819.847 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.819.847 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.819.848 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.819.848 I srv  get_availabl: prompt cache update took 27.84 ms
0.45.819.938 I slot launch_slot_: id  1 | task 73 | processing task, is_child = 0
0.45.819.940 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
0.45.819.964 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.819.969 I srv         alloc:  - prompt is already in the cache, skipping
0.45.819.971 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.45.819.997 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.819.999 I srv         alloc:  - prompt is already in the cache, skipping
0.45.819.999 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.45.820.023 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.025 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.026 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.45.820.051 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.053 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.058 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.45.820.058 I srv  get_availabl: updating prompt cache
0.45.820.082 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.084 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.084 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.45.820.086 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.820.087 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.820.089 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.820.089 I srv  get_availabl: prompt cache update took 0.03 ms
0.45.820.121 I slot launch_slot_: id  3 | task 74 | processing task, is_child = 0
0.45.820.122 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.45.820.151 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.152 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.152 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.45.820.175 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.177 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.177 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.45.820.242 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.245 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.253 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
0.45.820.255 I srv  get_availabl: updating prompt cache
0.45.820.289 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.290 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.291 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.120
0.45.820.294 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.820.295 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.820.296 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.820.297 I srv  get_availabl: prompt cache update took 0.04 ms
0.45.820.344 I slot launch_slot_: id  8 | task 75 | processing task, is_child = 0
0.45.820.345 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.45.820.364 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.365 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.365 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.45.820.382 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.383 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.384 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.45.820.384 I srv  get_availabl: updating prompt cache
0.45.820.400 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.401 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.401 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.45.820.402 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.820.403 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.820.403 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.820.404 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.820.422 I slot launch_slot_: id  9 | task 76 | processing task, is_child = 0
0.45.820.423 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.45.820.439 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.440 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.441 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.45.820.441 I srv  get_availabl: updating prompt cache
0.45.820.458 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.45.820.458 I srv         alloc:  - prompt is already in the cache, skipping
0.45.820.459 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.45.820.459 I srv        update:  - cache state: 2 prompts, 259.602 MiB (limits: 8192.000 MiB, 8192 tokens, 9214 est)
0.45.820.460 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.45.820.460 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.45.820.460 I srv  get_availabl: prompt cache update took 0.02 ms
0.45.820.476 I slot launch_slot_: id 14 | task 77 | processing task, is_child = 0
0.45.826.935 I slot   operator(): id  1 | task 73 | Checking checkpoint with [23, 23] against 3...
0.45.826.937 W slot   operator(): id  1 | task 73 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.826.939 W slot   operator(): id  1 | task 73 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.828.679 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.828.705 I slot   operator(): id  3 | task 74 | Checking checkpoint with [23, 23] against 27...
0.45.832.364 W slot   operator(): id  3 | task 74 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.45.832.409 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.832.429 I slot   operator(): id  8 | task 75 | Checking checkpoint with [23, 23] against 3...
0.45.832.430 W slot   operator(): id  8 | task 75 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.832.430 W slot   operator(): id  8 | task 75 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.834.526 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.834.556 I slot   operator(): id  9 | task 76 | Checking checkpoint with [23, 23] against 3...
0.45.834.556 W slot   operator(): id  9 | task 76 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.45.834.557 W slot   operator(): id  9 | task 76 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.45.836.981 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.45.836.998 I slot   operator(): id 14 | task 77 | Checking checkpoint with [23, 23] against 27...
0.45.840.484 W slot   operator(): id 14 | task 77 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.45.840.529 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.019.087 I slot create_check: id  1 | task 73 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.46.033.370 I slot create_check: id  8 | task 75 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.46.047.207 I slot create_check: id  9 | task 76 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.46.527.311 I slot print_timing: id  2 | task 14 | n_decoded =    100, tg =  14.86 t/s, tg_3s =  14.86 t/s
0.46.532.049 I slot print_timing: id 12 | task 1 | n_decoded =    100, tg =  14.27 t/s, tg_3s =  14.27 t/s
0.46.533.853 I slot print_timing: id 15 | task 0 | n_decoded =    100, tg =  14.27 t/s, tg_3s =  14.27 t/s
0.46.650.040 I slot print_timing: id  5 | task 11 | n_decoded =    100, tg =  14.60 t/s, tg_3s =  14.60 t/s
0.46.650.651 I slot print_timing: id  6 | task 9 | n_decoded =    100, tg =  14.60 t/s, tg_3s =  14.60 t/s
0.48.336.601 I slot print_timing: id  2 | task 14 | prompt eval time =     593.49 ms /    29 tokens (   20.47 ms per token,    48.86 tokens per second)
0.48.336.605 I slot print_timing: id  2 | task 14 |        eval time =    8538.72 ms /   128 tokens (   66.71 ms per token,    14.99 tokens per second)
0.48.336.606 I slot print_timing: id  2 | task 14 |       total time =    9132.20 ms /   157 tokens
0.48.336.607 I slot print_timing: id  2 | task 14 |    graphs reused =          1
0.48.336.610 I slot print_timing: id  2 | task 14 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
0.48.336.623 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1056, #gen drafts =   1069, #acc drafts =   786, #gen tokens =   2133, #acc tokens =  1469, #mean acc len = 2.39, #acc rate/pos = (0.744, 0.647), dur(b,g,a) = 0.018, 218.376, 1.397 ms
0.48.336.654 I slot      release: id  2 | task 14 | stop processing: n_tokens = 156, truncated = 0
0.48.336.863 I slot print_timing: id 12 | task 1 | prompt eval time =     540.64 ms /    29 tokens (   18.64 ms per token,    53.64 tokens per second)
0.48.336.866 I slot print_timing: id 12 | task 1 |        eval time =    8812.03 ms /   128 tokens (   68.84 ms per token,    14.53 tokens per second)
0.48.336.866 I slot print_timing: id 12 | task 1 |       total time =    9352.68 ms /   157 tokens
0.48.336.866 I slot print_timing: id 12 | task 1 |    graphs reused =          1
0.48.336.868 I slot print_timing: id 12 | task 1 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.336.872 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1056, #gen drafts =   1069, #acc drafts =   786, #gen tokens =   2133, #acc tokens =  1469, #mean acc len = 2.39, #acc rate/pos = (0.744, 0.647), dur(b,g,a) = 0.018, 218.376, 1.397 ms
0.48.336.894 I slot      release: id 12 | task 1 | stop processing: n_tokens = 156, truncated = 0
0.48.337.094 I slot print_timing: id 15 | task 0 | prompt eval time =     542.77 ms /    29 tokens (   18.72 ms per token,    53.43 tokens per second)
0.48.337.096 I slot print_timing: id 15 | task 0 |        eval time =    8810.12 ms /   128 tokens (   68.83 ms per token,    14.53 tokens per second)
0.48.337.097 I slot print_timing: id 15 | task 0 |       total time =    9352.89 ms /   157 tokens
0.48.337.097 I slot print_timing: id 15 | task 0 |    graphs reused =          1
0.48.337.098 I slot print_timing: id 15 | task 0 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.337.101 I statistics        draft-mtp: #calls(b,g,a) =   27     69   1056, #gen drafts =   1069, #acc drafts =   786, #gen tokens =   2133, #acc tokens =  1469, #mean acc len = 2.39, #acc rate/pos = (0.744, 0.647), dur(b,g,a) = 0.018, 218.376, 1.397 ms
0.48.337.123 I slot      release: id 15 | task 0 | stop processing: n_tokens = 156, truncated = 0
0.48.347.397 I srv    operator(): Chat format: peg-native
0.48.347.403 I srv    operator(): Chat format: peg-native
0.48.347.655 I srv    operator(): Chat format: peg-native
0.48.435.231 I slot print_timing: id  5 | task 11 | prompt eval time =     595.29 ms /    29 tokens (   20.53 ms per token,    48.72 tokens per second)
0.48.435.238 I slot print_timing: id  5 | task 11 |        eval time =    8635.49 ms /   128 tokens (   67.46 ms per token,    14.82 tokens per second)
0.48.435.239 I slot print_timing: id  5 | task 11 |       total time =    9230.78 ms /   157 tokens
0.48.435.240 I slot print_timing: id  5 | task 11 |    graphs reused =          1
0.48.435.244 I slot print_timing: id  5 | task 11 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.435.259 I statistics        draft-mtp: #calls(b,g,a) =   27     70   1069, #gen drafts =   1080, #acc drafts =   793, #gen tokens =   2155, #acc tokens =  1483, #mean acc len = 2.39, #acc rate/pos = (0.742, 0.645), dur(b,g,a) = 0.018, 223.699, 1.414 ms
0.48.435.289 I slot      release: id  5 | task 11 | stop processing: n_tokens = 156, truncated = 0
0.48.435.527 I slot print_timing: id  6 | task 9 | prompt eval time =     595.89 ms /    29 tokens (   20.55 ms per token,    48.67 tokens per second)
0.48.435.529 I slot print_timing: id  6 | task 9 |        eval time =    8635.20 ms /   128 tokens (   67.46 ms per token,    14.82 tokens per second)
0.48.435.530 I slot print_timing: id  6 | task 9 |       total time =    9231.09 ms /   157 tokens
0.48.435.530 I slot print_timing: id  6 | task 9 |    graphs reused =          1
0.48.435.532 I slot print_timing: id  6 | task 9 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.48.435.536 I statistics        draft-mtp: #calls(b,g,a) =   27     70   1069, #gen drafts =   1080, #acc drafts =   793, #gen tokens =   2155, #acc tokens =  1483, #mean acc len = 2.39, #acc rate/pos = (0.742, 0.645), dur(b,g,a) = 0.018, 223.699, 1.414 ms
0.48.435.553 I slot      release: id  6 | task 9 | stop processing: n_tokens = 156, truncated = 0
0.48.440.635 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.48.440.637 I srv  get_availabl: updating prompt cache
0.48.440.679 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.444.183 I srv    operator(): Chat format: peg-native
0.48.444.316 I srv    operator(): Chat format: peg-native
0.48.468.910 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.48.468.915 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.468.916 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.468.916 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.468.917 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.468.917 I srv  get_availabl: prompt cache update took 28.28 ms
0.48.468.998 I slot launch_slot_: id  2 | task 99 | processing task, is_child = 0
0.48.469.000 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.48.469.024 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.026 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.027 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.48.469.044 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.045 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.045 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.48.469.062 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.064 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.064 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.48.469.080 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.081 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.085 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.48.469.086 I srv  get_availabl: updating prompt cache
0.48.469.101 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.102 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.102 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.48.469.103 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.469.104 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.469.104 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.469.105 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.469.106 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.469.124 I slot launch_slot_: id  5 | task 100 | processing task, is_child = 0
0.48.469.126 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.48.469.145 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.146 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.147 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.48.469.161 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.162 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.162 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.48.469.176 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.177 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.179 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.48.469.180 I srv  get_availabl: updating prompt cache
0.48.469.194 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.195 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.195 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.48.469.196 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.469.196 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.469.196 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.469.198 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.469.198 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.469.213 I slot launch_slot_: id  6 | task 101 | processing task, is_child = 0
0.48.469.214 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.48.469.228 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.229 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.229 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.48.469.243 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.245 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.246 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.48.469.246 I srv  get_availabl: updating prompt cache
0.48.469.262 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.263 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.264 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.48.469.264 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.469.264 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.469.266 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.469.266 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.469.266 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.469.282 I slot launch_slot_: id 12 | task 102 | processing task, is_child = 0
0.48.469.283 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.48.469.297 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.298 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.300 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.48.469.301 I srv  get_availabl: updating prompt cache
0.48.469.315 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.48.469.316 I srv         alloc:  - prompt is already in the cache, skipping
0.48.469.316 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.48.469.317 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.48.469.317 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.48.469.318 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.48.469.319 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.48.469.319 I srv  get_availabl: prompt cache update took 0.02 ms
0.48.469.334 I slot launch_slot_: id 15 | task 103 | processing task, is_child = 0
0.48.474.681 I slot   operator(): id  2 | task 99 | Checking checkpoint with [24, 24] against 28...
0.48.478.270 W slot   operator(): id  2 | task 99 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.48.478.298 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.478.320 I slot   operator(): id  5 | task 100 | Checking checkpoint with [24, 24] against 3...
0.48.478.322 W slot   operator(): id  5 | task 100 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.478.322 W slot   operator(): id  5 | task 100 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.479.992 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.480.028 I slot   operator(): id  6 | task 101 | Checking checkpoint with [24, 24] against 3...
0.48.480.029 W slot   operator(): id  6 | task 101 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.480.032 W slot   operator(): id  6 | task 101 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.481.599 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.481.611 I slot   operator(): id 12 | task 102 | Checking checkpoint with [24, 24] against 3...
0.48.481.615 W slot   operator(): id 12 | task 102 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.481.617 W slot   operator(): id 12 | task 102 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.48.483.073 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.483.100 I slot   operator(): id 15 | task 103 | Checking checkpoint with [24, 24] against 28...
0.48.486.767 W slot   operator(): id 15 | task 103 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.48.486.793 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.659.588 I slot create_check: id  5 | task 100 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.48.672.736 I slot create_check: id  6 | task 101 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.48.685.995 I slot create_check: id 12 | task 102 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.49.412.228 I slot print_timing: id  7 | task 61 | n_decoded =    102, tg =  22.53 t/s, tg_3s =  22.53 t/s
0.49.415.315 I slot print_timing: id 13 | task 64 | n_decoded =    102, tg =  22.51 t/s, tg_3s =  22.51 t/s
0.50.017.173 I slot print_timing: id  4 | task 60 | n_decoded =    101, tg =  20.40 t/s, tg_3s =  20.40 t/s
0.50.018.975 I slot print_timing: id  7 | task 61 | prompt eval time =     175.94 ms /     4 tokens (   43.99 ms per token,    22.73 tokens per second)
0.50.018.977 I slot print_timing: id  7 | task 61 |        eval time =    5134.78 ms /   114 tokens (   45.04 ms per token,    22.20 tokens per second)
0.50.018.977 I slot print_timing: id  7 | task 61 |       total time =    5310.72 ms /   118 tokens
0.50.018.979 I slot print_timing: id  7 | task 61 |    graphs reused =          1
0.50.018.981 I slot print_timing: id  7 | task 61 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.50.018.996 I statistics        draft-mtp: #calls(b,g,a) =   32     82   1256, #gen drafts =   1264, #acc drafts =   932, #gen tokens =   2523, #acc tokens =  1737, #mean acc len = 2.38, #acc rate/pos = (0.742, 0.641), dur(b,g,a) = 0.020, 262.391, 1.667 ms
0.50.019.022 I slot      release: id  7 | task 61 | stop processing: n_tokens = 140, truncated = 0
0.50.021.442 I slot print_timing: id 11 | task 63 | n_decoded =    101, tg =  20.39 t/s, tg_3s =  20.39 t/s
0.50.022.614 I slot print_timing: id 13 | task 64 | prompt eval time =     168.34 ms /     4 tokens (   42.08 ms per token,    23.76 tokens per second)
0.50.022.616 I slot print_timing: id 13 | task 64 |        eval time =    5137.97 ms /   114 tokens (   45.07 ms per token,    22.19 tokens per second)
0.50.022.616 I slot print_timing: id 13 | task 64 |       total time =    5306.31 ms /   118 tokens
0.50.022.617 I slot print_timing: id 13 | task 64 |    graphs reused =          1
0.50.022.619 I slot print_timing: id 13 | task 64 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.50.022.626 I statistics        draft-mtp: #calls(b,g,a) =   32     82   1262, #gen drafts =   1264, #acc drafts =   938, #gen tokens =   2523, #acc tokens =  1749, #mean acc len = 2.39, #acc rate/pos = (0.743, 0.643), dur(b,g,a) = 0.020, 262.391, 1.673 ms
0.50.022.650 I slot      release: id 13 | task 64 | stop processing: n_tokens = 140, truncated = 0
0.50.028.023 I srv    operator(): Chat format: peg-native
0.50.031.236 I srv    operator(): Chat format: peg-native
0.50.134.497 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.50.134.499 I srv  get_availabl: updating prompt cache
0.50.134.539 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.134.541 I srv         alloc:  - prompt is already in the cache, skipping
0.50.134.542 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.50.134.545 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.50.134.546 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.50.134.547 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.50.134.548 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.50.134.548 I srv  get_availabl: prompt cache update took 0.05 ms
0.50.134.613 I slot launch_slot_: id  7 | task 117 | processing task, is_child = 0
0.50.134.615 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.50.134.632 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.134.633 I srv         alloc:  - prompt is already in the cache, skipping
0.50.134.635 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.50.134.635 I srv  get_availabl: updating prompt cache
0.50.134.648 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.50.134.649 I srv         alloc:  - prompt is already in the cache, skipping
0.50.134.650 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.50.134.650 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.50.134.652 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.50.134.652 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.50.134.653 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.50.134.653 I srv  get_availabl: prompt cache update took 0.02 ms
0.50.134.670 I slot launch_slot_: id 13 | task 118 | processing task, is_child = 0
0.50.139.944 I slot   operator(): id  7 | task 117 | Checking checkpoint with [20, 20] against 3...
0.50.139.947 W slot   operator(): id  7 | task 117 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.50.139.950 W slot   operator(): id  7 | task 117 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.50.141.545 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.50.141.569 I slot   operator(): id 13 | task 118 | Checking checkpoint with [20, 20] against 24...
0.50.145.486 W slot   operator(): id 13 | task 118 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.50.145.534 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.50.294.375 I slot create_check: id  7 | task 117 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.50.532.918 I slot print_timing: id  8 | task 75 | n_decoded =    102, tg =  23.34 t/s, tg_3s =  23.34 t/s
0.50.894.283 I slot print_timing: id  3 | task 74 | n_decoded =    101, tg =  20.61 t/s, tg_3s =  20.61 t/s
0.50.900.900 I slot print_timing: id 14 | task 77 | n_decoded =    101, tg =  20.58 t/s, tg_3s =  20.58 t/s
0.51.139.316 I slot print_timing: id  4 | task 60 | prompt eval time =     360.24 ms /    28 tokens (   12.87 ms per token,    77.73 tokens per second)
0.51.139.319 I slot print_timing: id  4 | task 60 |        eval time =    6072.29 ms /   125 tokens (   48.58 ms per token,    20.59 tokens per second)
0.51.139.320 I slot print_timing: id  4 | task 60 |       total time =    6432.53 ms /   153 tokens
0.51.139.321 I slot print_timing: id  4 | task 60 |    graphs reused =          1
0.51.139.324 I slot print_timing: id  4 | task 60 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.51.139.339 I statistics        draft-mtp: #calls(b,g,a) =   34     91   1392, #gen drafts =   1403, #acc drafts =  1031, #gen tokens =   2801, #acc tokens =  1922, #mean acc len = 2.38, #acc rate/pos = (0.741, 0.640), dur(b,g,a) = 0.022, 294.901, 1.846 ms
0.51.139.371 I slot      release: id  4 | task 60 | stop processing: n_tokens = 152, truncated = 0
0.51.141.740 I slot print_timing: id  8 | task 75 | prompt eval time =     330.54 ms /    25 tokens (   13.22 ms per token,    75.63 tokens per second)
0.51.141.742 I slot print_timing: id  8 | task 75 |        eval time =    4978.76 ms /   114 tokens (   43.67 ms per token,    22.90 tokens per second)
0.51.141.743 I slot print_timing: id  8 | task 75 |       total time =    5309.29 ms /   139 tokens
0.51.141.743 I slot print_timing: id  8 | task 75 |    graphs reused =          1
0.51.141.745 I slot print_timing: id  8 | task 75 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.51.141.752 I statistics        draft-mtp: #calls(b,g,a) =   34     91   1396, #gen drafts =   1403, #acc drafts =  1035, #gen tokens =   2801, #acc tokens =  1930, #mean acc len = 2.38, #acc rate/pos = (0.741, 0.641), dur(b,g,a) = 0.022, 294.901, 1.850 ms
0.51.141.773 I slot      release: id  8 | task 75 | stop processing: n_tokens = 140, truncated = 0
0.51.142.946 I slot print_timing: id 11 | task 63 | prompt eval time =     354.06 ms /    28 tokens (   12.65 ms per token,    79.08 tokens per second)
0.51.142.948 I slot print_timing: id 11 | task 63 |        eval time =    6074.69 ms /   125 tokens (   48.60 ms per token,    20.58 tokens per second)
0.51.142.949 I slot print_timing: id 11 | task 63 |       total time =    6428.76 ms /   153 tokens
0.51.142.950 I slot print_timing: id 11 | task 63 |    graphs reused =          1
0.51.142.951 I slot print_timing: id 11 | task 63 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.51.142.957 I statistics        draft-mtp: #calls(b,g,a) =   34     91   1399, #gen drafts =   1403, #acc drafts =  1036, #gen tokens =   2801, #acc tokens =  1932, #mean acc len = 2.38, #acc rate/pos = (0.741, 0.640), dur(b,g,a) = 0.022, 294.901, 1.854 ms
0.51.142.979 I slot      release: id 11 | task 63 | stop processing: n_tokens = 152, truncated = 0
0.51.148.101 I srv    operator(): Chat format: peg-native
0.51.151.536 I srv    operator(): Chat format: peg-native
0.51.152.001 I srv    operator(): Chat format: peg-native
0.51.249.805 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.51.249.807 I srv  get_availabl: updating prompt cache
0.51.249.850 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.249.852 I srv         alloc:  - prompt is already in the cache, skipping
0.51.249.853 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.51.249.856 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.249.857 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.249.858 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.249.859 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.249.860 I srv  get_availabl: prompt cache update took 0.05 ms
0.51.249.925 I slot launch_slot_: id  4 | task 128 | processing task, is_child = 0
0.51.249.927 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.51.249.943 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.249.944 I srv         alloc:  - prompt is already in the cache, skipping
0.51.249.944 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.51.249.960 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.249.961 I srv         alloc:  - prompt is already in the cache, skipping
0.51.249.963 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.51.249.963 I srv  get_availabl: updating prompt cache
0.51.249.977 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.51.249.978 I srv         alloc:  - prompt is already in the cache, skipping
0.51.249.978 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.51.249.979 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.249.979 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.249.979 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.249.980 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.249.980 I srv  get_availabl: prompt cache update took 0.02 ms
0.51.249.997 I slot launch_slot_: id 11 | task 129 | processing task, is_child = 0
0.51.250.000 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.51.250.013 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.250.014 I srv         alloc:  - prompt is already in the cache, skipping
0.51.250.016 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.51.250.016 I srv  get_availabl: updating prompt cache
0.51.250.029 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.250.030 I srv         alloc:  - prompt is already in the cache, skipping
0.51.250.030 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.51.250.031 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.51.250.031 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.51.250.033 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.51.250.033 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.51.250.033 I srv  get_availabl: prompt cache update took 0.02 ms
0.51.250.047 I slot launch_slot_: id  8 | task 130 | processing task, is_child = 0
0.51.255.986 I slot   operator(): id  4 | task 128 | Checking checkpoint with [23, 23] against 3...
0.51.255.989 W slot   operator(): id  4 | task 128 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.51.255.992 W slot   operator(): id  4 | task 128 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.51.257.595 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.257.635 I slot   operator(): id  8 | task 130 | Checking checkpoint with [20, 20] against 24...
0.51.261.540 W slot   operator(): id  8 | task 130 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.51.261.570 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.261.590 I slot   operator(): id 11 | task 129 | Checking checkpoint with [23, 23] against 27...
0.51.265.163 W slot   operator(): id 11 | task 129 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.51.265.196 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.418.658 I slot create_check: id  4 | task 128 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.51.895.916 I slot print_timing: id  0 | task 59 | n_decoded =    100, tg =  14.64 t/s, tg_3s =  14.64 t/s
0.51.901.037 I slot print_timing: id 10 | task 62 | n_decoded =    100, tg =  14.63 t/s, tg_3s =  14.63 t/s
0.52.018.630 I slot print_timing: id  3 | task 74 | prompt eval time =     164.37 ms /     4 tokens (   41.09 ms per token,    24.33 tokens per second)
0.52.018.634 I slot print_timing: id  3 | task 74 |        eval time =    6025.49 ms /   125 tokens (   48.20 ms per token,    20.75 tokens per second)
0.52.018.635 I slot print_timing: id  3 | task 74 |       total time =    6189.87 ms /   129 tokens
0.52.018.636 I slot print_timing: id  3 | task 74 |    graphs reused =          1
0.52.018.640 I slot print_timing: id  3 | task 74 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.52.018.654 I statistics        draft-mtp: #calls(b,g,a) =   37     98   1496, #gen drafts =   1508, #acc drafts =  1108, #gen tokens =   3011, #acc tokens =  2065, #mean acc len = 2.38, #acc rate/pos = (0.741, 0.640), dur(b,g,a) = 0.024, 323.782, 1.991 ms
0.52.018.682 I slot      release: id  3 | task 74 | stop processing: n_tokens = 152, truncated = 0
0.52.024.279 I slot print_timing: id 14 | task 77 | prompt eval time =     156.46 ms /     4 tokens (   39.11 ms per token,    25.57 tokens per second)
0.52.024.282 I slot print_timing: id 14 | task 77 |        eval time =    6030.80 ms /   125 tokens (   48.25 ms per token,    20.73 tokens per second)
0.52.024.282 I slot print_timing: id 14 | task 77 |       total time =    6187.26 ms /   129 tokens
0.52.024.283 I slot print_timing: id 14 | task 77 |    graphs reused =          1
0.52.024.286 I slot print_timing: id 14 | task 77 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.52.024.294 I statistics        draft-mtp: #calls(b,g,a) =   37     98   1507, #gen drafts =   1508, #acc drafts =  1116, #gen tokens =   3011, #acc tokens =  2080, #mean acc len = 2.38, #acc rate/pos = (0.741, 0.640), dur(b,g,a) = 0.024, 323.782, 2.004 ms
0.52.024.315 I slot      release: id 14 | task 77 | stop processing: n_tokens = 152, truncated = 0
0.52.027.335 I srv    operator(): Chat format: peg-native
0.52.032.664 I srv    operator(): Chat format: peg-native
0.52.133.472 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.52.133.474 I srv  get_availabl: updating prompt cache
0.52.133.517 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.133.520 I srv         alloc:  - prompt is already in the cache, skipping
0.52.133.520 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.52.133.523 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.52.133.525 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.52.133.525 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.52.133.526 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.52.133.526 I srv  get_availabl: prompt cache update took 0.05 ms
0.52.133.595 I slot launch_slot_: id  3 | task 138 | processing task, is_child = 0
0.52.133.596 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.52.133.613 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.133.614 I srv         alloc:  - prompt is already in the cache, skipping
0.52.133.616 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.52.133.616 I srv  get_availabl: updating prompt cache
0.52.133.630 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.52.133.631 I srv         alloc:  - prompt is already in the cache, skipping
0.52.133.632 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.52.133.632 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.52.133.633 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.52.133.633 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.52.133.633 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.52.133.633 I srv  get_availabl: prompt cache update took 0.02 ms
0.52.133.651 I slot launch_slot_: id 14 | task 139 | processing task, is_child = 0
0.52.138.367 I slot   operator(): id  3 | task 138 | Checking checkpoint with [23, 23] against 3...
0.52.138.369 W slot   operator(): id  3 | task 138 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.52.138.372 W slot   operator(): id  3 | task 138 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.52.139.996 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.140.026 I slot   operator(): id 14 | task 139 | Checking checkpoint with [23, 23] against 27...
0.52.143.914 W slot   operator(): id 14 | task 139 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.52.143.947 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.296.067 I slot create_check: id  3 | task 138 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.52.902.172 I slot print_timing: id  1 | task 73 | n_decoded =    100, tg =  14.84 t/s, tg_3s =  14.84 t/s
0.52.906.090 I slot print_timing: id  9 | task 76 | n_decoded =    100, tg =  14.83 t/s, tg_3s =  14.83 t/s
0.53.145.507 I slot print_timing: id  6 | task 101 | n_decoded =    102, tg =  23.48 t/s, tg_3s =  23.48 t/s
0.53.148.464 I slot print_timing: id 12 | task 102 | n_decoded =    102, tg =  23.47 t/s, tg_3s =  23.47 t/s
0.53.628.012 I slot print_timing: id  5 | task 100 | n_decoded =    101, tg =  20.92 t/s, tg_3s =  20.92 t/s
0.53.744.029 I slot print_timing: id  0 | task 59 | prompt eval time =     361.29 ms /    29 tokens (   12.46 ms per token,    80.27 tokens per second)
0.53.744.032 I slot print_timing: id  0 | task 59 |        eval time =    8677.62 ms /   128 tokens (   67.79 ms per token,    14.75 tokens per second)
0.53.744.033 I slot print_timing: id  0 | task 59 |       total time =    9038.92 ms /   157 tokens
0.53.744.033 I slot print_timing: id  0 | task 59 |    graphs reused =          1
0.53.744.037 I slot print_timing: id  0 | task 59 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.53.744.051 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1711, #gen drafts =   1725, #acc drafts =  1258, #gen tokens =   3443, #acc tokens =  2346, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.026, 369.612, 2.258 ms
0.53.744.073 I slot      release: id  0 | task 59 | stop processing: n_tokens = 156, truncated = 0
0.53.744.289 I slot print_timing: id 10 | task 62 | prompt eval time =     355.66 ms /    29 tokens (   12.26 ms per token,    81.54 tokens per second)
0.53.744.291 I slot print_timing: id 10 | task 62 |        eval time =    8676.64 ms /   128 tokens (   67.79 ms per token,    14.75 tokens per second)
0.53.744.291 I slot print_timing: id 10 | task 62 |       total time =    9032.30 ms /   157 tokens
0.53.744.292 I slot print_timing: id 10 | task 62 |    graphs reused =          1
0.53.744.293 I slot print_timing: id 10 | task 62 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.53.744.296 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1711, #gen drafts =   1725, #acc drafts =  1258, #gen tokens =   3443, #acc tokens =  2346, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.026, 369.612, 2.258 ms
0.53.744.308 I slot      release: id 10 | task 62 | stop processing: n_tokens = 156, truncated = 0
0.53.747.219 I slot print_timing: id  6 | task 101 | prompt eval time =     321.37 ms /    25 tokens (   12.85 ms per token,    77.79 tokens per second)
0.53.747.222 I slot print_timing: id  6 | task 101 |        eval time =    4945.81 ms /   114 tokens (   43.38 ms per token,    23.05 tokens per second)
0.53.747.222 I slot print_timing: id  6 | task 101 |       total time =    5267.17 ms /   139 tokens
0.53.747.223 I slot print_timing: id  6 | task 101 |    graphs reused =          1
0.53.747.226 I slot print_timing: id  6 | task 101 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.53.747.232 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1717, #gen drafts =   1725, #acc drafts =  1262, #gen tokens =   3443, #acc tokens =  2353, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.635), dur(b,g,a) = 0.026, 369.612, 2.268 ms
0.53.747.252 I slot      release: id  6 | task 101 | stop processing: n_tokens = 140, truncated = 0
0.53.749.903 I slot print_timing: id 12 | task 102 | prompt eval time =     320.04 ms /    25 tokens (   12.80 ms per token,    78.12 tokens per second)
0.53.749.905 I slot print_timing: id 12 | task 102 |        eval time =    4948.23 ms /   114 tokens (   43.41 ms per token,    23.04 tokens per second)
0.53.749.906 I slot print_timing: id 12 | task 102 |       total time =    5268.27 ms /   139 tokens
0.53.749.906 I slot print_timing: id 12 | task 102 |    graphs reused =          1
0.53.749.910 I slot print_timing: id 12 | task 102 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.53.749.916 I statistics        draft-mtp: #calls(b,g,a) =   39    112   1722, #gen drafts =   1725, #acc drafts =  1266, #gen tokens =   3443, #acc tokens =  2361, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.026, 369.612, 2.273 ms
0.53.749.933 I slot      release: id 12 | task 102 | stop processing: n_tokens = 140, truncated = 0
0.53.753.966 I srv    operator(): Chat format: peg-native
0.53.753.972 I srv    operator(): Chat format: peg-native
0.53.756.441 I srv    operator(): Chat format: peg-native
0.53.758.554 I srv    operator(): Chat format: peg-native
0.53.847.145 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.53.847.147 I srv  get_availabl: updating prompt cache
0.53.847.189 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.847.191 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.192 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.53.847.195 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.847.196 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.847.196 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.847.197 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.847.199 I srv  get_availabl: prompt cache update took 0.05 ms
0.53.847.266 I slot launch_slot_: id  0 | task 154 | processing task, is_child = 0
0.53.847.268 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
0.53.847.285 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.285 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.285 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.53.847.301 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.847.302 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.303 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.53.847.318 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.319 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.321 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.53.847.321 I srv  get_availabl: updating prompt cache
0.53.847.334 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.335 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.336 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.53.847.336 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.847.337 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.847.337 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.847.337 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.847.337 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.847.355 I slot launch_slot_: id  6 | task 155 | processing task, is_child = 0
0.53.847.357 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.53.847.370 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.847.371 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.371 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.53.847.384 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.386 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.387 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.53.847.389 I srv  get_availabl: updating prompt cache
0.53.847.402 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.53.847.403 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.404 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.53.847.404 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.847.404 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.847.404 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.847.405 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.847.405 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.847.420 I slot launch_slot_: id 10 | task 156 | processing task, is_child = 0
0.53.847.422 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.53.847.435 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.436 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.437 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.53.847.438 I srv  get_availabl: updating prompt cache
0.53.847.452 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.53.847.456 I srv         alloc:  - prompt is already in the cache, skipping
0.53.847.456 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.53.847.457 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.53.847.457 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.53.847.457 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.53.847.457 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.53.847.458 I srv  get_availabl: prompt cache update took 0.02 ms
0.53.847.472 I slot launch_slot_: id 12 | task 157 | processing task, is_child = 0
0.53.853.068 I slot   operator(): id  0 | task 154 | Checking checkpoint with [24, 24] against 28...
0.53.856.965 W slot   operator(): id  0 | task 154 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.53.856.998 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.857.021 I slot   operator(): id  6 | task 155 | Checking checkpoint with [20, 20] against 24...
0.53.860.632 W slot   operator(): id  6 | task 155 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.53.860.652 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.860.678 I slot   operator(): id 10 | task 156 | Checking checkpoint with [24, 24] against 3...
0.53.860.680 W slot   operator(): id 10 | task 156 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.53.860.681 W slot   operator(): id 10 | task 156 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.53.862.418 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.862.440 I slot   operator(): id 12 | task 157 | Checking checkpoint with [20, 20] against 24...
0.53.866.023 W slot   operator(): id 12 | task 157 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.53.866.049 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.018.981 I slot create_check: id 10 | task 156 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.54.621.533 I slot print_timing: id 13 | task 118 | n_decoded =    102, tg =  23.44 t/s, tg_3s =  23.44 t/s
0.54.734.856 I slot print_timing: id  1 | task 73 | prompt eval time =     335.39 ms /    29 tokens (   11.57 ms per token,    86.47 tokens per second)
0.54.734.859 I slot print_timing: id  1 | task 73 |        eval time =    8572.50 ms /   128 tokens (   66.97 ms per token,    14.93 tokens per second)
0.54.734.860 I slot print_timing: id  1 | task 73 |       total time =    8907.89 ms /   157 tokens
0.54.734.861 I slot print_timing: id  1 | task 73 |    graphs reused =          1
0.54.734.865 I slot print_timing: id  1 | task 73 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.54.734.879 I statistics        draft-mtp: #calls(b,g,a) =   43    120   1828, #gen drafts =   1842, #acc drafts =  1339, #gen tokens =   3675, #acc tokens =  2498, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.634), dur(b,g,a) = 0.028, 401.824, 2.412 ms
0.54.734.901 I slot      release: id  1 | task 73 | stop processing: n_tokens = 156, truncated = 0
0.54.735.147 I slot print_timing: id  9 | task 76 | prompt eval time =     329.04 ms /    29 tokens (   11.35 ms per token,    88.13 tokens per second)
0.54.735.152 I slot print_timing: id  9 | task 76 |        eval time =    8571.53 ms /   128 tokens (   66.97 ms per token,    14.93 tokens per second)
0.54.735.153 I slot print_timing: id  9 | task 76 |       total time =    8900.58 ms /   157 tokens
0.54.735.153 I slot print_timing: id  9 | task 76 |    graphs reused =          1
0.54.735.154 I slot print_timing: id  9 | task 76 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
0.54.735.156 I statistics        draft-mtp: #calls(b,g,a) =   43    120   1828, #gen drafts =   1842, #acc drafts =  1339, #gen tokens =   3675, #acc tokens =  2498, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.634), dur(b,g,a) = 0.028, 401.824, 2.412 ms
0.54.735.171 I slot      release: id  9 | task 76 | stop processing: n_tokens = 156, truncated = 0
0.54.737.281 I slot print_timing: id  5 | task 100 | prompt eval time =     322.80 ms /    28 tokens (   11.53 ms per token,    86.74 tokens per second)
0.54.737.283 I slot print_timing: id  5 | task 100 |        eval time =    5936.12 ms /   125 tokens (   47.49 ms per token,    21.06 tokens per second)
0.54.737.284 I slot print_timing: id  5 | task 100 |       total time =    6258.92 ms /   153 tokens
0.54.737.284 I slot print_timing: id  5 | task 100 |    graphs reused =          1
0.54.737.287 I slot print_timing: id  5 | task 100 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.54.737.294 I statistics        draft-mtp: #calls(b,g,a) =   43    120   1833, #gen drafts =   1842, #acc drafts =  1341, #gen tokens =   3675, #acc tokens =  2502, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.028, 401.824, 2.420 ms
0.54.737.312 I slot      release: id  5 | task 100 | stop processing: n_tokens = 152, truncated = 0
0.54.745.520 I srv    operator(): Chat format: peg-native
0.54.747.847 I srv    operator(): Chat format: peg-native
0.54.748.052 I srv    operator(): Chat format: peg-native
0.54.846.151 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.54.846.154 I srv  get_availabl: updating prompt cache
0.54.846.194 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.846.197 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.198 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.54.846.201 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.846.202 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.846.202 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.846.203 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.846.203 I srv  get_availabl: prompt cache update took 0.05 ms
0.54.846.271 I slot launch_slot_: id  1 | task 166 | processing task, is_child = 0
0.54.846.273 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
0.54.846.289 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.54.846.291 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.291 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.54.846.307 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.846.308 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.310 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.54.846.310 I srv  get_availabl: updating prompt cache
0.54.846.324 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.54.846.325 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.326 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.54.846.326 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.846.327 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.846.327 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.846.327 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.846.328 I srv  get_availabl: prompt cache update took 0.02 ms
0.54.846.345 I slot launch_slot_: id  5 | task 167 | processing task, is_child = 0
0.54.846.346 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
0.54.846.359 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.846.360 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.362 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.54.846.362 I srv  get_availabl: updating prompt cache
0.54.846.375 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.846.376 I srv         alloc:  - prompt is already in the cache, skipping
0.54.846.377 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.54.846.377 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.54.846.378 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.54.846.378 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.54.846.378 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.54.846.378 I srv  get_availabl: prompt cache update took 0.02 ms
0.54.846.393 I slot launch_slot_: id  9 | task 168 | processing task, is_child = 0
0.54.852.179 I slot   operator(): id  1 | task 166 | Checking checkpoint with [24, 24] against 28...
0.54.856.116 W slot   operator(): id  1 | task 166 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.54.856.179 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.856.185 I slot   operator(): id  5 | task 167 | Checking checkpoint with [23, 23] against 27...
0.54.859.796 W slot   operator(): id  5 | task 167 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.54.859.819 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.859.849 I slot   operator(): id  9 | task 168 | Checking checkpoint with [24, 24] against 3...
0.54.859.851 W slot   operator(): id  9 | task 168 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.54.859.851 W slot   operator(): id  9 | task 168 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.54.861.431 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.012.090 I slot create_check: id  9 | task 168 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.55.247.477 I slot print_timing: id  2 | task 99 | n_decoded =    100, tg =  15.13 t/s, tg_3s =  15.13 t/s
0.55.249.860 I slot print_timing: id  7 | task 117 | n_decoded =    101, tg =  20.86 t/s, tg_3s =  20.86 t/s
0.55.253.489 I slot print_timing: id 13 | task 118 | prompt eval time =     128.58 ms /     4 tokens (   32.14 ms per token,    31.11 tokens per second)
0.55.253.492 I slot print_timing: id 13 | task 118 |        eval time =    4983.32 ms /   114 tokens (   43.71 ms per token,    22.88 tokens per second)
0.55.253.492 I slot print_timing: id 13 | task 118 |       total time =    5111.90 ms /   118 tokens
0.55.253.494 I slot print_timing: id 13 | task 118 |    graphs reused =          1
0.55.253.497 I slot print_timing: id 13 | task 118 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.55.253.510 I statistics        draft-mtp: #calls(b,g,a) =   46    124   1897, #gen drafts =   1899, #acc drafts =  1391, #gen tokens =   3789, #acc tokens =  2596, #mean acc len = 2.37, #acc rate/pos = (0.733, 0.635), dur(b,g,a) = 0.030, 420.507, 2.500 ms
0.55.253.536 I slot      release: id 13 | task 118 | stop processing: n_tokens = 140, truncated = 0
0.55.254.750 I slot print_timing: id 15 | task 103 | n_decoded =    100, tg =  15.11 t/s, tg_3s =  15.11 t/s
0.55.262.681 I srv    operator(): Chat format: peg-native
0.55.372.862 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.55.372.864 I srv  get_availabl: updating prompt cache
0.55.372.905 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.55.372.908 I srv         alloc:  - prompt is already in the cache, skipping
0.55.372.909 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.55.372.911 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.55.372.912 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.55.372.913 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.55.372.913 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.55.372.914 I srv  get_availabl: prompt cache update took 0.05 ms
0.55.372.979 I slot launch_slot_: id 13 | task 173 | processing task, is_child = 0
0.55.376.807 I slot   operator(): id 13 | task 173 | Checking checkpoint with [20, 20] against 3...
0.55.376.810 W slot   operator(): id 13 | task 173 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.55.376.813 W slot   operator(): id 13 | task 173 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.55.378.307 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.526.365 I slot create_check: id 13 | task 173 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.55.764.057 I slot print_timing: id  8 | task 130 | n_decoded =    102, tg =  23.33 t/s, tg_3s =  23.33 t/s
0.56.252.468 I slot print_timing: id 11 | task 129 | n_decoded =    101, tg =  20.78 t/s, tg_3s =  20.78 t/s
0.56.373.302 I slot print_timing: id  7 | task 117 | prompt eval time =     267.19 ms /    28 tokens (    9.54 ms per token,   104.80 tokens per second)
0.56.373.305 I slot print_timing: id  7 | task 117 |        eval time =    5966.14 ms /   125 tokens (   47.73 ms per token,    20.95 tokens per second)
0.56.373.306 I slot print_timing: id  7 | task 117 |       total time =    6233.33 ms /   153 tokens
0.56.373.307 I slot print_timing: id  7 | task 117 |    graphs reused =          1
0.56.373.310 I slot print_timing: id  7 | task 117 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.56.373.321 I statistics        draft-mtp: #calls(b,g,a) =   47    133   2032, #gen drafts =   2040, #acc drafts =  1493, #gen tokens =   4071, #acc tokens =  2785, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.030, 450.648, 2.670 ms
0.56.373.349 I slot      release: id  7 | task 117 | stop processing: n_tokens = 152, truncated = 0
0.56.373.951 I slot print_timing: id  8 | task 130 | prompt eval time =     135.23 ms /     4 tokens (   33.81 ms per token,    29.58 tokens per second)
0.56.373.953 I slot print_timing: id  8 | task 130 |        eval time =    4981.07 ms /   114 tokens (   43.69 ms per token,    22.89 tokens per second)
0.56.373.954 I slot print_timing: id  8 | task 130 |       total time =    5116.30 ms /   118 tokens
0.56.373.954 I slot print_timing: id  8 | task 130 |    graphs reused =          1
0.56.373.956 I slot print_timing: id  8 | task 130 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.56.373.964 I statistics        draft-mtp: #calls(b,g,a) =   47    133   2033, #gen drafts =   2040, #acc drafts =  1494, #gen tokens =   4071, #acc tokens =  2787, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.030, 450.648, 2.672 ms
0.56.373.982 I slot      release: id  8 | task 130 | stop processing: n_tokens = 140, truncated = 0
0.56.382.654 I srv    operator(): Chat format: peg-native
0.56.383.359 I srv    operator(): Chat format: peg-native
0.56.486.855 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.56.486.857 I srv  get_availabl: updating prompt cache
0.56.486.899 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.56.486.901 I srv         alloc:  - prompt is already in the cache, skipping
0.56.486.903 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
0.56.486.905 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.56.486.906 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.56.486.907 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.56.486.908 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.56.486.909 I srv  get_availabl: prompt cache update took 0.05 ms
0.56.486.978 I slot launch_slot_: id  7 | task 183 | processing task, is_child = 0
0.56.486.980 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
0.56.486.995 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.56.486.995 I srv         alloc:  - prompt is already in the cache, skipping
0.56.486.997 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.56.486.997 I srv  get_availabl: updating prompt cache
0.56.487.010 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.56.487.011 I srv         alloc:  - prompt is already in the cache, skipping
0.56.487.012 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.56.487.013 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.56.487.013 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.56.487.013 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.56.487.014 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.56.487.014 I srv  get_availabl: prompt cache update took 0.02 ms
0.56.487.033 I slot launch_slot_: id  8 | task 184 | processing task, is_child = 0
0.56.491.597 I slot   operator(): id  7 | task 183 | Checking checkpoint with [23, 23] against 27...
0.56.495.532 W slot   operator(): id  7 | task 183 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.56.495.575 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.495.589 I slot   operator(): id  8 | task 184 | Checking checkpoint with [20, 20] against 24...
0.56.499.175 W slot   operator(): id  8 | task 184 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.56.499.216 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.090.626 I slot print_timing: id  2 | task 99 | prompt eval time =     161.42 ms /     4 tokens (   40.35 ms per token,    24.78 tokens per second)
0.57.090.630 I slot print_timing: id  2 | task 99 |        eval time =    8454.49 ms /   128 tokens (   66.05 ms per token,    15.14 tokens per second)
0.57.090.631 I slot print_timing: id  2 | task 99 |       total time =    8615.91 ms /   132 tokens
0.57.090.632 I slot print_timing: id  2 | task 99 |    graphs reused =          1
0.57.090.636 I slot print_timing: id  2 | task 99 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
0.57.090.654 I statistics        draft-mtp: #calls(b,g,a) =   49    139   2116, #gen drafts =   2130, #acc drafts =  1548, #gen tokens =   4249, #acc tokens =  2887, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.030, 471.923, 2.767 ms
0.57.090.683 I slot      release: id  2 | task 99 | stop processing: n_tokens = 156, truncated = 0
0.57.090.896 I slot print_timing: id 15 | task 103 | prompt eval time =     153.27 ms /     4 tokens (   38.32 ms per token,    26.10 tokens per second)
0.57.090.898 I slot print_timing: id 15 | task 103 |        eval time =    8454.51 ms /   128 tokens (   66.05 ms per token,    15.14 tokens per second)
0.57.090.899 I slot print_timing: id 15 | task 103 |       total time =    8607.78 ms /   132 tokens
0.57.090.900 I slot print_timing: id 15 | task 103 |    graphs reused =          1
0.57.090.902 I slot print_timing: id 15 | task 103 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
0.57.090.905 I statistics        draft-mtp: #calls(b,g,a) =   49    139   2116, #gen drafts =   2130, #acc drafts =  1548, #gen tokens =   4249, #acc tokens =  2887, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.030, 471.923, 2.767 ms
0.57.090.924 I slot      release: id 15 | task 103 | stop processing: n_tokens = 156, truncated = 0
0.57.098.844 I slot print_timing: id 14 | task 139 | n_decoded =    101, tg =  20.91 t/s, tg_3s =  20.91 t/s
0.57.100.579 I srv    operator(): Chat format: peg-native
0.57.100.587 I srv    operator(): Chat format: peg-native
0.57.207.363 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.57.207.365 I srv  get_availabl: updating prompt cache
0.57.207.408 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.207.411 I srv         alloc:  - prompt is already in the cache, skipping
0.57.207.412 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.57.207.415 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.207.416 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.207.417 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.207.417 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.207.418 I srv  get_availabl: prompt cache update took 0.05 ms
0.57.207.485 I slot launch_slot_: id  2 | task 191 | processing task, is_child = 0
0.57.207.486 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.57.207.503 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.207.505 I srv         alloc:  - prompt is already in the cache, skipping
0.57.207.506 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.57.207.506 I srv  get_availabl: updating prompt cache
0.57.207.520 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.57.207.521 I srv         alloc:  - prompt is already in the cache, skipping
0.57.207.522 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.57.207.522 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.207.523 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.207.523 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.207.523 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.207.523 I srv  get_availabl: prompt cache update took 0.02 ms
0.57.207.540 I slot launch_slot_: id 15 | task 192 | processing task, is_child = 0
0.57.211.349 I slot   operator(): id  2 | task 191 | Checking checkpoint with [24, 24] against 28...
0.57.215.407 W slot   operator(): id  2 | task 191 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.57.215.445 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.215.466 I slot   operator(): id 15 | task 192 | Checking checkpoint with [24, 24] against 3...
0.57.215.468 W slot   operator(): id 15 | task 192 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.57.215.469 W slot   operator(): id 15 | task 192 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.57.217.526 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.347.085 I slot print_timing: id 11 | task 129 | prompt eval time =     131.55 ms /     4 tokens (   32.89 ms per token,    30.41 tokens per second)
0.57.347.089 I slot print_timing: id 11 | task 129 |        eval time =    5953.90 ms /   125 tokens (   47.63 ms per token,    20.99 tokens per second)
0.57.347.089 I slot print_timing: id 11 | task 129 |       total time =    6085.45 ms /   129 tokens
0.57.347.090 I slot print_timing: id 11 | task 129 |    graphs reused =          1
0.57.347.093 I slot print_timing: id 11 | task 129 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.57.347.104 I statistics        draft-mtp: #calls(b,g,a) =   50    141   2155, #gen drafts =   2158, #acc drafts =  1579, #gen tokens =   4305, #acc tokens =  2948, #mean acc len = 2.37, #acc rate/pos = (0.733, 0.635), dur(b,g,a) = 0.031, 479.368, 2.824 ms
0.57.347.133 I slot      release: id 11 | task 129 | stop processing: n_tokens = 152, truncated = 0
0.57.355.539 I srv    operator(): Chat format: peg-native
0.57.365.654 I slot create_check: id 15 | task 192 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.57.477.960 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
0.57.477.962 I srv  get_availabl: updating prompt cache
0.57.478.004 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.57.478.006 I srv         alloc:  - prompt is already in the cache, skipping
0.57.478.007 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.120
0.57.478.010 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.57.478.011 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.57.478.011 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.57.478.013 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.57.478.013 I srv  get_availabl: prompt cache update took 0.05 ms
0.57.478.081 I slot launch_slot_: id 11 | task 195 | processing task, is_child = 0
0.57.482.259 I slot   operator(): id 11 | task 195 | Checking checkpoint with [23, 23] against 3...
0.57.482.262 W slot   operator(): id 11 | task 195 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.57.482.265 W slot   operator(): id 11 | task 195 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.57.483.952 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.631.553 I slot create_check: id 11 | task 195 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.58.231.659 I slot print_timing: id  4 | task 128 | n_decoded =    100, tg =  14.93 t/s, tg_3s =  14.93 t/s
0.58.237.072 I slot print_timing: id 14 | task 139 | prompt eval time =     129.52 ms /     4 tokens (   32.38 ms per token,    30.88 tokens per second)
0.58.237.074 I slot print_timing: id 14 | task 139 |        eval time =    5967.47 ms /   125 tokens (   47.74 ms per token,    20.95 tokens per second)
0.58.237.075 I slot print_timing: id 14 | task 139 |       total time =    6096.99 ms /   129 tokens
0.58.237.076 I slot print_timing: id 14 | task 139 |    graphs reused =          1
0.58.237.079 I slot print_timing: id 14 | task 139 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
0.58.237.091 I statistics        draft-mtp: #calls(b,g,a) =   52    148   2265, #gen drafts =   2266, #acc drafts =  1662, #gen tokens =   4521, #acc tokens =  3098, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.634), dur(b,g,a) = 0.032, 503.135, 2.960 ms
0.58.237.120 I slot      release: id 14 | task 139 | stop processing: n_tokens = 152, truncated = 0
0.58.246.373 I srv    operator(): Chat format: peg-native
0.58.348.003 I slot print_timing: id  6 | task 155 | n_decoded =    102, tg =  23.43 t/s, tg_3s =  23.43 t/s
0.58.351.579 I slot print_timing: id 12 | task 157 | n_decoded =    102, tg =  23.41 t/s, tg_3s =  23.41 t/s
0.58.352.807 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.58.352.809 I srv  get_availabl: updating prompt cache
0.58.352.851 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
0.58.352.854 I srv         alloc:  - prompt is already in the cache, skipping
0.58.352.855 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
0.58.352.857 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.58.352.858 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.58.352.860 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.58.352.860 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.58.352.860 I srv  get_availabl: prompt cache update took 0.05 ms
0.58.352.927 I slot launch_slot_: id 14 | task 203 | processing task, is_child = 0
0.58.356.576 I slot   operator(): id 14 | task 203 | Checking checkpoint with [23, 23] against 3...
0.58.356.579 W slot   operator(): id 14 | task 203 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.356.582 W slot   operator(): id 14 | task 203 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.58.358.163 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.505.741 I slot create_check: id 14 | task 203 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.58.984.845 I slot print_timing: id  6 | task 155 | prompt eval time =     137.78 ms /     4 tokens (   34.44 ms per token,    29.03 tokens per second)
0.58.984.850 I slot print_timing: id  6 | task 155 |        eval time =    4990.02 ms /   114 tokens (   43.77 ms per token,    22.85 tokens per second)
0.58.984.851 I slot print_timing: id  6 | task 155 |       total time =    5127.80 ms /   118 tokens
0.58.984.852 I slot print_timing: id  6 | task 155 |    graphs reused =          1
0.58.984.855 I slot print_timing: id  6 | task 155 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.58.984.868 I statistics        draft-mtp: #calls(b,g,a) =   53    154   2350, #gen drafts =   2359, #acc drafts =  1724, #gen tokens =   4707, #acc tokens =  3215, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.634), dur(b,g,a) = 0.034, 522.677, 3.072 ms
0.58.984.894 I slot      release: id  6 | task 155 | stop processing: n_tokens = 140, truncated = 0
0.58.987.410 I slot print_timing: id 10 | task 156 | n_decoded =    101, tg =  20.80 t/s, tg_3s =  20.80 t/s
0.58.988.655 I slot print_timing: id 12 | task 157 | prompt eval time =     132.61 ms /     4 tokens (   33.15 ms per token,    30.16 tokens per second)
0.58.988.658 I slot print_timing: id 12 | task 157 |        eval time =    4993.59 ms /   114 tokens (   43.80 ms per token,    22.83 tokens per second)
0.58.988.658 I slot print_timing: id 12 | task 157 |       total time =    5126.20 ms /   118 tokens
0.58.988.659 I slot print_timing: id 12 | task 157 |    graphs reused =          1
0.58.988.661 I slot print_timing: id 12 | task 157 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
0.58.988.670 I statistics        draft-mtp: #calls(b,g,a) =   53    154   2356, #gen drafts =   2359, #acc drafts =  1730, #gen tokens =   4707, #acc tokens =  3227, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.635), dur(b,g,a) = 0.034, 522.677, 3.079 ms
0.58.988.692 I slot      release: id 12 | task 157 | stop processing: n_tokens = 140, truncated = 0
0.58.993.516 I srv    operator(): Chat format: peg-native
0.58.997.042 I srv    operator(): Chat format: peg-native
0.59.094.650 I slot print_timing: id  3 | task 138 | n_decoded =    100, tg =  14.96 t/s, tg_3s =  14.96 t/s
0.59.099.806 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.59.099.808 I srv  get_availabl: updating prompt cache
0.59.099.848 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.59.099.850 I srv         alloc:  - prompt is already in the cache, skipping
0.59.099.851 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.59.099.854 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.59.099.855 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.59.099.856 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.59.099.857 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.59.099.858 I srv  get_availabl: prompt cache update took 0.05 ms
0.59.099.924 I slot launch_slot_: id  6 | task 210 | processing task, is_child = 0
0.59.099.926 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.59.099.942 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.59.099.943 I srv         alloc:  - prompt is already in the cache, skipping
0.59.099.945 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.59.099.945 I srv  get_availabl: updating prompt cache
0.59.099.958 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.59.099.959 I srv         alloc:  - prompt is already in the cache, skipping
0.59.099.960 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.59.099.961 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
0.59.099.961 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
0.59.099.961 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
0.59.099.961 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
0.59.099.962 I srv  get_availabl: prompt cache update took 0.02 ms
0.59.099.981 I slot launch_slot_: id 12 | task 211 | processing task, is_child = 0
0.59.105.268 I slot   operator(): id  6 | task 210 | Checking checkpoint with [20, 20] against 3...
0.59.105.271 W slot   operator(): id  6 | task 210 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.59.105.274 W slot   operator(): id  6 | task 210 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.59.106.665 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.106.687 I slot   operator(): id 12 | task 211 | Checking checkpoint with [20, 20] against 24...
0.59.110.554 W slot   operator(): id 12 | task 211 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.59.110.579 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.259.823 I slot create_check: id  6 | task 210 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.59.497.198 I slot print_timing: id  9 | task 168 | n_decoded =    102, tg =  23.33 t/s, tg_3s =  23.33 t/s
0.59.859.877 I slot print_timing: id  5 | task 167 | n_decoded =    101, tg =  20.73 t/s, tg_3s =  20.73 t/s
1.00.100.156 I slot print_timing: id  4 | task 128 | prompt eval time =     275.58 ms /    29 tokens (    9.50 ms per token,   105.23 tokens per second)
1.00.100.160 I slot print_timing: id  4 | task 128 |        eval time =    8568.55 ms /   128 tokens (   66.94 ms per token,    14.94 tokens per second)
1.00.100.161 I slot print_timing: id  4 | task 128 |       total time =    8844.13 ms /   157 tokens
1.00.100.162 I slot print_timing: id  4 | task 128 |    graphs reused =          1
1.00.100.165 I slot print_timing: id  4 | task 128 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.00.100.181 I statistics        draft-mtp: #calls(b,g,a) =   55    163   2482, #gen drafts =   2497, #acc drafts =  1818, #gen tokens =   4982, #acc tokens =  3389, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.034, 556.125, 3.230 ms
1.00.100.209 I slot      release: id  4 | task 128 | stop processing: n_tokens = 156, truncated = 0
1.00.104.649 I slot print_timing: id  9 | task 168 | prompt eval time =     264.78 ms /    25 tokens (   10.59 ms per token,    94.42 tokens per second)
1.00.104.651 I slot print_timing: id  9 | task 168 |        eval time =    4980.00 ms /   114 tokens (   43.68 ms per token,    22.89 tokens per second)
1.00.104.651 I slot print_timing: id  9 | task 168 |       total time =    5244.78 ms /   139 tokens
1.00.104.652 I slot print_timing: id  9 | task 168 |    graphs reused =          1
1.00.104.654 I slot print_timing: id  9 | task 168 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.00.104.662 I statistics        draft-mtp: #calls(b,g,a) =   55    163   2491, #gen drafts =   2497, #acc drafts =  1824, #gen tokens =   4982, #acc tokens =  3400, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.034, 556.125, 3.244 ms
1.00.104.686 I slot      release: id  9 | task 168 | stop processing: n_tokens = 140, truncated = 0
1.00.105.311 I slot print_timing: id 10 | task 156 | prompt eval time =     270.73 ms /    28 tokens (    9.67 ms per token,   103.42 tokens per second)
1.00.105.313 I slot print_timing: id 10 | task 156 |        eval time =    5973.86 ms /   125 tokens (   47.79 ms per token,    20.92 tokens per second)
1.00.105.313 I slot print_timing: id 10 | task 156 |       total time =    6244.60 ms /   153 tokens
1.00.105.314 I slot print_timing: id 10 | task 156 |    graphs reused =          1
1.00.105.316 I slot print_timing: id 10 | task 156 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.00.105.322 I statistics        draft-mtp: #calls(b,g,a) =   55    163   2492, #gen drafts =   2497, #acc drafts =  1825, #gen tokens =   4982, #acc tokens =  3402, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.034, 556.125, 3.245 ms
1.00.105.342 I slot      release: id 10 | task 156 | stop processing: n_tokens = 152, truncated = 0
1.00.110.011 I srv    operator(): Chat format: peg-native
1.00.114.628 I srv    operator(): Chat format: peg-native
1.00.114.776 I srv    operator(): Chat format: peg-native
1.00.210.083 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.00.210.085 I srv  get_availabl: updating prompt cache
1.00.210.128 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.00.210.131 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.131 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.00.210.134 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.210.135 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.210.141 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.210.141 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.210.142 I srv  get_availabl: prompt cache update took 0.06 ms
1.00.210.208 I slot launch_slot_: id  4 | task 221 | processing task, is_child = 0
1.00.210.210 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
1.00.210.226 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.210.227 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.228 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.00.210.243 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.00.210.244 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.246 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.00.210.246 I srv  get_availabl: updating prompt cache
1.00.210.260 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.00.210.261 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.261 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.00.210.262 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.210.262 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.210.263 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.210.264 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.210.265 I srv  get_availabl: prompt cache update took 0.02 ms
1.00.210.282 I slot launch_slot_: id 10 | task 222 | processing task, is_child = 0
1.00.210.284 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
1.00.210.297 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.210.298 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.300 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.00.210.300 I srv  get_availabl: updating prompt cache
1.00.210.313 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.00.210.314 I srv         alloc:  - prompt is already in the cache, skipping
1.00.210.315 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.00.210.316 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.00.210.316 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.00.210.318 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.00.210.318 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.00.210.318 I srv  get_availabl: prompt cache update took 0.02 ms
1.00.210.332 I slot launch_slot_: id  9 | task 223 | processing task, is_child = 0
1.00.215.600 I slot   operator(): id  4 | task 221 | Checking checkpoint with [24, 24] against 28...
1.00.219.470 W slot   operator(): id  4 | task 221 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.00.219.502 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.219.529 I slot   operator(): id  9 | task 223 | Checking checkpoint with [20, 20] against 24...
1.00.223.080 W slot   operator(): id  9 | task 223 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.00.223.108 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.223.123 I slot   operator(): id 10 | task 222 | Checking checkpoint with [23, 23] against 27...
1.00.226.689 W slot   operator(): id 10 | task 222 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.00.226.720 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.706.548 I slot print_timing: id  0 | task 154 | n_decoded =    100, tg =  14.90 t/s, tg_3s =  14.90 t/s
1.00.946.687 I slot print_timing: id  3 | task 138 | prompt eval time =     270.62 ms /    29 tokens (    9.33 ms per token,   107.16 tokens per second)
1.00.946.691 I slot print_timing: id  3 | task 138 |        eval time =    8537.67 ms /   128 tokens (   66.70 ms per token,    14.99 tokens per second)
1.00.946.691 I slot print_timing: id  3 | task 138 |       total time =    8808.29 ms /   157 tokens
1.00.946.692 I slot print_timing: id  3 | task 138 |    graphs reused =          1
1.00.946.695 I slot print_timing: id  3 | task 138 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.00.946.714 I statistics        draft-mtp: #calls(b,g,a) =   58    170   2587, #gen drafts =   2602, #acc drafts =  1890, #gen tokens =   5191, #acc tokens =  3522, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.631), dur(b,g,a) = 0.035, 582.236, 3.365 ms
1.00.946.741 I slot      release: id  3 | task 138 | stop processing: n_tokens = 156, truncated = 0
1.00.948.947 I slot print_timing: id  5 | task 167 | prompt eval time =     131.18 ms /     4 tokens (   32.80 ms per token,    30.49 tokens per second)
1.00.948.949 I slot print_timing: id  5 | task 167 |        eval time =    5961.54 ms /   125 tokens (   47.69 ms per token,    20.97 tokens per second)
1.00.948.949 I slot print_timing: id  5 | task 167 |       total time =    6092.73 ms /   129 tokens
1.00.948.950 I slot print_timing: id  5 | task 167 |    graphs reused =          1
1.00.948.952 I slot print_timing: id  5 | task 167 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.00.948.960 I statistics        draft-mtp: #calls(b,g,a) =   58    170   2592, #gen drafts =   2602, #acc drafts =  1892, #gen tokens =   5191, #acc tokens =  3526, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.630), dur(b,g,a) = 0.035, 582.236, 3.373 ms
1.00.948.982 I slot      release: id  5 | task 167 | stop processing: n_tokens = 152, truncated = 0
1.00.950.546 I slot print_timing: id  8 | task 184 | n_decoded =    102, tg =  23.51 t/s, tg_3s =  23.51 t/s
1.00.955.595 I srv    operator(): Chat format: peg-native
1.00.957.061 I srv    operator(): Chat format: peg-native
1.01.064.037 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.01.064.039 I srv  get_availabl: updating prompt cache
1.01.064.080 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.01.064.083 I srv         alloc:  - prompt is already in the cache, skipping
1.01.064.084 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.01.064.086 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.01.064.087 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.01.064.089 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.01.064.089 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.01.064.089 I srv  get_availabl: prompt cache update took 0.05 ms
1.01.064.160 I slot launch_slot_: id  3 | task 231 | processing task, is_child = 0
1.01.064.162 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.01.064.179 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.01.064.180 I srv         alloc:  - prompt is already in the cache, skipping
1.01.064.182 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.01.064.182 I srv  get_availabl: updating prompt cache
1.01.064.196 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.01.064.198 I srv         alloc:  - prompt is already in the cache, skipping
1.01.064.199 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.01.064.199 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.01.064.200 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.01.064.200 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.01.064.200 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.01.064.200 I srv  get_availabl: prompt cache update took 0.02 ms
1.01.064.218 I slot launch_slot_: id  5 | task 232 | processing task, is_child = 0
1.01.068.951 I slot   operator(): id  3 | task 231 | Checking checkpoint with [24, 24] against 28...
1.01.073.031 W slot   operator(): id  3 | task 231 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.01.073.057 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.073.084 I slot   operator(): id  5 | task 232 | Checking checkpoint with [23, 23] against 27...
1.01.076.631 W slot   operator(): id  5 | task 232 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.01.076.675 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.438.200 I slot print_timing: id  7 | task 183 | n_decoded =    101, tg =  20.93 t/s, tg_3s =  20.93 t/s
1.01.559.854 I slot print_timing: id  8 | task 184 | prompt eval time =     116.99 ms /     4 tokens (   29.25 ms per token,    34.19 tokens per second)
1.01.559.857 I slot print_timing: id  8 | task 184 |        eval time =    4947.25 ms /   114 tokens (   43.40 ms per token,    23.04 tokens per second)
1.01.559.858 I slot print_timing: id  8 | task 184 |       total time =    5064.24 ms /   118 tokens
1.01.559.859 I slot print_timing: id  8 | task 184 |    graphs reused =          1
1.01.559.862 I slot print_timing: id  8 | task 184 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.01.559.875 I statistics        draft-mtp: #calls(b,g,a) =   60    175   2671, #gen drafts =   2678, #acc drafts =  1953, #gen tokens =   5343, #acc tokens =  3642, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.036, 600.334, 3.474 ms
1.01.559.903 I slot      release: id  8 | task 184 | stop processing: n_tokens = 140, truncated = 0
1.01.569.253 I srv    operator(): Chat format: peg-native
1.01.673.845 I slot print_timing: id  1 | task 166 | n_decoded =    100, tg =  14.95 t/s, tg_3s =  14.95 t/s
1.01.680.541 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.01.680.543 I srv  get_availabl: updating prompt cache
1.01.680.582 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.01.680.584 I srv         alloc:  - prompt is already in the cache, skipping
1.01.680.585 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.01.680.588 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.01.680.589 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.01.680.591 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.01.680.591 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.01.680.592 I srv  get_availabl: prompt cache update took 0.05 ms
1.01.680.659 I slot launch_slot_: id  8 | task 238 | processing task, is_child = 0
1.01.685.597 I slot   operator(): id  8 | task 238 | Checking checkpoint with [20, 20] against 24...
1.01.689.462 W slot   operator(): id  8 | task 238 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.01.689.497 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.050.044 I slot print_timing: id 11 | task 195 | n_decoded =    102, tg =  23.69 t/s, tg_3s =  23.69 t/s
1.02.293.027 I slot print_timing: id 13 | task 173 | n_decoded =    100, tg =  15.03 t/s, tg_3s =  15.03 t/s
1.02.293.946 I slot print_timing: id 15 | task 192 | n_decoded =    101, tg =  20.94 t/s, tg_3s =  20.94 t/s
1.02.526.633 I slot print_timing: id  0 | task 154 | prompt eval time =     141.45 ms /     4 tokens (   35.36 ms per token,    28.28 tokens per second)
1.02.526.636 I slot print_timing: id  0 | task 154 |        eval time =    8532.08 ms /   128 tokens (   66.66 ms per token,    15.00 tokens per second)
1.02.526.637 I slot print_timing: id  0 | task 154 |       total time =    8673.54 ms /   132 tokens
1.02.526.637 I slot print_timing: id  0 | task 154 |    graphs reused =          1
1.02.526.640 I slot print_timing: id  0 | task 154 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.02.526.656 I statistics        draft-mtp: #calls(b,g,a) =   61    183   2788, #gen drafts =   2803, #acc drafts =  2037, #gen tokens =   5592, #acc tokens =  3799, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.037, 627.441, 3.625 ms
1.02.526.687 I slot      release: id  0 | task 154 | stop processing: n_tokens = 156, truncated = 0
1.02.530.789 I slot print_timing: id  7 | task 183 | prompt eval time =     120.70 ms /     4 tokens (   30.18 ms per token,    33.14 tokens per second)
1.02.530.793 I slot print_timing: id  7 | task 183 |        eval time =    5918.45 ms /   125 tokens (   47.35 ms per token,    21.12 tokens per second)
1.02.530.793 I slot print_timing: id  7 | task 183 |       total time =    6039.15 ms /   129 tokens
1.02.530.794 I slot print_timing: id  7 | task 183 |    graphs reused =          1
1.02.530.797 I slot print_timing: id  7 | task 183 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.02.530.804 I statistics        draft-mtp: #calls(b,g,a) =   61    183   2795, #gen drafts =   2803, #acc drafts =  2042, #gen tokens =   5592, #acc tokens =  3808, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.037, 627.441, 3.633 ms
1.02.530.824 I slot      release: id  7 | task 183 | stop processing: n_tokens = 152, truncated = 0
1.02.535.794 I srv    operator(): Chat format: peg-native
1.02.539.705 I srv    operator(): Chat format: peg-native
1.02.642.570 I slot print_timing: id 11 | task 195 | prompt eval time =     261.66 ms /    25 tokens (   10.47 ms per token,    95.55 tokens per second)
1.02.642.573 I slot print_timing: id 11 | task 195 |        eval time =    4898.63 ms /   114 tokens (   42.97 ms per token,    23.27 tokens per second)
1.02.642.573 I slot print_timing: id 11 | task 195 |       total time =    5160.29 ms /   139 tokens
1.02.642.574 I slot print_timing: id 11 | task 195 |    graphs reused =          1
1.02.642.577 I slot print_timing: id 11 | task 195 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.02.642.589 I statistics        draft-mtp: #calls(b,g,a) =   61    184   2813, #gen drafts =   2817, #acc drafts =  2057, #gen tokens =   5620, #acc tokens =  3836, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.037, 632.086, 3.654 ms
1.02.642.615 I slot      release: id 11 | task 195 | stop processing: n_tokens = 140, truncated = 0
1.02.644.590 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.02.644.592 I srv  get_availabl: updating prompt cache
1.02.644.634 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.02.644.636 I srv         alloc:  - prompt is already in the cache, skipping
1.02.644.637 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.02.644.640 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.644.641 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.644.642 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.644.644 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.644.644 I srv  get_availabl: prompt cache update took 0.05 ms
1.02.644.712 I slot launch_slot_: id  0 | task 247 | processing task, is_child = 0
1.02.644.714 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
1.02.644.731 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.02.644.732 I srv         alloc:  - prompt is already in the cache, skipping
1.02.644.732 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
1.02.644.747 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.644.748 I srv         alloc:  - prompt is already in the cache, skipping
1.02.644.750 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.02.644.750 I srv  get_availabl: updating prompt cache
1.02.644.764 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.02.644.765 I srv         alloc:  - prompt is already in the cache, skipping
1.02.644.765 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.02.644.766 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.644.766 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.644.768 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.644.768 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.644.768 I srv  get_availabl: prompt cache update took 0.02 ms
1.02.644.789 I slot launch_slot_: id  7 | task 248 | processing task, is_child = 0
1.02.644.790 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
1.02.644.803 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.644.804 I srv         alloc:  - prompt is already in the cache, skipping
1.02.650.004 I slot   operator(): id  0 | task 247 | Checking checkpoint with [24, 24] against 28...
1.02.650.658 I srv    operator(): Chat format: peg-native
1.02.653.819 W slot   operator(): id  0 | task 247 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.02.653.844 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.653.871 I slot   operator(): id  7 | task 248 | Checking checkpoint with [23, 23] against 27...
1.02.657.451 W slot   operator(): id  7 | task 248 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.02.657.494 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.772.021 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.02.772.024 I srv  get_availabl: updating prompt cache
1.02.772.065 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.772.067 I srv         alloc:  - prompt is already in the cache, skipping
1.02.772.068 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.02.772.071 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.02.772.072 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.02.772.072 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.02.772.073 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.02.772.074 I srv  get_availabl: prompt cache update took 0.05 ms
1.02.772.144 I slot launch_slot_: id 11 | task 250 | processing task, is_child = 0
1.02.776.341 I slot   operator(): id 11 | task 250 | Checking checkpoint with [20, 20] against 24...
1.02.780.209 W slot   operator(): id 11 | task 250 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.02.780.243 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.386.214 I slot print_timing: id 15 | task 192 | prompt eval time =     256.04 ms /    28 tokens (    9.14 ms per token,   109.36 tokens per second)
1.03.386.217 I slot print_timing: id 15 | task 192 |        eval time =    5914.67 ms /   125 tokens (   47.32 ms per token,    21.13 tokens per second)
1.03.386.217 I slot print_timing: id 15 | task 192 |       total time =    6170.70 ms /   153 tokens
1.03.386.218 I slot print_timing: id 15 | task 192 |    graphs reused =          1
1.03.386.221 I slot print_timing: id 15 | task 192 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.03.386.234 I statistics        draft-mtp: #calls(b,g,a) =   64    190   2909, #gen drafts =   2909, #acc drafts =  2124, #gen tokens =   5803, #acc tokens =  3961, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.631), dur(b,g,a) = 0.039, 653.084, 3.770 ms
1.03.386.264 I slot      release: id 15 | task 192 | stop processing: n_tokens = 152, truncated = 0
1.03.394.762 I srv    operator(): Chat format: peg-native
1.03.492.005 I slot print_timing: id  1 | task 166 | prompt eval time =     134.89 ms /     4 tokens (   33.72 ms per token,    29.65 tokens per second)
1.03.492.008 I slot print_timing: id  1 | task 166 |        eval time =    8504.90 ms /   128 tokens (   66.44 ms per token,    15.05 tokens per second)
1.03.492.008 I slot print_timing: id  1 | task 166 |       total time =    8639.80 ms /   132 tokens
1.03.492.009 I slot print_timing: id  1 | task 166 |    graphs reused =          1
1.03.492.012 I slot print_timing: id  1 | task 166 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.03.492.025 I statistics        draft-mtp: #calls(b,g,a) =   64    191   2909, #gen drafts =   2923, #acc drafts =  2124, #gen tokens =   5831, #acc tokens =  3961, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.631), dur(b,g,a) = 0.039, 656.769, 3.770 ms
1.03.492.052 I slot      release: id  1 | task 166 | stop processing: n_tokens = 156, truncated = 0
1.03.498.665 I slot print_timing: id 12 | task 211 | n_decoded =    102, tg =  23.92 t/s, tg_3s =  23.92 t/s
1.03.499.243 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.03.499.244 I srv  get_availabl: updating prompt cache
1.03.499.285 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.03.499.287 I srv         alloc:  - prompt is already in the cache, skipping
1.03.499.288 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.03.499.292 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.03.499.294 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.03.499.295 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.03.499.295 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.03.499.295 I srv  get_availabl: prompt cache update took 0.05 ms
1.03.499.362 I slot launch_slot_: id  1 | task 257 | processing task, is_child = 0
1.03.499.363 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
1.03.499.381 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.03.499.382 I srv         alloc:  - prompt is already in the cache, skipping
1.03.500.667 I srv    operator(): Chat format: peg-native
1.03.503.021 I slot   operator(): id  1 | task 257 | Checking checkpoint with [24, 24] against 28...
1.03.507.034 W slot   operator(): id  1 | task 257 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.03.507.074 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.620.171 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.03.620.174 I srv  get_availabl: updating prompt cache
1.03.620.214 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.03.620.216 I srv         alloc:  - prompt is already in the cache, skipping
1.03.620.217 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.03.620.220 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.03.620.221 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.03.620.222 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.03.620.223 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.03.620.223 I srv  get_availabl: prompt cache update took 0.05 ms
1.03.620.289 I slot launch_slot_: id 15 | task 259 | processing task, is_child = 0
1.03.623.160 I slot   operator(): id 15 | task 259 | Checking checkpoint with [23, 23] against 27...
1.03.627.086 W slot   operator(): id 15 | task 259 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.03.627.125 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.860.543 I slot print_timing: id  2 | task 191 | n_decoded =    100, tg =  15.34 t/s, tg_3s =  15.34 t/s
1.04.101.778 I slot print_timing: id 13 | task 173 | prompt eval time =     261.91 ms /    29 tokens (    9.03 ms per token,   110.72 tokens per second)
1.04.101.782 I slot print_timing: id 13 | task 173 |        eval time =    8463.03 ms /   128 tokens (   66.12 ms per token,    15.12 tokens per second)
1.04.101.782 I slot print_timing: id 13 | task 173 |       total time =    8724.94 ms /   157 tokens
1.04.101.783 I slot print_timing: id 13 | task 173 |    graphs reused =          1
1.04.101.786 I slot print_timing: id 13 | task 173 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.04.101.800 I statistics        draft-mtp: #calls(b,g,a) =   66    196   2984, #gen drafts =   2999, #acc drafts =  2176, #gen tokens =   5982, #acc tokens =  4059, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.041, 672.833, 3.869 ms
1.04.101.829 I slot      release: id 13 | task 173 | stop processing: n_tokens = 156, truncated = 0
1.04.105.078 I slot print_timing: id  6 | task 210 | n_decoded =    101, tg =  21.34 t/s, tg_3s =  21.34 t/s
1.04.108.719 I slot print_timing: id 12 | task 211 | prompt eval time =     128.53 ms /     4 tokens (   32.13 ms per token,    31.12 tokens per second)
1.04.108.721 I slot print_timing: id 12 | task 211 |        eval time =    4873.48 ms /   114 tokens (   42.75 ms per token,    23.39 tokens per second)
1.04.108.722 I slot print_timing: id 12 | task 211 |       total time =    5002.02 ms /   118 tokens
1.04.108.722 I slot print_timing: id 12 | task 211 |    graphs reused =          1
1.04.108.725 I slot print_timing: id 12 | task 211 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.04.108.732 I statistics        draft-mtp: #calls(b,g,a) =   66    196   2997, #gen drafts =   2999, #acc drafts =  2186, #gen tokens =   5982, #acc tokens =  4078, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.041, 672.833, 3.885 ms
1.04.108.756 I slot      release: id 12 | task 211 | stop processing: n_tokens = 140, truncated = 0
1.04.110.534 I srv    operator(): Chat format: peg-native
1.04.117.385 I srv    operator(): Chat format: peg-native
1.04.218.735 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.04.218.738 I srv  get_availabl: updating prompt cache
1.04.218.779 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.04.218.782 I srv         alloc:  - prompt is already in the cache, skipping
1.04.218.783 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.04.218.785 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.04.218.786 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.04.218.788 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.04.218.788 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.04.218.789 I srv  get_availabl: prompt cache update took 0.05 ms
1.04.218.854 I slot launch_slot_: id 12 | task 265 | processing task, is_child = 0
1.04.218.855 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
1.04.218.870 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.218.872 I srv         alloc:  - prompt is already in the cache, skipping
1.04.218.874 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.04.218.875 I srv  get_availabl: updating prompt cache
1.04.218.888 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.218.889 I srv         alloc:  - prompt is already in the cache, skipping
1.04.218.890 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.04.218.891 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.04.218.891 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.04.218.892 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.04.218.893 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.04.218.893 I srv  get_availabl: prompt cache update took 0.02 ms
1.04.218.912 I slot launch_slot_: id 13 | task 266 | processing task, is_child = 0
1.04.222.604 I slot   operator(): id 12 | task 265 | Checking checkpoint with [20, 20] against 24...
1.04.226.463 W slot   operator(): id 12 | task 265 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.04.226.502 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.226.512 I slot   operator(): id 13 | task 266 | Checking checkpoint with [24, 24] against 28...
1.04.230.022 W slot   operator(): id 13 | task 266 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.04.230.057 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.587.691 I slot print_timing: id  9 | task 223 | n_decoded =    102, tg =  24.03 t/s, tg_3s =  24.03 t/s
1.05.076.253 I slot print_timing: id 10 | task 222 | n_decoded =    101, tg =  21.34 t/s, tg_3s =  21.34 t/s
1.05.195.199 I slot print_timing: id  6 | task 210 | prompt eval time =     267.13 ms /    28 tokens (    9.54 ms per token,   104.82 tokens per second)
1.05.195.202 I slot print_timing: id  6 | task 210 |        eval time =    5822.75 ms /   125 tokens (   46.58 ms per token,    21.47 tokens per second)
1.05.195.203 I slot print_timing: id  6 | task 210 |       total time =    6089.89 ms /   153 tokens
1.05.195.204 I slot print_timing: id  6 | task 210 |    graphs reused =          1
1.05.195.207 I slot print_timing: id  6 | task 210 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.05.195.222 I statistics        draft-mtp: #calls(b,g,a) =   68    205   3130, #gen drafts =   3139, #acc drafts =  2283, #gen tokens =   6262, #acc tokens =  4259, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.042, 700.236, 4.051 ms
1.05.195.251 I slot      release: id  6 | task 210 | stop processing: n_tokens = 152, truncated = 0
1.05.196.913 I slot print_timing: id  9 | task 223 | prompt eval time =     122.67 ms /     4 tokens (   30.67 ms per token,    32.61 tokens per second)
1.05.196.916 I slot print_timing: id  9 | task 223 |        eval time =    4854.69 ms /   114 tokens (   42.59 ms per token,    23.48 tokens per second)
1.05.196.916 I slot print_timing: id  9 | task 223 |       total time =    4977.36 ms /   118 tokens
1.05.196.917 I slot print_timing: id  9 | task 223 |    graphs reused =          1
1.05.196.919 I slot print_timing: id  9 | task 223 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.05.196.925 I statistics        draft-mtp: #calls(b,g,a) =   68    205   3133, #gen drafts =   3139, #acc drafts =  2286, #gen tokens =   6262, #acc tokens =  4264, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.631), dur(b,g,a) = 0.042, 700.236, 4.054 ms
1.05.196.949 I slot      release: id  9 | task 223 | stop processing: n_tokens = 140, truncated = 0
1.05.200.055 I slot print_timing: id 14 | task 203 | n_decoded =    100, tg =  15.19 t/s, tg_3s =  15.19 t/s
1.05.205.983 I srv    operator(): Chat format: peg-native
1.05.206.119 I srv    operator(): Chat format: peg-native
1.05.309.973 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.05.309.976 I srv  get_availabl: updating prompt cache
1.05.310.018 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.05.310.021 I srv         alloc:  - prompt is already in the cache, skipping
1.05.310.022 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.05.310.025 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.310.026 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.310.026 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.310.027 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.310.027 I srv  get_availabl: prompt cache update took 0.05 ms
1.05.310.095 I slot launch_slot_: id  6 | task 276 | processing task, is_child = 0
1.05.310.096 I slot process_sing: id  9 | task -1 | saving idle slot to prompt cache
1.05.310.112 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.05.310.113 I srv         alloc:  - prompt is already in the cache, skipping
1.05.310.115 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.05.310.116 I srv  get_availabl: updating prompt cache
1.05.310.128 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.05.310.129 I srv         alloc:  - prompt is already in the cache, skipping
1.05.310.130 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.05.310.130 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.310.131 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.310.132 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.310.133 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.310.133 I srv  get_availabl: prompt cache update took 0.02 ms
1.05.310.156 I slot launch_slot_: id  9 | task 277 | processing task, is_child = 0
1.05.315.467 I slot   operator(): id  6 | task 276 | Checking checkpoint with [23, 23] against 27...
1.05.319.411 W slot   operator(): id  6 | task 276 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.05.319.453 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.319.465 I slot   operator(): id  9 | task 277 | Checking checkpoint with [20, 20] against 24...
1.05.323.009 W slot   operator(): id  9 | task 277 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.05.323.047 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.677.985 I slot print_timing: id  2 | task 191 | prompt eval time =     129.87 ms /     4 tokens (   32.47 ms per token,    30.80 tokens per second)
1.05.677.991 I slot print_timing: id  2 | task 191 |        eval time =    8336.74 ms /   128 tokens (   65.13 ms per token,    15.35 tokens per second)
1.05.677.991 I slot print_timing: id  2 | task 191 |       total time =    8466.61 ms /   132 tokens
1.05.677.992 I slot print_timing: id  2 | task 191 |    graphs reused =          1
1.05.677.995 I slot print_timing: id  2 | task 191 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
1.05.678.010 I statistics        draft-mtp: #calls(b,g,a) =   70    209   3183, #gen drafts =   3198, #acc drafts =  2318, #gen tokens =   6379, #acc tokens =  4325, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.631), dur(b,g,a) = 0.043, 717.425, 4.110 ms
1.05.678.036 I slot      release: id  2 | task 191 | stop processing: n_tokens = 156, truncated = 0
1.05.686.392 I srv    operator(): Chat format: peg-native
1.05.801.170 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.05.801.172 I srv  get_availabl: updating prompt cache
1.05.801.214 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.05.801.217 I srv         alloc:  - prompt is already in the cache, skipping
1.05.801.218 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.05.801.220 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.05.801.221 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.05.801.222 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.05.801.224 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.05.801.224 I srv  get_availabl: prompt cache update took 0.05 ms
1.05.801.292 I slot launch_slot_: id  2 | task 282 | processing task, is_child = 0
1.05.805.074 I slot   operator(): id  2 | task 282 | Checking checkpoint with [24, 24] against 28...
1.05.809.032 W slot   operator(): id  2 | task 282 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.05.809.065 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.924.922 I slot print_timing: id  5 | task 232 | n_decoded =    101, tg =  21.34 t/s, tg_3s =  21.34 t/s
1.06.047.543 I slot print_timing: id  8 | task 238 | n_decoded =    102, tg =  24.02 t/s, tg_3s =  24.02 t/s
1.06.169.592 I slot print_timing: id 10 | task 222 | prompt eval time =     119.34 ms /     4 tokens (   29.84 ms per token,    33.52 tokens per second)
1.06.169.594 I slot print_timing: id 10 | task 222 |        eval time =    5827.09 ms /   125 tokens (   46.62 ms per token,    21.45 tokens per second)
1.06.169.595 I slot print_timing: id 10 | task 222 |       total time =    5946.43 ms /   129 tokens
1.06.169.596 I slot print_timing: id 10 | task 222 |    graphs reused =          1
1.06.169.599 I slot print_timing: id 10 | task 222 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.06.169.611 I statistics        draft-mtp: #calls(b,g,a) =   71    213   3255, #gen drafts =   3260, #acc drafts =  2370, #gen tokens =   6503, #acc tokens =  4420, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.630), dur(b,g,a) = 0.044, 730.753, 4.193 ms
1.06.169.649 I slot      release: id 10 | task 222 | stop processing: n_tokens = 152, truncated = 0
1.06.177.806 I srv    operator(): Chat format: peg-native
1.06.289.601 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.06.289.604 I srv  get_availabl: updating prompt cache
1.06.289.647 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.06.289.649 I srv         alloc:  - prompt is already in the cache, skipping
1.06.289.650 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.06.289.654 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.06.289.656 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.06.289.656 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.06.289.657 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.06.289.658 I srv  get_availabl: prompt cache update took 0.05 ms
1.06.289.726 I slot launch_slot_: id 10 | task 287 | processing task, is_child = 0
1.06.294.382 I slot   operator(): id 10 | task 287 | Checking checkpoint with [23, 23] against 27...
1.06.298.213 W slot   operator(): id 10 | task 287 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.06.298.260 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.657.935 I slot print_timing: id  8 | task 238 | prompt eval time =     116.21 ms /     4 tokens (   29.05 ms per token,    34.42 tokens per second)
1.06.657.940 I slot print_timing: id  8 | task 238 |        eval time =    4856.10 ms /   114 tokens (   42.60 ms per token,    23.48 tokens per second)
1.06.657.941 I slot print_timing: id  8 | task 238 |       total time =    4972.32 ms /   118 tokens
1.06.657.942 I slot print_timing: id  8 | task 238 |    graphs reused =          1
1.06.657.945 I slot print_timing: id  8 | task 238 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.06.657.958 I statistics        draft-mtp: #calls(b,g,a) =   72    217   3315, #gen drafts =   3322, #acc drafts =  2415, #gen tokens =   6627, #acc tokens =  4505, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.045, 745.409, 4.276 ms
1.06.657.984 I slot      release: id  8 | task 238 | stop processing: n_tokens = 140, truncated = 0
1.06.666.819 I srv    operator(): Chat format: peg-native
1.06.778.238 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.06.778.240 I srv  get_availabl: updating prompt cache
1.06.778.281 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.06.778.283 I srv         alloc:  - prompt is already in the cache, skipping
1.06.778.284 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.06.778.286 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.06.778.287 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.06.778.288 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.06.778.288 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.06.778.290 I srv  get_availabl: prompt cache update took 0.05 ms
1.06.778.356 I slot launch_slot_: id  8 | task 292 | processing task, is_child = 0
1.06.783.207 I slot   operator(): id  8 | task 292 | Checking checkpoint with [20, 20] against 24...
1.06.787.158 W slot   operator(): id  8 | task 292 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.06.787.197 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.901.118 I slot print_timing: id  4 | task 221 | n_decoded =    100, tg =  15.25 t/s, tg_3s =  15.25 t/s
1.07.019.766 I slot print_timing: id 14 | task 203 | prompt eval time =     261.21 ms /    29 tokens (    9.01 ms per token,   111.02 tokens per second)
1.07.019.769 I slot print_timing: id 14 | task 203 |        eval time =    8401.95 ms /   128 tokens (   65.64 ms per token,    15.23 tokens per second)
1.07.019.770 I slot print_timing: id 14 | task 203 |       total time =    8663.16 ms /   157 tokens
1.07.019.771 I slot print_timing: id 14 | task 203 |    graphs reused =          1
1.07.019.774 I slot print_timing: id 14 | task 203 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.07.019.789 I statistics        draft-mtp: #calls(b,g,a) =   73    220   3352, #gen drafts =   3367, #acc drafts =  2442, #gen tokens =   6716, #acc tokens =  4553, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.045, 758.803, 4.326 ms
1.07.019.818 I slot      release: id 14 | task 203 | stop processing: n_tokens = 156, truncated = 0
1.07.022.663 I slot print_timing: id  5 | task 232 | prompt eval time =     118.23 ms /     4 tokens (   29.56 ms per token,    33.83 tokens per second)
1.07.022.668 I slot print_timing: id  5 | task 232 |        eval time =    5831.30 ms /   125 tokens (   46.65 ms per token,    21.44 tokens per second)
1.07.022.668 I slot print_timing: id  5 | task 232 |       total time =    5949.53 ms /   129 tokens
1.07.022.668 I slot print_timing: id  5 | task 232 |    graphs reused =          1
1.07.022.671 I slot print_timing: id  5 | task 232 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.07.022.678 I statistics        draft-mtp: #calls(b,g,a) =   73    220   3358, #gen drafts =   3367, #acc drafts =  2446, #gen tokens =   6716, #acc tokens =  4559, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.629), dur(b,g,a) = 0.045, 758.803, 4.337 ms
1.07.022.698 I slot      release: id  5 | task 232 | stop processing: n_tokens = 152, truncated = 0
1.07.028.394 I srv    operator(): Chat format: peg-native
1.07.031.316 I srv    operator(): Chat format: peg-native
1.07.136.261 I slot print_timing: id 11 | task 250 | n_decoded =    102, tg =  24.04 t/s, tg_3s =  24.04 t/s
1.07.137.449 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.07.137.451 I srv  get_availabl: updating prompt cache
1.07.137.492 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.07.137.494 I srv         alloc:  - prompt is already in the cache, skipping
1.07.137.495 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.07.137.498 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.07.137.499 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.07.137.500 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.07.137.500 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.07.137.500 I srv  get_availabl: prompt cache update took 0.05 ms
1.07.137.565 I slot launch_slot_: id 14 | task 296 | processing task, is_child = 0
1.07.137.566 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.07.137.584 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.07.137.585 I srv         alloc:  - prompt is already in the cache, skipping
1.07.137.587 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.07.137.588 I srv  get_availabl: updating prompt cache
1.07.137.601 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.07.137.602 I srv         alloc:  - prompt is already in the cache, skipping
1.07.137.603 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.07.137.603 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.07.137.604 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.07.137.605 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.07.137.605 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.07.137.606 I srv  get_availabl: prompt cache update took 0.02 ms
1.07.137.623 I slot launch_slot_: id  5 | task 297 | processing task, is_child = 0
1.07.143.030 I slot   operator(): id  5 | task 297 | Checking checkpoint with [23, 23] against 27...
1.07.146.886 W slot   operator(): id  5 | task 297 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.07.146.940 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.146.947 I slot   operator(): id 14 | task 296 | Checking checkpoint with [24, 24] against 28...
1.07.150.511 W slot   operator(): id 14 | task 296 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.07.150.559 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.510.991 I slot print_timing: id  7 | task 248 | n_decoded =    101, tg =  21.29 t/s, tg_3s =  21.28 t/s
1.07.751.832 I slot print_timing: id  3 | task 231 | n_decoded =    100, tg =  15.24 t/s, tg_3s =  15.24 t/s
1.07.756.353 I slot print_timing: id 11 | task 250 | prompt eval time =     116.56 ms /     4 tokens (   29.14 ms per token,    34.32 tokens per second)
1.07.756.358 I slot print_timing: id 11 | task 250 |        eval time =    4863.44 ms /   114 tokens (   42.66 ms per token,    23.44 tokens per second)
1.07.756.358 I slot print_timing: id 11 | task 250 |       total time =    4980.00 ms /   118 tokens
1.07.756.360 I slot print_timing: id 11 | task 250 |    graphs reused =          1
1.07.756.362 I slot print_timing: id 11 | task 250 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.07.756.375 I statistics        draft-mtp: #calls(b,g,a) =   75    226   3455, #gen drafts =   3459, #acc drafts =  2517, #gen tokens =   6900, #acc tokens =  4694, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.046, 781.262, 4.466 ms
1.07.756.402 I slot      release: id 11 | task 250 | stop processing: n_tokens = 140, truncated = 0
1.07.764.993 I srv    operator(): Chat format: peg-native
1.07.874.116 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.07.874.118 I srv  get_availabl: updating prompt cache
1.07.874.162 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.07.874.164 I srv         alloc:  - prompt is already in the cache, skipping
1.07.874.165 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.07.874.168 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.07.874.169 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.07.874.170 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.07.874.170 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.07.874.171 I srv  get_availabl: prompt cache update took 0.05 ms
1.07.874.238 I slot launch_slot_: id 11 | task 304 | processing task, is_child = 0
1.07.878.459 I slot   operator(): id 11 | task 304 | Checking checkpoint with [20, 20] against 24...
1.07.882.300 W slot   operator(): id 11 | task 304 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.07.882.340 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.487.253 I slot print_timing: id 15 | task 259 | n_decoded =    101, tg =  21.27 t/s, tg_3s =  21.27 t/s
1.08.604.191 I slot print_timing: id  7 | task 248 | prompt eval time =     111.97 ms /     4 tokens (   27.99 ms per token,    35.72 tokens per second)
1.08.604.194 I slot print_timing: id  7 | task 248 |        eval time =    5838.31 ms /   125 tokens (   46.71 ms per token,    21.41 tokens per second)
1.08.604.194 I slot print_timing: id  7 | task 248 |       total time =    5950.29 ms /   129 tokens
1.08.604.195 I slot print_timing: id  7 | task 248 |    graphs reused =          1
1.08.604.198 I slot print_timing: id  7 | task 248 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.08.604.211 I statistics        draft-mtp: #calls(b,g,a) =   76    233   3561, #gen drafts =   3569, #acc drafts =  2592, #gen tokens =   7119, #acc tokens =  4833, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.629), dur(b,g,a) = 0.047, 803.976, 4.597 ms
1.08.604.237 I slot      release: id  7 | task 248 | stop processing: n_tokens = 152, truncated = 0
1.08.607.256 I slot print_timing: id 12 | task 265 | n_decoded =    102, tg =  23.92 t/s, tg_3s =  23.92 t/s
1.08.612.970 I srv    operator(): Chat format: peg-native
1.08.716.159 I slot print_timing: id  4 | task 221 | prompt eval time =     126.34 ms /     4 tokens (   31.58 ms per token,    31.66 tokens per second)
1.08.716.162 I slot print_timing: id  4 | task 221 |        eval time =    8374.19 ms /   128 tokens (   65.42 ms per token,    15.29 tokens per second)
1.08.716.162 I slot print_timing: id  4 | task 221 |       total time =    8500.53 ms /   132 tokens
1.08.716.163 I slot print_timing: id  4 | task 221 |    graphs reused =          1
1.08.716.166 I slot print_timing: id  4 | task 221 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.08.716.180 I statistics        draft-mtp: #calls(b,g,a) =   76    234   3569, #gen drafts =   3583, #acc drafts =  2599, #gen tokens =   7147, #acc tokens =  4846, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.630), dur(b,g,a) = 0.047, 809.381, 4.605 ms
1.08.716.202 I slot      release: id  4 | task 221 | stop processing: n_tokens = 156, truncated = 0
1.08.722.976 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.08.722.978 I srv  get_availabl: updating prompt cache
1.08.723.020 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.08.723.023 I srv         alloc:  - prompt is already in the cache, skipping
1.08.723.024 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.08.723.027 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.08.723.028 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.08.723.029 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.08.723.030 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.08.723.032 I srv  get_availabl: prompt cache update took 0.05 ms
1.08.723.099 I slot launch_slot_: id  4 | task 312 | processing task, is_child = 0
1.08.723.100 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
1.08.723.116 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.08.723.117 I srv         alloc:  - prompt is already in the cache, skipping
1.08.724.872 I srv    operator(): Chat format: peg-native
1.08.728.539 I slot   operator(): id  4 | task 312 | Checking checkpoint with [24, 24] against 28...
1.08.732.436 W slot   operator(): id  4 | task 312 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.08.732.478 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.845.085 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.08.845.087 I srv  get_availabl: updating prompt cache
1.08.845.126 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.08.845.128 I srv         alloc:  - prompt is already in the cache, skipping
1.08.845.129 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.08.845.132 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.08.845.133 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.08.845.133 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.08.845.135 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.08.845.135 I srv  get_availabl: prompt cache update took 0.05 ms
1.08.845.205 I slot launch_slot_: id  7 | task 314 | processing task, is_child = 0
1.08.850.059 I slot   operator(): id  7 | task 314 | Checking checkpoint with [23, 23] against 27...
1.08.853.982 W slot   operator(): id  7 | task 314 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.08.854.008 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.215.955 I slot print_timing: id 12 | task 265 | prompt eval time =     120.88 ms /     4 tokens (   30.22 ms per token,    33.09 tokens per second)
1.09.215.958 I slot print_timing: id 12 | task 265 |        eval time =    4872.45 ms /   114 tokens (   42.74 ms per token,    23.40 tokens per second)
1.09.215.958 I slot print_timing: id 12 | task 265 |       total time =    4993.33 ms /   118 tokens
1.09.215.960 I slot print_timing: id 12 | task 265 |    graphs reused =          1
1.09.215.963 I slot print_timing: id 12 | task 265 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.09.215.975 I statistics        draft-mtp: #calls(b,g,a) =   78    238   3641, #gen drafts =   3644, #acc drafts =  2647, #gen tokens =   7269, #acc tokens =  4936, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.049, 825.368, 4.691 ms
1.09.216.003 I slot      release: id 12 | task 265 | stop processing: n_tokens = 140, truncated = 0
1.09.224.194 I srv    operator(): Chat format: peg-native
1.09.326.624 I slot print_timing: id  0 | task 247 | n_decoded =    100, tg =  15.24 t/s, tg_3s =  15.24 t/s
1.09.332.884 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.09.332.886 I srv  get_availabl: updating prompt cache
1.09.332.926 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.09.332.928 I srv         alloc:  - prompt is already in the cache, skipping
1.09.332.929 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.09.332.933 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.332.935 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.332.935 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.332.936 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.332.937 I srv  get_availabl: prompt cache update took 0.05 ms
1.09.333.002 I slot launch_slot_: id 12 | task 319 | processing task, is_child = 0
1.09.336.890 I slot   operator(): id 12 | task 319 | Checking checkpoint with [20, 20] against 24...
1.09.340.851 W slot   operator(): id 12 | task 319 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.09.340.889 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.573.988 I slot print_timing: id  3 | task 231 | prompt eval time =     122.08 ms /     4 tokens (   30.52 ms per token,    32.76 tokens per second)
1.09.573.992 I slot print_timing: id  3 | task 231 |        eval time =    8382.92 ms /   128 tokens (   65.49 ms per token,    15.27 tokens per second)
1.09.573.993 I slot print_timing: id  3 | task 231 |       total time =    8505.01 ms /   132 tokens
1.09.573.994 I slot print_timing: id  3 | task 231 |    graphs reused =          1
1.09.573.997 I slot print_timing: id  3 | task 231 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.09.574.014 I statistics        draft-mtp: #calls(b,g,a) =   79    241   3674, #gen drafts =   3689, #acc drafts =  2668, #gen tokens =   7358, #acc tokens =  4974, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.050, 836.956, 4.729 ms
1.09.574.046 I slot      release: id  3 | task 231 | stop processing: n_tokens = 156, truncated = 0
1.09.582.038 I slot print_timing: id 15 | task 259 | prompt eval time =     116.03 ms /     4 tokens (   29.01 ms per token,    34.47 tokens per second)
1.09.582.040 I slot print_timing: id 15 | task 259 |        eval time =    5842.83 ms /   125 tokens (   46.74 ms per token,    21.39 tokens per second)
1.09.582.040 I slot print_timing: id 15 | task 259 |       total time =    5958.86 ms /   129 tokens
1.09.582.041 I slot print_timing: id 15 | task 259 |    graphs reused =          1
1.09.582.043 I slot print_timing: id 15 | task 259 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.09.582.050 I statistics        draft-mtp: #calls(b,g,a) =   79    241   3689, #gen drafts =   3689, #acc drafts =  2680, #gen tokens =   7358, #acc tokens =  4997, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.050, 836.956, 4.748 ms
1.09.582.074 I slot      release: id 15 | task 259 | stop processing: n_tokens = 152, truncated = 0
1.09.583.431 I srv    operator(): Chat format: peg-native
1.09.590.560 I srv    operator(): Chat format: peg-native
1.09.689.338 I slot print_timing: id  9 | task 277 | n_decoded =    102, tg =  23.99 t/s, tg_3s =  23.99 t/s
1.09.691.929 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.09.691.932 I srv  get_availabl: updating prompt cache
1.09.691.972 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.09.691.975 I srv         alloc:  - prompt is already in the cache, skipping
1.09.691.976 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.09.691.987 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.691.989 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.691.989 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.691.990 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.691.990 I srv  get_availabl: prompt cache update took 0.06 ms
1.09.692.055 I slot launch_slot_: id  3 | task 323 | processing task, is_child = 0
1.09.692.057 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
1.09.692.074 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.09.692.075 I srv         alloc:  - prompt is already in the cache, skipping
1.09.692.078 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.09.692.078 I srv  get_availabl: updating prompt cache
1.09.692.090 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.09.692.092 I srv         alloc:  - prompt is already in the cache, skipping
1.09.692.092 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.09.692.093 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.09.692.093 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.09.692.094 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.09.692.094 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.09.692.094 I srv  get_availabl: prompt cache update took 0.02 ms
1.09.692.113 I slot launch_slot_: id 15 | task 324 | processing task, is_child = 0
1.09.695.999 I slot   operator(): id  3 | task 323 | Checking checkpoint with [24, 24] against 28...
1.09.699.979 W slot   operator(): id  3 | task 323 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.09.700.019 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.700.037 I slot   operator(): id 15 | task 324 | Checking checkpoint with [23, 23] against 27...
1.09.703.524 W slot   operator(): id 15 | task 324 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.09.703.549 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.181.019 I slot print_timing: id  1 | task 257 | n_decoded =    100, tg =  15.22 t/s, tg_3s =  15.22 t/s
1.10.183.237 I slot print_timing: id  6 | task 276 | n_decoded =    101, tg =  21.28 t/s, tg_3s =  21.28 t/s
1.10.305.774 I slot print_timing: id  9 | task 277 | prompt eval time =     118.10 ms /     4 tokens (   29.52 ms per token,    33.87 tokens per second)
1.10.305.777 I slot print_timing: id  9 | task 277 |        eval time =    4868.19 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
1.10.305.777 I slot print_timing: id  9 | task 277 |       total time =    4986.28 ms /   118 tokens
1.10.305.778 I slot print_timing: id  9 | task 277 |    graphs reused =          1
1.10.305.781 I slot print_timing: id  9 | task 277 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.10.305.794 I statistics        draft-mtp: #calls(b,g,a) =   81    247   3775, #gen drafts =   3781, #acc drafts =  2744, #gen tokens =   7542, #acc tokens =  5115, #mean acc len = 2.35, #acc rate/pos = (0.727, 0.628), dur(b,g,a) = 0.051, 856.315, 4.862 ms
1.10.305.822 I slot      release: id  9 | task 277 | stop processing: n_tokens = 140, truncated = 0
1.10.314.724 I srv    operator(): Chat format: peg-native
1.10.425.277 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.10.425.280 I srv  get_availabl: updating prompt cache
1.10.425.320 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.10.425.323 I srv         alloc:  - prompt is already in the cache, skipping
1.10.425.324 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.10.425.327 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.10.425.328 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.10.425.329 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.10.425.330 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.10.425.331 I srv  get_availabl: prompt cache update took 0.05 ms
1.10.425.396 I slot launch_slot_: id  9 | task 331 | processing task, is_child = 0
1.10.430.042 I slot   operator(): id  9 | task 331 | Checking checkpoint with [20, 20] against 24...
1.10.433.920 W slot   operator(): id  9 | task 331 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.10.433.948 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.914.669 I slot print_timing: id 13 | task 266 | n_decoded =    100, tg =  15.22 t/s, tg_3s =  15.22 t/s
1.11.148.189 I slot print_timing: id  0 | task 247 | prompt eval time =     115.56 ms /     4 tokens (   28.89 ms per token,    34.61 tokens per second)
1.11.148.192 I slot print_timing: id  0 | task 247 |        eval time =    8382.59 ms /   128 tokens (   65.49 ms per token,    15.27 tokens per second)
1.11.148.192 I slot print_timing: id  0 | task 247 |       total time =    8498.16 ms /   132 tokens
1.11.148.193 I slot print_timing: id  0 | task 247 |    graphs reused =          1
1.11.148.196 I slot print_timing: id  0 | task 247 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.11.148.210 I statistics        draft-mtp: #calls(b,g,a) =   82    254   3875, #gen drafts =   3890, #acc drafts =  2813, #gen tokens =   7759, #acc tokens =  5245, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.052, 879.556, 4.983 ms
1.11.148.233 I slot      release: id  0 | task 247 | stop processing: n_tokens = 156, truncated = 0
1.11.153.006 I slot print_timing: id  8 | task 292 | n_decoded =    102, tg =  23.98 t/s, tg_3s =  23.98 t/s
1.11.154.222 I slot print_timing: id 10 | task 287 | n_decoded =    101, tg =  21.29 t/s, tg_3s =  21.29 t/s
1.11.156.932 I srv    operator(): Chat format: peg-native
1.11.266.130 I slot print_timing: id  6 | task 276 | prompt eval time =     121.82 ms /     4 tokens (   30.45 ms per token,    32.84 tokens per second)
1.11.266.133 I slot print_timing: id  6 | task 276 |        eval time =    5828.81 ms /   125 tokens (   46.63 ms per token,    21.45 tokens per second)
1.11.266.134 I slot print_timing: id  6 | task 276 |       total time =    5950.63 ms /   129 tokens
1.11.266.135 I slot print_timing: id  6 | task 276 |    graphs reused =          1
1.11.266.143 I slot print_timing: id  6 | task 276 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.11.266.154 I statistics        draft-mtp: #calls(b,g,a) =   82    255   3896, #gen drafts =   3905, #acc drafts =  2832, #gen tokens =   7789, #acc tokens =  5281, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.052, 882.354, 5.008 ms
1.11.266.177 I slot      release: id  6 | task 276 | stop processing: n_tokens = 152, truncated = 0
1.11.271.090 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.11.271.092 I srv  get_availabl: updating prompt cache
1.11.271.132 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.11.271.135 I srv         alloc:  - prompt is already in the cache, skipping
1.11.271.135 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.11.271.141 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.271.142 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.271.142 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.271.143 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.271.143 I srv  get_availabl: prompt cache update took 0.05 ms
1.11.271.208 I slot launch_slot_: id  0 | task 338 | processing task, is_child = 0
1.11.271.209 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
1.11.271.227 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.11.271.228 I srv         alloc:  - prompt is already in the cache, skipping
1.11.275.200 I srv    operator(): Chat format: peg-native
1.11.275.321 I slot   operator(): id  0 | task 338 | Checking checkpoint with [24, 24] against 28...
1.11.279.258 W slot   operator(): id  0 | task 338 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.11.279.290 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.392.015 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.11.392.017 I srv  get_availabl: updating prompt cache
1.11.392.057 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.11.392.060 I srv         alloc:  - prompt is already in the cache, skipping
1.11.392.061 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.11.392.063 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.392.064 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.392.065 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.392.066 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.392.067 I srv  get_availabl: prompt cache update took 0.05 ms
1.11.392.129 I slot launch_slot_: id  6 | task 341 | processing task, is_child = 0
1.11.396.496 I slot   operator(): id  6 | task 341 | Checking checkpoint with [23, 23] against 27...
1.11.400.396 W slot   operator(): id  6 | task 341 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.11.400.427 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.759.367 I slot print_timing: id  8 | task 292 | prompt eval time =     115.89 ms /     4 tokens (   28.97 ms per token,    34.51 tokens per second)
1.11.759.370 I slot print_timing: id  8 | task 292 |        eval time =    4860.25 ms /   114 tokens (   42.63 ms per token,    23.46 tokens per second)
1.11.759.371 I slot print_timing: id  8 | task 292 |       total time =    4976.14 ms /   118 tokens
1.11.759.372 I slot print_timing: id  8 | task 292 |    graphs reused =          1
1.11.759.376 I slot print_timing: id  8 | task 292 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.11.759.389 I statistics        draft-mtp: #calls(b,g,a) =   84    259   3959, #gen drafts =   3966, #acc drafts =  2875, #gen tokens =   7911, #acc tokens =  5363, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.053, 896.543, 5.098 ms
1.11.759.415 I slot      release: id  8 | task 292 | stop processing: n_tokens = 140, truncated = 0
1.11.768.060 I srv    operator(): Chat format: peg-native
1.11.879.233 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.11.879.236 I srv  get_availabl: updating prompt cache
1.11.879.275 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.11.879.277 I srv         alloc:  - prompt is already in the cache, skipping
1.11.879.278 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.11.879.281 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.11.879.282 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.11.879.283 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.11.879.284 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.11.879.285 I srv  get_availabl: prompt cache update took 0.05 ms
1.11.879.350 I slot launch_slot_: id  8 | task 346 | processing task, is_child = 0
1.11.884.749 I slot   operator(): id  8 | task 346 | Checking checkpoint with [20, 20] against 24...
1.11.888.643 W slot   operator(): id  8 | task 346 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.11.888.666 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.999.807 I slot print_timing: id  1 | task 257 | prompt eval time =     109.56 ms /     4 tokens (   27.39 ms per token,    36.51 tokens per second)
1.11.999.809 I slot print_timing: id  1 | task 257 |        eval time =    8387.20 ms /   128 tokens (   65.52 ms per token,    15.26 tokens per second)
1.11.999.810 I slot print_timing: id  1 | task 257 |       total time =    8496.76 ms /   132 tokens
1.11.999.811 I slot print_timing: id  1 | task 257 |    graphs reused =          1
1.11.999.814 I slot print_timing: id  1 | task 257 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.11.999.830 I statistics        draft-mtp: #calls(b,g,a) =   84    261   3981, #gen drafts =   3995, #acc drafts =  2891, #gen tokens =   7968, #acc tokens =  5392, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.053, 906.799, 5.128 ms
1.11.999.854 I slot      release: id  1 | task 257 | stop processing: n_tokens = 156, truncated = 0
1.12.002.437 I slot print_timing: id  5 | task 297 | n_decoded =    101, tg =  21.32 t/s, tg_3s =  21.32 t/s
1.12.008.616 I srv    operator(): Chat format: peg-native
1.12.123.642 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.12.123.644 I srv  get_availabl: updating prompt cache
1.12.123.686 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.123.689 I srv         alloc:  - prompt is already in the cache, skipping
1.12.123.690 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.12.123.692 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.123.693 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.123.694 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.123.696 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.123.696 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.123.759 I slot launch_slot_: id  1 | task 349 | processing task, is_child = 0
1.12.127.596 I slot   operator(): id  1 | task 349 | Checking checkpoint with [24, 24] against 28...
1.12.131.619 W slot   operator(): id  1 | task 349 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.12.131.645 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.248.852 I slot print_timing: id 10 | task 287 | prompt eval time =     116.79 ms /     4 tokens (   29.20 ms per token,    34.25 tokens per second)
1.12.248.854 I slot print_timing: id 10 | task 287 |        eval time =    5837.66 ms /   125 tokens (   46.70 ms per token,    21.41 tokens per second)
1.12.248.855 I slot print_timing: id 10 | task 287 |       total time =    5954.45 ms /   129 tokens
1.12.248.856 I slot print_timing: id 10 | task 287 |    graphs reused =          1
1.12.248.859 I slot print_timing: id 10 | task 287 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.12.248.871 I statistics        draft-mtp: #calls(b,g,a) =   86    263   4020, #gen drafts =   4025, #acc drafts =  2921, #gen tokens =   8028, #acc tokens =  5447, #mean acc len = 2.35, #acc rate/pos = (0.727, 0.628), dur(b,g,a) = 0.054, 914.274, 5.178 ms
1.12.248.897 I slot      release: id 10 | task 287 | stop processing: n_tokens = 152, truncated = 0
1.12.249.544 I slot print_timing: id 11 | task 304 | n_decoded =    102, tg =  23.97 t/s, tg_3s =  23.97 t/s
1.12.257.692 I srv    operator(): Chat format: peg-native
1.12.361.808 I slot print_timing: id  2 | task 282 | n_decoded =    100, tg =  15.53 t/s, tg_3s =  15.53 t/s
1.12.367.359 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.12.367.361 I srv  get_availabl: updating prompt cache
1.12.367.403 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.12.367.405 I srv         alloc:  - prompt is already in the cache, skipping
1.12.367.406 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.12.367.409 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.367.410 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.367.411 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.367.412 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.367.413 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.367.481 I slot launch_slot_: id 10 | task 352 | processing task, is_child = 0
1.12.372.120 I slot   operator(): id 10 | task 352 | Checking checkpoint with [23, 23] against 27...
1.12.376.053 W slot   operator(): id 10 | task 352 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.12.376.098 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.730.269 I slot print_timing: id 13 | task 266 | prompt eval time =     117.24 ms /     4 tokens (   29.31 ms per token,    34.12 tokens per second)
1.12.730.272 I slot print_timing: id 13 | task 266 |        eval time =    8386.48 ms /   128 tokens (   65.52 ms per token,    15.26 tokens per second)
1.12.730.272 I slot print_timing: id 13 | task 266 |       total time =    8503.72 ms /   132 tokens
1.12.730.273 I slot print_timing: id 13 | task 266 |    graphs reused =          1
1.12.730.276 I slot print_timing: id 13 | task 266 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.12.730.292 I statistics        draft-mtp: #calls(b,g,a) =   87    267   4071, #gen drafts =   4086, #acc drafts =  2955, #gen tokens =   8149, #acc tokens =  5512, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.054, 929.731, 5.244 ms
1.12.730.316 I slot      release: id 13 | task 266 | stop processing: n_tokens = 156, truncated = 0
1.12.739.688 I srv    operator(): Chat format: peg-native
1.12.853.384 I slot print_timing: id 11 | task 304 | prompt eval time =     116.41 ms /     4 tokens (   29.10 ms per token,    34.36 tokens per second)
1.12.853.386 I slot print_timing: id 11 | task 304 |        eval time =    4858.50 ms /   114 tokens (   42.62 ms per token,    23.46 tokens per second)
1.12.853.387 I slot print_timing: id 11 | task 304 |       total time =    4974.91 ms /   118 tokens
1.12.853.388 I slot print_timing: id 11 | task 304 |    graphs reused =          1
1.12.853.390 I slot print_timing: id 11 | task 304 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.12.853.402 I statistics        draft-mtp: #calls(b,g,a) =   87    268   4098, #gen drafts =   4101, #acc drafts =  2977, #gen tokens =   8179, #acc tokens =  5553, #mean acc len = 2.36, #acc rate/pos = (0.726, 0.629), dur(b,g,a) = 0.054, 933.511, 5.277 ms
1.12.853.429 I slot      release: id 11 | task 304 | stop processing: n_tokens = 140, truncated = 0
1.12.854.974 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.12.854.976 I srv  get_availabl: updating prompt cache
1.12.855.017 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.12.855.020 I srv         alloc:  - prompt is already in the cache, skipping
1.12.855.020 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.12.855.023 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.855.024 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.855.025 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.855.026 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.855.026 I srv  get_availabl: prompt cache update took 0.05 ms
1.12.855.089 I slot launch_slot_: id 11 | task 357 | processing task, is_child = 0
1.12.855.090 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
1.12.855.106 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.855.108 I srv         alloc:  - prompt is already in the cache, skipping
1.12.859.618 I slot   operator(): id 11 | task 357 | Checking checkpoint with [20, 20] against 24...
1.12.861.819 I srv    operator(): Chat format: peg-native
1.12.863.523 W slot   operator(): id 11 | task 357 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.12.863.553 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.976.932 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.12.976.934 I srv  get_availabl: updating prompt cache
1.12.976.985 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.976.988 I srv         alloc:  - prompt is already in the cache, skipping
1.12.976.989 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.12.976.992 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.12.976.994 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.12.976.995 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.12.976.997 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.12.976.997 I srv  get_availabl: prompt cache update took 0.06 ms
1.12.977.076 I slot launch_slot_: id 13 | task 359 | processing task, is_child = 0
1.12.980.899 I slot   operator(): id 13 | task 359 | Checking checkpoint with [24, 24] against 28...
1.12.984.866 W slot   operator(): id 13 | task 359 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.12.984.894 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.099.871 I slot print_timing: id  5 | task 297 | prompt eval time =     121.76 ms /     4 tokens (   30.44 ms per token,    32.85 tokens per second)
1.13.099.874 I slot print_timing: id  5 | task 297 |        eval time =    5835.04 ms /   125 tokens (   46.68 ms per token,    21.42 tokens per second)
1.13.099.874 I slot print_timing: id  5 | task 297 |       total time =    5956.81 ms /   129 tokens
1.13.099.875 I slot print_timing: id  5 | task 297 |    graphs reused =          1
1.13.099.878 I slot print_timing: id  5 | task 297 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.13.099.890 I statistics        draft-mtp: #calls(b,g,a) =   89    270   4121, #gen drafts =   4130, #acc drafts =  2992, #gen tokens =   8237, #acc tokens =  5579, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.056, 941.770, 5.311 ms
1.13.099.913 I slot      release: id  5 | task 297 | stop processing: n_tokens = 152, truncated = 0
1.13.108.257 I srv    operator(): Chat format: peg-native
1.13.221.287 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.13.221.290 I srv  get_availabl: updating prompt cache
1.13.221.332 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.13.221.335 I srv         alloc:  - prompt is already in the cache, skipping
1.13.221.337 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.13.221.341 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.13.221.343 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.13.221.344 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.13.221.345 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.13.221.345 I srv  get_availabl: prompt cache update took 0.05 ms
1.13.221.422 I slot launch_slot_: id  5 | task 362 | processing task, is_child = 0
1.13.225.873 I slot   operator(): id  5 | task 362 | Checking checkpoint with [23, 23] against 27...
1.13.229.783 W slot   operator(): id  5 | task 362 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.13.229.817 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.710.154 I slot print_timing: id  7 | task 314 | n_decoded =    101, tg =  21.30 t/s, tg_3s =  21.29 t/s
1.13.713.294 I slot print_timing: id 12 | task 319 | n_decoded =    102, tg =  23.95 t/s, tg_3s =  23.95 t/s
1.13.835.316 I slot print_timing: id 14 | task 296 | n_decoded =    100, tg =  15.22 t/s, tg_3s =  15.22 t/s
1.14.189.539 I slot print_timing: id  2 | task 282 | prompt eval time =     116.99 ms /     4 tokens (   29.25 ms per token,    34.19 tokens per second)
1.14.189.542 I slot print_timing: id  2 | task 282 |        eval time =    8267.45 ms /   128 tokens (   64.59 ms per token,    15.48 tokens per second)
1.14.189.542 I slot print_timing: id  2 | task 282 |       total time =    8384.44 ms /   132 tokens
1.14.189.543 I slot print_timing: id  2 | task 282 |    graphs reused =          1
1.14.189.546 I slot print_timing: id  2 | task 282 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
1.14.189.559 I statistics        draft-mtp: #calls(b,g,a) =   90    279   4256, #gen drafts =   4271, #acc drafts =  3089, #gen tokens =   8518, #acc tokens =  5761, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.057, 971.501, 5.484 ms
1.14.189.581 I slot      release: id  2 | task 282 | stop processing: n_tokens = 156, truncated = 0
1.14.198.858 I srv    operator(): Chat format: peg-native
1.14.311.072 I slot print_timing: id 12 | task 319 | prompt eval time =     116.78 ms /     4 tokens (   29.19 ms per token,    34.25 tokens per second)
1.14.311.075 I slot print_timing: id 12 | task 319 |        eval time =    4857.38 ms /   114 tokens (   42.61 ms per token,    23.47 tokens per second)
1.14.311.075 I slot print_timing: id 12 | task 319 |       total time =    4974.16 ms /   118 tokens
1.14.311.076 I slot print_timing: id 12 | task 319 |    graphs reused =          1
1.14.311.079 I slot print_timing: id 12 | task 319 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.14.311.090 I statistics        draft-mtp: #calls(b,g,a) =   90    280   4283, #gen drafts =   4286, #acc drafts =  3108, #gen tokens =   8548, #acc tokens =  5796, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.057, 975.315, 5.524 ms
1.14.311.117 I slot      release: id 12 | task 319 | stop processing: n_tokens = 140, truncated = 0
1.14.312.660 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.14.312.662 I srv  get_availabl: updating prompt cache
1.14.312.703 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.14.312.705 I srv         alloc:  - prompt is already in the cache, skipping
1.14.312.706 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.14.312.709 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.14.312.710 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.14.312.710 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.14.312.711 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.14.312.711 I srv  get_availabl: prompt cache update took 0.05 ms
1.14.312.774 I slot launch_slot_: id 12 | task 372 | processing task, is_child = 0
1.14.312.775 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.14.312.792 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.14.312.792 I srv         alloc:  - prompt is already in the cache, skipping
1.14.317.587 I slot   operator(): id 12 | task 372 | Checking checkpoint with [20, 20] against 24...
1.14.319.855 I srv    operator(): Chat format: peg-native
1.14.321.538 W slot   operator(): id 12 | task 372 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.14.321.572 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.435.960 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.14.435.962 I srv  get_availabl: updating prompt cache
1.14.436.002 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.14.436.005 I srv         alloc:  - prompt is already in the cache, skipping
1.14.436.006 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.14.436.010 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.14.436.012 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.14.436.012 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.14.436.013 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.14.436.013 I srv  get_availabl: prompt cache update took 0.05 ms
1.14.436.077 I slot launch_slot_: id  2 | task 374 | processing task, is_child = 0
1.14.439.886 I slot   operator(): id  2 | task 374 | Checking checkpoint with [24, 24] against 28...
1.14.443.755 W slot   operator(): id  2 | task 374 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.14.443.785 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.565.281 I slot print_timing: id 15 | task 324 | n_decoded =    101, tg =  21.27 t/s, tg_3s =  21.27 t/s
1.14.803.229 I slot print_timing: id  7 | task 314 | prompt eval time =     117.16 ms /     4 tokens (   29.29 ms per token,    34.14 tokens per second)
1.14.803.232 I slot print_timing: id  7 | task 314 |        eval time =    5835.98 ms /   125 tokens (   46.69 ms per token,    21.42 tokens per second)
1.14.803.233 I slot print_timing: id  7 | task 314 |       total time =    5953.14 ms /   129 tokens
1.14.803.234 I slot print_timing: id  7 | task 314 |    graphs reused =          1
1.14.803.238 I slot print_timing: id  7 | task 314 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.14.803.250 I statistics        draft-mtp: #calls(b,g,a) =   92    284   4339, #gen drafts =   4347, #acc drafts =  3151, #gen tokens =   8670, #acc tokens =  5876, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.059, 989.640, 5.598 ms
1.14.803.276 I slot      release: id  7 | task 314 | stop processing: n_tokens = 152, truncated = 0
1.14.804.505 I slot print_timing: id  9 | task 331 | n_decoded =    102, tg =  23.95 t/s, tg_3s =  23.95 t/s
1.14.812.320 I srv    operator(): Chat format: peg-native
1.14.923.522 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.14.923.525 I srv  get_availabl: updating prompt cache
1.14.923.565 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.14.923.568 I srv         alloc:  - prompt is already in the cache, skipping
1.14.923.569 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.14.923.572 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.14.923.574 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.14.923.574 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.14.923.575 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.14.923.576 I srv  get_availabl: prompt cache update took 0.05 ms
1.14.923.640 I slot launch_slot_: id  7 | task 379 | processing task, is_child = 0
1.14.928.493 I slot   operator(): id  7 | task 379 | Checking checkpoint with [23, 23] against 27...
1.14.932.337 W slot   operator(): id  7 | task 379 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.14.932.365 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.411.073 I slot print_timing: id  4 | task 312 | n_decoded =    100, tg =  15.22 t/s, tg_3s =  15.21 t/s
1.15.413.892 I slot print_timing: id  9 | task 331 | prompt eval time =     116.15 ms /     4 tokens (   29.04 ms per token,    34.44 tokens per second)
1.15.413.895 I slot print_timing: id  9 | task 331 |        eval time =    4867.68 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
1.15.413.895 I slot print_timing: id  9 | task 331 |       total time =    4983.83 ms /   118 tokens
1.15.413.896 I slot print_timing: id  9 | task 331 |    graphs reused =          1
1.15.413.900 I slot print_timing: id  9 | task 331 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.15.413.911 I statistics        draft-mtp: #calls(b,g,a) =   93    289   4419, #gen drafts =   4425, #acc drafts =  3206, #gen tokens =   8826, #acc tokens =  5979, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.060, 1007.928, 5.699 ms
1.15.413.937 I slot      release: id  9 | task 331 | stop processing: n_tokens = 140, truncated = 0
1.15.422.655 I srv    operator(): Chat format: peg-native
1.15.532.898 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.15.532.901 I srv  get_availabl: updating prompt cache
1.15.532.943 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.15.532.945 I srv         alloc:  - prompt is already in the cache, skipping
1.15.532.946 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.15.532.949 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.532.950 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.532.950 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.532.951 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.532.951 I srv  get_availabl: prompt cache update took 0.05 ms
1.15.533.015 I slot launch_slot_: id  9 | task 385 | processing task, is_child = 0
1.15.538.012 I slot   operator(): id  9 | task 385 | Checking checkpoint with [20, 20] against 24...
1.15.541.885 W slot   operator(): id  9 | task 385 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.15.541.905 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.653.055 I slot print_timing: id 14 | task 296 | prompt eval time =     118.13 ms /     4 tokens (   29.53 ms per token,    33.86 tokens per second)
1.15.653.058 I slot print_timing: id 14 | task 296 |        eval time =    8387.97 ms /   128 tokens (   65.53 ms per token,    15.26 tokens per second)
1.15.653.059 I slot print_timing: id 14 | task 296 |       total time =    8506.10 ms /   132 tokens
1.15.653.059 I slot print_timing: id 14 | task 296 |    graphs reused =          1
1.15.653.063 I slot print_timing: id 14 | task 296 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.15.653.075 I statistics        draft-mtp: #calls(b,g,a) =   94    291   4440, #gen drafts =   4454, #acc drafts =  3222, #gen tokens =   8883, #acc tokens =  6007, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.627), dur(b,g,a) = 0.060, 1017.248, 5.726 ms
1.15.653.099 I slot      release: id 14 | task 296 | stop processing: n_tokens = 156, truncated = 0
1.15.660.612 I slot print_timing: id 15 | task 324 | prompt eval time =     117.18 ms /     4 tokens (   29.30 ms per token,    34.14 tokens per second)
1.15.660.615 I slot print_timing: id 15 | task 324 |        eval time =    5843.36 ms /   125 tokens (   46.75 ms per token,    21.39 tokens per second)
1.15.660.615 I slot print_timing: id 15 | task 324 |       total time =    5960.54 ms /   129 tokens
1.15.660.616 I slot print_timing: id 15 | task 324 |    graphs reused =          1
1.15.660.619 I slot print_timing: id 15 | task 324 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.15.660.626 I statistics        draft-mtp: #calls(b,g,a) =   94    291   4454, #gen drafts =   4454, #acc drafts =  3233, #gen tokens =   8883, #acc tokens =  6028, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.060, 1017.248, 5.749 ms
1.15.660.648 I slot      release: id 15 | task 324 | stop processing: n_tokens = 152, truncated = 0
1.15.662.827 I srv    operator(): Chat format: peg-native
1.15.669.080 I srv    operator(): Chat format: peg-native
1.15.767.227 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.15.767.229 I srv  get_availabl: updating prompt cache
1.15.767.270 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.15.767.273 I srv         alloc:  - prompt is already in the cache, skipping
1.15.767.274 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.15.767.276 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.767.277 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.767.278 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.767.279 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.767.279 I srv  get_availabl: prompt cache update took 0.05 ms
1.15.767.345 I slot launch_slot_: id 14 | task 388 | processing task, is_child = 0
1.15.767.346 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
1.15.767.361 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.15.767.363 I srv         alloc:  - prompt is already in the cache, skipping
1.15.767.364 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.15.767.365 I srv  get_availabl: updating prompt cache
1.15.767.378 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.15.767.379 I srv         alloc:  - prompt is already in the cache, skipping
1.15.767.380 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.15.767.380 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.15.767.381 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.15.767.381 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.15.767.381 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.15.767.382 I srv  get_availabl: prompt cache update took 0.02 ms
1.15.767.401 I slot launch_slot_: id 15 | task 389 | processing task, is_child = 0
1.15.770.202 I slot   operator(): id 14 | task 388 | Checking checkpoint with [24, 24] against 28...
1.15.774.128 W slot   operator(): id 14 | task 388 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.15.774.166 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.774.189 I slot   operator(): id 15 | task 389 | Checking checkpoint with [23, 23] against 27...
1.15.777.694 W slot   operator(): id 15 | task 389 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.15.777.726 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.256.278 I slot print_timing: id  6 | task 341 | n_decoded =    101, tg =  21.29 t/s, tg_3s =  21.29 t/s
1.16.257.523 I slot print_timing: id  8 | task 346 | n_decoded =    102, tg =  23.96 t/s, tg_3s =  23.96 t/s
1.16.375.940 I slot print_timing: id  3 | task 323 | n_decoded =    100, tg =  15.25 t/s, tg_3s =  15.25 t/s
1.16.862.551 I slot print_timing: id  8 | task 346 | prompt eval time =     115.37 ms /     4 tokens (   28.84 ms per token,    34.67 tokens per second)
1.16.862.554 I slot print_timing: id  8 | task 346 |        eval time =    4862.41 ms /   114 tokens (   42.65 ms per token,    23.45 tokens per second)
1.16.862.554 I slot print_timing: id  8 | task 346 |       total time =    4977.78 ms /   118 tokens
1.16.862.555 I slot print_timing: id  8 | task 346 |    graphs reused =          1
1.16.862.558 I slot print_timing: id  8 | task 346 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.16.862.570 I statistics        draft-mtp: #calls(b,g,a) =   96    301   4603, #gen drafts =   4610, #acc drafts =  3337, #gen tokens =   9195, #acc tokens =  6221, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.062, 1045.894, 5.932 ms
1.16.862.594 I slot      release: id  8 | task 346 | stop processing: n_tokens = 140, truncated = 0
1.16.872.277 I srv    operator(): Chat format: peg-native
1.16.983.353 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.16.983.356 I srv  get_availabl: updating prompt cache
1.16.983.396 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.16.983.398 I srv         alloc:  - prompt is already in the cache, skipping
1.16.983.399 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.16.983.402 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.16.983.402 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.16.983.403 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.16.983.404 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.16.983.404 I srv  get_availabl: prompt cache update took 0.05 ms
1.16.983.466 I slot launch_slot_: id  8 | task 400 | processing task, is_child = 0
1.16.988.328 I slot   operator(): id  8 | task 400 | Checking checkpoint with [20, 20] against 24...
1.16.992.262 W slot   operator(): id  8 | task 400 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.16.992.297 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.225.373 I slot print_timing: id  4 | task 312 | prompt eval time =     110.05 ms /     4 tokens (   27.51 ms per token,    36.35 tokens per second)
1.17.225.377 I slot print_timing: id  4 | task 312 |        eval time =    8386.75 ms /   128 tokens (   65.52 ms per token,    15.26 tokens per second)
1.17.225.377 I slot print_timing: id  4 | task 312 |       total time =    8496.80 ms /   132 tokens
1.17.225.378 I slot print_timing: id  4 | task 312 |    graphs reused =          1
1.17.225.383 I slot print_timing: id  4 | task 312 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.17.225.397 I statistics        draft-mtp: #calls(b,g,a) =   97    304   4640, #gen drafts =   4655, #acc drafts =  3364, #gen tokens =   9284, #acc tokens =  6273, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.062, 1059.768, 5.975 ms
1.17.225.423 I slot      release: id  4 | task 312 | stop processing: n_tokens = 156, truncated = 0
1.17.230.916 I slot print_timing: id 10 | task 352 | n_decoded =    101, tg =  21.30 t/s, tg_3s =  21.30 t/s
1.17.231.536 I slot print_timing: id 11 | task 357 | n_decoded =    102, tg =  23.94 t/s, tg_3s =  23.93 t/s
1.17.234.645 I srv    operator(): Chat format: peg-native
1.17.344.633 I slot print_timing: id  6 | task 341 | prompt eval time =     116.51 ms /     4 tokens (   29.13 ms per token,    34.33 tokens per second)
1.17.344.636 I slot print_timing: id  6 | task 341 |        eval time =    5831.60 ms /   125 tokens (   46.65 ms per token,    21.43 tokens per second)
1.17.344.637 I slot print_timing: id  6 | task 341 |       total time =    5948.11 ms /   129 tokens
1.17.344.638 I slot print_timing: id  6 | task 341 |    graphs reused =          1
1.17.344.640 I slot print_timing: id  6 | task 341 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.17.344.651 I statistics        draft-mtp: #calls(b,g,a) =   97    305   4661, #gen drafts =   4670, #acc drafts =  3381, #gen tokens =   9314, #acc tokens =  6305, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.062, 1063.952, 6.005 ms
1.17.344.672 I slot      release: id  6 | task 341 | stop processing: n_tokens = 152, truncated = 0
1.17.349.677 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.17.349.679 I srv  get_availabl: updating prompt cache
1.17.349.721 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.17.349.723 I srv         alloc:  - prompt is already in the cache, skipping
1.17.349.724 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.17.349.727 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.349.728 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.349.729 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.349.729 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.349.730 I srv  get_availabl: prompt cache update took 0.05 ms
1.17.349.794 I slot launch_slot_: id  4 | task 404 | processing task, is_child = 0
1.17.349.796 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
1.17.349.812 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.17.349.814 I srv         alloc:  - prompt is already in the cache, skipping
1.17.353.480 I srv    operator(): Chat format: peg-native
1.17.354.776 I slot   operator(): id  4 | task 404 | Checking checkpoint with [24, 24] against 28...
1.17.358.665 W slot   operator(): id  4 | task 404 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.17.358.707 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.471.450 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.17.471.453 I srv  get_availabl: updating prompt cache
1.17.471.494 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.17.471.497 I srv         alloc:  - prompt is already in the cache, skipping
1.17.471.498 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.17.471.500 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.471.501 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.471.502 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.471.503 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.471.504 I srv  get_availabl: prompt cache update took 0.05 ms
1.17.471.568 I slot launch_slot_: id  6 | task 406 | processing task, is_child = 0
1.17.475.899 I slot   operator(): id  6 | task 406 | Checking checkpoint with [23, 23] against 27...
1.17.479.818 W slot   operator(): id  6 | task 406 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.17.479.845 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.840.269 I slot print_timing: id 11 | task 357 | prompt eval time =     110.36 ms /     4 tokens (   27.59 ms per token,    36.25 tokens per second)
1.17.840.272 I slot print_timing: id 11 | task 357 |        eval time =    4870.28 ms /   114 tokens (   42.72 ms per token,    23.41 tokens per second)
1.17.840.273 I slot print_timing: id 11 | task 357 |       total time =    4980.63 ms /   118 tokens
1.17.840.274 I slot print_timing: id 11 | task 357 |    graphs reused =          1
1.17.840.277 I slot print_timing: id 11 | task 357 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.17.840.287 I statistics        draft-mtp: #calls(b,g,a) =   99    309   4727, #gen drafts =   4731, #acc drafts =  3428, #gen tokens =   9436, #acc tokens =  6393, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.064, 1078.971, 6.091 ms
1.17.840.311 I slot      release: id 11 | task 357 | stop processing: n_tokens = 140, truncated = 0
1.17.848.962 I srv    operator(): Chat format: peg-native
1.17.951.938 I slot print_timing: id  0 | task 338 | n_decoded =    100, tg =  15.23 t/s, tg_3s =  15.23 t/s
1.17.958.375 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.17.958.378 I srv  get_availabl: updating prompt cache
1.17.958.417 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.17.958.419 I srv         alloc:  - prompt is already in the cache, skipping
1.17.958.420 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.17.958.422 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.17.958.423 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.17.958.424 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.17.958.424 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.17.958.425 I srv  get_availabl: prompt cache update took 0.05 ms
1.17.958.490 I slot launch_slot_: id 11 | task 411 | processing task, is_child = 0
1.17.962.645 I slot   operator(): id 11 | task 411 | Checking checkpoint with [20, 20] against 24...
1.17.966.553 W slot   operator(): id 11 | task 411 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.17.966.579 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.081.318 I slot print_timing: id  5 | task 362 | n_decoded =    101, tg =  21.31 t/s, tg_3s =  21.31 t/s
1.18.198.525 I slot print_timing: id  3 | task 323 | prompt eval time =     120.95 ms /     4 tokens (   30.24 ms per token,    33.07 tokens per second)
1.18.198.529 I slot print_timing: id  3 | task 323 |        eval time =    8381.55 ms /   128 tokens (   65.48 ms per token,    15.27 tokens per second)
1.18.198.529 I slot print_timing: id  3 | task 323 |       total time =    8502.50 ms /   132 tokens
1.18.198.530 I slot print_timing: id  3 | task 323 |    graphs reused =          1
1.18.198.534 I slot print_timing: id  3 | task 323 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.18.198.550 I statistics        draft-mtp: #calls(b,g,a) =  100    312   4761, #gen drafts =   4776, #acc drafts =  3450, #gen tokens =   9525, #acc tokens =  6433, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.064, 1091.108, 6.131 ms
1.18.198.573 I slot      release: id  3 | task 323 | stop processing: n_tokens = 156, truncated = 0
1.18.207.767 I srv    operator(): Chat format: peg-native
1.18.319.919 I slot print_timing: id 10 | task 352 | prompt eval time =     116.38 ms /     4 tokens (   29.10 ms per token,    34.37 tokens per second)
1.18.319.922 I slot print_timing: id 10 | task 352 |        eval time =    5831.39 ms /   125 tokens (   46.65 ms per token,    21.44 tokens per second)
1.18.319.923 I slot print_timing: id 10 | task 352 |       total time =    5947.77 ms /   129 tokens
1.18.319.924 I slot print_timing: id 10 | task 352 |    graphs reused =          1
1.18.319.928 I slot print_timing: id 10 | task 352 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.18.319.942 I statistics        draft-mtp: #calls(b,g,a) =  100    313   4786, #gen drafts =   4791, #acc drafts =  3469, #gen tokens =   9555, #acc tokens =  6471, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.064, 1095.023, 6.160 ms
1.18.319.968 I slot      release: id 10 | task 352 | stop processing: n_tokens = 152, truncated = 0
1.18.322.571 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.18.322.573 I srv  get_availabl: updating prompt cache
1.18.322.615 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.18.322.618 I srv         alloc:  - prompt is already in the cache, skipping
1.18.322.619 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.18.322.621 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.18.322.622 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.18.322.623 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.18.322.623 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.18.322.624 I srv  get_availabl: prompt cache update took 0.05 ms
1.18.322.690 I slot launch_slot_: id  3 | task 415 | processing task, is_child = 0
1.18.322.692 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.18.322.708 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.18.322.710 I srv         alloc:  - prompt is already in the cache, skipping
1.18.328.100 I slot   operator(): id  3 | task 415 | Checking checkpoint with [24, 24] against 28...
1.18.329.468 I srv    operator(): Chat format: peg-native
1.18.332.042 W slot   operator(): id  3 | task 415 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.18.332.079 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.445.099 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.18.445.101 I srv  get_availabl: updating prompt cache
1.18.445.145 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.18.445.148 I srv         alloc:  - prompt is already in the cache, skipping
1.18.445.149 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.18.445.152 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.18.445.154 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.18.445.154 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.18.445.154 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.18.445.155 I srv  get_availabl: prompt cache update took 0.05 ms
1.18.445.220 I slot launch_slot_: id 10 | task 417 | processing task, is_child = 0
1.18.449.428 I slot   operator(): id 10 | task 417 | Checking checkpoint with [23, 23] against 27...
1.18.453.302 W slot   operator(): id 10 | task 417 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.18.453.332 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.693.755 I slot print_timing: id 12 | task 372 | n_decoded =    102, tg =  23.91 t/s, tg_3s =  23.91 t/s
1.18.810.764 I slot print_timing: id  1 | task 349 | n_decoded =    100, tg =  15.23 t/s, tg_3s =  15.23 t/s
1.19.176.938 I slot print_timing: id  5 | task 362 | prompt eval time =     116.48 ms /     4 tokens (   29.12 ms per token,    34.34 tokens per second)
1.19.176.941 I slot print_timing: id  5 | task 362 |        eval time =    5834.54 ms /   125 tokens (   46.68 ms per token,    21.42 tokens per second)
1.19.176.941 I slot print_timing: id  5 | task 362 |       total time =    5951.03 ms /   129 tokens
1.19.176.942 I slot print_timing: id  5 | task 362 |    graphs reused =          1
1.19.176.946 I slot print_timing: id  5 | task 362 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.19.176.958 I statistics        draft-mtp: #calls(b,g,a) =  102    320   4890, #gen drafts =   4900, #acc drafts =  3546, #gen tokens =   9773, #acc tokens =  6612, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.065, 1119.013, 6.301 ms
1.19.176.985 I slot      release: id  5 | task 362 | stop processing: n_tokens = 152, truncated = 0
1.19.185.385 I srv    operator(): Chat format: peg-native
1.19.296.533 I slot print_timing: id 12 | task 372 | prompt eval time =     110.84 ms /     4 tokens (   27.71 ms per token,    36.09 tokens per second)
1.19.296.536 I slot print_timing: id 12 | task 372 |        eval time =    4868.09 ms /   114 tokens (   42.70 ms per token,    23.42 tokens per second)
1.19.296.536 I slot print_timing: id 12 | task 372 |       total time =    4978.93 ms /   118 tokens
1.19.296.537 I slot print_timing: id 12 | task 372 |    graphs reused =          1
1.19.296.540 I slot print_timing: id 12 | task 372 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.19.296.552 I statistics        draft-mtp: #calls(b,g,a) =  102    321   4912, #gen drafts =   4915, #acc drafts =  3561, #gen tokens =   9803, #acc tokens =  6641, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.065, 1123.222, 6.327 ms
1.19.296.575 I slot      release: id 12 | task 372 | stop processing: n_tokens = 140, truncated = 0
1.19.297.745 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.19.297.747 I srv  get_availabl: updating prompt cache
1.19.297.787 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.19.297.789 I srv         alloc:  - prompt is already in the cache, skipping
1.19.297.790 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.19.297.793 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.297.794 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.297.795 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.297.795 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.297.797 I srv  get_availabl: prompt cache update took 0.05 ms
1.19.297.861 I slot launch_slot_: id 12 | task 425 | processing task, is_child = 0
1.19.297.863 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.19.297.880 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.19.297.882 I srv         alloc:  - prompt is already in the cache, skipping
1.19.303.374 I slot   operator(): id 12 | task 425 | Checking checkpoint with [20, 20] against 24...
1.19.305.282 I srv    operator(): Chat format: peg-native
1.19.307.243 W slot   operator(): id 12 | task 425 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.19.307.277 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.420.223 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
1.19.420.226 I srv  get_availabl: updating prompt cache
1.19.420.269 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.19.420.271 I srv         alloc:  - prompt is already in the cache, skipping
1.19.420.272 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
1.19.420.275 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.420.277 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.420.277 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.420.278 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.420.278 I srv  get_availabl: prompt cache update took 0.05 ms
1.19.420.342 I slot launch_slot_: id  5 | task 427 | processing task, is_child = 0
1.19.424.563 I slot   operator(): id  5 | task 427 | Checking checkpoint with [23, 23] against 3...
1.19.424.565 W slot   operator(): id  5 | task 427 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.19.424.568 W slot   operator(): id  5 | task 427 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.19.426.474 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.575.554 I slot create_check: id  5 | task 427 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.19.694.363 I slot print_timing: id 13 | task 359 | n_decoded =    100, tg =  15.16 t/s, tg_3s =  15.16 t/s
1.19.807.387 I slot print_timing: id  0 | task 338 | prompt eval time =     110.10 ms /     4 tokens (   27.53 ms per token,    36.33 tokens per second)
1.19.807.390 I slot print_timing: id  0 | task 338 |        eval time =    8421.93 ms /   128 tokens (   65.80 ms per token,    15.20 tokens per second)
1.19.807.391 I slot print_timing: id  0 | task 338 |       total time =    8532.03 ms /   132 tokens
1.19.807.392 I slot print_timing: id  0 | task 338 |    graphs reused =          1
1.19.807.395 I slot print_timing: id  0 | task 338 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.19.807.412 I statistics        draft-mtp: #calls(b,g,a) =  104    325   4959, #gen drafts =   4974, #acc drafts =  3591, #gen tokens =   9920, #acc tokens =  6698, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.627), dur(b,g,a) = 0.066, 1140.018, 6.389 ms
1.19.807.438 I slot      release: id  0 | task 338 | stop processing: n_tokens = 156, truncated = 0
1.19.811.670 I slot print_timing: id  7 | task 379 | n_decoded =    101, tg =  21.19 t/s, tg_3s =  21.19 t/s
1.19.817.039 I srv    operator(): Chat format: peg-native
1.19.927.400 I slot print_timing: id  9 | task 385 | n_decoded =    102, tg =  23.86 t/s, tg_3s =  23.86 t/s
1.19.930.412 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
1.19.930.414 I srv  get_availabl: updating prompt cache
1.19.930.454 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.19.930.456 I srv         alloc:  - prompt is already in the cache, skipping
1.19.930.457 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.107
1.19.930.460 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.19.930.461 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.19.930.462 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.19.930.462 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.19.930.463 I srv  get_availabl: prompt cache update took 0.05 ms
1.19.930.532 I slot launch_slot_: id  0 | task 432 | processing task, is_child = 0
1.19.933.366 I slot   operator(): id  0 | task 432 | Checking checkpoint with [24, 24] against 3...
1.19.933.369 W slot   operator(): id  0 | task 432 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.19.933.372 W slot   operator(): id  0 | task 432 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.19.934.971 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.080.991 I slot create_check: id  0 | task 432 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.20.560.868 I slot print_timing: id  9 | task 385 | prompt eval time =     114.81 ms /     4 tokens (   28.70 ms per token,    34.84 tokens per second)
1.20.560.871 I slot print_timing: id  9 | task 385 |        eval time =    4908.02 ms /   114 tokens (   43.05 ms per token,    23.23 tokens per second)
1.20.560.871 I slot print_timing: id  9 | task 385 |       total time =    5022.84 ms /   118 tokens
1.20.560.872 I slot print_timing: id  9 | task 385 |    graphs reused =          1
1.20.560.875 I slot print_timing: id  9 | task 385 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.20.560.887 I statistics        draft-mtp: #calls(b,g,a) =  105    331   5061, #gen drafts =   5067, #acc drafts =  3665, #gen tokens =  10105, #acc tokens =  6835, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.067, 1157.216, 6.513 ms
1.20.560.912 I slot      release: id  9 | task 385 | stop processing: n_tokens = 140, truncated = 0
1.20.569.314 I srv    operator(): Chat format: peg-native
1.20.672.335 I slot print_timing: id  1 | task 349 | prompt eval time =     116.44 ms /     4 tokens (   29.11 ms per token,    34.35 tokens per second)
1.20.672.337 I slot print_timing: id  1 | task 349 |        eval time =    8428.28 ms /   128 tokens (   65.85 ms per token,    15.19 tokens per second)
1.20.672.338 I slot print_timing: id  1 | task 349 |       total time =    8544.71 ms /   132 tokens
1.20.672.338 I slot print_timing: id  1 | task 349 |    graphs reused =          1
1.20.672.341 I slot print_timing: id  1 | task 349 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.20.672.353 I statistics        draft-mtp: #calls(b,g,a) =  105    332   5067, #gen drafts =   5081, #acc drafts =  3670, #gen tokens =  10133, #acc tokens =  6845, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.627), dur(b,g,a) = 0.067, 1162.551, 6.520 ms
1.20.672.375 I slot      release: id  1 | task 349 | stop processing: n_tokens = 156, truncated = 0
1.20.679.600 I slot print_timing: id 15 | task 389 | n_decoded =    101, tg =  21.09 t/s, tg_3s =  21.09 t/s
1.20.679.611 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.20.679.611 I srv  get_availabl: updating prompt cache
1.20.679.651 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.20.679.653 I srv         alloc:  - prompt is already in the cache, skipping
1.20.679.654 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.20.679.657 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.20.679.658 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.20.679.659 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.20.679.659 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.20.679.660 I srv  get_availabl: prompt cache update took 0.05 ms
1.20.679.727 I slot launch_slot_: id  9 | task 439 | processing task, is_child = 0
1.20.679.729 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.20.679.746 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.20.679.747 I srv         alloc:  - prompt is already in the cache, skipping
1.20.681.506 I srv    operator(): Chat format: peg-native
1.20.685.186 I slot   operator(): id  9 | task 439 | Checking checkpoint with [20, 20] against 24...
1.20.689.089 W slot   operator(): id  9 | task 439 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.20.689.125 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.803.059 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.20.803.061 I srv  get_availabl: updating prompt cache
1.20.803.100 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.20.803.102 I srv         alloc:  - prompt is already in the cache, skipping
1.20.803.103 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.20.803.106 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.20.803.106 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.20.803.107 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.20.803.107 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.20.803.108 I srv  get_availabl: prompt cache update took 0.05 ms
1.20.803.176 I slot launch_slot_: id  1 | task 441 | processing task, is_child = 0
1.20.806.882 I slot   operator(): id  1 | task 441 | Checking checkpoint with [24, 24] against 28...
1.20.810.804 W slot   operator(): id  1 | task 441 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.20.810.828 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.926.681 I slot print_timing: id  7 | task 379 | prompt eval time =     116.35 ms /     4 tokens (   29.09 ms per token,    34.38 tokens per second)
1.20.926.683 I slot print_timing: id  7 | task 379 |        eval time =    5881.80 ms /   125 tokens (   47.05 ms per token,    21.25 tokens per second)
1.20.926.684 I slot print_timing: id  7 | task 379 |       total time =    5998.14 ms /   129 tokens
1.20.926.685 I slot print_timing: id  7 | task 379 |    graphs reused =          1
1.20.926.688 I slot print_timing: id  7 | task 379 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.20.926.700 I statistics        draft-mtp: #calls(b,g,a) =  107    334   5102, #gen drafts =   5110, #acc drafts =  3697, #gen tokens =  10191, #acc tokens =  6894, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.068, 1171.632, 6.562 ms
1.20.926.723 I slot      release: id  7 | task 379 | stop processing: n_tokens = 152, truncated = 0
1.20.935.111 I srv    operator(): Chat format: peg-native
1.21.041.493 I slot print_timing: id  2 | task 374 | n_decoded =    100, tg =  15.42 t/s, tg_3s =  15.42 t/s
1.21.047.641 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.21.047.643 I srv  get_availabl: updating prompt cache
1.21.047.682 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.21.047.685 I srv         alloc:  - prompt is already in the cache, skipping
1.21.047.686 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.21.047.688 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.21.047.689 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.21.047.690 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.21.047.690 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.21.047.691 I srv  get_availabl: prompt cache update took 0.05 ms
1.21.047.754 I slot launch_slot_: id  7 | task 444 | processing task, is_child = 0
1.21.052.659 I slot   operator(): id  7 | task 444 | Checking checkpoint with [23, 23] against 27...
1.21.056.532 W slot   operator(): id  7 | task 444 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.21.056.567 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.414.605 I slot print_timing: id  8 | task 400 | n_decoded =    102, tg =  23.67 t/s, tg_3s =  23.66 t/s
1.21.531.024 I slot print_timing: id 13 | task 359 | prompt eval time =     116.18 ms /     4 tokens (   29.04 ms per token,    34.43 tokens per second)
1.21.531.027 I slot print_timing: id 13 | task 359 |        eval time =    8433.92 ms /   128 tokens (   65.89 ms per token,    15.18 tokens per second)
1.21.531.028 I slot print_timing: id 13 | task 359 |       total time =    8550.10 ms /   132 tokens
1.21.531.028 I slot print_timing: id 13 | task 359 |    graphs reused =          1
1.21.531.031 I slot print_timing: id 13 | task 359 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.21.531.044 I statistics        draft-mtp: #calls(b,g,a) =  108    339   5172, #gen drafts =   5187, #acc drafts =  3745, #gen tokens =  10344, #acc tokens =  6985, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.069, 1190.838, 6.652 ms
1.21.531.069 I slot      release: id 13 | task 359 | stop processing: n_tokens = 156, truncated = 0
1.21.540.438 I srv    operator(): Chat format: peg-native
1.21.654.573 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.21.654.575 I srv  get_availabl: updating prompt cache
1.21.654.616 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.21.654.618 I srv         alloc:  - prompt is already in the cache, skipping
1.21.654.619 I srv          load:  - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.21.654.622 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.21.654.623 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.21.654.624 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.21.654.624 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.21.654.625 I srv  get_availabl: prompt cache update took 0.05 ms
1.21.654.690 I slot launch_slot_: id 13 | task 450 | processing task, is_child = 0
1.21.658.462 I slot   operator(): id 13 | task 450 | Checking checkpoint with [24, 24] against 3...
1.21.658.465 W slot   operator(): id 13 | task 450 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.21.658.468 W slot   operator(): id 13 | task 450 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.21.660.016 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.790.064 I slot print_timing: id 15 | task 389 | prompt eval time =     116.68 ms /     4 tokens (   29.17 ms per token,    34.28 tokens per second)
1.21.790.069 I slot print_timing: id 15 | task 389 |        eval time =    5899.16 ms /   125 tokens (   47.19 ms per token,    21.19 tokens per second)
1.21.790.069 I slot print_timing: id 15 | task 389 |       total time =    6015.84 ms /   129 tokens
1.21.790.070 I slot print_timing: id 15 | task 389 |    graphs reused =          1
1.21.790.074 I slot print_timing: id 15 | task 389 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.21.790.086 I statistics        draft-mtp: #calls(b,g,a) =  108    341   5217, #gen drafts =   5217, #acc drafts =  3778, #gen tokens =  10404, #acc tokens =  7046, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.069, 1198.330, 6.709 ms
1.21.790.116 I slot      release: id 15 | task 389 | stop processing: n_tokens = 152, truncated = 0
1.21.798.087 I srv    operator(): Chat format: peg-native
1.21.807.652 I slot create_check: id 13 | task 450 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.21.920.862 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
1.21.920.864 I srv  get_availabl: updating prompt cache
1.21.920.906 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.21.920.908 I srv         alloc:  - prompt is already in the cache, skipping
1.21.920.909 I srv          load:  - looking for better prompt, base f_keep = 0.020, sim = 0.103
1.21.920.912 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.21.920.914 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.21.920.914 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.21.920.915 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.21.920.915 I srv  get_availabl: prompt cache update took 0.05 ms
1.21.920.979 I slot launch_slot_: id 15 | task 453 | processing task, is_child = 0
1.21.923.859 I slot   operator(): id 15 | task 453 | Checking checkpoint with [23, 23] against 3...
1.21.923.862 W slot   operator(): id 15 | task 453 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.21.923.865 W slot   operator(): id 15 | task 453 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.21.925.309 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.052.529 I slot print_timing: id  8 | task 400 | prompt eval time =     116.10 ms /     4 tokens (   29.03 ms per token,    34.45 tokens per second)
1.22.052.532 I slot print_timing: id  8 | task 400 |        eval time =    4948.08 ms /   114 tokens (   43.40 ms per token,    23.04 tokens per second)
1.22.052.532 I slot print_timing: id  8 | task 400 |       total time =    5064.18 ms /   118 tokens
1.22.052.533 I slot print_timing: id  8 | task 400 |    graphs reused =          1
1.22.052.536 I slot print_timing: id  8 | task 400 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.22.052.549 I statistics        draft-mtp: #calls(b,g,a) =  109    343   5240, #gen drafts =   5246, #acc drafts =  3794, #gen tokens =  10462, #acc tokens =  7076, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.070, 1204.805, 6.740 ms
1.22.052.576 I slot      release: id  8 | task 400 | stop processing: n_tokens = 140, truncated = 0
1.22.060.654 I srv    operator(): Chat format: peg-native
1.22.073.514 I slot create_check: id 15 | task 453 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.22.186.486 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.22.186.488 I srv  get_availabl: updating prompt cache
1.22.186.528 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.22.186.530 I srv         alloc:  - prompt is already in the cache, skipping
1.22.186.531 I srv          load:  - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.22.186.534 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.22.186.535 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.22.186.535 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.22.186.536 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.22.186.536 I srv  get_availabl: prompt cache update took 0.05 ms
1.22.186.604 I slot launch_slot_: id  8 | task 456 | processing task, is_child = 0
1.22.191.568 I slot   operator(): id  8 | task 456 | Checking checkpoint with [20, 20] against 3...
1.22.191.571 W slot   operator(): id  8 | task 456 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.22.191.575 W slot   operator(): id  8 | task 456 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.22.193.200 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.342.801 I slot create_check: id  8 | task 456 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.22.459.279 I slot print_timing: id  6 | task 406 | n_decoded =    101, tg =  20.75 t/s, tg_3s =  20.75 t/s
1.22.461.724 I slot print_timing: id 11 | task 411 | n_decoded =    102, tg =  23.27 t/s, tg_3s =  23.27 t/s
1.22.584.465 I slot print_timing: id 14 | task 388 | n_decoded =    100, tg =  14.94 t/s, tg_3s =  14.94 t/s
1.22.938.376 I slot print_timing: id  2 | task 374 | prompt eval time =     116.95 ms /     4 tokens (   29.24 ms per token,    34.20 tokens per second)
1.22.938.379 I slot print_timing: id  2 | task 374 |        eval time =    8381.51 ms /   128 tokens (   65.48 ms per token,    15.27 tokens per second)
1.22.938.379 I slot print_timing: id  2 | task 374 |       total time =    8498.46 ms /   132 tokens
1.22.938.380 I slot print_timing: id  2 | task 374 |    graphs reused =          1
1.22.938.383 I slot print_timing: id  2 | task 374 | draft acceptance = 0.44361 (   59 accepted /   133 generated), mean acceptance length =  1.88, acceptance rate per position = (0.493, 0.388)
1.22.938.399 I statistics        draft-mtp: #calls(b,g,a) =  111    350   5338, #gen drafts =   5353, #acc drafts =  3865, #gen tokens =  10675, #acc tokens =  7208, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.072, 1231.780, 6.865 ms
1.22.938.426 I slot      release: id  2 | task 374 | stop processing: n_tokens = 156, truncated = 0
1.22.947.519 I srv    operator(): Chat format: peg-native
1.23.059.033 I slot print_timing: id 11 | task 411 | prompt eval time =     116.02 ms /     4 tokens (   29.01 ms per token,    34.48 tokens per second)
1.23.059.036 I slot print_timing: id 11 | task 411 |        eval time =    4980.34 ms /   114 tokens (   43.69 ms per token,    22.89 tokens per second)
1.23.059.037 I slot print_timing: id 11 | task 411 |       total time =    5096.37 ms /   118 tokens
1.23.059.037 I slot print_timing: id 11 | task 411 |    graphs reused =          1
1.23.059.040 I slot print_timing: id 11 | task 411 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.23.059.052 I statistics        draft-mtp: #calls(b,g,a) =  111    351   5364, #gen drafts =   5368, #acc drafts =  3883, #gen tokens =  10705, #acc tokens =  7241, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.072, 1235.558, 6.898 ms
1.23.059.079 I slot      release: id 11 | task 411 | stop processing: n_tokens = 140, truncated = 0
1.23.061.190 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.23.061.191 I srv  get_availabl: updating prompt cache
1.23.061.233 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.23.061.235 I srv         alloc:  - prompt is already in the cache, skipping
1.23.061.236 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.23.061.239 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.23.061.240 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.23.061.240 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.23.061.241 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.23.061.242 I srv  get_availabl: prompt cache update took 0.05 ms
1.23.061.308 I slot launch_slot_: id 11 | task 464 | processing task, is_child = 0
1.23.061.309 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
1.23.061.327 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.23.061.328 I srv         alloc:  - prompt is already in the cache, skipping
1.23.066.833 I slot   operator(): id 11 | task 464 | Checking checkpoint with [20, 20] against 24...
1.23.067.220 I srv    operator(): Chat format: peg-native
1.23.070.729 W slot   operator(): id 11 | task 464 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.23.070.771 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.184.676 I slot get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.23.184.678 I srv  get_availabl: updating prompt cache
1.23.184.720 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.23.184.722 I srv         alloc:  - prompt is already in the cache, skipping
1.23.184.723 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.23.184.726 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.23.184.727 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.23.184.727 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.23.184.728 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.23.184.730 I srv  get_availabl: prompt cache update took 0.05 ms
1.23.184.796 I slot launch_slot_: id  2 | task 466 | processing task, is_child = 0
1.23.188.591 I slot   operator(): id  2 | task 466 | Checking checkpoint with [24, 24] against 28...
1.23.192.590 W slot   operator(): id  2 | task 466 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.23.192.619 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.432.264 I slot print_timing: id 10 | task 417 | n_decoded =    101, tg =  20.76 t/s, tg_3s =  20.75 t/s
1.23.551.769 I slot print_timing: id  6 | task 406 | prompt eval time =     116.28 ms /     4 tokens (   29.07 ms per token,    34.40 tokens per second)
1.23.551.771 I slot print_timing: id  6 | task 406 |        eval time =    5959.55 ms /   125 tokens (   47.68 ms per token,    20.97 tokens per second)
1.23.551.772 I slot print_timing: id  6 | task 406 |       total time =    6075.82 ms /   129 tokens
1.23.551.773 I slot print_timing: id  6 | task 406 |    graphs reused =          1
1.23.551.776 I slot print_timing: id  6 | task 406 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.23.551.788 I statistics        draft-mtp: #calls(b,g,a) =  113    355   5420, #gen drafts =   5429, #acc drafts =  3925, #gen tokens =  10827, #acc tokens =  7321, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.627), dur(b,g,a) = 0.074, 1250.587, 6.968 ms
1.23.551.814 I slot      release: id  6 | task 406 | stop processing: n_tokens = 152, truncated = 0
1.23.559.926 I srv    operator(): Chat format: peg-native
1.23.672.684 I slot get_availabl: id  6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.23.672.686 I srv  get_availabl: updating prompt cache
1.23.672.728 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.23.672.730 I srv         alloc:  - prompt is already in the cache, skipping
1.23.672.731 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.23.672.734 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.23.672.735 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.23.672.736 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.23.672.736 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.23.672.737 I srv  get_availabl: prompt cache update took 0.05 ms
1.23.672.801 I slot launch_slot_: id  6 | task 471 | processing task, is_child = 0
1.23.677.539 I slot   operator(): id  6 | task 471 | Checking checkpoint with [23, 23] against 27...
1.23.681.473 W slot   operator(): id  6 | task 471 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.23.681.499 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.799.752 I slot print_timing: id 12 | task 425 | n_decoded =    102, tg =  23.26 t/s, tg_3s =  23.26 t/s
1.24.158.700 I slot print_timing: id  4 | task 404 | n_decoded =    100, tg =  14.94 t/s, tg_3s =  14.94 t/s
1.24.398.347 I slot print_timing: id 14 | task 388 | prompt eval time =     120.39 ms /     4 tokens (   30.10 ms per token,    33.22 tokens per second)
1.24.398.350 I slot print_timing: id 14 | task 388 |        eval time =    8507.72 ms /   128 tokens (   66.47 ms per token,    15.05 tokens per second)
1.24.398.350 I slot print_timing: id 14 | task 388 |       total time =    8628.12 ms /   132 tokens
1.24.398.351 I slot print_timing: id 14 | task 388 |    graphs reused =          1
1.24.398.354 I slot print_timing: id 14 | task 388 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.24.398.367 I statistics        draft-mtp: #calls(b,g,a) =  114    362   5523, #gen drafts =   5538, #acc drafts =  3998, #gen tokens =  11044, #acc tokens =  7454, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.074, 1274.769, 7.091 ms
1.24.398.392 I slot      release: id 14 | task 388 | stop processing: n_tokens = 156, truncated = 0
1.24.405.291 I slot print_timing: id 12 | task 425 | prompt eval time =     110.33 ms /     4 tokens (   27.58 ms per token,    36.25 tokens per second)
1.24.405.293 I slot print_timing: id 12 | task 425 |        eval time =    4991.57 ms /   114 tokens (   43.79 ms per token,    22.84 tokens per second)
1.24.405.294 I slot print_timing: id 12 | task 425 |       total time =    5101.90 ms /   118 tokens
1.24.405.294 I slot print_timing: id 12 | task 425 |    graphs reused =          1
1.24.405.297 I slot print_timing: id 12 | task 425 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.24.405.304 I statistics        draft-mtp: #calls(b,g,a) =  114    362   5536, #gen drafts =   5538, #acc drafts =  4008, #gen tokens =  11044, #acc tokens =  7473, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.074, 1274.769, 7.108 ms
1.24.405.327 I slot      release: id 12 | task 425 | stop processing: n_tokens = 140, truncated = 0
1.24.407.107 I srv    operator(): Chat format: peg-native
1.24.414.839 I srv    operator(): Chat format: peg-native
1.24.514.336 I slot print_timing: id 10 | task 417 | prompt eval time =     116.53 ms /     4 tokens (   29.13 ms per token,    34.33 tokens per second)
1.24.514.339 I slot print_timing: id 10 | task 417 |        eval time =    5948.33 ms /   125 tokens (   47.59 ms per token,    21.01 tokens per second)
1.24.514.340 I slot print_timing: id 10 | task 417 |       total time =    6064.86 ms /   129 tokens
1.24.514.341 I slot print_timing: id 10 | task 417 |    graphs reused =          1
1.24.514.344 I slot print_timing: id 10 | task 417 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.24.514.356 I statistics        draft-mtp: #calls(b,g,a) =  114    363   5549, #gen drafts =   5552, #acc drafts =  4018, #gen tokens =  11072, #acc tokens =  7492, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.074, 1279.473, 7.125 ms
1.24.514.385 I slot      release: id 10 | task 417 | stop processing: n_tokens = 152, truncated = 0
1.24.516.044 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.24.516.046 I srv  get_availabl: updating prompt cache
1.24.516.086 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.24.516.089 I srv         alloc:  - prompt is already in the cache, skipping
1.24.516.090 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.24.516.092 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.24.516.093 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.24.516.094 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.24.516.094 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.24.516.095 I srv  get_availabl: prompt cache update took 0.05 ms
1.24.516.165 I slot launch_slot_: id 12 | task 479 | processing task, is_child = 0
1.24.516.167 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.24.516.182 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.24.516.184 I srv         alloc:  - prompt is already in the cache, skipping
1.24.516.184 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
1.24.516.199 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.24.516.200 I srv         alloc:  - prompt is already in the cache, skipping
1.24.516.202 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.24.516.202 I srv  get_availabl: updating prompt cache
1.24.516.216 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.24.516.217 I srv         alloc:  - prompt is already in the cache, skipping
1.24.516.217 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.24.516.218 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.24.516.219 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.24.516.219 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.24.516.220 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.24.516.220 I srv  get_availabl: prompt cache update took 0.02 ms
1.24.516.240 I slot launch_slot_: id 14 | task 480 | processing task, is_child = 0
1.24.516.241 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.24.516.255 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.24.516.256 I srv         alloc:  - prompt is already in the cache, skipping
1.24.521.478 I slot   operator(): id 12 | task 479 | Checking checkpoint with [20, 20] against 24...
1.24.523.079 I srv    operator(): Chat format: peg-native
1.24.525.332 W slot   operator(): id 12 | task 479 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.24.525.380 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.525.386 I slot   operator(): id 14 | task 480 | Checking checkpoint with [24, 24] against 28...
1.24.528.969 W slot   operator(): id 14 | task 480 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.24.529.003 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.642.753 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.24.642.755 I srv  get_availabl: updating prompt cache
1.24.642.794 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.24.642.796 I srv         alloc:  - prompt is already in the cache, skipping
1.24.642.797 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.24.642.800 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.24.642.801 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.24.642.801 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.24.642.802 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.24.642.803 I srv  get_availabl: prompt cache update took 0.05 ms
1.24.642.869 I slot launch_slot_: id 10 | task 482 | processing task, is_child = 0
1.24.647.085 I slot   operator(): id 10 | task 482 | Checking checkpoint with [23, 23] against 27...
1.24.650.951 W slot   operator(): id 10 | task 482 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.24.650.979 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.007.916 I slot print_timing: id  0 | task 432 | n_decoded =    101, tg =  20.98 t/s, tg_3s =  20.98 t/s
1.25.130.648 I slot print_timing: id  3 | task 415 | n_decoded =    100, tg =  14.94 t/s, tg_3s =  14.94 t/s
1.25.134.285 I slot print_timing: id  9 | task 439 | n_decoded =    102, tg =  23.51 t/s, tg_3s =  23.51 t/s
1.25.739.108 I slot print_timing: id  9 | task 439 | prompt eval time =     110.37 ms /     4 tokens (   27.59 ms per token,    36.24 tokens per second)
1.25.739.111 I slot print_timing: id  9 | task 439 |        eval time =    4943.53 ms /   114 tokens (   43.36 ms per token,    23.06 tokens per second)
1.25.739.111 I slot print_timing: id  9 | task 439 |       total time =    5053.90 ms /   118 tokens
1.25.739.112 I slot print_timing: id  9 | task 439 |    graphs reused =          1
1.25.739.115 I slot print_timing: id  9 | task 439 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.25.739.129 I statistics        draft-mtp: #calls(b,g,a) =  117    373   5702, #gen drafts =   5708, #acc drafts =  4128, #gen tokens =  11384, #acc tokens =  7697, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.076, 1311.957, 7.321 ms
1.25.739.159 I slot      release: id  9 | task 439 | stop processing: n_tokens = 140, truncated = 0
1.25.747.785 I srv    operator(): Chat format: peg-native
1.25.857.529 I slot get_availabl: id  9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.25.857.532 I srv  get_availabl: updating prompt cache
1.25.857.573 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.25.857.575 I srv         alloc:  - prompt is already in the cache, skipping
1.25.857.576 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.25.857.579 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.25.857.580 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.25.857.580 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.25.857.581 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.25.857.582 I srv  get_availabl: prompt cache update took 0.05 ms
1.25.857.649 I slot launch_slot_: id  9 | task 493 | processing task, is_child = 0
1.25.863.170 I slot   operator(): id  9 | task 493 | Checking checkpoint with [20, 20] against 24...
1.25.867.076 W slot   operator(): id  9 | task 493 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.25.867.116 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.978.274 I slot print_timing: id  4 | task 404 | prompt eval time =     110.12 ms /     4 tokens (   27.53 ms per token,    36.32 tokens per second)
1.25.978.277 I slot print_timing: id  4 | task 404 |        eval time =    8513.35 ms /   128 tokens (   66.51 ms per token,    15.04 tokens per second)
1.25.978.278 I slot print_timing: id  4 | task 404 |       total time =    8623.47 ms /   132 tokens
1.25.978.278 I slot print_timing: id  4 | task 404 |    graphs reused =          1
1.25.978.282 I slot print_timing: id  4 | task 404 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.25.978.301 I statistics        draft-mtp: #calls(b,g,a) =  117    375   5723, #gen drafts =   5737, #acc drafts =  4141, #gen tokens =  11441, #acc tokens =  7723, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.076, 1321.715, 7.346 ms
1.25.978.326 I slot      release: id  4 | task 404 | stop processing: n_tokens = 156, truncated = 0
1.25.982.491 I slot print_timing: id  7 | task 444 | n_decoded =    101, tg =  20.98 t/s, tg_3s =  20.98 t/s
1.25.987.634 I srv    operator(): Chat format: peg-native
1.26.094.776 I slot print_timing: id  0 | task 432 | prompt eval time =     259.56 ms /    28 tokens (    9.27 ms per token,   107.88 tokens per second)
1.26.094.779 I slot print_timing: id  0 | task 432 |        eval time =    5901.80 ms /   125 tokens (   47.21 ms per token,    21.18 tokens per second)
1.26.094.779 I slot print_timing: id  0 | task 432 |       total time =    6161.35 ms /   153 tokens
1.26.094.780 I slot print_timing: id  0 | task 432 |    graphs reused =          1
1.26.094.783 I slot print_timing: id  0 | task 432 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.26.094.795 I statistics        draft-mtp: #calls(b,g,a) =  118    376   5738, #gen drafts =   5752, #acc drafts =  4154, #gen tokens =  11471, #acc tokens =  7746, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.077, 1325.944, 7.365 ms
1.26.094.813 I slot      release: id  0 | task 432 | stop processing: n_tokens = 152, truncated = 0
1.26.102.657 I slot get_availabl: id  4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.26.102.659 I srv  get_availabl: updating prompt cache
1.26.102.702 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.26.102.705 I srv         alloc:  - prompt is already in the cache, skipping
1.26.102.706 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.26.102.708 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.102.709 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.102.710 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.102.710 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.102.711 I srv  get_availabl: prompt cache update took 0.05 ms
1.26.102.778 I slot launch_slot_: id  4 | task 496 | processing task, is_child = 0
1.26.102.780 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
1.26.102.796 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.26.102.798 I srv         alloc:  - prompt is already in the cache, skipping
1.26.103.841 I srv    operator(): Chat format: peg-native
1.26.106.633 I slot   operator(): id  4 | task 496 | Checking checkpoint with [24, 24] against 28...
1.26.110.520 W slot   operator(): id  4 | task 496 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.26.110.562 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.222.457 I slot print_timing: id 13 | task 450 | n_decoded =    102, tg =  23.67 t/s, tg_3s =  23.67 t/s
1.26.223.029 I slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.26.223.032 I srv  get_availabl: updating prompt cache
1.26.223.072 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.26.223.075 I srv         alloc:  - prompt is already in the cache, skipping
1.26.223.075 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.26.223.078 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.223.079 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.223.080 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.223.080 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.223.081 I srv  get_availabl: prompt cache update took 0.05 ms
1.26.223.150 I slot launch_slot_: id  0 | task 498 | processing task, is_child = 0
1.26.225.975 I slot   operator(): id  0 | task 498 | Checking checkpoint with [23, 23] against 27...
1.26.229.810 W slot   operator(): id  0 | task 498 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.26.229.848 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.344.393 I slot print_timing: id  5 | task 427 | n_decoded =    100, tg =  15.02 t/s, tg_3s =  15.02 t/s
1.26.832.712 I slot print_timing: id 13 | task 450 | prompt eval time =     255.40 ms /    25 tokens (   10.22 ms per token,    97.89 tokens per second)
1.26.832.715 I slot print_timing: id 13 | task 450 |        eval time =    4918.84 ms /   114 tokens (   43.15 ms per token,    23.18 tokens per second)
1.26.832.716 I slot print_timing: id 13 | task 450 |       total time =    5174.23 ms /   139 tokens
1.26.832.717 I slot print_timing: id 13 | task 450 |    graphs reused =          1
1.26.832.719 I slot print_timing: id 13 | task 450 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.26.832.731 I statistics        draft-mtp: #calls(b,g,a) =  120    382   5843, #gen drafts =   5845, #acc drafts =  4227, #gen tokens =  11656, #acc tokens =  7881, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.078, 1344.036, 7.498 ms
1.26.832.759 I slot      release: id 13 | task 450 | stop processing: n_tokens = 140, truncated = 0
1.26.841.951 I srv    operator(): Chat format: peg-native
1.26.941.438 I slot print_timing: id  3 | task 415 | prompt eval time =     110.52 ms /     4 tokens (   27.63 ms per token,    36.19 tokens per second)
1.26.941.441 I slot print_timing: id  3 | task 415 |        eval time =    8502.79 ms /   128 tokens (   66.43 ms per token,    15.05 tokens per second)
1.26.941.442 I slot print_timing: id  3 | task 415 |       total time =    8613.31 ms /   132 tokens
1.26.941.443 I slot print_timing: id  3 | task 415 |    graphs reused =          1
1.26.941.446 I slot print_timing: id  3 | task 415 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.26.941.459 I statistics        draft-mtp: #calls(b,g,a) =  120    383   5845, #gen drafts =   5859, #acc drafts =  4228, #gen tokens =  11684, #acc tokens =  7883, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.078, 1348.821, 7.500 ms
1.26.941.484 I slot      release: id  3 | task 415 | stop processing: n_tokens = 156, truncated = 0
1.26.949.455 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.26.949.457 I srv  get_availabl: updating prompt cache
1.26.949.498 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.26.949.500 I srv         alloc:  - prompt is already in the cache, skipping
1.26.949.501 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.26.949.504 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.26.949.504 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.26.949.505 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.26.949.507 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.26.949.507 I srv  get_availabl: prompt cache update took 0.05 ms
1.26.949.570 I slot launch_slot_: id 13 | task 505 | processing task, is_child = 0
1.26.949.572 I slot process_sing: id  3 | task -1 | saving idle slot to prompt cache
1.26.949.589 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.26.949.591 I srv         alloc:  - prompt is already in the cache, skipping
1.26.950.838 I srv    operator(): Chat format: peg-native
1.26.954.393 I slot   operator(): id 13 | task 505 | Checking checkpoint with [20, 20] against 24...
1.26.958.254 W slot   operator(): id 13 | task 505 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.26.958.276 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.068.540 I slot print_timing: id  7 | task 444 | prompt eval time =     116.31 ms /     4 tokens (   29.08 ms per token,    34.39 tokens per second)
1.27.068.543 I slot print_timing: id  7 | task 444 |        eval time =    5899.54 ms /   125 tokens (   47.20 ms per token,    21.19 tokens per second)
1.27.068.543 I slot print_timing: id  7 | task 444 |       total time =    6015.85 ms /   129 tokens
1.27.068.544 I slot print_timing: id  7 | task 444 |    graphs reused =          1
1.27.068.547 I slot print_timing: id  7 | task 444 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.27.068.562 I statistics        draft-mtp: #calls(b,g,a) =  121    384   5866, #gen drafts =   5873, #acc drafts =  4245, #gen tokens =  11712, #acc tokens =  7917, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.078, 1353.592, 7.531 ms
1.27.068.592 I slot      release: id  7 | task 444 | stop processing: n_tokens = 152, truncated = 0
1.27.072.418 I slot get_availabl: id  3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.27.072.420 I srv  get_availabl: updating prompt cache
1.27.072.461 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.27.072.463 I srv         alloc:  - prompt is already in the cache, skipping
1.27.072.464 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.27.072.467 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.27.072.468 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.27.072.469 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.27.072.469 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.27.072.471 I srv  get_availabl: prompt cache update took 0.05 ms
1.27.072.536 I slot launch_slot_: id  3 | task 507 | processing task, is_child = 0
1.27.072.538 I slot process_sing: id  7 | task -1 | saving idle slot to prompt cache
1.27.072.554 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.27.072.555 I srv         alloc:  - prompt is already in the cache, skipping
1.27.076.825 I srv    operator(): Chat format: peg-native
1.27.077.808 I slot   operator(): id  3 | task 507 | Checking checkpoint with [24, 24] against 28...
1.27.081.756 W slot   operator(): id  3 | task 507 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.27.081.791 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.191.216 I slot print_timing: id  8 | task 456 | n_decoded =    101, tg =  21.33 t/s, tg_3s =  21.33 t/s
1.27.195.198 I slot get_availabl: id  7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.27.195.200 I srv  get_availabl: updating prompt cache
1.27.195.239 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.27.195.241 I srv         alloc:  - prompt is already in the cache, skipping
1.27.195.242 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.27.195.245 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.27.195.246 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.27.195.247 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.27.195.247 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.27.195.248 I srv  get_availabl: prompt cache update took 0.05 ms
1.27.195.311 I slot launch_slot_: id  7 | task 509 | processing task, is_child = 0
1.27.200.183 I slot   operator(): id  7 | task 509 | Checking checkpoint with [23, 23] against 27...
1.27.204.034 W slot   operator(): id  7 | task 509 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.27.204.067 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.443.628 I slot print_timing: id 11 | task 464 | n_decoded =    102, tg =  23.91 t/s, tg_3s =  23.91 t/s
1.27.559.620 I slot print_timing: id  1 | task 441 | n_decoded =    100, tg =  15.07 t/s, tg_3s =  15.07 t/s
1.28.050.075 I slot print_timing: id 11 | task 464 | prompt eval time =     110.60 ms /     4 tokens (   27.65 ms per token,    36.17 tokens per second)
1.28.050.080 I slot print_timing: id 11 | task 464 |        eval time =    4872.62 ms /   114 tokens (   42.74 ms per token,    23.40 tokens per second)
1.28.050.080 I slot print_timing: id 11 | task 464 |       total time =    4983.22 ms /   118 tokens
1.28.050.081 I slot print_timing: id 11 | task 464 |    graphs reused =          1
1.28.050.084 I slot print_timing: id 11 | task 464 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.28.050.097 I statistics        draft-mtp: #calls(b,g,a) =  123    392   5994, #gen drafts =   5998, #acc drafts =  4336, #gen tokens =  11961, #acc tokens =  8086, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.080, 1380.828, 7.686 ms
1.28.050.121 I slot      release: id 11 | task 464 | stop processing: n_tokens = 140, truncated = 0
1.28.058.595 I srv    operator(): Chat format: peg-native
1.28.162.524 I slot print_timing: id  5 | task 427 | prompt eval time =     262.77 ms /    29 tokens (    9.06 ms per token,   110.36 tokens per second)
1.28.162.527 I slot print_timing: id  5 | task 427 |        eval time =    8475.16 ms /   128 tokens (   66.21 ms per token,    15.10 tokens per second)
1.28.162.528 I slot print_timing: id  5 | task 427 |       total time =    8737.93 ms /   157 tokens
1.28.162.529 I slot print_timing: id  5 | task 427 |    graphs reused =          1
1.28.162.532 I slot print_timing: id  5 | task 427 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.28.162.547 I statistics        draft-mtp: #calls(b,g,a) =  123    393   5998, #gen drafts =   6012, #acc drafts =  4339, #gen tokens =  11989, #acc tokens =  8090, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.080, 1386.210, 7.690 ms
1.28.162.571 I slot      release: id  5 | task 427 | stop processing: n_tokens = 156, truncated = 0
1.28.169.472 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.28.169.474 I srv  get_availabl: updating prompt cache
1.28.169.514 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.28.169.516 I srv         alloc:  - prompt is already in the cache, skipping
1.28.169.517 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.28.169.520 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.169.522 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.169.522 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.169.522 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.169.523 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.169.591 I slot launch_slot_: id 11 | task 518 | processing task, is_child = 0
1.28.169.593 I slot process_sing: id  5 | task -1 | saving idle slot to prompt cache
1.28.169.611 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.28.169.612 I srv         alloc:  - prompt is already in the cache, skipping
1.28.171.384 I srv    operator(): Chat format: peg-native
1.28.175.107 I slot   operator(): id 11 | task 518 | Checking checkpoint with [20, 20] against 24...
1.28.178.929 W slot   operator(): id 11 | task 518 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.28.178.962 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.289.306 I slot print_timing: id  8 | task 456 | prompt eval time =     263.64 ms /    28 tokens (    9.42 ms per token,   106.21 tokens per second)
1.28.289.309 I slot print_timing: id  8 | task 456 |        eval time =    5834.05 ms /   125 tokens (   46.67 ms per token,    21.43 tokens per second)
1.28.289.309 I slot print_timing: id  8 | task 456 |       total time =    6097.68 ms /   153 tokens
1.28.289.310 I slot print_timing: id  8 | task 456 |    graphs reused =          1
1.28.289.313 I slot print_timing: id  8 | task 456 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.28.289.324 I statistics        draft-mtp: #calls(b,g,a) =  124    394   6020, #gen drafts =   6026, #acc drafts =  4352, #gen tokens =  12017, #acc tokens =  8116, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.081, 1391.667, 7.718 ms
1.28.289.352 I slot      release: id  8 | task 456 | stop processing: n_tokens = 152, truncated = 0
1.28.292.076 I slot get_availabl: id  5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.28.292.078 I srv  get_availabl: updating prompt cache
1.28.292.120 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.28.292.123 I srv         alloc:  - prompt is already in the cache, skipping
1.28.292.124 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.28.292.127 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.292.128 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.292.128 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.292.129 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.292.131 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.292.202 I slot launch_slot_: id  5 | task 520 | processing task, is_child = 0
1.28.292.203 I slot process_sing: id  8 | task -1 | saving idle slot to prompt cache
1.28.292.219 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.28.292.220 I srv         alloc:  - prompt is already in the cache, skipping
1.28.297.601 I slot   operator(): id  5 | task 520 | Checking checkpoint with [24, 24] against 28...
1.28.297.935 I srv    operator(): Chat format: peg-native
1.28.301.547 W slot   operator(): id  5 | task 520 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.28.301.580 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.415.515 I slot get_availabl: id  8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.28.415.518 I srv  get_availabl: updating prompt cache
1.28.415.557 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.28.415.559 I srv         alloc:  - prompt is already in the cache, skipping
1.28.415.560 I srv          load:  - looking for better prompt, base f_keep = 0.184, sim = 1.000
1.28.415.563 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.28.415.563 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.28.415.564 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.28.415.565 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.28.415.565 I srv  get_availabl: prompt cache update took 0.05 ms
1.28.415.631 I slot launch_slot_: id  8 | task 522 | processing task, is_child = 0
1.28.420.517 I slot   operator(): id  8 | task 522 | Checking checkpoint with [23, 23] against 27...
1.28.424.357 W slot   operator(): id  8 | task 522 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.28.424.387 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.540.955 I slot print_timing: id  6 | task 471 | n_decoded =    101, tg =  21.28 t/s, tg_3s =  21.28 t/s
1.28.788.279 I slot print_timing: id 15 | task 453 | n_decoded =    100, tg =  15.13 t/s, tg_3s =  15.13 t/s
1.28.907.459 I slot print_timing: id 12 | task 479 | n_decoded =    102, tg =  23.88 t/s, tg_3s =  23.88 t/s
1.29.384.983 I slot print_timing: id  1 | task 441 | prompt eval time =     116.38 ms /     4 tokens (   29.09 ms per token,    34.37 tokens per second)
1.29.384.986 I slot print_timing: id  1 | task 441 |        eval time =    8461.69 ms /   128 tokens (   66.11 ms per token,    15.13 tokens per second)
1.29.384.987 I slot print_timing: id  1 | task 441 |       total time =    8578.07 ms /   132 tokens
1.29.384.988 I slot print_timing: id  1 | task 441 |    graphs reused =          1
1.29.384.991 I slot print_timing: id  1 | task 441 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.29.385.007 I statistics        draft-mtp: #calls(b,g,a) =  126    403   6151, #gen drafts =   6166, #acc drafts =  4449, #gen tokens =  12296, #acc tokens =  8299, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.083, 1422.794, 7.870 ms
1.29.385.031 I slot      release: id  1 | task 441 | stop processing: n_tokens = 156, truncated = 0
1.29.393.752 I srv    operator(): Chat format: peg-native
1.29.505.114 I slot print_timing: id 10 | task 482 | n_decoded =    101, tg =  21.30 t/s, tg_3s =  21.30 t/s
1.29.506.351 I slot print_timing: id 12 | task 479 | prompt eval time =     114.74 ms /     4 tokens (   28.69 ms per token,    34.86 tokens per second)
1.29.506.353 I slot print_timing: id 12 | task 479 |        eval time =    4870.11 ms /   114 tokens (   42.72 ms per token,    23.41 tokens per second)
1.29.506.354 I slot print_timing: id 12 | task 479 |       total time =    4984.86 ms /   118 tokens
1.29.506.355 I slot print_timing: id 12 | task 479 |    graphs reused =          1
1.29.506.358 I slot print_timing: id 12 | task 479 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.29.506.370 I statistics        draft-mtp: #calls(b,g,a) =  126    404   6178, #gen drafts =   6181, #acc drafts =  4470, #gen tokens =  12326, #acc tokens =  8337, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.083, 1426.558, 7.904 ms
1.29.506.396 I slot      release: id 12 | task 479 | stop processing: n_tokens = 140, truncated = 0
1.29.507.900 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.29.507.902 I srv  get_availabl: updating prompt cache
1.29.507.943 W srv   prompt_save:  - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.29.507.945 I srv         alloc:  - prompt is already in the cache, skipping
1.29.507.946 I srv          load:  - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.29.507.949 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.29.507.949 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.29.507.950 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.29.507.951 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.29.507.951 I srv  get_availabl: prompt cache update took 0.05 ms
1.29.508.013 I slot launch_slot_: id 12 | task 532 | processing task, is_child = 0
1.29.508.015 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
1.29.508.032 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.29.508.034 I srv         alloc:  - prompt is already in the cache, skipping
1.29.512.721 I slot   operator(): id 12 | task 532 | Checking checkpoint with [20, 20] against 24...
1.29.514.348 I srv    operator(): Chat format: peg-native
1.29.516.628 W slot   operator(): id 12 | task 532 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.29.516.661 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.625.616 I slot print_timing: id  6 | task 471 | prompt eval time =     116.57 ms /     4 tokens (   29.14 ms per token,    34.31 tokens per second)
1.29.625.618 I slot print_timing: id  6 | task 471 |        eval time =    5831.47 ms /   125 tokens (   46.65 ms per token,    21.44 tokens per second)
1.29.625.619 I slot print_timing: id  6 | task 471 |       total time =    5948.04 ms /   129 tokens
1.29.625.620 I slot print_timing: id  6 | task 471 |    graphs reused =          1
1.29.625.623 I slot print_timing: id  6 | task 471 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.29.625.635 I statistics        draft-mtp: #calls(b,g,a) =  127    405   6187, #gen drafts =   6195, #acc drafts =  4476, #gen tokens =  12354, #acc tokens =  8347, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.084, 1431.216, 7.913 ms
1.29.625.660 I slot      release: id  6 | task 471 | stop processing: n_tokens = 152, truncated = 0
1.29.630.038 I slot get_availabl: id  1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.29.630.040 I srv  get_availabl: updating prompt cache
1.29.630.080 W srv   prompt_save:  - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.29.630.083 I srv         alloc:  - prompt is already in the cache, skipping
1.29.630.083 I srv          load:  - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.29.630.087 I srv        update:  - cache state: 3 prompts, 389.697 MiB (limits: 8192.000 MiB, 8192 tokens, 9417 est)
1.29.630.087 I srv        update:    - prompt 0x5f1c3ad9c7a0:     140 tokens, checkpoints:  1,   129.625 MiB
1.29.630.088 I srv        update:    - prompt 0x5f1c4478ecf0:     152 tokens, checkpoints:  1,   129.977 MiB
1.29.630.089 I srv        update:    - prompt 0x5f1c4479fe70:     156 tokens, checkpoints:  1,   130.094 MiB
1.29.630.089 I srv  get_availabl: prompt cache update took 0.05 ms
1.29.630.157 I slot launch_slot_: id  1 | task 534 | processing task, is_child = 0
1.29.630.159 I slot process_sing: id  6 | task -1 | saving idle slot to prompt cache
1.29.630.176 W srv   prompt_save:  - saving prompt with length 152, total state size = 66.976 MiB (draft: 1.191 MiB)
1.29.630.178 I srv         alloc:  - prompt is already in the cache, skipping
1.29.635.162 I slot   operator(): id  1 | task 534 | Checking checkpoint with [24, 24] against 28...
1.29.639.005 W slot   operator(): id  1 | task 534 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.29.639.039 I srv  stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.745.913 I slot print_timing: id  2 | task 466 | n_decoded =    100, tg =  15.53 t/s, tg_3s =  15.53 t/s
1.30.213.528 I slot print_timing: id  9 | task 493 | n_decoded =    102, tg =  24.09 t/s, tg_3s =  24.09 t/s
1.30.555.076 I slot print_timing: id 15 | task 453 | prompt eval time =     255.49 ms /    29 tokens (    8.81 ms per token,   113.51 tokens per second)
1.30.555.080 I slot print_timing: id 15 | task 453 |        eval time =    8375.69 ms /   128 tokens (   65.44 ms per token,    15.28 tokens per second)
1.30.555.081 I slot print_timing: id 15 | task 453 |       total time =    8631.19 ms /   157 tokens
1.30.555.082 I slot print_timing: id 15 | task 453 |    graphs reused =          1
1.30.555.085 I slot print_timing: id 15 | task 453 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.30.555.100 I statistics        draft-mtp: #calls(b,g,a) =  128    413   6299, #gen drafts =   6313, #acc drafts =  4552, #gen tokens =  12589, #acc tokens =  8489, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.085, 1466.681, 8.046 ms
1.30.555.129 I slot      release: id 15 | task 453 | stop processing: n_tokens = 156, truncated = 0
1.30.560.098 I slot print_timing: id 10 | task 482 | prompt eval time =     116.85 ms /     4 tokens (   29.21 ms per token,    34.23 tokens per second)
1.30.560.100 I slot print_timing: id 10 | task 482 |        eval time =    5796.14 ms /   125 tokens (   46.37 ms per token,    21.57 tokens per second)
1.30.560.101 I slot print_timing: id 10 | task 482 |       total time =    5912.99 ms /   129 tokens
1.30.560.101 I slot print_timing: id 10 | task 482 |    graphs reused =          1
1.30.560.104 I slot print_timing: id 10 | task 482 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.30.560.112 I statistics        draft-mtp: #calls(b,g,a) =  128    413   6309, #gen drafts =   6313, #acc drafts =  4559, #gen tokens =  12589, #acc tokens =  8502, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.085, 1466.681, 8.060 ms
1.30.560.135 I slot      release: id 10 | task 482 | stop processing: n_tokens = 152, truncated = 0
1.30.767.218 I slot print_timing: id  9 | task 493 | prompt eval time =     115.42 ms /     4 tokens (   28.86 ms per token,    34.66 tokens per second)
1.30.767.221 I slot print_timing: id  9 | task 493 |        eval time =    4788.61 ms /   114 tokens (   42.01 ms per token,    23.81 tokens per second)
1.30.767.222 I slot print_timing: id  9 | task 493 |       total time =    4904.03 ms /   118 tokens
1.30.767.223 I slot print_timing: id  9 | task 493 |    graphs reused =          1
1.30.767.226 I slot print_timing: id  9 | task 493 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.30.767.239 I statistics        draft-mtp: #calls(b,g,a) =  128    415   6335, #gen drafts =   6339, #acc drafts =  4577, #gen tokens =  12641, #acc tokens =  8535, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.085, 1477.122, 8.096 ms
1.30.767.263 I slot      release: id  9 | task 493 | stop processing: n_tokens = 140, truncated = 0
1.30.957.347 I slot print_timing: id  0 | task 498 | n_decoded =    101, tg =  21.88 t/s, tg_3s =  21.88 t/s
1.31.058.040 I slot print_timing: id 14 | task 480 | n_decoded =    100, tg =  15.57 t/s, tg_3s =  15.57 t/s
1.31.151.850 I slot print_timing: id 13 | task 505 | n_decoded =    102, tg =  24.96 t/s, tg_3s =  24.96 t/s
1.31.429.399 I slot print_timing: id  2 | task 466 | prompt eval time =     117.09 ms /     4 tokens (   29.27 ms per token,    34.16 tokens per second)
1.31.429.402 I slot print_timing: id  2 | task 466 |        eval time =    8123.69 ms /   128 tokens (   63.47 ms per token,    15.76 tokens per second)
1.31.429.403 I slot print_timing: id  2 | task 466 |       total time =    8240.78 ms /   132 tokens
1.31.429.403 I slot print_timing: id  2 | task 466 |    graphs reused =          1
1.31.429.406 I slot print_timing: id  2 | task 466 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.485, 0.368)
1.31.429.420 I statistics        draft-mtp: #calls(b,g,a) =  128    422   6411, #gen drafts =   6422, #acc drafts =  4631, #gen tokens =  12806, #acc tokens =  8636, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.085, 1512.521, 8.191 ms
1.31.429.449 I slot      release: id  2 | task 466 | stop processing: n_tokens = 156, truncated = 0
1.31.612.279 I slot print_timing: id 13 | task 505 | prompt eval time =     110.41 ms /     4 tokens (   27.60 ms per token,    36.23 tokens per second)
1.31.612.282 I slot print_timing: id 13 | task 505 |        eval time =    4547.46 ms /   114 tokens (   39.89 ms per token,    25.07 tokens per second)
1.31.612.283 I slot print_timing: id 13 | task 505 |       total time =    4657.87 ms /   118 tokens
1.31.612.284 I slot print_timing: id 13 | task 505 |    graphs reused =          1
1.31.612.287 I slot print_timing: id 13 | task 505 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.31.612.298 I statistics        draft-mtp: #calls(b,g,a) =  128    424   6443, #gen drafts =   6444, #acc drafts =  4652, #gen tokens =  12850, #acc tokens =  8674, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.085, 1523.390, 8.233 ms
1.31.612.322 I slot      release: id 13 | task 505 | stop processing: n_tokens = 140, truncated = 0
1.31.693.167 I slot print_timing: id  7 | task 509 | n_decoded =    101, tg =  23.08 t/s, tg_3s =  23.08 t/s
1.31.772.803 I slot print_timing: id  0 | task 498 | prompt eval time =     115.91 ms /     4 tokens (   28.98 ms per token,    34.51 tokens per second)
1.31.772.806 I slot print_timing: id  0 | task 498 |        eval time =    5430.89 ms /   125 tokens (   43.45 ms per token,    23.02 tokens per second)
1.31.772.806 I slot print_timing: id  0 | task 498 |       total time =    5546.80 ms /   129 tokens
1.31.772.807 I slot print_timing: id  0 | task 498 |    graphs reused =          1
1.31.772.810 I slot print_timing: id  0 | task 498 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.31.772.823 I statistics        draft-mtp: #calls(b,g,a) =  128    426   6455, #gen drafts =   6464, #acc drafts =  4660, #gen tokens =  12890, #acc tokens =  8690, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.085, 1534.920, 8.250 ms
1.31.772.842 I slot      release: id  0 | task 498 | stop processing: n_tokens = 152, truncated = 0
1.32.000.236 I slot print_timing: id 11 | task 518 | n_decoded =    102, tg =  27.46 t/s, tg_3s =  27.46 t/s
1.32.146.885 I slot print_timing: id  4 | task 496 | n_decoded =    100, tg =  16.86 t/s, tg_3s =  16.86 t/s
1.32.289.212 I slot print_timing: id 14 | task 480 | prompt eval time =     111.14 ms /     4 tokens (   27.78 ms per token,    35.99 tokens per second)
1.32.289.215 I slot print_timing: id 14 | task 480 |        eval time =    7652.66 ms /   128 tokens (   59.79 ms per token,    16.73 tokens per second)
1.32.289.215 I slot print_timing: id 14 | task 480 |       total time =    7763.80 ms /   132 tokens
1.32.289.216 I slot print_timing: id 14 | task 480 |    graphs reused =          1
1.32.289.220 I slot print_timing: id 14 | task 480 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.32.289.233 I statistics        draft-mtp: #calls(b,g,a) =  128    433   6518, #gen drafts =   6526, #acc drafts =  4704, #gen tokens =  13013, #acc tokens =  8772, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.085, 1573.033, 8.333 ms
1.32.289.256 I slot      release: id 14 | task 480 | stop processing: n_tokens = 156, truncated = 0
1.32.357.607 I slot print_timing: id  7 | task 509 | prompt eval time =     116.34 ms /     4 tokens (   29.09 ms per token,    34.38 tokens per second)
1.32.357.610 I slot print_timing: id  7 | task 509 |        eval time =    5041.06 ms /   125 tokens (   40.33 ms per token,    24.80 tokens per second)
1.32.357.611 I slot print_timing: id  7 | task 509 |       total time =    5157.40 ms /   129 tokens
1.32.357.612 I slot print_timing: id  7 | task 509 |    graphs reused =          1
1.32.357.615 I slot print_timing: id  7 | task 509 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.32.357.627 I statistics        draft-mtp: #calls(b,g,a) =  128    434   6531, #gen drafts =   6534, #acc drafts =  4714, #gen tokens =  13029, #acc tokens =  8789, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.085, 1577.653, 8.348 ms
1.32.357.648 I slot      release: id  7 | task 509 | stop processing: n_tokens = 152, truncated = 0
1.32.358.837 I slot print_timing: id 11 | task 518 | prompt eval time =     110.50 ms /     4 tokens (   27.62 ms per token,    36.20 tokens per second)
1.32.358.840 I slot print_timing: id 11 | task 518 |        eval time =    4073.22 ms /   114 tokens (   35.73 ms per token,    27.99 tokens per second)
1.32.358.840 I slot print_timing: id 11 | task 518 |       total time =    4183.72 ms /   118 tokens
1.32.358.841 I slot print_timing: id 11 | task 518 |    graphs reused =          1
1.32.358.843 I slot print_timing: id 11 | task 518 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.32.358.849 I statistics        draft-mtp: #calls(b,g,a) =  128    434   6533, #gen drafts =   6534, #acc drafts =  4716, #gen tokens =  13029, #acc tokens =  8793, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.085, 1577.653, 8.350 ms
1.32.358.866 I slot      release: id 11 | task 518 | stop processing: n_tokens = 140, truncated = 0
1.32.409.279 I slot print_timing: id  8 | task 522 | n_decoded =    101, tg =  26.08 t/s, tg_3s =  26.08 t/s
1.32.610.818 I slot print_timing: id  3 | task 507 | n_decoded =    100, tg =  18.44 t/s, tg_3s =  18.44 t/s
1.32.664.152 I slot print_timing: id 12 | task 532 | n_decoded =    102, tg =  33.54 t/s, tg_3s =  33.54 t/s
1.32.866.335 I slot print_timing: id  8 | task 522 | prompt eval time =     116.26 ms /     4 tokens (   29.06 ms per token,    34.41 tokens per second)
1.32.866.337 I slot print_timing: id  8 | task 522 |        eval time =    4329.54 ms /   125 tokens (   34.64 ms per token,    28.87 tokens per second)
1.32.866.338 I slot print_timing: id  8 | task 522 |       total time =    4445.80 ms /   129 tokens
1.32.866.339 I slot print_timing: id  8 | task 522 |    graphs reused =          1
1.32.866.342 I slot print_timing: id  8 | task 522 | draft acceptance = 0.79167 (   76 accepted /    96 generated), mean acceptance length =  2.58, acceptance rate per position = (0.833, 0.750)
1.32.866.354 I statistics        draft-mtp: #calls(b,g,a) =  128    444   6593, #gen drafts =   6594, #acc drafts =  4752, #gen tokens =  13149, #acc tokens =  8860, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.623), dur(b,g,a) = 0.085, 1620.753, 8.419 ms
1.32.866.374 I slot      release: id  8 | task 522 | stop processing: n_tokens = 152, truncated = 0
1.32.908.067 I slot print_timing: id 12 | task 532 | prompt eval time =     110.15 ms /     4 tokens (   27.54 ms per token,    36.31 tokens per second)
1.32.908.070 I slot print_timing: id 12 | task 532 |        eval time =    3285.18 ms /   114 tokens (   28.82 ms per token,    34.70 tokens per second)
1.32.908.070 I slot print_timing: id 12 | task 532 |       total time =    3395.33 ms /   118 tokens
1.32.908.071 I slot print_timing: id 12 | task 532 |    graphs reused =          1
1.32.908.074 I slot print_timing: id 12 | task 532 | draft acceptance = 0.93750 (   75 accepted /    80 generated), mean acceptance length =  2.88, acceptance rate per position = (1.000, 0.875)
1.32.908.085 I statistics        draft-mtp: #calls(b,g,a) =  128    445   6599, #gen drafts =   6599, #acc drafts =  4756, #gen tokens =  13158, #acc tokens =  8867, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.623), dur(b,g,a) = 0.085, 1624.148, 8.426 ms
1.32.908.102 I slot      release: id 12 | task 532 | stop processing: n_tokens = 140, truncated = 0
1.32.938.438 I slot print_timing: id  4 | task 496 | prompt eval time =     109.49 ms /     4 tokens (   27.37 ms per token,    36.53 tokens per second)
1.32.938.441 I slot print_timing: id  4 | task 496 |        eval time =    6722.30 ms /   128 tokens (   52.52 ms per token,    19.04 tokens per second)
1.32.938.441 I slot print_timing: id  4 | task 496 |       total time =    6831.79 ms /   132 tokens
1.32.938.442 I slot print_timing: id  4 | task 496 |    graphs reused =          1
1.32.938.445 I slot print_timing: id  4 | task 496 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.32.938.456 I statistics        draft-mtp: #calls(b,g,a) =  128    446   6599, #gen drafts =   6602, #acc drafts =  4756, #gen tokens =  13164, #acc tokens =  8867, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.623), dur(b,g,a) = 0.085, 1627.087, 8.426 ms
1.32.938.472 I slot      release: id  4 | task 496 | stop processing: n_tokens = 156, truncated = 0
1.33.019.499 I slot print_timing: id  5 | task 520 | n_decoded =    100, tg =  21.68 t/s, tg_3s =  21.68 t/s
1.33.145.950 I slot print_timing: id  3 | task 507 | prompt eval time =     110.18 ms /     4 tokens (   27.55 ms per token,    36.30 tokens per second)
1.33.145.952 I slot print_timing: id  3 | task 507 |        eval time =    5957.95 ms /   128 tokens (   46.55 ms per token,    21.48 tokens per second)
1.33.145.953 I slot print_timing: id  3 | task 507 |       total time =    6068.13 ms /   132 tokens
1.33.145.954 I slot print_timing: id  3 | task 507 |    graphs reused =          1
1.33.145.957 I slot print_timing: id  3 | task 507 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.145.968 I statistics        draft-mtp: #calls(b,g,a) =  128    454   6623, #gen drafts =   6625, #acc drafts =  4767, #gen tokens =  13209, #acc tokens =  8887, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.622), dur(b,g,a) = 0.085, 1649.177, 8.449 ms
1.33.145.983 I slot      release: id  3 | task 507 | stop processing: n_tokens = 156, truncated = 0
1.33.254.785 I slot print_timing: id  1 | task 534 | n_decoded =    100, tg =  28.49 t/s, tg_3s =  28.49 t/s
1.33.321.190 I slot print_timing: id  5 | task 520 | prompt eval time =     110.25 ms /     4 tokens (   27.56 ms per token,    36.28 tokens per second)
1.33.321.193 I slot print_timing: id  5 | task 520 |        eval time =    4913.33 ms /   128 tokens (   38.39 ms per token,    26.05 tokens per second)
1.33.321.193 I slot print_timing: id  5 | task 520 |       total time =    5023.58 ms /   132 tokens
1.33.321.194 I slot print_timing: id  5 | task 520 |    graphs reused =          1
1.33.321.198 I slot print_timing: id  5 | task 520 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.321.210 I statistics        draft-mtp: #calls(b,g,a) =  128    464   6643, #gen drafts =   6644, #acc drafts =  4778, #gen tokens =  13246, #acc tokens =  8907, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.622), dur(b,g,a) = 0.085, 1667.615, 8.465 ms
1.33.321.229 I slot      release: id  5 | task 520 | stop processing: n_tokens = 156, truncated = 0
1.33.413.673 I slot print_timing: id  1 | task 534 | prompt eval time =     109.72 ms /     4 tokens (   27.43 ms per token,    36.45 tokens per second)
1.33.413.676 I slot print_timing: id  1 | task 534 |        eval time =    3668.78 ms /   128 tokens (   28.66 ms per token,    34.89 tokens per second)
1.33.413.676 I slot print_timing: id  1 | task 534 |       total time =    3778.50 ms /   132 tokens
1.33.413.677 I slot print_timing: id  1 | task 534 |    graphs reused =          9
1.33.413.680 I slot print_timing: id  1 | task 534 | draft acceptance = 0.42963 (   58 accepted /   135 generated), mean acceptance length =  1.85, acceptance rate per position = (0.471, 0.382)
1.33.413.690 I statistics        draft-mtp: #calls(b,g,a) =  128    474   6654, #gen drafts =   6654, #acc drafts =  4783, #gen tokens =  13265, #acc tokens =  8917, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.621), dur(b,g,a) = 0.085, 1677.501, 8.474 ms
1.33.413.702 I slot      release: id  1 | task 534 | stop processing: n_tokens = 156, truncated = 0
1.33.413.713 I srv  update_slots: all slots are idle