0.00.139.446 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.139.449 I device_info:
0.00.220.874 I - CUDA0 : NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (97244 MiB, 96677 MiB free)
0.00.220.888 I - CPU : AMD Ryzen Threadripper 9960X 24-Cores (257066 MiB, 257066 MiB free)
0.00.220.977 I system_info: n_threads = 24 (n_threads_batch = 24) / 48 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.221.062 I srv init: using 47 threads for HTTP server
0.00.221.269 I srv start: binding port with default address family
0.00.222.468 I srv llama_server: loading model
0.00.222.531 I srv load_model: loading model '/home/ripper/ornith-35b-lab/artifacts/quant/ornith-1.0-35b-IQ4_XS.gguf'
0.00.499.404 I srv load_model: [spec] estimated memory usage of draft model is 3176.45 MiB
0.00.499.439 I common_init_result: fitting params to device memory ...
0.00.499.441 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.528.998 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.543.875 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.576.363 I srv load_model: loading draft model '/home/ripper/ornith-35b-lab/artifacts/mtp/ornith-1.0-35b-mtp-chaincorr-h2r9fw125-h1r2-h3r6ce375-lr5e7-Q6_K.gguf'
0.02.923.515 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.932.282 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.940.049 W srv operator(): LLAMA_MTP_PER_HEAD_CONTEXT disables prompt cache; per-head MTP state is not prompt-cache serializable yet
0.02.940.051 I srv operator(): [spec] created 2 MTP draft contexts, one per active head
0.02.940.065 I srv load_model: initializing slots, n_slots = 16
0.02.949.032 I common_context_can_seq_rm: the context supports bounded partial sequence removal
0.02.955.736 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
0.02.955.739 I common_speculative_impl_draft_mtp: - n_max=2, n_min=0, p_min=0.00, n_embd=2048, backend_sampling=1
0.02.955.741 I common_speculative_impl_draft_mtp: - gpu_layers=99, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[default]
0.02.955.873 I common_speculative_impl_draft_mtp: - draft sampler top_k=1 top_p=1.000 temp=1.000
0.02.955.898 I common_speculative_impl_draft_mtp: - per-head MTP contexts enabled (2 active heads)
0.02.955.906 I common_speculative_impl_draft_mtp: - backend draft sampler top_k=1 top_p=1.000 temp=1.000 contexts=2
0.02.973.488 I srv load_model: speculative decoding context initialized
0.02.973.491 I slot load_model: id 0 | task -1 | new slot, n_ctx = 512
0.02.973.501 I slot load_model: id 1 | task -1 | new slot, n_ctx = 512
0.02.973.501 I slot load_model: id 2 | task -1 | new slot, n_ctx = 512
0.02.973.502 I slot load_model: id 3 | task -1 | new slot, n_ctx = 512
0.02.973.502 I slot load_model: id 4 | task -1 | new slot, n_ctx = 512
0.02.973.502 I slot load_model: id 5 | task -1 | new slot, n_ctx = 512
0.02.973.502 I slot load_model: id 6 | task -1 | new slot, n_ctx = 512
0.02.973.502 I slot load_model: id 7 | task -1 | new slot, n_ctx = 512
0.02.973.503 I slot load_model: id 8 | task -1 | new slot, n_ctx = 512
0.02.973.503 I slot load_model: id 9 | task -1 | new slot, n_ctx = 512
0.02.973.503 I slot load_model: id 10 | task -1 | new slot, n_ctx = 512
0.02.973.503 I slot load_model: id 11 | task -1 | new slot, n_ctx = 512
0.02.973.504 I slot load_model: id 12 | task -1 | new slot, n_ctx = 512
0.02.973.504 I slot load_model: id 13 | task -1 | new slot, n_ctx = 512
0.02.973.504 I slot load_model: id 14 | task -1 | new slot, n_ctx = 512
0.02.973.504 I slot load_model: id 15 | task -1 | new slot, n_ctx = 512
0.02.973.551 I srv load_model: prompt cache is disabled - use `--cache-ram N` to enable it
0.02.973.552 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.02.973.552 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.02.973.570 W srv init: --cache-idle-slots requires --cache-ram, disabling
0.02.982.126 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
0.02.988.410 I srv init: init: chat template, thinking = 0
0.02.988.456 I srv llama_server: model loaded
0.02.988.461 I srv llama_server: server is listening on http://127.0.0.1:18236
0.02.988.468 I srv update_slots: all slots are idle
0.42.767.444 I srv operator(): Chat format: peg-native
0.42.767.452 I srv operator(): Chat format: peg-native
0.42.767.577 I srv operator(): Chat format: peg-native
0.42.767.582 I srv operator(): Chat format: peg-native
0.42.767.590 I srv operator(): Chat format: peg-native
0.42.767.602 I srv operator(): Chat format: peg-native
0.42.767.612 I slot get_availabl: id 15 | task -1 | selected slot by LRU, t_last = -1
0.42.767.615 I srv operator(): Chat format: peg-native
0.42.767.661 I slot launch_slot_: id 15 | task 1 | processing task, is_child = 0
0.42.767.664 I slot get_availabl: id 14 | task -1 | selected slot by LRU, t_last = -1
0.42.767.678 I slot launch_slot_: id 14 | task 0 | processing task, is_child = 0
0.42.767.680 I slot get_availabl: id 13 | task -1 | selected slot by LRU, t_last = -1
0.42.767.693 I slot launch_slot_: id 13 | task 2 | processing task, is_child = 0
0.42.767.695 I slot get_availabl: id 12 | task -1 | selected slot by LRU, t_last = -1
0.42.767.707 I slot launch_slot_: id 12 | task 5 | processing task, is_child = 0
0.42.767.710 I slot get_availabl: id 11 | task -1 | selected slot by LRU, t_last = -1
0.42.767.722 I slot launch_slot_: id 11 | task 4 | processing task, is_child = 0
0.42.767.724 I slot get_availabl: id 10 | task -1 | selected slot by LRU, t_last = -1
0.42.767.736 I srv operator(): Chat format: peg-native
0.42.767.741 I slot launch_slot_: id 10 | task 7 | processing task, is_child = 0
0.42.767.742 I slot get_availabl: id 9 | task -1 | selected slot by LRU, t_last = -1
0.42.767.756 I slot launch_slot_: id 9 | task 6 | processing task, is_child = 0
0.42.767.793 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.807 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.852 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.856 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.889 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.891 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.767.931 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.768.326 I srv operator(): Chat format: peg-native
0.42.771.992 I srv operator(): Chat format: peg-native
0.42.773.666 I srv operator(): Chat format: peg-native
0.42.782.510 I srv operator(): Chat format: peg-native
0.42.783.282 I srv operator(): Chat format: peg-native
0.42.783.329 I srv operator(): Chat format: peg-native
0.42.783.343 I srv operator(): Chat format: peg-native
0.42.785.001 I srv operator(): Chat format: peg-native
0.42.999.508 I slot get_availabl: id 8 | task -1 | selected slot by LRU, t_last = -1
0.42.999.580 I slot launch_slot_: id 8 | task 3 | processing task, is_child = 0
0.42.999.582 I slot get_availabl: id 7 | task -1 | selected slot by LRU, t_last = -1
0.42.999.596 I slot launch_slot_: id 7 | task 9 | processing task, is_child = 0
0.42.999.598 I slot get_availabl: id 6 | task -1 | selected slot by LRU, t_last = -1
0.42.999.611 I slot launch_slot_: id 6 | task 10 | processing task, is_child = 0
0.42.999.614 I slot get_availabl: id 5 | task -1 | selected slot by LRU, t_last = -1
0.42.999.625 I slot launch_slot_: id 5 | task 11 | processing task, is_child = 0
0.42.999.628 I slot get_availabl: id 4 | task -1 | selected slot by LRU, t_last = -1
0.42.999.640 I slot launch_slot_: id 4 | task 12 | processing task, is_child = 0
0.42.999.643 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.42.999.653 I slot launch_slot_: id 3 | task 13 | processing task, is_child = 0
0.42.999.656 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
0.42.999.667 I slot launch_slot_: id 2 | task 14 | processing task, is_child = 0
0.42.999.669 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
0.42.999.680 I slot launch_slot_: id 1 | task 15 | processing task, is_child = 0
0.42.999.682 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
0.42.999.694 I slot launch_slot_: id 0 | task 16 | processing task, is_child = 0
0.42.999.772 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.778 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.781 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.783 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.796 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.798 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.804 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.806 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.42.999.807 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.43.014.125 I slot create_check: id 9 | task 6 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.027.421 I slot create_check: id 10 | task 7 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.43.040.819 I slot create_check: id 11 | task 4 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.054.490 I slot create_check: id 12 | task 5 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.068.134 I slot create_check: id 13 | task 2 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.081.749 I slot create_check: id 14 | task 0 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.43.095.363 I slot create_check: id 15 | task 1 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.43.341.392 I slot create_check: id 0 | task 16 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.43.354.678 I slot create_check: id 1 | task 15 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.368.034 I slot create_check: id 2 | task 14 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.381.344 I slot create_check: id 3 | task 13 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.394.985 I slot create_check: id 4 | task 12 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.408.418 I slot create_check: id 5 | task 11 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.43.421.877 I slot create_check: id 6 | task 10 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.435.368 I slot create_check: id 7 | task 9 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.43.448.847 I slot create_check: id 8 | task 3 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.47.727.448 I slot print_timing: id 9 | task 6 | n_decoded = 102, tg = 23.12 t/s, tg_3s = 23.12 t/s
0.47.732.139 I slot print_timing: id 13 | task 2 | n_decoded = 102, tg = 23.11 t/s, tg_3s = 23.11 t/s
0.47.846.908 I slot print_timing: id 1 | task 15 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.47.847.927 I slot print_timing: id 3 | task 13 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.47.849.184 I slot print_timing: id 6 | task 10 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.47.850.212 I slot print_timing: id 8 | task 3 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.48.216.177 I slot print_timing: id 11 | task 4 | n_decoded = 101, tg = 20.62 t/s, tg_3s = 20.62 t/s
0.48.216.689 I slot print_timing: id 12 | task 5 | n_decoded = 101, tg = 20.62 t/s, tg_3s = 20.62 t/s
0.48.333.691 I slot print_timing: id 2 | task 14 | n_decoded = 101, tg = 21.26 t/s, tg_3s = 21.26 t/s
0.48.334.624 I slot print_timing: id 4 | task 12 | n_decoded = 101, tg = 21.26 t/s, tg_3s = 21.26 t/s
0.48.337.616 I slot print_timing: id 7 | task 9 | n_decoded = 101, tg = 21.26 t/s, tg_3s = 21.26 t/s
0.48.338.519 I slot print_timing: id 9 | task 6 | prompt eval time = 548.46 ms / 25 tokens ( 21.94 ms per token, 45.58 tokens per second)
0.48.338.522 I slot print_timing: id 9 | task 6 | eval time = 5022.28 ms / 114 tokens ( 44.06 ms per token, 22.70 tokens per second)
0.48.338.522 I slot print_timing: id 9 | task 6 | total time = 5570.73 ms / 139 tokens
0.48.338.528 I slot print_timing: id 9 | task 6 | graphs reused = 1
0.48.338.531 I slot print_timing: id 9 | task 6 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.338.550 I statistics draft-mtp: #calls(b,g,a) = 16 40 625, #gen drafts = 631, #acc drafts = 491, #gen tokens = 1262, #acc tokens = 916, #mean acc len = 2.47, #acc rate/pos = (0.786, 0.680), dur(b,g,a) = 0.008, 121.173, 0.839 ms
0.48.338.583 I slot release: id 9 | task 6 | stop processing: n_tokens = 140, truncated = 0
0.48.340.424 I slot print_timing: id 13 | task 2 | prompt eval time = 550.77 ms / 25 tokens ( 22.03 ms per token, 45.39 tokens per second)
0.48.340.426 I slot print_timing: id 13 | task 2 | eval time = 5021.80 ms / 114 tokens ( 44.05 ms per token, 22.70 tokens per second)
0.48.340.427 I slot print_timing: id 13 | task 2 | total time = 5572.57 ms / 139 tokens
0.48.340.428 I slot print_timing: id 13 | task 2 | graphs reused = 1
0.48.340.431 I slot print_timing: id 13 | task 2 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.340.437 I statistics draft-mtp: #calls(b,g,a) = 16 40 629, #gen drafts = 631, #acc drafts = 494, #gen tokens = 1262, #acc tokens = 922, #mean acc len = 2.47, #acc rate/pos = (0.785, 0.680), dur(b,g,a) = 0.008, 121.173, 0.844 ms
0.48.340.454 I slot release: id 13 | task 2 | stop processing: n_tokens = 140, truncated = 0
0.48.347.953 I srv operator(): Chat format: peg-native
0.48.348.882 I srv operator(): Chat format: peg-native
0.48.444.214 I slot print_timing: id 1 | task 15 | prompt eval time = 582.66 ms / 25 tokens ( 23.31 ms per token, 42.91 tokens per second)
0.48.444.217 I slot print_timing: id 1 | task 15 | eval time = 4861.79 ms / 114 tokens ( 42.65 ms per token, 23.45 tokens per second)
0.48.444.217 I slot print_timing: id 1 | task 15 | total time = 5444.45 ms / 139 tokens
0.48.444.218 I slot print_timing: id 1 | task 15 | graphs reused = 1
0.48.444.221 I slot print_timing: id 1 | task 15 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.444.232 I statistics draft-mtp: #calls(b,g,a) = 16 41 633, #gen drafts = 645, #acc drafts = 495, #gen tokens = 1290, #acc tokens = 924, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.678), dur(b,g,a) = 0.008, 125.870, 0.849 ms
0.48.444.253 I slot release: id 1 | task 15 | stop processing: n_tokens = 140, truncated = 0
0.48.445.302 I slot print_timing: id 3 | task 13 | prompt eval time = 583.88 ms / 25 tokens ( 23.36 ms per token, 42.82 tokens per second)
0.48.445.303 I slot print_timing: id 3 | task 13 | eval time = 4861.65 ms / 114 tokens ( 42.65 ms per token, 23.45 tokens per second)
0.48.445.304 I slot print_timing: id 3 | task 13 | total time = 5445.53 ms / 139 tokens
0.48.445.304 I slot print_timing: id 3 | task 13 | graphs reused = 1
0.48.445.306 I slot print_timing: id 3 | task 13 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.445.312 I statistics draft-mtp: #calls(b,g,a) = 16 41 635, #gen drafts = 645, #acc drafts = 497, #gen tokens = 1290, #acc tokens = 928, #mean acc len = 2.46, #acc rate/pos = (0.783, 0.679), dur(b,g,a) = 0.008, 125.870, 0.851 ms
0.48.445.330 I slot release: id 3 | task 13 | stop processing: n_tokens = 140, truncated = 0
0.48.446.613 I slot print_timing: id 6 | task 10 | prompt eval time = 585.63 ms / 25 tokens ( 23.43 ms per token, 42.69 tokens per second)
0.48.446.615 I slot print_timing: id 6 | task 10 | eval time = 4861.17 ms / 114 tokens ( 42.64 ms per token, 23.45 tokens per second)
0.48.446.615 I slot print_timing: id 6 | task 10 | total time = 5446.81 ms / 139 tokens
0.48.446.616 I slot print_timing: id 6 | task 10 | graphs reused = 1
0.48.446.617 I slot print_timing: id 6 | task 10 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.446.622 I statistics draft-mtp: #calls(b,g,a) = 16 41 638, #gen drafts = 645, #acc drafts = 499, #gen tokens = 1290, #acc tokens = 932, #mean acc len = 2.46, #acc rate/pos = (0.782, 0.679), dur(b,g,a) = 0.008, 125.870, 0.856 ms
0.48.446.637 I slot release: id 6 | task 10 | stop processing: n_tokens = 140, truncated = 0
0.48.447.668 I slot print_timing: id 8 | task 3 | prompt eval time = 586.76 ms / 25 tokens ( 23.47 ms per token, 42.61 tokens per second)
0.48.447.670 I slot print_timing: id 8 | task 3 | eval time = 4861.10 ms / 114 tokens ( 42.64 ms per token, 23.45 tokens per second)
0.48.447.670 I slot print_timing: id 8 | task 3 | total time = 5447.86 ms / 139 tokens
0.48.447.670 I slot print_timing: id 8 | task 3 | graphs reused = 1
0.48.447.672 I slot print_timing: id 8 | task 3 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.48.447.676 I statistics draft-mtp: #calls(b,g,a) = 16 41 640, #gen drafts = 645, #acc drafts = 501, #gen tokens = 1290, #acc tokens = 936, #mean acc len = 2.46, #acc rate/pos = (0.783, 0.680), dur(b,g,a) = 0.008, 125.870, 0.858 ms
0.48.447.689 I slot release: id 8 | task 3 | stop processing: n_tokens = 140, truncated = 0
0.48.448.970 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.48.449.037 I slot launch_slot_: id 1 | task 59 | processing task, is_child = 0
0.48.449.039 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.48.449.056 I slot launch_slot_: id 3 | task 60 | processing task, is_child = 0
0.48.453.864 I srv operator(): Chat format: peg-native
0.48.454.693 I srv operator(): Chat format: peg-native
0.48.455.564 I srv operator(): Chat format: peg-native
0.48.455.696 I slot operator(): id 1 | task 59 | Checking checkpoint with [20, 20] against 3...
0.48.455.699 W slot operator(): id 1 | task 59 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.455.701 W slot operator(): id 1 | task 59 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.48.456.717 I srv operator(): Chat format: peg-native
0.48.457.595 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.457.625 I slot operator(): id 3 | task 60 | Checking checkpoint with [20, 20] against 3...
0.48.457.627 W slot operator(): id 3 | task 60 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.457.628 W slot operator(): id 3 | task 60 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.48.459.439 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.572.376 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.48.572.444 I slot launch_slot_: id 6 | task 62 | processing task, is_child = 0
0.48.572.447 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.48.572.466 I slot launch_slot_: id 8 | task 63 | processing task, is_child = 0
0.48.572.469 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.48.572.484 I slot launch_slot_: id 9 | task 64 | processing task, is_child = 0
0.48.572.486 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.48.572.500 I slot launch_slot_: id 13 | task 65 | processing task, is_child = 0
0.48.593.308 I slot create_check: id 1 | task 59 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.48.606.510 I slot create_check: id 3 | task 60 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.48.606.516 I slot operator(): id 6 | task 62 | Checking checkpoint with [20, 20] against 24...
0.48.609.970 W slot operator(): id 6 | task 62 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.48.610.002 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.610.016 I slot operator(): id 8 | task 63 | Checking checkpoint with [20, 20] against 3...
0.48.610.019 W slot operator(): id 8 | task 63 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.610.020 W slot operator(): id 8 | task 63 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.48.611.558 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.611.578 I slot operator(): id 9 | task 64 | Checking checkpoint with [20, 20] against 3...
0.48.611.580 W slot operator(): id 9 | task 64 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.48.611.581 W slot operator(): id 9 | task 64 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.48.613.448 I slot operator(): id 13 | task 65 | Checking checkpoint with [20, 20] against 24...
0.48.613.450 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.616.938 W slot operator(): id 13 | task 65 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.48.616.969 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.48.784.413 I slot create_check: id 8 | task 63 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.48.797.577 I slot create_check: id 9 | task 64 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.49.398.492 I slot print_timing: id 11 | task 4 | prompt eval time = 549.64 ms / 28 tokens ( 19.63 ms per token, 50.94 tokens per second)
0.49.398.495 I slot print_timing: id 11 | task 4 | eval time = 6081.00 ms / 125 tokens ( 48.65 ms per token, 20.56 tokens per second)
0.49.398.496 I slot print_timing: id 11 | task 4 | total time = 6630.64 ms / 153 tokens
0.49.398.497 I slot print_timing: id 11 | task 4 | graphs reused = 1
0.49.398.499 I slot print_timing: id 11 | task 4 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.49.398.511 I statistics draft-mtp: #calls(b,g,a) = 22 48 739, #gen drafts = 743, #acc drafts = 566, #gen tokens = 1486, #acc tokens = 1057, #mean acc len = 2.43, #acc rate/pos = (0.766, 0.664), dur(b,g,a) = 0.010, 155.200, 0.988 ms
0.49.398.535 I slot release: id 11 | task 4 | stop processing: n_tokens = 152, truncated = 0
0.49.399.086 I slot print_timing: id 12 | task 5 | prompt eval time = 550.23 ms / 28 tokens ( 19.65 ms per token, 50.89 tokens per second)
0.49.399.087 I slot print_timing: id 12 | task 5 | eval time = 6080.98 ms / 125 tokens ( 48.65 ms per token, 20.56 tokens per second)
0.49.399.088 I slot print_timing: id 12 | task 5 | total time = 6631.20 ms / 153 tokens
0.49.399.089 I slot print_timing: id 12 | task 5 | graphs reused = 1
0.49.399.092 I slot print_timing: id 12 | task 5 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.49.399.097 I statistics draft-mtp: #calls(b,g,a) = 22 48 740, #gen drafts = 743, #acc drafts = 567, #gen tokens = 1486, #acc tokens = 1059, #mean acc len = 2.43, #acc rate/pos = (0.766, 0.665), dur(b,g,a) = 0.010, 155.200, 0.989 ms
0.49.399.116 I slot release: id 12 | task 5 | stop processing: n_tokens = 152, truncated = 0
0.49.407.993 I srv operator(): Chat format: peg-native
0.49.408.066 I srv operator(): Chat format: peg-native
0.49.504.619 I slot print_timing: id 2 | task 14 | prompt eval time = 583.25 ms / 28 tokens ( 20.83 ms per token, 48.01 tokens per second)
0.49.504.623 I slot print_timing: id 2 | task 14 | eval time = 5921.57 ms / 125 tokens ( 47.37 ms per token, 21.11 tokens per second)
0.49.504.623 I slot print_timing: id 2 | task 14 | total time = 6504.82 ms / 153 tokens
0.49.504.624 I slot print_timing: id 2 | task 14 | graphs reused = 1
0.49.504.627 I slot print_timing: id 2 | task 14 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.49.504.638 I statistics draft-mtp: #calls(b,g,a) = 22 49 746, #gen drafts = 757, #acc drafts = 569, #gen tokens = 1514, #acc tokens = 1063, #mean acc len = 2.42, #acc rate/pos = (0.763, 0.662), dur(b,g,a) = 0.010, 160.611, 0.998 ms
0.49.504.660 I slot release: id 2 | task 14 | stop processing: n_tokens = 152, truncated = 0
0.49.505.727 I slot print_timing: id 4 | task 12 | prompt eval time = 584.43 ms / 28 tokens ( 20.87 ms per token, 47.91 tokens per second)
0.49.505.729 I slot print_timing: id 4 | task 12 | eval time = 5921.50 ms / 125 tokens ( 47.37 ms per token, 21.11 tokens per second)
0.49.505.730 I slot print_timing: id 4 | task 12 | total time = 6505.93 ms / 153 tokens
0.49.505.730 I slot print_timing: id 4 | task 12 | graphs reused = 1
0.49.505.732 I slot print_timing: id 4 | task 12 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.49.505.735 I statistics draft-mtp: #calls(b,g,a) = 22 49 748, #gen drafts = 757, #acc drafts = 571, #gen tokens = 1514, #acc tokens = 1067, #mean acc len = 2.43, #acc rate/pos = (0.763, 0.663), dur(b,g,a) = 0.010, 160.611, 1.000 ms
0.49.505.752 I slot release: id 4 | task 12 | stop processing: n_tokens = 152, truncated = 0
0.49.507.062 I slot print_timing: id 7 | task 9 | prompt eval time = 586.19 ms / 28 tokens ( 20.94 ms per token, 47.77 tokens per second)
0.49.507.064 I slot print_timing: id 7 | task 9 | eval time = 5921.06 ms / 125 tokens ( 47.37 ms per token, 21.11 tokens per second)
0.49.507.064 I slot print_timing: id 7 | task 9 | total time = 6507.24 ms / 153 tokens
0.49.507.065 I slot print_timing: id 7 | task 9 | graphs reused = 1
0.49.507.067 I slot print_timing: id 7 | task 9 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.49.507.070 I statistics draft-mtp: #calls(b,g,a) = 22 49 751, #gen drafts = 757, #acc drafts = 573, #gen tokens = 1514, #acc tokens = 1071, #mean acc len = 2.43, #acc rate/pos = (0.763, 0.663), dur(b,g,a) = 0.010, 160.611, 1.003 ms
0.49.507.089 I slot release: id 7 | task 9 | stop processing: n_tokens = 152, truncated = 0
0.49.510.027 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.49.510.094 I slot launch_slot_: id 2 | task 73 | processing task, is_child = 0
0.49.510.099 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.49.510.115 I slot launch_slot_: id 4 | task 74 | processing task, is_child = 0
0.49.514.064 I srv operator(): Chat format: peg-native
0.49.514.757 I srv operator(): Chat format: peg-native
0.49.515.573 I srv operator(): Chat format: peg-native
0.49.516.116 I slot operator(): id 2 | task 73 | Checking checkpoint with [23, 23] against 3...
0.49.516.120 W slot operator(): id 2 | task 73 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.516.122 W slot operator(): id 2 | task 73 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.49.517.566 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.517.590 I slot operator(): id 4 | task 74 | Checking checkpoint with [23, 23] against 27...
0.49.521.383 W slot operator(): id 4 | task 74 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.49.521.409 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.633.028 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
0.49.633.101 I slot launch_slot_: id 7 | task 76 | processing task, is_child = 0
0.49.633.105 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.49.633.121 I slot launch_slot_: id 11 | task 77 | processing task, is_child = 0
0.49.633.123 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.49.633.138 I slot launch_slot_: id 12 | task 78 | processing task, is_child = 0
0.49.652.049 I slot create_check: id 2 | task 73 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.49.652.058 I slot operator(): id 7 | task 76 | Checking checkpoint with [23, 23] against 3...
0.49.652.058 W slot operator(): id 7 | task 76 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.652.059 W slot operator(): id 7 | task 76 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.49.653.590 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.653.610 I slot operator(): id 11 | task 77 | Checking checkpoint with [23, 23] against 3...
0.49.653.612 W slot operator(): id 11 | task 77 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.653.612 W slot operator(): id 11 | task 77 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.49.654.977 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.654.988 I slot operator(): id 12 | task 78 | Checking checkpoint with [23, 23] against 27...
0.49.658.599 W slot operator(): id 12 | task 78 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.49.658.619 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.824.224 I slot create_check: id 7 | task 76 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.49.837.492 I slot create_check: id 11 | task 77 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.50.077.441 I slot print_timing: id 10 | task 7 | n_decoded = 100, tg = 14.79 t/s, tg_3s = 14.79 t/s
0.50.199.501 I slot print_timing: id 14 | task 0 | n_decoded = 100, tg = 14.53 t/s, tg_3s = 14.53 t/s
0.50.200.025 I slot print_timing: id 15 | task 1 | n_decoded = 100, tg = 14.53 t/s, tg_3s = 14.53 t/s
0.50.314.100 I slot print_timing: id 0 | task 16 | n_decoded = 100, tg = 14.85 t/s, tg_3s = 14.85 t/s
0.50.316.450 I slot print_timing: id 5 | task 11 | n_decoded = 100, tg = 14.86 t/s, tg_3s = 14.86 t/s
0.51.882.861 I slot print_timing: id 10 | task 7 | prompt eval time = 549.06 ms / 29 tokens ( 18.93 ms per token, 52.82 tokens per second)
0.51.882.864 I slot print_timing: id 10 | task 7 | eval time = 8565.96 ms / 128 tokens ( 66.92 ms per token, 14.94 tokens per second)
0.51.882.865 I slot print_timing: id 10 | task 7 | total time = 9115.03 ms / 157 tokens
0.51.882.866 I slot print_timing: id 10 | task 7 | graphs reused = 1
0.51.882.869 I slot print_timing: id 10 | task 7 | draft acceptance = 0.44361 ( 59 accepted / 133 generated), mean acceptance length = 1.88, acceptance rate per position = (0.493, 0.388)
0.51.882.887 I statistics draft-mtp: #calls(b,g,a) = 27 68 1034, #gen drafts = 1049, #acc drafts = 771, #gen tokens = 2095, #acc tokens = 1441, #mean acc len = 2.39, #acc rate/pos = (0.746, 0.648), dur(b,g,a) = 0.013, 225.433, 1.334 ms
0.51.882.915 I slot release: id 10 | task 7 | stop processing: n_tokens = 156, truncated = 0
0.51.892.319 I srv operator(): Chat format: peg-native
0.51.990.899 I slot print_timing: id 14 | task 0 | prompt eval time = 551.31 ms / 29 tokens ( 19.01 ms per token, 52.60 tokens per second)
0.51.990.902 I slot print_timing: id 14 | task 0 | eval time = 8671.72 ms / 128 tokens ( 67.75 ms per token, 14.76 tokens per second)
0.51.990.903 I slot print_timing: id 14 | task 0 | total time = 9223.03 ms / 157 tokens
0.51.990.904 I slot print_timing: id 14 | task 0 | graphs reused = 1
0.51.990.907 I slot print_timing: id 14 | task 0 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.51.990.920 I statistics draft-mtp: #calls(b,g,a) = 27 69 1049, #gen drafts = 1062, #acc drafts = 780, #gen tokens = 2119, #acc tokens = 1458, #mean acc len = 2.39, #acc rate/pos = (0.744, 0.646), dur(b,g,a) = 0.013, 229.207, 1.351 ms
0.51.990.945 I slot release: id 14 | task 0 | stop processing: n_tokens = 156, truncated = 0
0.51.991.130 I slot print_timing: id 15 | task 1 | prompt eval time = 551.93 ms / 29 tokens ( 19.03 ms per token, 52.54 tokens per second)
0.51.991.132 I slot print_timing: id 15 | task 1 | eval time = 8671.34 ms / 128 tokens ( 67.74 ms per token, 14.76 tokens per second)
0.51.991.132 I slot print_timing: id 15 | task 1 | total time = 9223.27 ms / 157 tokens
0.51.991.133 I slot print_timing: id 15 | task 1 | graphs reused = 1
0.51.991.134 I slot print_timing: id 15 | task 1 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.51.991.136 I statistics draft-mtp: #calls(b,g,a) = 27 69 1049, #gen drafts = 1062, #acc drafts = 780, #gen tokens = 2119, #acc tokens = 1458, #mean acc len = 2.39, #acc rate/pos = (0.744, 0.646), dur(b,g,a) = 0.013, 229.207, 1.351 ms
0.51.991.155 I slot release: id 15 | task 1 | stop processing: n_tokens = 156, truncated = 0
0.51.996.648 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.51.996.722 I slot launch_slot_: id 10 | task 98 | processing task, is_child = 0
0.52.001.543 I srv operator(): Chat format: peg-native
0.52.001.586 I srv operator(): Chat format: peg-native
0.52.001.640 I slot operator(): id 10 | task 98 | Checking checkpoint with [24, 24] against 3...
0.52.001.642 W slot operator(): id 10 | task 98 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.52.001.645 W slot operator(): id 10 | task 98 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.52.003.183 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.108.515 I slot print_timing: id 0 | task 16 | prompt eval time = 582.03 ms / 29 tokens ( 20.07 ms per token, 49.83 tokens per second)
0.52.108.518 I slot print_timing: id 0 | task 16 | eval time = 8526.75 ms / 128 tokens ( 66.62 ms per token, 15.01 tokens per second)
0.52.108.519 I slot print_timing: id 0 | task 16 | total time = 9108.78 ms / 157 tokens
0.52.108.520 I slot print_timing: id 0 | task 16 | graphs reused = 1
0.52.108.523 I slot print_timing: id 0 | task 16 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.52.108.537 I statistics draft-mtp: #calls(b,g,a) = 27 70 1062, #gen drafts = 1073, #acc drafts = 788, #gen tokens = 2141, #acc tokens = 1473, #mean acc len = 2.39, #acc rate/pos = (0.742, 0.645), dur(b,g,a) = 0.013, 234.088, 1.366 ms
0.52.108.563 I slot release: id 0 | task 16 | stop processing: n_tokens = 156, truncated = 0
0.52.108.777 I slot print_timing: id 5 | task 11 | prompt eval time = 584.99 ms / 29 tokens ( 20.17 ms per token, 49.57 tokens per second)
0.52.108.778 I slot print_timing: id 5 | task 11 | eval time = 8523.99 ms / 128 tokens ( 66.59 ms per token, 15.02 tokens per second)
0.52.108.779 I slot print_timing: id 5 | task 11 | total time = 9108.98 ms / 157 tokens
0.52.108.779 I slot print_timing: id 5 | task 11 | graphs reused = 1
0.52.108.779 I slot print_timing: id 5 | task 11 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.52.108.782 I statistics draft-mtp: #calls(b,g,a) = 27 70 1062, #gen drafts = 1073, #acc drafts = 788, #gen tokens = 2141, #acc tokens = 1473, #mean acc len = 2.39, #acc rate/pos = (0.742, 0.645), dur(b,g,a) = 0.013, 234.088, 1.366 ms
0.52.108.795 I slot release: id 5 | task 11 | stop processing: n_tokens = 156, truncated = 0
0.52.113.653 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.52.113.723 I slot launch_slot_: id 0 | task 100 | processing task, is_child = 0
0.52.113.726 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.52.113.743 I slot launch_slot_: id 5 | task 101 | processing task, is_child = 0
0.52.117.796 I srv operator(): Chat format: peg-native
0.52.117.989 I srv operator(): Chat format: peg-native
0.52.118.665 I slot operator(): id 0 | task 100 | Checking checkpoint with [24, 24] against 28...
0.52.122.490 W slot operator(): id 0 | task 100 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.52.122.515 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.122.541 I slot operator(): id 5 | task 101 | Checking checkpoint with [24, 24] against 3...
0.52.122.542 W slot operator(): id 5 | task 101 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.52.122.543 W slot operator(): id 5 | task 101 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.52.124.237 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.137.279 I slot create_check: id 10 | task 98 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.52.254.853 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.52.254.923 I slot launch_slot_: id 14 | task 103 | processing task, is_child = 0
0.52.254.927 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.52.254.945 I slot launch_slot_: id 15 | task 104 | processing task, is_child = 0
0.52.273.080 I slot create_check: id 5 | task 101 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.52.273.088 I slot operator(): id 14 | task 103 | Checking checkpoint with [24, 24] against 3...
0.52.273.089 W slot operator(): id 14 | task 103 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.52.273.089 W slot operator(): id 14 | task 103 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.52.274.585 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.274.615 I slot operator(): id 15 | task 104 | Checking checkpoint with [24, 24] against 28...
0.52.278.145 W slot operator(): id 15 | task 104 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.52.278.167 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.52.426.633 I slot create_check: id 14 | task 103 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.53.147.481 I slot print_timing: id 6 | task 62 | n_decoded = 102, tg = 23.25 t/s, tg_3s = 23.25 t/s
0.53.150.605 I slot print_timing: id 13 | task 65 | n_decoded = 102, tg = 23.24 t/s, tg_3s = 23.24 t/s
0.53.630.377 I slot print_timing: id 3 | task 60 | n_decoded = 101, tg = 20.74 t/s, tg_3s = 20.74 t/s
0.53.752.907 I slot print_timing: id 6 | task 62 | prompt eval time = 153.87 ms / 4 tokens ( 38.47 ms per token, 26.00 tokens per second)
0.53.752.910 I slot print_timing: id 6 | task 62 | eval time = 4992.50 ms / 114 tokens ( 43.79 ms per token, 22.83 tokens per second)
0.53.752.910 I slot print_timing: id 6 | task 62 | total time = 5146.37 ms / 118 tokens
0.53.752.911 I slot print_timing: id 6 | task 62 | graphs reused = 1
0.53.752.914 I slot print_timing: id 6 | task 62 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.53.752.931 I statistics draft-mtp: #calls(b,g,a) = 32 83 1263, #gen drafts = 1272, #acc drafts = 935, #gen tokens = 2539, #acc tokens = 1744, #mean acc len = 2.38, #acc rate/pos = (0.740, 0.641), dur(b,g,a) = 0.016, 276.809, 1.624 ms
0.53.752.962 I slot release: id 6 | task 62 | stop processing: n_tokens = 140, truncated = 0
0.53.754.626 I slot print_timing: id 9 | task 64 | n_decoded = 101, tg = 20.85 t/s, tg_3s = 20.85 t/s
0.53.756.747 I slot print_timing: id 13 | task 65 | prompt eval time = 147.49 ms / 4 tokens ( 36.87 ms per token, 27.12 tokens per second)
0.53.756.750 I slot print_timing: id 13 | task 65 | eval time = 4995.80 ms / 114 tokens ( 43.82 ms per token, 22.82 tokens per second)
0.53.756.750 I slot print_timing: id 13 | task 65 | total time = 5143.28 ms / 118 tokens
0.53.756.750 I slot print_timing: id 13 | task 65 | graphs reused = 1
0.53.756.753 I slot print_timing: id 13 | task 65 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.53.756.760 I statistics draft-mtp: #calls(b,g,a) = 32 83 1270, #gen drafts = 1272, #acc drafts = 942, #gen tokens = 2539, #acc tokens = 1758, #mean acc len = 2.38, #acc rate/pos = (0.742, 0.643), dur(b,g,a) = 0.016, 276.809, 1.633 ms
0.53.756.783 I slot release: id 13 | task 65 | stop processing: n_tokens = 140, truncated = 0
0.53.761.382 I srv operator(): Chat format: peg-native
0.53.765.442 I srv operator(): Chat format: peg-native
0.53.867.969 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.53.868.044 I slot launch_slot_: id 6 | task 118 | processing task, is_child = 0
0.53.868.047 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.53.868.064 I slot launch_slot_: id 13 | task 119 | processing task, is_child = 0
0.53.873.437 I slot operator(): id 6 | task 118 | Checking checkpoint with [20, 20] against 3...
0.53.873.440 W slot operator(): id 6 | task 118 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.53.873.443 W slot operator(): id 6 | task 118 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.53.874.951 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.874.973 I slot operator(): id 13 | task 119 | Checking checkpoint with [20, 20] against 24...
0.53.878.805 W slot operator(): id 13 | task 119 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.53.878.832 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.027.435 I slot create_check: id 6 | task 118 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.54.264.688 I slot print_timing: id 7 | task 76 | n_decoded = 102, tg = 23.65 t/s, tg_3s = 23.65 t/s
0.54.505.595 I slot print_timing: id 4 | task 74 | n_decoded = 101, tg = 20.71 t/s, tg_3s = 20.71 t/s
0.54.631.272 I slot print_timing: id 12 | task 78 | n_decoded = 101, tg = 20.90 t/s, tg_3s = 20.90 t/s
0.54.748.062 I slot print_timing: id 3 | task 60 | prompt eval time = 302.53 ms / 28 tokens ( 10.80 ms per token, 92.55 tokens per second)
0.54.748.065 I slot print_timing: id 3 | task 60 | eval time = 5987.87 ms / 125 tokens ( 47.90 ms per token, 20.88 tokens per second)
0.54.748.066 I slot print_timing: id 3 | task 60 | total time = 6290.40 ms / 153 tokens
0.54.748.067 I slot print_timing: id 3 | task 60 | graphs reused = 1
0.54.748.070 I slot print_timing: id 3 | task 60 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.54.748.091 I statistics draft-mtp: #calls(b,g,a) = 34 91 1383, #gen drafts = 1395, #acc drafts = 1022, #gen tokens = 2785, #acc tokens = 1907, #mean acc len = 2.38, #acc rate/pos = (0.739, 0.640), dur(b,g,a) = 0.018, 306.784, 1.772 ms
0.54.748.124 I slot release: id 3 | task 60 | stop processing: n_tokens = 152, truncated = 0
0.54.756.550 I srv operator(): Chat format: peg-native
0.54.865.835 I slot print_timing: id 7 | task 76 | prompt eval time = 299.56 ms / 25 tokens ( 11.98 ms per token, 83.46 tokens per second)
0.54.865.838 I slot print_timing: id 7 | task 76 | eval time = 4914.20 ms / 114 tokens ( 43.11 ms per token, 23.20 tokens per second)
0.54.865.839 I slot print_timing: id 7 | task 76 | total time = 5213.76 ms / 139 tokens
0.54.865.840 I slot print_timing: id 7 | task 76 | graphs reused = 1
0.54.865.843 I slot print_timing: id 7 | task 76 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.54.865.860 I statistics draft-mtp: #calls(b,g,a) = 34 92 1402, #gen drafts = 1410, #acc drafts = 1037, #gen tokens = 2815, #acc tokens = 1936, #mean acc len = 2.38, #acc rate/pos = (0.740, 0.641), dur(b,g,a) = 0.018, 310.804, 1.795 ms
0.54.865.890 I slot release: id 7 | task 76 | stop processing: n_tokens = 140, truncated = 0
0.54.866.702 I slot print_timing: id 9 | task 64 | prompt eval time = 299.83 ms / 28 tokens ( 10.71 ms per token, 93.39 tokens per second)
0.54.866.704 I slot print_timing: id 9 | task 64 | eval time = 5955.26 ms / 125 tokens ( 47.64 ms per token, 20.99 tokens per second)
0.54.866.704 I slot print_timing: id 9 | task 64 | total time = 6255.09 ms / 153 tokens
0.54.866.705 I slot print_timing: id 9 | task 64 | graphs reused = 1
0.54.866.707 I slot print_timing: id 9 | task 64 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.54.866.713 I statistics draft-mtp: #calls(b,g,a) = 34 92 1404, #gen drafts = 1410, #acc drafts = 1038, #gen tokens = 2815, #acc tokens = 1938, #mean acc len = 2.38, #acc rate/pos = (0.739, 0.641), dur(b,g,a) = 0.018, 310.804, 1.797 ms
0.54.866.734 I slot release: id 9 | task 64 | stop processing: n_tokens = 152, truncated = 0
0.54.869.286 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.54.869.356 I slot launch_slot_: id 3 | task 128 | processing task, is_child = 0
0.54.875.774 I slot operator(): id 3 | task 128 | Checking checkpoint with [23, 23] against 3...
0.54.875.779 W slot operator(): id 3 | task 128 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.54.875.782 W slot operator(): id 3 | task 128 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.54.876.264 I srv operator(): Chat format: peg-native
0.54.876.443 I srv operator(): Chat format: peg-native
0.54.877.309 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.995.198 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.54.995.270 I slot launch_slot_: id 9 | task 130 | processing task, is_child = 0
0.54.995.273 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.54.995.293 I slot launch_slot_: id 7 | task 131 | processing task, is_child = 0
0.55.014.598 I slot create_check: id 3 | task 128 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.55.014.605 I slot operator(): id 7 | task 131 | Checking checkpoint with [20, 20] against 24...
0.55.018.267 W slot operator(): id 7 | task 131 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.55.018.310 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.018.322 I slot operator(): id 9 | task 130 | Checking checkpoint with [23, 23] against 27...
0.55.021.910 W slot operator(): id 9 | task 130 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.55.021.952 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.501.646 I slot print_timing: id 1 | task 59 | n_decoded = 100, tg = 14.83 t/s, tg_3s = 14.83 t/s
0.55.623.258 I slot print_timing: id 4 | task 74 | prompt eval time = 110.39 ms / 4 tokens ( 27.60 ms per token, 36.24 tokens per second)
0.55.623.261 I slot print_timing: id 4 | task 74 | eval time = 5995.24 ms / 125 tokens ( 47.96 ms per token, 20.85 tokens per second)
0.55.623.261 I slot print_timing: id 4 | task 74 | total time = 6105.63 ms / 129 tokens
0.55.623.262 I slot print_timing: id 4 | task 74 | graphs reused = 1
0.55.623.265 I slot print_timing: id 4 | task 74 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.55.623.284 I statistics draft-mtp: #calls(b,g,a) = 37 98 1489, #gen drafts = 1500, #acc drafts = 1100, #gen tokens = 2995, #acc tokens = 2051, #mean acc len = 2.38, #acc rate/pos = (0.739, 0.639), dur(b,g,a) = 0.020, 334.984, 1.904 ms
0.55.623.316 I slot release: id 4 | task 74 | stop processing: n_tokens = 152, truncated = 0
0.55.625.351 I slot print_timing: id 8 | task 63 | n_decoded = 100, tg = 14.89 t/s, tg_3s = 14.89 t/s
0.55.631.466 I srv operator(): Chat format: peg-native
0.55.741.890 I slot print_timing: id 12 | task 78 | prompt eval time = 144.55 ms / 4 tokens ( 36.14 ms per token, 27.67 tokens per second)
0.55.741.893 I slot print_timing: id 12 | task 78 | eval time = 5942.32 ms / 125 tokens ( 47.54 ms per token, 21.04 tokens per second)
0.55.741.893 I slot print_timing: id 12 | task 78 | total time = 6086.87 ms / 129 tokens
0.55.741.895 I slot print_timing: id 12 | task 78 | graphs reused = 1
0.55.741.898 I slot print_timing: id 12 | task 78 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.55.741.910 I statistics draft-mtp: #calls(b,g,a) = 37 99 1512, #gen drafts = 1515, #acc drafts = 1116, #gen tokens = 3025, #acc tokens = 2080, #mean acc len = 2.38, #acc rate/pos = (0.738, 0.638), dur(b,g,a) = 0.020, 339.278, 1.933 ms
0.55.741.939 I slot release: id 12 | task 78 | stop processing: n_tokens = 152, truncated = 0
0.55.743.304 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
0.55.743.375 I slot launch_slot_: id 4 | task 138 | processing task, is_child = 0
0.55.749.576 I slot operator(): id 4 | task 138 | Checking checkpoint with [23, 23] against 3...
0.55.749.580 W slot operator(): id 4 | task 138 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.55.749.583 W slot operator(): id 4 | task 138 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.55.750.444 I srv operator(): Chat format: peg-native
0.55.751.115 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.875.270 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.55.875.342 I slot launch_slot_: id 12 | task 140 | processing task, is_child = 0
0.55.894.280 I slot create_check: id 4 | task 138 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.55.894.287 I slot operator(): id 12 | task 140 | Checking checkpoint with [23, 23] against 27...
0.55.897.789 W slot operator(): id 12 | task 140 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.55.897.835 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.497.864 I slot print_timing: id 2 | task 73 | n_decoded = 100, tg = 14.93 t/s, tg_3s = 14.93 t/s
0.56.622.574 I slot print_timing: id 10 | task 98 | n_decoded = 102, tg = 23.33 t/s, tg_3s = 23.33 t/s
0.56.623.127 I slot print_timing: id 11 | task 77 | n_decoded = 100, tg = 14.99 t/s, tg_3s = 14.99 t/s
0.56.864.912 I slot print_timing: id 14 | task 103 | n_decoded = 102, tg = 23.58 t/s, tg_3s = 23.58 t/s
0.57.221.795 I slot print_timing: id 5 | task 101 | n_decoded = 101, tg = 20.96 t/s, tg_3s = 20.96 t/s
0.57.224.098 I slot print_timing: id 10 | task 98 | prompt eval time = 248.86 ms / 25 tokens ( 9.95 ms per token, 100.46 tokens per second)
0.57.224.100 I slot print_timing: id 10 | task 98 | eval time = 4973.58 ms / 114 tokens ( 43.63 ms per token, 22.92 tokens per second)
0.57.224.101 I slot print_timing: id 10 | task 98 | total time = 5222.44 ms / 139 tokens
0.57.224.102 I slot print_timing: id 10 | task 98 | graphs reused = 1
0.57.224.104 I slot print_timing: id 10 | task 98 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.57.224.116 I statistics draft-mtp: #calls(b,g,a) = 39 111 1698, #gen drafts = 1703, #acc drafts = 1247, #gen tokens = 3400, #acc tokens = 2326, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.635), dur(b,g,a) = 0.021, 380.819, 2.162 ms
0.57.224.142 I slot release: id 10 | task 98 | stop processing: n_tokens = 140, truncated = 0
0.57.233.112 I srv operator(): Chat format: peg-native
0.57.334.463 I slot print_timing: id 1 | task 59 | prompt eval time = 304.18 ms / 29 tokens ( 10.49 ms per token, 95.34 tokens per second)
0.57.334.466 I slot print_timing: id 1 | task 59 | eval time = 8574.56 ms / 128 tokens ( 66.99 ms per token, 14.93 tokens per second)
0.57.334.466 I slot print_timing: id 1 | task 59 | total time = 8878.74 ms / 157 tokens
0.57.334.467 I slot print_timing: id 1 | task 59 | graphs reused = 1
0.57.334.470 I slot print_timing: id 1 | task 59 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.57.334.484 I statistics draft-mtp: #calls(b,g,a) = 39 112 1703, #gen drafts = 1717, #acc drafts = 1251, #gen tokens = 3427, #acc tokens = 2334, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.636), dur(b,g,a) = 0.021, 386.347, 2.166 ms
0.57.334.510 I slot release: id 1 | task 59 | stop processing: n_tokens = 156, truncated = 0
0.57.340.787 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.57.340.857 I slot launch_slot_: id 10 | task 153 | processing task, is_child = 0
0.57.343.497 I srv operator(): Chat format: peg-native
0.57.346.907 I slot operator(): id 10 | task 153 | Checking checkpoint with [20, 20] against 24...
0.57.350.695 W slot operator(): id 10 | task 153 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.57.350.728 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.455.532 I slot print_timing: id 8 | task 63 | prompt eval time = 300.79 ms / 29 tokens ( 10.37 ms per token, 96.41 tokens per second)
0.57.455.535 I slot print_timing: id 8 | task 63 | eval time = 8544.70 ms / 128 tokens ( 66.76 ms per token, 14.98 tokens per second)
0.57.455.536 I slot print_timing: id 8 | task 63 | total time = 8845.49 ms / 157 tokens
0.57.455.537 I slot print_timing: id 8 | task 63 | graphs reused = 1
0.57.455.539 I slot print_timing: id 8 | task 63 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.57.455.552 I statistics draft-mtp: #calls(b,g,a) = 39 113 1717, #gen drafts = 1730, #acc drafts = 1261, #gen tokens = 3453, #acc tokens = 2352, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.635), dur(b,g,a) = 0.021, 392.356, 2.183 ms
0.57.455.578 I slot release: id 8 | task 63 | stop processing: n_tokens = 156, truncated = 0
0.57.461.290 I slot print_timing: id 14 | task 103 | prompt eval time = 266.80 ms / 25 tokens ( 10.67 ms per token, 93.70 tokens per second)
0.57.461.293 I slot print_timing: id 14 | task 103 | eval time = 4921.39 ms / 114 tokens ( 43.17 ms per token, 23.16 tokens per second)
0.57.461.293 I slot print_timing: id 14 | task 103 | total time = 5188.18 ms / 139 tokens
0.57.461.294 I slot print_timing: id 14 | task 103 | graphs reused = 1
0.57.461.296 I slot print_timing: id 14 | task 103 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.57.461.304 I statistics draft-mtp: #calls(b,g,a) = 40 113 1729, #gen drafts = 1730, #acc drafts = 1270, #gen tokens = 3453, #acc tokens = 2368, #mean acc len = 2.37, #acc rate/pos = (0.735, 0.635), dur(b,g,a) = 0.022, 392.356, 2.202 ms
0.57.461.328 I slot release: id 14 | task 103 | stop processing: n_tokens = 140, truncated = 0
0.57.461.605 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.57.461.673 I slot launch_slot_: id 1 | task 155 | processing task, is_child = 0
0.57.464.804 I srv operator(): Chat format: peg-native
0.57.467.725 I slot operator(): id 1 | task 155 | Checking checkpoint with [24, 24] against 28...
0.57.469.497 I srv operator(): Chat format: peg-native
0.57.471.573 W slot operator(): id 1 | task 155 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.57.471.615 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.577.732 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.57.577.799 I slot launch_slot_: id 8 | task 157 | processing task, is_child = 0
0.57.577.802 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.57.577.822 I slot launch_slot_: id 14 | task 158 | processing task, is_child = 0
0.57.583.356 I slot operator(): id 8 | task 157 | Checking checkpoint with [24, 24] against 3...
0.57.583.359 W slot operator(): id 8 | task 157 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.57.583.362 W slot operator(): id 8 | task 157 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.57.585.513 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.585.536 I slot operator(): id 14 | task 158 | Checking checkpoint with [20, 20] against 24...
0.57.589.381 W slot operator(): id 14 | task 158 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.57.589.412 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.740.058 I slot create_check: id 8 | task 157 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.58.337.613 I slot print_timing: id 2 | task 73 | prompt eval time = 282.80 ms / 29 tokens ( 9.75 ms per token, 102.55 tokens per second)
0.58.337.616 I slot print_timing: id 2 | task 73 | eval time = 8538.67 ms / 128 tokens ( 66.71 ms per token, 14.99 tokens per second)
0.58.337.617 I slot print_timing: id 2 | task 73 | total time = 8821.46 ms / 157 tokens
0.58.337.617 I slot print_timing: id 2 | task 73 | graphs reused = 1
0.58.337.620 I slot print_timing: id 2 | task 73 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.58.337.638 I statistics draft-mtp: #calls(b,g,a) = 43 120 1820, #gen drafts = 1835, #acc drafts = 1331, #gen tokens = 3661, #acc tokens = 2483, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.633), dur(b,g,a) = 0.024, 421.661, 2.318 ms
0.58.337.663 I slot release: id 2 | task 73 | stop processing: n_tokens = 156, truncated = 0
0.58.339.563 I slot print_timing: id 5 | task 101 | prompt eval time = 280.92 ms / 28 tokens ( 10.03 ms per token, 99.67 tokens per second)
0.58.339.565 I slot print_timing: id 5 | task 101 | eval time = 5936.07 ms / 125 tokens ( 47.49 ms per token, 21.06 tokens per second)
0.58.339.566 I slot print_timing: id 5 | task 101 | total time = 6216.99 ms / 153 tokens
0.58.339.566 I slot print_timing: id 5 | task 101 | graphs reused = 1
0.58.339.568 I slot print_timing: id 5 | task 101 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
0.58.339.574 I statistics draft-mtp: #calls(b,g,a) = 43 120 1825, #gen drafts = 1835, #acc drafts = 1333, #gen tokens = 3661, #acc tokens = 2487, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.632), dur(b,g,a) = 0.024, 421.661, 2.325 ms
0.58.339.596 I slot release: id 5 | task 101 | stop processing: n_tokens = 152, truncated = 0
0.58.343.616 I slot print_timing: id 13 | task 119 | n_decoded = 102, tg = 23.51 t/s, tg_3s = 23.51 t/s
0.58.346.686 I srv operator(): Chat format: peg-native
0.58.347.536 I srv operator(): Chat format: peg-native
0.58.446.788 I slot print_timing: id 11 | task 77 | prompt eval time = 298.62 ms / 29 tokens ( 10.30 ms per token, 97.11 tokens per second)
0.58.446.791 I slot print_timing: id 11 | task 77 | eval time = 8494.52 ms / 128 tokens ( 66.36 ms per token, 15.07 tokens per second)
0.58.446.792 I slot print_timing: id 11 | task 77 | total time = 8793.15 ms / 157 tokens
0.58.446.793 I slot print_timing: id 11 | task 77 | graphs reused = 1
0.58.446.796 I slot print_timing: id 11 | task 77 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
0.58.446.809 I statistics draft-mtp: #calls(b,g,a) = 43 121 1835, #gen drafts = 1848, #acc drafts = 1341, #gen tokens = 3687, #acc tokens = 2503, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.633), dur(b,g,a) = 0.024, 427.600, 2.336 ms
0.58.446.838 I slot release: id 11 | task 77 | stop processing: n_tokens = 156, truncated = 0
0.58.453.145 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.58.453.216 I slot launch_slot_: id 2 | task 166 | processing task, is_child = 0
0.58.453.219 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
0.58.453.236 I slot launch_slot_: id 5 | task 167 | processing task, is_child = 0
0.58.455.882 I srv operator(): Chat format: peg-native
0.58.459.219 I slot operator(): id 2 | task 166 | Checking checkpoint with [24, 24] against 28...
0.58.463.023 W slot operator(): id 2 | task 166 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.58.463.054 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.463.087 I slot operator(): id 5 | task 167 | Checking checkpoint with [23, 23] against 27...
0.58.466.571 W slot operator(): id 5 | task 167 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.58.466.600 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.581.183 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.58.581.253 I slot launch_slot_: id 11 | task 169 | processing task, is_child = 0
0.58.585.506 I slot operator(): id 11 | task 169 | Checking checkpoint with [24, 24] against 3...
0.58.585.509 W slot operator(): id 11 | task 169 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.585.510 W slot operator(): id 11 | task 169 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.58.587.008 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.734.299 I slot create_check: id 11 | task 169 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.58.848.221 I slot print_timing: id 0 | task 100 | n_decoded = 100, tg = 15.16 t/s, tg_3s = 15.16 t/s
0.58.970.965 I slot print_timing: id 6 | task 118 | n_decoded = 101, tg = 20.91 t/s, tg_3s = 20.91 t/s
0.58.974.744 I slot print_timing: id 13 | task 119 | prompt eval time = 129.15 ms / 4 tokens ( 32.29 ms per token, 30.97 tokens per second)
0.58.974.746 I slot print_timing: id 13 | task 119 | eval time = 4970.60 ms / 114 tokens ( 43.60 ms per token, 22.93 tokens per second)
0.58.974.747 I slot print_timing: id 13 | task 119 | total time = 5099.75 ms / 118 tokens
0.58.974.748 I slot print_timing: id 13 | task 119 | graphs reused = 1
0.58.974.751 I slot print_timing: id 13 | task 119 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.58.974.766 I statistics draft-mtp: #calls(b,g,a) = 46 125 1905, #gen drafts = 1907, #acc drafts = 1396, #gen tokens = 3805, #acc tokens = 2603, #mean acc len = 2.37, #acc rate/pos = (0.733, 0.634), dur(b,g,a) = 0.026, 445.188, 2.425 ms
0.58.974.793 I slot release: id 13 | task 119 | stop processing: n_tokens = 140, truncated = 0
0.58.983.055 I srv operator(): Chat format: peg-native
0.59.091.397 I slot print_timing: id 15 | task 104 | n_decoded = 100, tg = 14.95 t/s, tg_3s = 14.95 t/s
0.59.091.408 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.59.091.477 I slot launch_slot_: id 13 | task 174 | processing task, is_child = 0
0.59.095.368 I slot operator(): id 13 | task 174 | Checking checkpoint with [20, 20] against 3...
0.59.095.370 W slot operator(): id 13 | task 174 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.59.095.372 W slot operator(): id 13 | task 174 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.59.096.987 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.245.322 I slot create_check: id 13 | task 174 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.59.481.363 I slot print_timing: id 7 | task 131 | n_decoded = 102, tg = 23.49 t/s, tg_3s = 23.49 t/s
0.59.967.628 I slot print_timing: id 9 | task 130 | n_decoded = 101, tg = 20.92 t/s, tg_3s = 20.92 t/s
1.00.087.111 I slot print_timing: id 6 | task 118 | prompt eval time = 266.88 ms / 28 tokens ( 9.53 ms per token, 104.92 tokens per second)
1.00.087.114 I slot print_timing: id 6 | task 118 | eval time = 5946.74 ms / 125 tokens ( 47.57 ms per token, 21.02 tokens per second)
1.00.087.115 I slot print_timing: id 6 | task 118 | total time = 6213.62 ms / 153 tokens
1.00.087.115 I slot print_timing: id 6 | task 118 | graphs reused = 1
1.00.087.118 I slot print_timing: id 6 | task 118 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.00.087.137 I statistics draft-mtp: #calls(b,g,a) = 47 134 2039, #gen drafts = 2048, #acc drafts = 1495, #gen tokens = 4087, #acc tokens = 2788, #mean acc len = 2.37, #acc rate/pos = (0.733, 0.634), dur(b,g,a) = 0.027, 474.531, 2.589 ms
1.00.087.164 I slot release: id 6 | task 118 | stop processing: n_tokens = 152, truncated = 0
1.00.087.687 I slot print_timing: id 7 | task 131 | prompt eval time = 125.12 ms / 4 tokens ( 31.28 ms per token, 31.97 tokens per second)
1.00.087.689 I slot print_timing: id 7 | task 131 | eval time = 4947.94 ms / 114 tokens ( 43.40 ms per token, 23.04 tokens per second)
1.00.087.690 I slot print_timing: id 7 | task 131 | total time = 5073.07 ms / 118 tokens
1.00.087.691 I slot print_timing: id 7 | task 131 | graphs reused = 1
1.00.087.693 I slot print_timing: id 7 | task 131 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.00.087.698 I statistics draft-mtp: #calls(b,g,a) = 47 134 2040, #gen drafts = 2048, #acc drafts = 1496, #gen tokens = 4087, #acc tokens = 2790, #mean acc len = 2.37, #acc rate/pos = (0.733, 0.634), dur(b,g,a) = 0.027, 474.531, 2.590 ms
1.00.087.718 I slot release: id 7 | task 131 | stop processing: n_tokens = 140, truncated = 0
1.00.096.869 I srv operator(): Chat format: peg-native
1.00.097.136 I srv operator(): Chat format: peg-native
1.00.200.679 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.00.200.753 I slot launch_slot_: id 7 | task 184 | processing task, is_child = 0
1.00.200.756 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.00.200.773 I slot launch_slot_: id 6 | task 185 | processing task, is_child = 0
1.00.205.515 I slot operator(): id 6 | task 185 | Checking checkpoint with [23, 23] against 27...
1.00.209.313 W slot operator(): id 6 | task 185 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.00.209.343 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.209.367 I slot operator(): id 7 | task 184 | Checking checkpoint with [20, 20] against 24...
1.00.212.868 W slot operator(): id 7 | task 184 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.00.212.897 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.684.759 I slot print_timing: id 0 | task 100 | prompt eval time = 131.58 ms / 4 tokens ( 32.90 ms per token, 30.40 tokens per second)
1.00.684.763 I slot print_timing: id 0 | task 100 | eval time = 8434.49 ms / 128 tokens ( 65.89 ms per token, 15.18 tokens per second)
1.00.684.763 I slot print_timing: id 0 | task 100 | total time = 8566.07 ms / 132 tokens
1.00.684.764 I slot print_timing: id 0 | task 100 | graphs reused = 1
1.00.684.767 I slot print_timing: id 0 | task 100 | draft acceptance = 0.44361 ( 59 accepted / 133 generated), mean acceptance length = 1.88, acceptance rate per position = (0.493, 0.388)
1.00.684.783 I statistics draft-mtp: #calls(b,g,a) = 49 139 2108, #gen drafts = 2123, #acc drafts = 1541, #gen tokens = 4236, #acc tokens = 2877, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.634), dur(b,g,a) = 0.028, 492.776, 2.667 ms
1.00.684.810 I slot release: id 0 | task 100 | stop processing: n_tokens = 156, truncated = 0
1.00.693.570 I srv operator(): Chat format: peg-native
1.00.803.664 I slot print_timing: id 12 | task 140 | n_decoded = 101, tg = 21.08 t/s, tg_3s = 21.08 t/s
1.00.804.985 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.00.805.051 I slot launch_slot_: id 0 | task 191 | processing task, is_child = 0
1.00.807.931 I slot operator(): id 0 | task 191 | Checking checkpoint with [24, 24] against 28...
1.00.811.806 W slot operator(): id 0 | task 191 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.00.811.834 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.922.754 I slot print_timing: id 15 | task 104 | prompt eval time = 129.10 ms / 4 tokens ( 32.27 ms per token, 30.98 tokens per second)
1.00.922.758 I slot print_timing: id 15 | task 104 | eval time = 8519.02 ms / 128 tokens ( 66.55 ms per token, 15.03 tokens per second)
1.00.922.758 I slot print_timing: id 15 | task 104 | total time = 8648.12 ms / 132 tokens
1.00.922.759 I slot print_timing: id 15 | task 104 | graphs reused = 1
1.00.922.762 I slot print_timing: id 15 | task 104 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.00.922.777 I statistics draft-mtp: #calls(b,g,a) = 50 141 2138, #gen drafts = 2152, #acc drafts = 1564, #gen tokens = 4293, #acc tokens = 2920, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.634), dur(b,g,a) = 0.029, 498.523, 2.701 ms
1.00.922.804 I slot release: id 15 | task 104 | stop processing: n_tokens = 156, truncated = 0
1.00.931.611 I srv operator(): Chat format: peg-native
1.01.039.992 I slot print_timing: id 9 | task 130 | prompt eval time = 121.63 ms / 4 tokens ( 30.41 ms per token, 32.89 tokens per second)
1.01.039.996 I slot print_timing: id 9 | task 130 | eval time = 5899.99 ms / 125 tokens ( 47.20 ms per token, 21.19 tokens per second)
1.01.039.996 I slot print_timing: id 9 | task 130 | total time = 6021.62 ms / 129 tokens
1.01.039.997 I slot print_timing: id 9 | task 130 | graphs reused = 1
1.01.040.000 I slot print_timing: id 9 | task 130 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.01.040.016 I statistics draft-mtp: #calls(b,g,a) = 50 142 2162, #gen drafts = 2167, #acc drafts = 1581, #gen tokens = 4323, #acc tokens = 2950, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.633), dur(b,g,a) = 0.029, 501.462, 2.728 ms
1.01.040.051 I slot release: id 9 | task 130 | stop processing: n_tokens = 152, truncated = 0
1.01.042.163 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.01.042.228 I slot launch_slot_: id 9 | task 194 | processing task, is_child = 0
1.01.046.457 I slot operator(): id 9 | task 194 | Checking checkpoint with [23, 23] against 27...
1.01.048.798 I srv operator(): Chat format: peg-native
1.01.050.303 W slot operator(): id 9 | task 194 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.01.050.336 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.162.697 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.01.162.765 I slot launch_slot_: id 15 | task 196 | processing task, is_child = 0
1.01.165.686 I slot operator(): id 15 | task 196 | Checking checkpoint with [24, 24] against 3...
1.01.165.688 W slot operator(): id 15 | task 196 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.01.165.691 W slot operator(): id 15 | task 196 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.01.167.714 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.314.375 I slot create_check: id 15 | task 196 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.01.791.616 I slot print_timing: id 3 | task 128 | n_decoded = 100, tg = 15.03 t/s, tg_3s = 15.03 t/s
1.01.794.994 I slot print_timing: id 10 | task 153 | n_decoded = 102, tg = 23.51 t/s, tg_3s = 23.51 t/s
1.01.916.352 I slot print_timing: id 12 | task 140 | prompt eval time = 118.73 ms / 4 tokens ( 29.68 ms per token, 33.69 tokens per second)
1.01.916.354 I slot print_timing: id 12 | task 140 | eval time = 5903.31 ms / 125 tokens ( 47.23 ms per token, 21.17 tokens per second)
1.01.916.354 I slot print_timing: id 12 | task 140 | total time = 6022.03 ms / 129 tokens
1.01.916.355 I slot print_timing: id 12 | task 140 | graphs reused = 1
1.01.916.358 I slot print_timing: id 12 | task 140 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.01.916.371 I statistics draft-mtp: #calls(b,g,a) = 52 149 2272, #gen drafts = 2275, #acc drafts = 1662, #gen tokens = 4539, #acc tokens = 3099, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.632), dur(b,g,a) = 0.031, 523.458, 2.868 ms
1.01.916.399 I slot release: id 12 | task 140 | stop processing: n_tokens = 152, truncated = 0
1.01.924.963 I srv operator(): Chat format: peg-native
1.02.033.300 I slot print_timing: id 14 | task 158 | n_decoded = 102, tg = 23.62 t/s, tg_3s = 23.62 t/s
1.02.033.853 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
1.02.033.926 I slot launch_slot_: id 12 | task 204 | processing task, is_child = 0
1.02.038.432 I slot operator(): id 12 | task 204 | Checking checkpoint with [23, 23] against 3...
1.02.038.438 W slot operator(): id 12 | task 204 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.02.038.440 W slot operator(): id 12 | task 204 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.02.040.020 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.188.093 I slot create_check: id 12 | task 204 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.02.427.647 I slot print_timing: id 10 | task 153 | prompt eval time = 108.91 ms / 4 tokens ( 27.23 ms per token, 36.73 tokens per second)
1.02.427.650 I slot print_timing: id 10 | task 153 | eval time = 4971.81 ms / 114 tokens ( 43.61 ms per token, 22.93 tokens per second)
1.02.427.650 I slot print_timing: id 10 | task 153 | total time = 5080.72 ms / 118 tokens
1.02.427.651 I slot print_timing: id 10 | task 153 | graphs reused = 1
1.02.427.655 I slot print_timing: id 10 | task 153 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.02.427.670 I statistics draft-mtp: #calls(b,g,a) = 53 153 2331, #gen drafts = 2336, #acc drafts = 1707, #gen tokens = 4661, #acc tokens = 3183, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.031, 538.852, 2.934 ms
1.02.427.697 I slot release: id 10 | task 153 | stop processing: n_tokens = 140, truncated = 0
1.02.436.484 I srv operator(): Chat format: peg-native
1.02.546.548 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.02.546.621 I slot launch_slot_: id 10 | task 209 | processing task, is_child = 0
1.02.551.220 I slot operator(): id 10 | task 209 | Checking checkpoint with [20, 20] against 3...
1.02.551.222 W slot operator(): id 10 | task 209 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.02.551.224 W slot operator(): id 10 | task 209 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.02.552.659 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.677.515 I slot print_timing: id 4 | task 138 | n_decoded = 100, tg = 15.00 t/s, tg_3s = 15.00 t/s
1.02.679.628 I slot print_timing: id 8 | task 157 | n_decoded = 101, tg = 20.92 t/s, tg_3s = 20.92 t/s
1.02.681.698 I slot print_timing: id 14 | task 158 | prompt eval time = 129.49 ms / 4 tokens ( 32.37 ms per token, 30.89 tokens per second)
1.02.681.699 I slot print_timing: id 14 | task 158 | eval time = 4966.66 ms / 114 tokens ( 43.57 ms per token, 22.95 tokens per second)
1.02.681.699 I slot print_timing: id 14 | task 158 | total time = 5096.14 ms / 118 tokens
1.02.681.701 I slot print_timing: id 14 | task 158 | graphs reused = 1
1.02.681.704 I slot print_timing: id 14 | task 158 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.02.681.723 I statistics draft-mtp: #calls(b,g,a) = 53 155 2365, #gen drafts = 2366, #acc drafts = 1732, #gen tokens = 4721, #acc tokens = 3231, #mean acc len = 2.37, #acc rate/pos = (0.732, 0.634), dur(b,g,a) = 0.031, 547.682, 2.976 ms
1.02.681.756 I slot release: id 14 | task 158 | stop processing: n_tokens = 140, truncated = 0
1.02.690.523 I srv operator(): Chat format: peg-native
1.02.700.460 I slot create_check: id 10 | task 209 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.02.813.817 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.02.813.890 I slot launch_slot_: id 14 | task 212 | processing task, is_child = 0
1.02.817.714 I slot operator(): id 14 | task 212 | Checking checkpoint with [20, 20] against 24...
1.02.821.551 W slot operator(): id 14 | task 212 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.02.821.580 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.180.425 I slot print_timing: id 11 | task 169 | n_decoded = 102, tg = 23.54 t/s, tg_3s = 23.54 t/s
1.03.420.414 I slot print_timing: id 5 | task 167 | n_decoded = 101, tg = 20.85 t/s, tg_3s = 20.85 t/s
1.03.659.414 I slot print_timing: id 3 | task 128 | prompt eval time = 263.72 ms / 29 tokens ( 9.09 ms per token, 109.96 tokens per second)
1.03.659.417 I slot print_timing: id 3 | task 128 | eval time = 8519.89 ms / 128 tokens ( 66.56 ms per token, 15.02 tokens per second)
1.03.659.418 I slot print_timing: id 3 | task 128 | total time = 8783.61 ms / 157 tokens
1.03.659.418 I slot print_timing: id 3 | task 128 | graphs reused = 1
1.03.659.421 I slot print_timing: id 3 | task 128 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.03.659.436 I statistics draft-mtp: #calls(b,g,a) = 55 163 2475, #gen drafts = 2490, #acc drafts = 1811, #gen tokens = 4968, #acc tokens = 3378, #mean acc len = 2.36, #acc rate/pos = (0.732, 0.633), dur(b,g,a) = 0.032, 575.156, 3.101 ms
1.03.659.462 I slot release: id 3 | task 128 | stop processing: n_tokens = 156, truncated = 0
1.03.668.488 I srv operator(): Chat format: peg-native
1.03.778.116 I slot print_timing: id 8 | task 157 | prompt eval time = 269.44 ms / 28 tokens ( 9.62 ms per token, 103.92 tokens per second)
1.03.778.119 I slot print_timing: id 8 | task 157 | eval time = 5925.28 ms / 125 tokens ( 47.40 ms per token, 21.10 tokens per second)
1.03.778.120 I slot print_timing: id 8 | task 157 | total time = 6194.72 ms / 153 tokens
1.03.778.121 I slot print_timing: id 8 | task 157 | graphs reused = 1
1.03.778.124 I slot print_timing: id 8 | task 157 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.03.778.142 I statistics draft-mtp: #calls(b,g,a) = 55 164 2498, #gen drafts = 2505, #acc drafts = 1826, #gen tokens = 4998, #acc tokens = 3405, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.032, 579.127, 3.129 ms
1.03.778.172 I slot release: id 8 | task 157 | stop processing: n_tokens = 152, truncated = 0
1.03.779.493 I slot print_timing: id 11 | task 169 | prompt eval time = 262.09 ms / 25 tokens ( 10.48 ms per token, 95.39 tokens per second)
1.03.779.495 I slot print_timing: id 11 | task 169 | eval time = 4931.88 ms / 114 tokens ( 43.26 ms per token, 23.11 tokens per second)
1.03.779.496 I slot print_timing: id 11 | task 169 | total time = 5193.97 ms / 139 tokens
1.03.779.496 I slot print_timing: id 11 | task 169 | graphs reused = 1
1.03.779.498 I slot print_timing: id 11 | task 169 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.03.779.502 I statistics draft-mtp: #calls(b,g,a) = 55 164 2501, #gen drafts = 2505, #acc drafts = 1828, #gen tokens = 4998, #acc tokens = 3409, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.632), dur(b,g,a) = 0.032, 579.127, 3.132 ms
1.03.779.522 I slot release: id 11 | task 169 | stop processing: n_tokens = 140, truncated = 0
1.03.781.384 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.03.781.455 I slot launch_slot_: id 3 | task 221 | processing task, is_child = 0
1.03.786.842 I srv operator(): Chat format: peg-native
1.03.787.639 I srv operator(): Chat format: peg-native
1.03.788.003 I slot operator(): id 3 | task 221 | Checking checkpoint with [24, 24] against 28...
1.03.791.813 W slot operator(): id 3 | task 221 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.03.791.841 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.897.786 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.03.897.858 I slot launch_slot_: id 8 | task 223 | processing task, is_child = 0
1.03.897.861 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.03.897.881 I slot launch_slot_: id 11 | task 224 | processing task, is_child = 0
1.03.903.435 I slot operator(): id 8 | task 223 | Checking checkpoint with [23, 23] against 27...
1.03.907.315 W slot operator(): id 8 | task 223 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.03.907.347 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.907.362 I slot operator(): id 11 | task 224 | Checking checkpoint with [20, 20] against 24...
1.03.910.837 W slot operator(): id 11 | task 224 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.03.910.871 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.267.542 I slot print_timing: id 1 | task 155 | n_decoded = 100, tg = 14.94 t/s, tg_3s = 14.94 t/s
1.04.507.218 I slot print_timing: id 4 | task 138 | prompt eval time = 263.19 ms / 29 tokens ( 9.08 ms per token, 110.19 tokens per second)
1.04.507.221 I slot print_timing: id 4 | task 138 | eval time = 8494.42 ms / 128 tokens ( 66.36 ms per token, 15.07 tokens per second)
1.04.507.222 I slot print_timing: id 4 | task 138 | total time = 8757.62 ms / 157 tokens
1.04.507.223 I slot print_timing: id 4 | task 138 | graphs reused = 1
1.04.507.225 I slot print_timing: id 4 | task 138 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.04.507.239 I statistics draft-mtp: #calls(b,g,a) = 58 170 2580, #gen drafts = 2595, #acc drafts = 1883, #gen tokens = 5177, #acc tokens = 3510, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.631), dur(b,g,a) = 0.034, 604.354, 3.227 ms
1.04.507.265 I slot release: id 4 | task 138 | stop processing: n_tokens = 156, truncated = 0
1.04.509.247 I slot print_timing: id 5 | task 167 | prompt eval time = 112.33 ms / 4 tokens ( 28.08 ms per token, 35.61 tokens per second)
1.04.509.249 I slot print_timing: id 5 | task 167 | eval time = 5933.80 ms / 125 tokens ( 47.47 ms per token, 21.07 tokens per second)
1.04.509.250 I slot print_timing: id 5 | task 167 | total time = 6046.12 ms / 129 tokens
1.04.509.250 I slot print_timing: id 5 | task 167 | graphs reused = 1
1.04.509.252 I slot print_timing: id 5 | task 167 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.04.509.257 I statistics draft-mtp: #calls(b,g,a) = 58 170 2585, #gen drafts = 2595, #acc drafts = 1885, #gen tokens = 5177, #acc tokens = 3514, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.034, 604.354, 3.234 ms
1.04.509.279 I slot release: id 5 | task 167 | stop processing: n_tokens = 152, truncated = 0
1.04.515.576 I srv operator(): Chat format: peg-native
1.04.517.421 I srv operator(): Chat format: peg-native
1.04.618.325 I slot print_timing: id 7 | task 184 | n_decoded = 102, tg = 23.77 t/s, tg_3s = 23.77 t/s
1.04.622.189 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.04.622.259 I slot launch_slot_: id 4 | task 231 | processing task, is_child = 0
1.04.622.262 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.04.622.281 I slot launch_slot_: id 5 | task 232 | processing task, is_child = 0
1.04.627.107 I slot operator(): id 4 | task 231 | Checking checkpoint with [24, 24] against 28...
1.04.630.925 W slot operator(): id 4 | task 231 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.04.630.978 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.630.983 I slot operator(): id 5 | task 232 | Checking checkpoint with [23, 23] against 27...
1.04.634.520 W slot operator(): id 5 | task 232 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.04.634.559 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.114.955 I slot print_timing: id 6 | task 185 | n_decoded = 101, tg = 21.09 t/s, tg_3s = 21.09 t/s
1.05.233.834 I slot print_timing: id 2 | task 166 | n_decoded = 100, tg = 15.02 t/s, tg_3s = 15.02 t/s
1.05.236.061 I slot print_timing: id 7 | task 184 | prompt eval time = 117.74 ms / 4 tokens ( 29.44 ms per token, 33.97 tokens per second)
1.05.236.064 I slot print_timing: id 7 | task 184 | eval time = 4908.93 ms / 114 tokens ( 43.06 ms per token, 23.22 tokens per second)
1.05.236.064 I slot print_timing: id 7 | task 184 | total time = 5026.67 ms / 118 tokens
1.05.236.065 I slot print_timing: id 7 | task 184 | graphs reused = 1
1.05.236.068 I slot print_timing: id 7 | task 184 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.05.236.086 I statistics draft-mtp: #calls(b,g,a) = 60 176 2679, #gen drafts = 2687, #acc drafts = 1957, #gen tokens = 5361, #acc tokens = 3650, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.632), dur(b,g,a) = 0.035, 625.176, 3.349 ms
1.05.236.111 I slot release: id 7 | task 184 | stop processing: n_tokens = 140, truncated = 0
1.05.244.241 I srv operator(): Chat format: peg-native
1.05.356.626 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.05.356.699 I slot launch_slot_: id 7 | task 239 | processing task, is_child = 0
1.05.361.684 I slot operator(): id 7 | task 239 | Checking checkpoint with [20, 20] against 24...
1.05.365.503 W slot operator(): id 7 | task 239 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.05.365.532 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.725.798 I slot print_timing: id 15 | task 196 | n_decoded = 102, tg = 23.73 t/s, tg_3s = 23.73 t/s
1.05.964.073 I slot print_timing: id 9 | task 194 | n_decoded = 101, tg = 21.01 t/s, tg_3s = 21.01 t/s
1.05.965.954 I slot print_timing: id 13 | task 174 | n_decoded = 100, tg = 15.13 t/s, tg_3s = 15.13 t/s
1.06.079.862 I slot print_timing: id 1 | task 155 | prompt eval time = 104.25 ms / 4 tokens ( 26.06 ms per token, 38.37 tokens per second)
1.06.079.865 I slot print_timing: id 1 | task 155 | eval time = 8507.85 ms / 128 tokens ( 66.47 ms per token, 15.04 tokens per second)
1.06.079.865 I slot print_timing: id 1 | task 155 | total time = 8612.11 ms / 132 tokens
1.06.079.866 I slot print_timing: id 1 | task 155 | graphs reused = 1
1.06.079.869 I slot print_timing: id 1 | task 155 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.06.079.883 I statistics draft-mtp: #calls(b,g,a) = 61 183 2781, #gen drafts = 2796, #acc drafts = 2028, #gen tokens = 5578, #acc tokens = 3783, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.036, 650.619, 3.457 ms
1.06.079.909 I slot release: id 1 | task 155 | stop processing: n_tokens = 156, truncated = 0
1.06.089.450 I srv operator(): Chat format: peg-native
1.06.197.678 I slot print_timing: id 6 | task 185 | prompt eval time = 121.34 ms / 4 tokens ( 30.34 ms per token, 32.96 tokens per second)
1.06.197.681 I slot print_timing: id 6 | task 185 | eval time = 5870.78 ms / 125 tokens ( 46.97 ms per token, 21.29 tokens per second)
1.06.197.681 I slot print_timing: id 6 | task 185 | total time = 5992.12 ms / 129 tokens
1.06.197.682 I slot print_timing: id 6 | task 185 | graphs reused = 1
1.06.197.686 I slot print_timing: id 6 | task 185 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.06.197.700 I statistics draft-mtp: #calls(b,g,a) = 61 184 2802, #gen drafts = 2811, #acc drafts = 2046, #gen tokens = 5608, #acc tokens = 3816, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.632), dur(b,g,a) = 0.036, 654.403, 3.482 ms
1.06.197.732 I slot release: id 6 | task 185 | stop processing: n_tokens = 152, truncated = 0
1.06.201.586 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.06.201.655 I slot launch_slot_: id 1 | task 247 | processing task, is_child = 0
1.06.206.515 I srv operator(): Chat format: peg-native
1.06.206.718 I slot operator(): id 1 | task 247 | Checking checkpoint with [24, 24] against 28...
1.06.210.580 W slot operator(): id 1 | task 247 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.06.210.614 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.323.238 I slot print_timing: id 15 | task 196 | prompt eval time = 261.24 ms / 25 tokens ( 10.45 ms per token, 95.70 tokens per second)
1.06.323.241 I slot print_timing: id 15 | task 196 | eval time = 4896.30 ms / 114 tokens ( 42.95 ms per token, 23.28 tokens per second)
1.06.323.241 I slot print_timing: id 15 | task 196 | total time = 5157.53 ms / 139 tokens
1.06.323.242 I slot print_timing: id 15 | task 196 | graphs reused = 1
1.06.323.245 I slot print_timing: id 15 | task 196 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.06.323.258 I statistics draft-mtp: #calls(b,g,a) = 62 185 2825, #gen drafts = 2825, #acc drafts = 2061, #gen tokens = 5636, #acc tokens = 3843, #mean acc len = 2.36, #acc rate/pos = (0.730, 0.631), dur(b,g,a) = 0.037, 659.430, 3.509 ms
1.06.323.288 I slot release: id 15 | task 196 | stop processing: n_tokens = 140, truncated = 0
1.06.323.304 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.06.323.375 I slot launch_slot_: id 6 | task 249 | processing task, is_child = 0
1.06.328.103 I slot operator(): id 6 | task 249 | Checking checkpoint with [23, 23] against 27...
1.06.331.229 I srv operator(): Chat format: peg-native
1.06.331.906 W slot operator(): id 6 | task 249 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.06.331.948 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.444.772 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.06.444.841 I slot launch_slot_: id 15 | task 251 | processing task, is_child = 0
1.06.447.754 I slot operator(): id 15 | task 251 | Checking checkpoint with [20, 20] against 24...
1.06.451.737 W slot operator(): id 15 | task 251 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.06.451.780 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.045.801 I slot print_timing: id 2 | task 166 | prompt eval time = 115.95 ms / 4 tokens ( 28.99 ms per token, 34.50 tokens per second)
1.07.045.805 I slot print_timing: id 2 | task 166 | eval time = 8470.60 ms / 128 tokens ( 66.18 ms per token, 15.11 tokens per second)
1.07.045.805 I slot print_timing: id 2 | task 166 | total time = 8586.55 ms / 132 tokens
1.07.045.806 I slot print_timing: id 2 | task 166 | graphs reused = 1
1.07.045.809 I slot print_timing: id 2 | task 166 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.07.045.823 I statistics draft-mtp: #calls(b,g,a) = 64 191 2902, #gen drafts = 2917, #acc drafts = 2115, #gen tokens = 5819, #acc tokens = 3946, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.037, 679.610, 3.588 ms
1.07.045.848 I slot release: id 2 | task 166 | stop processing: n_tokens = 156, truncated = 0
1.07.050.037 I slot print_timing: id 9 | task 194 | prompt eval time = 110.16 ms / 4 tokens ( 27.54 ms per token, 36.31 tokens per second)
1.07.050.039 I slot print_timing: id 9 | task 194 | eval time = 5893.38 ms / 125 tokens ( 47.15 ms per token, 21.21 tokens per second)
1.07.050.040 I slot print_timing: id 9 | task 194 | total time = 6003.55 ms / 129 tokens
1.07.050.041 I slot print_timing: id 9 | task 194 | graphs reused = 1
1.07.050.043 I slot print_timing: id 9 | task 194 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.07.050.051 I statistics draft-mtp: #calls(b,g,a) = 64 191 2911, #gen drafts = 2917, #acc drafts = 2122, #gen tokens = 5819, #acc tokens = 3958, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.037, 679.610, 3.598 ms
1.07.050.086 I slot release: id 9 | task 194 | stop processing: n_tokens = 152, truncated = 0
1.07.054.916 I srv operator(): Chat format: peg-native
1.07.058.526 I srv operator(): Chat format: peg-native
1.07.162.194 I slot print_timing: id 14 | task 212 | n_decoded = 102, tg = 24.13 t/s, tg_3s = 24.13 t/s
1.07.162.714 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.07.162.787 I slot launch_slot_: id 2 | task 258 | processing task, is_child = 0
1.07.162.792 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.07.162.811 I slot launch_slot_: id 9 | task 259 | processing task, is_child = 0
1.07.168.134 I slot operator(): id 2 | task 258 | Checking checkpoint with [24, 24] against 28...
1.07.171.954 W slot operator(): id 2 | task 258 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.07.171.981 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.172.005 I slot operator(): id 9 | task 259 | Checking checkpoint with [23, 23] against 27...
1.07.175.482 W slot operator(): id 9 | task 259 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.07.175.511 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.410.389 I slot print_timing: id 0 | task 191 | n_decoded = 100, tg = 15.41 t/s, tg_3s = 15.41 t/s
1.07.534.759 I slot print_timing: id 10 | task 209 | n_decoded = 101, tg = 21.37 t/s, tg_3s = 21.37 t/s
1.07.771.281 I slot print_timing: id 13 | task 174 | prompt eval time = 262.69 ms / 29 tokens ( 9.06 ms per token, 110.39 tokens per second)
1.07.771.283 I slot print_timing: id 13 | task 174 | eval time = 8413.19 ms / 128 tokens ( 65.73 ms per token, 15.21 tokens per second)
1.07.771.284 I slot print_timing: id 13 | task 174 | total time = 8675.89 ms / 157 tokens
1.07.771.285 I slot print_timing: id 13 | task 174 | graphs reused = 1
1.07.771.288 I slot print_timing: id 13 | task 174 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.07.771.302 I statistics draft-mtp: #calls(b,g,a) = 66 197 2993, #gen drafts = 3008, #acc drafts = 2181, #gen tokens = 6000, #acc tokens = 4068, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.038, 703.074, 3.693 ms
1.07.771.330 I slot release: id 13 | task 174 | stop processing: n_tokens = 156, truncated = 0
1.07.777.933 I slot print_timing: id 14 | task 212 | prompt eval time = 117.36 ms / 4 tokens ( 29.34 ms per token, 34.08 tokens per second)
1.07.777.935 I slot print_timing: id 14 | task 212 | eval time = 4842.85 ms / 114 tokens ( 42.48 ms per token, 23.54 tokens per second)
1.07.777.935 I slot print_timing: id 14 | task 212 | total time = 4960.20 ms / 118 tokens
1.07.777.936 I slot print_timing: id 14 | task 212 | graphs reused = 1
1.07.777.938 I slot print_timing: id 14 | task 212 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.07.777.946 I statistics draft-mtp: #calls(b,g,a) = 66 197 3007, #gen drafts = 3008, #acc drafts = 2192, #gen tokens = 6000, #acc tokens = 4089, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.038, 703.074, 3.711 ms
1.07.777.970 I slot release: id 14 | task 212 | stop processing: n_tokens = 140, truncated = 0
1.07.779.923 I srv operator(): Chat format: peg-native
1.07.786.882 I srv operator(): Chat format: peg-native
1.07.886.994 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.07.887.065 I slot launch_slot_: id 14 | task 266 | processing task, is_child = 0
1.07.887.068 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.07.887.091 I slot launch_slot_: id 13 | task 267 | processing task, is_child = 0
1.07.890.821 I slot operator(): id 13 | task 267 | Checking checkpoint with [24, 24] against 28...
1.07.894.658 W slot operator(): id 13 | task 267 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.07.894.690 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.894.710 I slot operator(): id 14 | task 266 | Checking checkpoint with [20, 20] against 24...
1.07.898.213 W slot operator(): id 14 | task 266 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.07.898.243 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.257.004 I slot print_timing: id 11 | task 224 | n_decoded = 102, tg = 24.11 t/s, tg_3s = 24.11 t/s
1.08.620.330 I slot print_timing: id 10 | task 209 | prompt eval time = 256.42 ms / 28 tokens ( 9.16 ms per token, 109.20 tokens per second)
1.08.620.335 I slot print_timing: id 10 | task 209 | eval time = 5812.65 ms / 125 tokens ( 46.50 ms per token, 21.50 tokens per second)
1.08.620.335 I slot print_timing: id 10 | task 209 | total time = 6069.07 ms / 153 tokens
1.08.620.337 I slot print_timing: id 10 | task 209 | graphs reused = 1
1.08.620.341 I slot print_timing: id 10 | task 209 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.08.620.362 I statistics draft-mtp: #calls(b,g,a) = 68 204 3111, #gen drafts = 3116, #acc drafts = 2267, #gen tokens = 6216, #acc tokens = 4230, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.631), dur(b,g,a) = 0.039, 725.251, 3.837 ms
1.08.620.391 I slot release: id 10 | task 209 | stop processing: n_tokens = 152, truncated = 0
1.08.629.261 I srv operator(): Chat format: peg-native
1.08.736.019 I slot print_timing: id 8 | task 223 | n_decoded = 101, tg = 21.44 t/s, tg_3s = 21.44 t/s
1.08.738.497 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.08.738.567 I slot launch_slot_: id 10 | task 275 | processing task, is_child = 0
1.08.743.152 I slot operator(): id 10 | task 275 | Checking checkpoint with [23, 23] against 27...
1.08.747.005 W slot operator(): id 10 | task 275 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.08.747.028 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.864.631 I slot print_timing: id 11 | task 224 | prompt eval time = 118.34 ms / 4 tokens ( 29.58 ms per token, 33.80 tokens per second)
1.08.864.634 I slot print_timing: id 11 | task 224 | eval time = 4838.91 ms / 114 tokens ( 42.45 ms per token, 23.56 tokens per second)
1.08.864.635 I slot print_timing: id 11 | task 224 | total time = 4957.24 ms / 118 tokens
1.08.864.636 I slot print_timing: id 11 | task 224 | graphs reused = 1
1.08.864.639 I slot print_timing: id 11 | task 224 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.08.864.656 I statistics draft-mtp: #calls(b,g,a) = 69 206 3142, #gen drafts = 3146, #acc drafts = 2290, #gen tokens = 6276, #acc tokens = 4271, #mean acc len = 2.36, #acc rate/pos = (0.729, 0.630), dur(b,g,a) = 0.040, 734.064, 3.872 ms
1.08.864.687 I slot release: id 11 | task 224 | stop processing: n_tokens = 140, truncated = 0
1.08.865.248 I slot print_timing: id 12 | task 204 | n_decoded = 100, tg = 15.24 t/s, tg_3s = 15.24 t/s
1.08.873.208 I srv operator(): Chat format: peg-native
1.08.981.546 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.08.981.617 I slot launch_slot_: id 11 | task 278 | processing task, is_child = 0
1.08.986.404 I slot operator(): id 11 | task 278 | Checking checkpoint with [20, 20] against 24...
1.08.990.195 W slot operator(): id 11 | task 278 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.08.990.232 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.220.138 I slot print_timing: id 0 | task 191 | prompt eval time = 114.60 ms / 4 tokens ( 28.65 ms per token, 34.90 tokens per second)
1.09.220.140 I slot print_timing: id 0 | task 191 | eval time = 8297.58 ms / 128 tokens ( 64.82 ms per token, 15.43 tokens per second)
1.09.220.141 I slot print_timing: id 0 | task 191 | total time = 8412.18 ms / 132 tokens
1.09.220.141 I slot print_timing: id 0 | task 191 | graphs reused = 1
1.09.220.144 I slot print_timing: id 0 | task 191 | draft acceptance = 0.44361 ( 59 accepted / 133 generated), mean acceptance length = 1.88, acceptance rate per position = (0.493, 0.388)
1.09.220.158 I statistics draft-mtp: #calls(b,g,a) = 70 209 3176, #gen drafts = 3191, #acc drafts = 2310, #gen tokens = 6365, #acc tokens = 4310, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.630), dur(b,g,a) = 0.041, 745.983, 3.914 ms
1.09.220.182 I slot release: id 0 | task 191 | stop processing: n_tokens = 156, truncated = 0
1.09.228.850 I srv operator(): Chat format: peg-native
1.09.341.252 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.09.341.323 I slot launch_slot_: id 0 | task 282 | processing task, is_child = 0
1.09.344.304 I slot operator(): id 0 | task 282 | Checking checkpoint with [24, 24] against 28...
1.09.348.168 W slot operator(): id 0 | task 282 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.09.348.203 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.463.542 I slot print_timing: id 5 | task 232 | n_decoded = 101, tg = 21.42 t/s, tg_3s = 21.42 t/s
1.09.704.405 I slot print_timing: id 7 | task 239 | n_decoded = 102, tg = 24.14 t/s, tg_3s = 24.14 t/s
1.09.825.771 I slot print_timing: id 8 | task 223 | prompt eval time = 122.01 ms / 4 tokens ( 30.50 ms per token, 32.78 tokens per second)
1.09.825.775 I slot print_timing: id 8 | task 223 | eval time = 5800.28 ms / 125 tokens ( 46.40 ms per token, 21.55 tokens per second)
1.09.825.775 I slot print_timing: id 8 | task 223 | total time = 5922.29 ms / 129 tokens
1.09.825.776 I slot print_timing: id 8 | task 223 | graphs reused = 1
1.09.825.779 I slot print_timing: id 8 | task 223 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.09.825.799 I statistics draft-mtp: #calls(b,g,a) = 71 214 3262, #gen drafts = 3269, #acc drafts = 2373, #gen tokens = 6521, #acc tokens = 4427, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.630), dur(b,g,a) = 0.042, 760.798, 4.015 ms
1.09.825.830 I slot release: id 8 | task 223 | stop processing: n_tokens = 152, truncated = 0
1.09.834.209 I srv operator(): Chat format: peg-native
1.09.948.261 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.09.948.332 I slot launch_slot_: id 8 | task 288 | processing task, is_child = 0
1.09.953.262 I slot operator(): id 8 | task 288 | Checking checkpoint with [23, 23] against 27...
1.09.957.109 W slot operator(): id 8 | task 288 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.09.957.145 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.314.589 I slot print_timing: id 7 | task 239 | prompt eval time = 116.84 ms / 4 tokens ( 29.21 ms per token, 34.23 tokens per second)
1.10.314.592 I slot print_timing: id 7 | task 239 | eval time = 4836.04 ms / 114 tokens ( 42.42 ms per token, 23.57 tokens per second)
1.10.314.593 I slot print_timing: id 7 | task 239 | total time = 4952.88 ms / 118 tokens
1.10.314.594 I slot print_timing: id 7 | task 239 | graphs reused = 1
1.10.314.597 I slot print_timing: id 7 | task 239 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.10.314.615 I statistics draft-mtp: #calls(b,g,a) = 72 218 3323, #gen drafts = 3331, #acc drafts = 2418, #gen tokens = 6645, #acc tokens = 4512, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.630), dur(b,g,a) = 0.043, 776.630, 4.087 ms
1.10.314.645 I slot release: id 7 | task 239 | stop processing: n_tokens = 140, truncated = 0
1.10.323.450 I srv operator(): Chat format: peg-native
1.10.429.195 I slot print_timing: id 3 | task 221 | n_decoded = 100, tg = 15.30 t/s, tg_3s = 15.30 t/s
1.10.433.824 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.10.433.897 I slot launch_slot_: id 7 | task 293 | processing task, is_child = 0
1.10.438.953 I slot operator(): id 7 | task 293 | Checking checkpoint with [20, 20] against 24...
1.10.442.832 W slot operator(): id 7 | task 293 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.10.442.862 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.558.448 I slot print_timing: id 5 | task 232 | prompt eval time = 117.94 ms / 4 tokens ( 29.49 ms per token, 33.91 tokens per second)
1.10.558.450 I slot print_timing: id 5 | task 232 | eval time = 5809.49 ms / 125 tokens ( 46.48 ms per token, 21.52 tokens per second)
1.10.558.451 I slot print_timing: id 5 | task 232 | total time = 5927.43 ms / 129 tokens
1.10.558.452 I slot print_timing: id 5 | task 232 | graphs reused = 1
1.10.558.454 I slot print_timing: id 5 | task 232 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.10.558.472 I statistics draft-mtp: #calls(b,g,a) = 73 220 3352, #gen drafts = 3361, #acc drafts = 2438, #gen tokens = 6704, #acc tokens = 4545, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.044, 786.560, 4.123 ms
1.10.558.509 I slot release: id 5 | task 232 | stop processing: n_tokens = 152, truncated = 0
1.10.566.818 I srv operator(): Chat format: peg-native
1.10.671.355 I slot print_timing: id 12 | task 204 | prompt eval time = 263.22 ms / 29 tokens ( 9.08 ms per token, 110.18 tokens per second)
1.10.671.358 I slot print_timing: id 12 | task 204 | eval time = 8369.68 ms / 128 tokens ( 65.39 ms per token, 15.29 tokens per second)
1.10.671.358 I slot print_timing: id 12 | task 204 | total time = 8632.90 ms / 157 tokens
1.10.671.359 I slot print_timing: id 12 | task 204 | graphs reused = 1
1.10.671.362 I slot print_timing: id 12 | task 204 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.10.671.373 I statistics draft-mtp: #calls(b,g,a) = 73 221 3361, #gen drafts = 3375, #acc drafts = 2445, #gen tokens = 6732, #acc tokens = 4559, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.044, 792.021, 4.133 ms
1.10.671.399 I slot release: id 12 | task 204 | stop processing: n_tokens = 156, truncated = 0
1.10.677.801 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.10.677.870 I slot launch_slot_: id 12 | task 296 | processing task, is_child = 0
1.10.680.354 I srv operator(): Chat format: peg-native
1.10.683.302 I slot operator(): id 12 | task 296 | Checking checkpoint with [24, 24] against 28...
1.10.687.189 W slot operator(): id 12 | task 296 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.10.687.219 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.10.799.620 I slot print_timing: id 15 | task 251 | n_decoded = 102, tg = 24.09 t/s, tg_3s = 24.09 t/s
1.10.799.631 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.10.799.697 I slot launch_slot_: id 5 | task 298 | processing task, is_child = 0
1.10.803.955 I slot operator(): id 5 | task 298 | Checking checkpoint with [23, 23] against 27...
1.10.807.841 W slot operator(): id 5 | task 298 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.10.807.866 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.167.289 I slot print_timing: id 6 | task 249 | n_decoded = 101, tg = 21.36 t/s, tg_3s = 21.36 t/s
1.11.286.737 I slot print_timing: id 4 | task 231 | n_decoded = 100, tg = 15.30 t/s, tg_3s = 15.30 t/s
1.11.411.865 I slot print_timing: id 15 | task 251 | prompt eval time = 116.93 ms / 4 tokens ( 29.23 ms per token, 34.21 tokens per second)
1.11.411.868 I slot print_timing: id 15 | task 251 | eval time = 4847.16 ms / 114 tokens ( 42.52 ms per token, 23.52 tokens per second)
1.11.411.868 I slot print_timing: id 15 | task 251 | total time = 4964.09 ms / 118 tokens
1.11.411.869 I slot print_timing: id 15 | task 251 | graphs reused = 1
1.11.411.872 I slot print_timing: id 15 | task 251 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.11.411.884 I statistics draft-mtp: #calls(b,g,a) = 75 227 3468, #gen drafts = 3468, #acc drafts = 2524, #gen tokens = 6918, #acc tokens = 4706, #mean acc len = 2.36, #acc rate/pos = (0.728, 0.629), dur(b,g,a) = 0.045, 813.573, 4.263 ms
1.11.411.909 I slot release: id 15 | task 251 | stop processing: n_tokens = 140, truncated = 0
1.11.420.274 I srv operator(): Chat format: peg-native
1.11.524.388 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.11.524.459 I slot launch_slot_: id 15 | task 305 | processing task, is_child = 0
1.11.527.369 I slot operator(): id 15 | task 305 | Checking checkpoint with [20, 20] against 24...
1.11.531.292 W slot operator(): id 15 | task 305 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.11.531.339 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.008.965 I slot print_timing: id 9 | task 259 | n_decoded = 101, tg = 21.41 t/s, tg_3s = 21.41 t/s
1.12.244.821 I slot print_timing: id 3 | task 221 | prompt eval time = 104.33 ms / 4 tokens ( 26.08 ms per token, 38.34 tokens per second)
1.12.244.823 I slot print_timing: id 3 | task 221 | eval time = 8352.45 ms / 128 tokens ( 65.25 ms per token, 15.32 tokens per second)
1.12.244.824 I slot print_timing: id 3 | task 221 | total time = 8456.79 ms / 132 tokens
1.12.244.825 I slot print_timing: id 3 | task 221 | graphs reused = 1
1.12.244.828 I slot print_timing: id 3 | task 221 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.12.244.842 I statistics draft-mtp: #calls(b,g,a) = 76 234 3562, #gen drafts = 3577, #acc drafts = 2591, #gen tokens = 7135, #acc tokens = 4831, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.046, 835.006, 4.364 ms
1.12.244.868 I slot release: id 3 | task 221 | stop processing: n_tokens = 156, truncated = 0
1.12.247.594 I slot print_timing: id 6 | task 249 | prompt eval time = 110.17 ms / 4 tokens ( 27.54 ms per token, 36.31 tokens per second)
1.12.247.598 I slot print_timing: id 6 | task 249 | eval time = 5809.30 ms / 125 tokens ( 46.47 ms per token, 21.52 tokens per second)
1.12.247.598 I slot print_timing: id 6 | task 249 | total time = 5919.46 ms / 129 tokens
1.12.247.599 I slot print_timing: id 6 | task 249 | graphs reused = 1
1.12.247.601 I slot print_timing: id 6 | task 249 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.12.247.607 I statistics draft-mtp: #calls(b,g,a) = 76 234 3568, #gen drafts = 3577, #acc drafts = 2595, #gen tokens = 7135, #acc tokens = 4839, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.629), dur(b,g,a) = 0.046, 835.006, 4.371 ms
1.12.247.630 I slot release: id 6 | task 249 | stop processing: n_tokens = 152, truncated = 0
1.12.251.048 I slot print_timing: id 14 | task 266 | n_decoded = 102, tg = 24.07 t/s, tg_3s = 24.07 t/s
1.12.253.984 I srv operator(): Chat format: peg-native
1.12.255.779 I srv operator(): Chat format: peg-native
1.12.360.059 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.12.360.135 I slot launch_slot_: id 3 | task 313 | processing task, is_child = 0
1.12.360.140 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.12.360.158 I slot launch_slot_: id 6 | task 314 | processing task, is_child = 0
1.12.365.055 I slot operator(): id 3 | task 313 | Checking checkpoint with [24, 24] against 28...
1.12.368.886 W slot operator(): id 3 | task 313 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.12.368.910 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.368.936 I slot operator(): id 6 | task 314 | Checking checkpoint with [23, 23] against 27...
1.12.372.464 W slot operator(): id 6 | task 314 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.12.372.489 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.850.105 I slot print_timing: id 1 | task 247 | n_decoded = 100, tg = 15.31 t/s, tg_3s = 15.31 t/s
1.12.855.339 I slot print_timing: id 14 | task 266 | prompt eval time = 118.14 ms / 4 tokens ( 29.54 ms per token, 33.86 tokens per second)
1.12.855.341 I slot print_timing: id 14 | task 266 | eval time = 4842.47 ms / 114 tokens ( 42.48 ms per token, 23.54 tokens per second)
1.12.855.342 I slot print_timing: id 14 | task 266 | total time = 4960.61 ms / 118 tokens
1.12.855.343 I slot print_timing: id 14 | task 266 | graphs reused = 1
1.12.855.346 I slot print_timing: id 14 | task 266 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.12.855.358 I statistics draft-mtp: #calls(b,g,a) = 78 239 3652, #gen drafts = 3653, #acc drafts = 2652, #gen tokens = 7287, #acc tokens = 4946, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.047, 853.629, 4.470 ms
1.12.855.386 I slot release: id 14 | task 266 | stop processing: n_tokens = 140, truncated = 0
1.12.863.581 I srv operator(): Chat format: peg-native
1.12.971.002 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.12.971.078 I slot launch_slot_: id 14 | task 320 | processing task, is_child = 0
1.12.976.143 I slot operator(): id 14 | task 320 | Checking checkpoint with [20, 20] against 24...
1.12.980.027 W slot operator(): id 14 | task 320 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.12.980.059 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.091.418 I slot print_timing: id 4 | task 231 | prompt eval time = 121.57 ms / 4 tokens ( 30.39 ms per token, 32.90 tokens per second)
1.13.091.420 I slot print_timing: id 4 | task 231 | eval time = 8342.71 ms / 128 tokens ( 65.18 ms per token, 15.34 tokens per second)
1.13.091.421 I slot print_timing: id 4 | task 231 | total time = 8464.28 ms / 132 tokens
1.13.091.422 I slot print_timing: id 4 | task 231 | graphs reused = 1
1.13.091.424 I slot print_timing: id 4 | task 231 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.13.091.438 I statistics draft-mtp: #calls(b,g,a) = 78 241 3668, #gen drafts = 3682, #acc drafts = 2662, #gen tokens = 7344, #acc tokens = 4964, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.047, 862.412, 4.489 ms
1.13.091.464 I slot release: id 4 | task 231 | stop processing: n_tokens = 156, truncated = 0
1.13.095.916 I slot print_timing: id 9 | task 259 | prompt eval time = 118.66 ms / 4 tokens ( 29.66 ms per token, 33.71 tokens per second)
1.13.095.918 I slot print_timing: id 9 | task 259 | eval time = 5805.23 ms / 125 tokens ( 46.44 ms per token, 21.53 tokens per second)
1.13.095.919 I slot print_timing: id 9 | task 259 | total time = 5923.88 ms / 129 tokens
1.13.095.920 I slot print_timing: id 9 | task 259 | graphs reused = 1
1.13.095.923 I slot print_timing: id 9 | task 259 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.13.095.929 I statistics draft-mtp: #calls(b,g,a) = 79 241 3677, #gen drafts = 3682, #acc drafts = 2669, #gen tokens = 7344, #acc tokens = 4977, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.048, 862.412, 4.500 ms
1.13.095.954 I slot release: id 9 | task 259 | stop processing: n_tokens = 152, truncated = 0
1.13.099.851 I srv operator(): Chat format: peg-native
1.13.103.893 I srv operator(): Chat format: peg-native
1.13.208.961 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.13.209.033 I slot launch_slot_: id 4 | task 323 | processing task, is_child = 0
1.13.209.036 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.13.209.054 I slot launch_slot_: id 9 | task 324 | processing task, is_child = 0
1.13.214.684 I slot operator(): id 4 | task 323 | Checking checkpoint with [24, 24] against 28...
1.13.218.532 W slot operator(): id 4 | task 323 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.13.218.562 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.218.581 I slot operator(): id 9 | task 324 | Checking checkpoint with [23, 23] against 27...
1.13.222.063 W slot operator(): id 9 | task 324 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.13.222.096 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.341.036 I slot print_timing: id 11 | task 278 | n_decoded = 102, tg = 24.06 t/s, tg_3s = 24.06 t/s
1.13.581.646 I slot print_timing: id 10 | task 275 | n_decoded = 101, tg = 21.39 t/s, tg_3s = 21.39 t/s
1.13.819.408 I slot print_timing: id 2 | task 258 | n_decoded = 100, tg = 15.32 t/s, tg_3s = 15.32 t/s
1.13.944.320 I slot print_timing: id 11 | task 278 | prompt eval time = 116.01 ms / 4 tokens ( 29.00 ms per token, 34.48 tokens per second)
1.13.944.323 I slot print_timing: id 11 | task 278 | eval time = 4841.89 ms / 114 tokens ( 42.47 ms per token, 23.54 tokens per second)
1.13.944.323 I slot print_timing: id 11 | task 278 | total time = 4957.90 ms / 118 tokens
1.13.944.325 I slot print_timing: id 11 | task 278 | graphs reused = 1
1.13.944.327 I slot print_timing: id 11 | task 278 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.13.944.340 I statistics draft-mtp: #calls(b,g,a) = 81 248 3786, #gen drafts = 3790, #acc drafts = 2752, #gen tokens = 7560, #acc tokens = 5131, #mean acc len = 2.36, #acc rate/pos = (0.727, 0.628), dur(b,g,a) = 0.049, 888.492, 4.632 ms
1.13.944.367 I slot release: id 11 | task 278 | stop processing: n_tokens = 140, truncated = 0
1.13.953.465 I srv operator(): Chat format: peg-native
1.14.061.680 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.14.061.753 I slot launch_slot_: id 11 | task 332 | processing task, is_child = 0
1.14.066.303 I slot operator(): id 11 | task 332 | Checking checkpoint with [20, 20] against 24...
1.14.070.145 W slot operator(): id 11 | task 332 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.14.070.177 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.550.536 I slot print_timing: id 13 | task 267 | n_decoded = 100, tg = 15.30 t/s, tg_3s = 15.30 t/s
1.14.664.629 I slot print_timing: id 1 | task 247 | prompt eval time = 110.52 ms / 4 tokens ( 27.63 ms per token, 36.19 tokens per second)
1.14.664.632 I slot print_timing: id 1 | task 247 | eval time = 8347.36 ms / 128 tokens ( 65.21 ms per token, 15.33 tokens per second)
1.14.664.632 I slot print_timing: id 1 | task 247 | total time = 8457.88 ms / 132 tokens
1.14.664.633 I slot print_timing: id 1 | task 247 | graphs reused = 1
1.14.664.637 I slot print_timing: id 1 | task 247 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.14.664.652 I statistics draft-mtp: #calls(b,g,a) = 82 254 3868, #gen drafts = 3883, #acc drafts = 2805, #gen tokens = 7745, #acc tokens = 5232, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.050, 909.942, 4.730 ms
1.14.664.679 I slot release: id 1 | task 247 | stop processing: n_tokens = 156, truncated = 0
1.14.669.538 I slot print_timing: id 10 | task 275 | prompt eval time = 116.84 ms / 4 tokens ( 29.21 ms per token, 34.24 tokens per second)
1.14.669.540 I slot print_timing: id 10 | task 275 | eval time = 5809.51 ms / 125 tokens ( 46.48 ms per token, 21.52 tokens per second)
1.14.669.540 I slot print_timing: id 10 | task 275 | total time = 5926.35 ms / 129 tokens
1.14.669.540 I slot print_timing: id 10 | task 275 | graphs reused = 1
1.14.669.543 I slot print_timing: id 10 | task 275 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.14.669.550 I statistics draft-mtp: #calls(b,g,a) = 82 254 3878, #gen drafts = 3883, #acc drafts = 2814, #gen tokens = 7745, #acc tokens = 5248, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.050, 909.942, 4.742 ms
1.14.669.575 I slot release: id 10 | task 275 | stop processing: n_tokens = 152, truncated = 0
1.14.673.577 I srv operator(): Chat format: peg-native
1.14.678.022 I srv operator(): Chat format: peg-native
1.14.778.999 I slot print_timing: id 7 | task 293 | n_decoded = 102, tg = 24.15 t/s, tg_3s = 24.15 t/s
1.14.779.634 I slot print_timing: id 8 | task 288 | n_decoded = 101, tg = 21.45 t/s, tg_3s = 21.45 t/s
1.14.782.156 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.14.782.229 I slot launch_slot_: id 1 | task 339 | processing task, is_child = 0
1.14.782.232 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.14.782.250 I slot launch_slot_: id 10 | task 340 | processing task, is_child = 0
1.14.787.771 I slot operator(): id 1 | task 339 | Checking checkpoint with [24, 24] against 28...
1.14.791.631 W slot operator(): id 1 | task 339 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.14.791.666 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.14.791.682 I slot operator(): id 10 | task 340 | Checking checkpoint with [23, 23] against 27...
1.14.795.163 W slot operator(): id 10 | task 340 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.14.795.198 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.394.594 I slot print_timing: id 7 | task 293 | prompt eval time = 117.19 ms / 4 tokens ( 29.30 ms per token, 34.13 tokens per second)
1.15.394.597 I slot print_timing: id 7 | task 293 | eval time = 4838.43 ms / 114 tokens ( 42.44 ms per token, 23.56 tokens per second)
1.15.394.598 I slot print_timing: id 7 | task 293 | total time = 4955.62 ms / 118 tokens
1.15.394.599 I slot print_timing: id 7 | task 293 | graphs reused = 1
1.15.394.602 I slot print_timing: id 7 | task 293 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.15.394.619 I statistics draft-mtp: #calls(b,g,a) = 84 260 3967, #gen drafts = 3975, #acc drafts = 2876, #gen tokens = 7929, #acc tokens = 5365, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.051, 932.888, 4.849 ms
1.15.394.647 I slot release: id 7 | task 293 | stop processing: n_tokens = 140, truncated = 0
1.15.402.715 I srv operator(): Chat format: peg-native
1.15.514.821 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.15.514.891 I slot launch_slot_: id 7 | task 347 | processing task, is_child = 0
1.15.520.449 I slot operator(): id 7 | task 347 | Checking checkpoint with [20, 20] against 24...
1.15.524.247 W slot operator(): id 7 | task 347 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.15.524.283 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.635.323 I slot print_timing: id 2 | task 258 | prompt eval time = 122.30 ms / 4 tokens ( 30.57 ms per token, 32.71 tokens per second)
1.15.635.326 I slot print_timing: id 2 | task 258 | eval time = 8344.86 ms / 128 tokens ( 65.19 ms per token, 15.34 tokens per second)
1.15.635.326 I slot print_timing: id 2 | task 258 | total time = 8467.16 ms / 132 tokens
1.15.635.327 I slot print_timing: id 2 | task 258 | graphs reused = 1
1.15.635.330 I slot print_timing: id 2 | task 258 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.15.635.344 I statistics draft-mtp: #calls(b,g,a) = 84 262 3990, #gen drafts = 4004, #acc drafts = 2897, #gen tokens = 7986, #acc tokens = 5403, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.051, 943.360, 4.881 ms
1.15.635.370 I slot release: id 2 | task 258 | stop processing: n_tokens = 156, truncated = 0
1.15.637.835 I slot print_timing: id 5 | task 298 | n_decoded = 101, tg = 21.41 t/s, tg_3s = 21.41 t/s
1.15.644.425 I srv operator(): Chat format: peg-native
1.15.756.706 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.15.756.775 I slot launch_slot_: id 2 | task 350 | processing task, is_child = 0
1.15.760.705 I slot operator(): id 2 | task 350 | Checking checkpoint with [24, 24] against 28...
1.15.764.557 W slot operator(): id 2 | task 350 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.15.764.589 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.877.926 I slot print_timing: id 0 | task 282 | n_decoded = 100, tg = 15.58 t/s, tg_3s = 15.58 t/s
1.15.880.783 I slot print_timing: id 8 | task 288 | prompt eval time = 116.89 ms / 4 tokens ( 29.22 ms per token, 34.22 tokens per second)
1.15.880.786 I slot print_timing: id 8 | task 288 | eval time = 5810.59 ms / 125 tokens ( 46.48 ms per token, 21.51 tokens per second)
1.15.880.786 I slot print_timing: id 8 | task 288 | total time = 5927.48 ms / 129 tokens
1.15.880.787 I slot print_timing: id 8 | task 288 | graphs reused = 1
1.15.880.790 I slot print_timing: id 8 | task 288 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.15.880.802 I statistics draft-mtp: #calls(b,g,a) = 86 264 4027, #gen drafts = 4034, #acc drafts = 2923, #gen tokens = 8046, #acc tokens = 5452, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.628), dur(b,g,a) = 0.052, 951.146, 4.926 ms
1.15.880.830 I slot release: id 8 | task 288 | stop processing: n_tokens = 152, truncated = 0
1.15.884.107 I slot print_timing: id 15 | task 305 | n_decoded = 102, tg = 24.05 t/s, tg_3s = 24.05 t/s
1.15.890.104 I srv operator(): Chat format: peg-native
1.15.999.427 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.15.999.500 I slot launch_slot_: id 8 | task 353 | processing task, is_child = 0
1.16.004.488 I slot operator(): id 8 | task 353 | Checking checkpoint with [23, 23] against 27...
1.16.008.389 W slot operator(): id 8 | task 353 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.16.008.419 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.363.317 I slot print_timing: id 13 | task 267 | prompt eval time = 121.79 ms / 4 tokens ( 30.45 ms per token, 32.84 tokens per second)
1.16.363.320 I slot print_timing: id 13 | task 267 | eval time = 8350.67 ms / 128 tokens ( 65.24 ms per token, 15.33 tokens per second)
1.16.363.321 I slot print_timing: id 13 | task 267 | total time = 8472.47 ms / 132 tokens
1.16.363.322 I slot print_timing: id 13 | task 267 | graphs reused = 1
1.16.363.325 I slot print_timing: id 13 | task 267 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.16.363.339 I statistics draft-mtp: #calls(b,g,a) = 87 268 4080, #gen drafts = 4095, #acc drafts = 2960, #gen tokens = 8167, #acc tokens = 5522, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.628), dur(b,g,a) = 0.053, 967.835, 4.993 ms
1.16.363.366 I slot release: id 13 | task 267 | stop processing: n_tokens = 156, truncated = 0
1.16.372.232 I srv operator(): Chat format: peg-native
1.16.485.457 I slot print_timing: id 15 | task 305 | prompt eval time = 116.33 ms / 4 tokens ( 29.08 ms per token, 34.39 tokens per second)
1.16.485.460 I slot print_timing: id 15 | task 305 | eval time = 4841.74 ms / 114 tokens ( 42.47 ms per token, 23.55 tokens per second)
1.16.485.460 I slot print_timing: id 15 | task 305 | total time = 4958.07 ms / 118 tokens
1.16.485.461 I slot print_timing: id 15 | task 305 | graphs reused = 1
1.16.485.465 I slot print_timing: id 15 | task 305 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.16.485.476 I statistics draft-mtp: #calls(b,g,a) = 87 269 4110, #gen drafts = 4110, #acc drafts = 2982, #gen tokens = 8197, #acc tokens = 5561, #mean acc len = 2.35, #acc rate/pos = (0.726, 0.627), dur(b,g,a) = 0.053, 971.707, 5.025 ms
1.16.485.504 I slot release: id 15 | task 305 | stop processing: n_tokens = 140, truncated = 0
1.16.485.520 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.16.485.587 I slot launch_slot_: id 15 | task 358 | processing task, is_child = 0
1.16.489.391 I slot operator(): id 15 | task 358 | Checking checkpoint with [20, 20] against 24...
1.16.493.420 W slot operator(): id 15 | task 358 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.16.493.451 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.493.603 I srv operator(): Chat format: peg-native
1.16.605.980 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.16.606.050 I slot launch_slot_: id 13 | task 360 | processing task, is_child = 0
1.16.609.937 I slot operator(): id 13 | task 360 | Checking checkpoint with [24, 24] against 28...
1.16.613.829 W slot operator(): id 13 | task 360 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.16.613.869 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.729.639 I slot print_timing: id 5 | task 298 | prompt eval time = 117.04 ms / 4 tokens ( 29.26 ms per token, 34.18 tokens per second)
1.16.729.642 I slot print_timing: id 5 | task 298 | eval time = 5808.59 ms / 125 tokens ( 46.47 ms per token, 21.52 tokens per second)
1.16.729.642 I slot print_timing: id 5 | task 298 | total time = 5925.63 ms / 129 tokens
1.16.729.643 I slot print_timing: id 5 | task 298 | graphs reused = 1
1.16.729.646 I slot print_timing: id 5 | task 298 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.16.729.661 I statistics draft-mtp: #calls(b,g,a) = 89 271 4130, #gen drafts = 4139, #acc drafts = 2994, #gen tokens = 8255, #acc tokens = 5583, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.055, 979.302, 5.050 ms
1.16.729.692 I slot release: id 5 | task 298 | stop processing: n_tokens = 152, truncated = 0
1.16.738.746 I srv operator(): Chat format: peg-native
1.16.848.644 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.16.848.712 I slot launch_slot_: id 5 | task 363 | processing task, is_child = 0
1.16.853.415 I slot operator(): id 5 | task 363 | Checking checkpoint with [23, 23] against 27...
1.16.857.296 W slot operator(): id 5 | task 363 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.16.857.330 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.215.397 I slot print_timing: id 6 | task 314 | n_decoded = 101, tg = 21.36 t/s, tg_3s = 21.36 t/s
1.17.339.186 I slot print_timing: id 12 | task 296 | n_decoded = 100, tg = 15.28 t/s, tg_3s = 15.28 t/s
1.17.340.286 I slot print_timing: id 14 | task 320 | n_decoded = 102, tg = 24.01 t/s, tg_3s = 24.01 t/s
1.17.812.796 I slot print_timing: id 0 | task 282 | prompt eval time = 116.98 ms / 4 tokens ( 29.25 ms per token, 34.19 tokens per second)
1.17.812.799 I slot print_timing: id 0 | task 282 | eval time = 8351.48 ms / 128 tokens ( 65.25 ms per token, 15.33 tokens per second)
1.17.812.800 I slot print_timing: id 0 | task 282 | total time = 8468.46 ms / 132 tokens
1.17.812.800 I slot print_timing: id 0 | task 282 | graphs reused = 1
1.17.812.803 I slot print_timing: id 0 | task 282 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.485, 0.368)
1.17.812.818 I statistics draft-mtp: #calls(b,g,a) = 90 280 4265, #gen drafts = 4280, #acc drafts = 3092, #gen tokens = 8536, #acc tokens = 5767, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.055, 1009.130, 5.217 ms
1.17.812.848 I slot release: id 0 | task 282 | stop processing: n_tokens = 156, truncated = 0
1.17.821.777 I srv operator(): Chat format: peg-native
1.17.933.055 I slot print_timing: id 14 | task 320 | prompt eval time = 115.56 ms / 4 tokens ( 28.89 ms per token, 34.61 tokens per second)
1.17.933.058 I slot print_timing: id 14 | task 320 | eval time = 4841.33 ms / 114 tokens ( 42.47 ms per token, 23.55 tokens per second)
1.17.933.058 I slot print_timing: id 14 | task 320 | total time = 4956.89 ms / 118 tokens
1.17.933.059 I slot print_timing: id 14 | task 320 | graphs reused = 1
1.17.933.062 I slot print_timing: id 14 | task 320 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.17.933.074 I statistics draft-mtp: #calls(b,g,a) = 90 281 4294, #gen drafts = 4295, #acc drafts = 3114, #gen tokens = 8566, #acc tokens = 5808, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.055, 1012.023, 5.254 ms
1.17.933.106 I slot release: id 14 | task 320 | stop processing: n_tokens = 140, truncated = 0
1.17.933.521 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.17.933.588 I slot launch_slot_: id 14 | task 373 | processing task, is_child = 0
1.17.937.375 I slot operator(): id 14 | task 373 | Checking checkpoint with [20, 20] against 24...
1.17.941.216 W slot operator(): id 14 | task 373 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.17.941.242 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.942.145 I srv operator(): Chat format: peg-native
1.18.051.924 I slot print_timing: id 9 | task 324 | n_decoded = 101, tg = 21.42 t/s, tg_3s = 21.42 t/s
1.18.054.527 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.18.054.592 I slot launch_slot_: id 0 | task 375 | processing task, is_child = 0
1.18.057.560 I slot operator(): id 0 | task 375 | Checking checkpoint with [24, 24] against 28...
1.18.061.396 W slot operator(): id 0 | task 375 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.18.061.436 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.296.690 I slot print_timing: id 6 | task 314 | prompt eval time = 118.77 ms / 4 tokens ( 29.69 ms per token, 33.68 tokens per second)
1.18.296.693 I slot print_timing: id 6 | task 314 | eval time = 5808.94 ms / 125 tokens ( 46.47 ms per token, 21.52 tokens per second)
1.18.296.693 I slot print_timing: id 6 | task 314 | total time = 5927.71 ms / 129 tokens
1.18.296.694 I slot print_timing: id 6 | task 314 | graphs reused = 1
1.18.296.697 I slot print_timing: id 6 | task 314 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.18.296.709 I statistics draft-mtp: #calls(b,g,a) = 92 284 4331, #gen drafts = 4340, #acc drafts = 3141, #gen tokens = 8656, #acc tokens = 5858, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.058, 1021.672, 5.301 ms
1.18.296.736 I slot release: id 6 | task 314 | stop processing: n_tokens = 152, truncated = 0
1.18.305.254 I srv operator(): Chat format: peg-native
1.18.414.049 I slot print_timing: id 11 | task 332 | n_decoded = 102, tg = 24.11 t/s, tg_3s = 24.11 t/s
1.18.415.831 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.18.415.905 I slot launch_slot_: id 6 | task 379 | processing task, is_child = 0
1.18.420.816 I slot operator(): id 6 | task 379 | Checking checkpoint with [23, 23] against 27...
1.18.424.636 W slot operator(): id 6 | task 379 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.18.424.668 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.022.929 I slot print_timing: id 3 | task 313 | n_decoded = 100, tg = 15.30 t/s, tg_3s = 15.30 t/s
1.19.026.798 I slot print_timing: id 11 | task 332 | prompt eval time = 116.93 ms / 4 tokens ( 29.23 ms per token, 34.21 tokens per second)
1.19.026.800 I slot print_timing: id 11 | task 332 | eval time = 4843.55 ms / 114 tokens ( 42.49 ms per token, 23.54 tokens per second)
1.19.026.801 I slot print_timing: id 11 | task 332 | total time = 4960.48 ms / 118 tokens
1.19.026.802 I slot print_timing: id 11 | task 332 | graphs reused = 1
1.19.026.804 I slot print_timing: id 11 | task 332 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.19.026.818 I statistics draft-mtp: #calls(b,g,a) = 93 290 4430, #gen drafts = 4434, #acc drafts = 3211, #gen tokens = 8843, #acc tokens = 5989, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.059, 1042.868, 5.426 ms
1.19.026.848 I slot release: id 11 | task 332 | stop processing: n_tokens = 140, truncated = 0
1.19.035.509 I srv operator(): Chat format: peg-native
1.19.135.006 I slot print_timing: id 12 | task 296 | prompt eval time = 110.34 ms / 4 tokens ( 27.59 ms per token, 36.25 tokens per second)
1.19.135.009 I slot print_timing: id 12 | task 296 | eval time = 8341.33 ms / 128 tokens ( 65.17 ms per token, 15.35 tokens per second)
1.19.135.009 I slot print_timing: id 12 | task 296 | total time = 8451.67 ms / 132 tokens
1.19.135.010 I slot print_timing: id 12 | task 296 | graphs reused = 1
1.19.135.013 I slot print_timing: id 12 | task 296 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.19.135.026 I statistics draft-mtp: #calls(b,g,a) = 93 291 4434, #gen drafts = 4448, #acc drafts = 3214, #gen tokens = 8871, #acc tokens = 5995, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.059, 1046.772, 5.431 ms
1.19.135.055 I slot release: id 12 | task 296 | stop processing: n_tokens = 156, truncated = 0
1.19.139.155 I slot print_timing: id 9 | task 324 | prompt eval time = 118.06 ms / 4 tokens ( 29.52 ms per token, 33.88 tokens per second)
1.19.139.158 I slot print_timing: id 9 | task 324 | eval time = 5802.49 ms / 125 tokens ( 46.42 ms per token, 21.54 tokens per second)
1.19.139.158 I slot print_timing: id 9 | task 324 | total time = 5920.55 ms / 129 tokens
1.19.139.159 I slot print_timing: id 9 | task 324 | graphs reused = 1
1.19.139.161 I slot print_timing: id 9 | task 324 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.19.139.167 I statistics draft-mtp: #calls(b,g,a) = 93 291 4444, #gen drafts = 4448, #acc drafts = 3220, #gen tokens = 8871, #acc tokens = 6005, #mean acc len = 2.35, #acc rate/pos = (0.725, 0.627), dur(b,g,a) = 0.059, 1046.772, 5.444 ms
1.19.139.191 I slot release: id 9 | task 324 | stop processing: n_tokens = 152, truncated = 0
1.19.141.153 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.19.141.220 I slot launch_slot_: id 11 | task 386 | processing task, is_child = 0
1.19.144.608 I srv operator(): Chat format: peg-native
1.19.145.966 I slot operator(): id 11 | task 386 | Checking checkpoint with [20, 20] against 24...
1.19.148.014 I srv operator(): Chat format: peg-native
1.19.149.745 W slot operator(): id 11 | task 386 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.19.149.783 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.255.531 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.19.255.600 I slot launch_slot_: id 12 | task 388 | processing task, is_child = 0
1.19.255.604 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.19.255.624 I slot launch_slot_: id 9 | task 389 | processing task, is_child = 0
1.19.260.524 I slot operator(): id 9 | task 389 | Checking checkpoint with [23, 23] against 27...
1.19.264.386 W slot operator(): id 9 | task 389 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.19.264.417 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.264.435 I slot operator(): id 12 | task 388 | Checking checkpoint with [24, 24] against 28...
1.19.267.957 W slot operator(): id 12 | task 388 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.19.267.997 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.628.852 I slot print_timing: id 10 | task 340 | n_decoded = 101, tg = 21.40 t/s, tg_3s = 21.40 t/s
1.19.867.103 I slot print_timing: id 4 | task 323 | n_decoded = 100, tg = 15.31 t/s, tg_3s = 15.31 t/s
1.19.868.732 I slot print_timing: id 7 | task 347 | n_decoded = 102, tg = 24.10 t/s, tg_3s = 24.10 t/s
1.20.469.822 I slot print_timing: id 7 | task 347 | prompt eval time = 115.16 ms / 4 tokens ( 28.79 ms per token, 34.74 tokens per second)
1.20.469.826 I slot print_timing: id 7 | task 347 | eval time = 4834.19 ms / 114 tokens ( 42.41 ms per token, 23.58 tokens per second)
1.20.469.826 I slot print_timing: id 7 | task 347 | total time = 4949.35 ms / 118 tokens
1.20.469.827 I slot print_timing: id 7 | task 347 | graphs reused = 1
1.20.469.830 I slot print_timing: id 7 | task 347 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.20.469.844 I statistics draft-mtp: #calls(b,g,a) = 96 302 4611, #gen drafts = 4619, #acc drafts = 3337, #gen tokens = 9213, #acc tokens = 6223, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.059, 1083.244, 5.647 ms
1.20.469.871 I slot release: id 7 | task 347 | stop processing: n_tokens = 140, truncated = 0
1.20.478.784 I srv operator(): Chat format: peg-native
1.20.591.066 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.20.591.141 I slot launch_slot_: id 7 | task 401 | processing task, is_child = 0
1.20.596.089 I slot operator(): id 7 | task 401 | Checking checkpoint with [20, 20] against 24...
1.20.599.871 W slot operator(): id 7 | task 401 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.20.599.905 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.717.172 I slot print_timing: id 10 | task 340 | prompt eval time = 118.12 ms / 4 tokens ( 29.53 ms per token, 33.86 tokens per second)
1.20.717.175 I slot print_timing: id 10 | task 340 | eval time = 5807.34 ms / 125 tokens ( 46.46 ms per token, 21.52 tokens per second)
1.20.717.175 I slot print_timing: id 10 | task 340 | total time = 5925.45 ms / 129 tokens
1.20.717.176 I slot print_timing: id 10 | task 340 | graphs reused = 1
1.20.717.179 I slot print_timing: id 10 | task 340 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.20.717.192 I statistics draft-mtp: #calls(b,g,a) = 97 304 4644, #gen drafts = 4649, #acc drafts = 3364, #gen tokens = 9272, #acc tokens = 6275, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.627), dur(b,g,a) = 0.060, 1093.062, 5.692 ms
1.20.717.222 I slot release: id 10 | task 340 | stop processing: n_tokens = 152, truncated = 0
1.20.725.524 I srv operator(): Chat format: peg-native
1.20.827.907 I slot print_timing: id 3 | task 313 | prompt eval time = 122.36 ms / 4 tokens ( 30.59 ms per token, 32.69 tokens per second)
1.20.827.910 I slot print_timing: id 3 | task 313 | eval time = 8340.46 ms / 128 tokens ( 65.16 ms per token, 15.35 tokens per second)
1.20.827.910 I slot print_timing: id 3 | task 313 | total time = 8462.82 ms / 132 tokens
1.20.827.911 I slot print_timing: id 3 | task 313 | graphs reused = 1
1.20.827.913 I slot print_timing: id 3 | task 313 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.20.827.927 I statistics draft-mtp: #calls(b,g,a) = 97 305 4649, #gen drafts = 4663, #acc drafts = 3368, #gen tokens = 9300, #acc tokens = 6281, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.627), dur(b,g,a) = 0.060, 1098.424, 5.697 ms
1.20.827.957 I slot release: id 3 | task 313 | stop processing: n_tokens = 156, truncated = 0
1.20.831.795 I slot print_timing: id 8 | task 353 | n_decoded = 101, tg = 21.44 t/s, tg_3s = 21.44 t/s
1.20.834.730 I slot print_timing: id 15 | task 358 | n_decoded = 102, tg = 24.09 t/s, tg_3s = 24.09 t/s
1.20.834.743 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.20.834.812 I slot launch_slot_: id 3 | task 404 | processing task, is_child = 0
1.20.836.316 I srv operator(): Chat format: peg-native
1.20.840.210 I slot operator(): id 3 | task 404 | Checking checkpoint with [24, 24] against 28...
1.20.844.058 W slot operator(): id 3 | task 404 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.20.844.096 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.956.585 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.20.956.653 I slot launch_slot_: id 10 | task 406 | processing task, is_child = 0
1.20.960.902 I slot operator(): id 10 | task 406 | Checking checkpoint with [23, 23] against 27...
1.20.964.737 W slot operator(): id 10 | task 406 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.20.964.810 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.439.548 I slot print_timing: id 1 | task 339 | n_decoded = 100, tg = 15.31 t/s, tg_3s = 15.31 t/s
1.21.445.773 I slot print_timing: id 15 | task 358 | prompt eval time = 110.60 ms / 4 tokens ( 27.65 ms per token, 36.17 tokens per second)
1.21.445.775 I slot print_timing: id 15 | task 358 | eval time = 4845.76 ms / 114 tokens ( 42.51 ms per token, 23.53 tokens per second)
1.21.445.776 I slot print_timing: id 15 | task 358 | total time = 4956.36 ms / 118 tokens
1.21.445.777 I slot print_timing: id 15 | task 358 | graphs reused = 1
1.21.445.780 I slot print_timing: id 15 | task 358 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.21.445.792 I statistics draft-mtp: #calls(b,g,a) = 99 310 4740, #gen drafts = 4740, #acc drafts = 3432, #gen tokens = 9454, #acc tokens = 6401, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.062, 1116.992, 5.804 ms
1.21.445.818 I slot release: id 15 | task 358 | stop processing: n_tokens = 140, truncated = 0
1.21.454.519 I srv operator(): Chat format: peg-native
1.21.559.339 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.21.559.409 I slot launch_slot_: id 15 | task 412 | processing task, is_child = 0
1.21.563.631 I slot operator(): id 15 | task 412 | Checking checkpoint with [20, 20] against 24...
1.21.567.485 W slot operator(): id 15 | task 412 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.21.567.527 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.678.350 I slot print_timing: id 4 | task 323 | prompt eval time = 121.71 ms / 4 tokens ( 30.43 ms per token, 32.86 tokens per second)
1.21.678.353 I slot print_timing: id 4 | task 323 | eval time = 8341.92 ms / 128 tokens ( 65.17 ms per token, 15.34 tokens per second)
1.21.678.353 I slot print_timing: id 4 | task 323 | total time = 8463.64 ms / 132 tokens
1.21.678.354 I slot print_timing: id 4 | task 323 | graphs reused = 1
1.21.678.357 I slot print_timing: id 4 | task 323 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.21.678.372 I statistics draft-mtp: #calls(b,g,a) = 99 312 4755, #gen drafts = 4769, #acc drafts = 3443, #gen tokens = 9511, #acc tokens = 6420, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.062, 1124.086, 5.818 ms
1.21.678.402 I slot release: id 4 | task 323 | stop processing: n_tokens = 156, truncated = 0
1.21.680.345 I slot print_timing: id 5 | task 363 | n_decoded = 101, tg = 21.44 t/s, tg_3s = 21.44 t/s
1.21.688.095 I srv operator(): Chat format: peg-native
1.21.800.620 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.21.800.692 I slot launch_slot_: id 4 | task 415 | processing task, is_child = 0
1.21.805.552 I slot operator(): id 4 | task 415 | Checking checkpoint with [24, 24] against 28...
1.21.809.404 W slot operator(): id 4 | task 415 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.21.809.439 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.925.157 I slot print_timing: id 8 | task 353 | prompt eval time = 117.13 ms / 4 tokens ( 29.28 ms per token, 34.15 tokens per second)
1.21.925.160 I slot print_timing: id 8 | task 353 | eval time = 5803.52 ms / 125 tokens ( 46.43 ms per token, 21.54 tokens per second)
1.21.925.161 I slot print_timing: id 8 | task 353 | total time = 5920.65 ms / 129 tokens
1.21.925.162 I slot print_timing: id 8 | task 353 | graphs reused = 1
1.21.925.164 I slot print_timing: id 8 | task 353 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.21.925.177 I statistics draft-mtp: #calls(b,g,a) = 101 314 4792, #gen drafts = 4799, #acc drafts = 3467, #gen tokens = 9571, #acc tokens = 6467, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.064, 1133.197, 5.863 ms
1.21.925.204 I slot release: id 8 | task 353 | stop processing: n_tokens = 152, truncated = 0
1.21.934.762 I srv operator(): Chat format: peg-native
1.22.044.725 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.22.044.794 I slot launch_slot_: id 8 | task 418 | processing task, is_child = 0
1.22.049.785 I slot operator(): id 8 | task 418 | Checking checkpoint with [23, 23] against 27...
1.22.053.695 W slot operator(): id 8 | task 418 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.22.053.724 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.294.078 I slot print_timing: id 14 | task 373 | n_decoded = 102, tg = 24.02 t/s, tg_3s = 24.02 t/s
1.22.409.189 I slot print_timing: id 2 | task 350 | n_decoded = 100, tg = 15.31 t/s, tg_3s = 15.31 t/s
1.22.772.213 I slot print_timing: id 5 | task 363 | prompt eval time = 117.07 ms / 4 tokens ( 29.27 ms per token, 34.17 tokens per second)
1.22.772.216 I slot print_timing: id 5 | task 363 | eval time = 5801.70 ms / 125 tokens ( 46.41 ms per token, 21.55 tokens per second)
1.22.772.216 I slot print_timing: id 5 | task 363 | total time = 5918.77 ms / 129 tokens
1.22.772.218 I slot print_timing: id 5 | task 363 | graphs reused = 1
1.22.772.220 I slot print_timing: id 5 | task 363 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.22.772.232 I statistics draft-mtp: #calls(b,g,a) = 102 321 4899, #gen drafts = 4909, #acc drafts = 3546, #gen tokens = 9791, #acc tokens = 6613, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.065, 1157.979, 5.997 ms
1.22.772.260 I slot release: id 5 | task 363 | stop processing: n_tokens = 152, truncated = 0
1.22.780.842 I srv operator(): Chat format: peg-native
1.22.891.578 I slot print_timing: id 14 | task 373 | prompt eval time = 110.42 ms / 4 tokens ( 27.60 ms per token, 36.23 tokens per second)
1.22.891.581 I slot print_timing: id 14 | task 373 | eval time = 4843.76 ms / 114 tokens ( 42.49 ms per token, 23.54 tokens per second)
1.22.891.581 I slot print_timing: id 14 | task 373 | total time = 4954.18 ms / 118 tokens
1.22.891.582 I slot print_timing: id 14 | task 373 | graphs reused = 1
1.22.891.585 I slot print_timing: id 14 | task 373 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.22.891.597 I statistics draft-mtp: #calls(b,g,a) = 102 322 4923, #gen drafts = 4924, #acc drafts = 3563, #gen tokens = 9821, #acc tokens = 6644, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.065, 1162.242, 6.025 ms
1.22.891.622 I slot release: id 14 | task 373 | stop processing: n_tokens = 140, truncated = 0
1.22.892.212 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.22.892.278 I slot launch_slot_: id 14 | task 426 | processing task, is_child = 0
1.22.898.320 I slot operator(): id 14 | task 426 | Checking checkpoint with [20, 20] against 24...
1.22.899.646 I srv operator(): Chat format: peg-native
1.22.902.201 W slot operator(): id 14 | task 426 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.22.902.241 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.015.193 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.020
1.23.015.259 I slot launch_slot_: id 5 | task 428 | processing task, is_child = 0
1.23.019.573 I slot operator(): id 5 | task 428 | Checking checkpoint with [23, 23] against 3...
1.23.019.576 W slot operator(): id 5 | task 428 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.23.019.579 W slot operator(): id 5 | task 428 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.23.021.758 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.171.562 I slot create_check: id 5 | task 428 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.23.282.951 I slot print_timing: id 1 | task 339 | prompt eval time = 121.79 ms / 4 tokens ( 30.45 ms per token, 32.84 tokens per second)
1.23.282.954 I slot print_timing: id 1 | task 339 | eval time = 8373.36 ms / 128 tokens ( 65.42 ms per token, 15.29 tokens per second)
1.23.282.955 I slot print_timing: id 1 | task 339 | total time = 8495.15 ms / 132 tokens
1.23.282.955 I slot print_timing: id 1 | task 339 | graphs reused = 1
1.23.282.958 I slot print_timing: id 1 | task 339 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.23.282.973 I statistics draft-mtp: #calls(b,g,a) = 103 325 4953, #gen drafts = 4967, #acc drafts = 3582, #gen tokens = 9906, #acc tokens = 6681, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.066, 1177.300, 6.059 ms
1.23.282.997 I slot release: id 1 | task 339 | stop processing: n_tokens = 156, truncated = 0
1.23.285.591 I slot print_timing: id 6 | task 379 | n_decoded = 101, tg = 21.27 t/s, tg_3s = 21.27 t/s
1.23.289.302 I slot print_timing: id 13 | task 360 | n_decoded = 100, tg = 15.24 t/s, tg_3s = 15.24 t/s
1.23.292.595 I srv operator(): Chat format: peg-native
1.23.405.037 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
1.23.405.120 I slot launch_slot_: id 1 | task 432 | processing task, is_child = 0
1.23.408.955 I slot operator(): id 1 | task 432 | Checking checkpoint with [24, 24] against 3...
1.23.408.958 W slot operator(): id 1 | task 432 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.23.408.960 W slot operator(): id 1 | task 432 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.23.410.914 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.538.985 I slot print_timing: id 11 | task 386 | n_decoded = 102, tg = 23.78 t/s, tg_3s = 23.78 t/s
1.23.558.260 I slot create_check: id 1 | task 432 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.24.157.523 I slot print_timing: id 9 | task 389 | n_decoded = 101, tg = 21.15 t/s, tg_3s = 21.15 t/s
1.24.158.460 I slot print_timing: id 11 | task 386 | prompt eval time = 103.42 ms / 4 tokens ( 25.85 ms per token, 38.68 tokens per second)
1.24.158.462 I slot print_timing: id 11 | task 386 | eval time = 4909.07 ms / 114 tokens ( 43.06 ms per token, 23.22 tokens per second)
1.24.158.463 I slot print_timing: id 11 | task 386 | total time = 5012.48 ms / 118 tokens
1.24.158.463 I slot print_timing: id 11 | task 386 | graphs reused = 1
1.24.158.466 I slot print_timing: id 11 | task 386 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.24.158.480 I statistics draft-mtp: #calls(b,g,a) = 105 332 5072, #gen drafts = 5076, #acc drafts = 3669, #gen tokens = 10123, #acc tokens = 6843, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.068, 1200.606, 6.211 ms
1.24.158.507 I slot release: id 11 | task 386 | stop processing: n_tokens = 140, truncated = 0
1.24.167.298 I srv operator(): Chat format: peg-native
1.24.269.038 I slot print_timing: id 2 | task 350 | prompt eval time = 116.59 ms / 4 tokens ( 29.15 ms per token, 34.31 tokens per second)
1.24.269.041 I slot print_timing: id 2 | task 350 | eval time = 8391.72 ms / 128 tokens ( 65.56 ms per token, 15.25 tokens per second)
1.24.269.041 I slot print_timing: id 2 | task 350 | total time = 8508.31 ms / 132 tokens
1.24.269.042 I slot print_timing: id 2 | task 350 | graphs reused = 1
1.24.269.045 I slot print_timing: id 2 | task 350 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.24.269.056 I statistics draft-mtp: #calls(b,g,a) = 105 333 5076, #gen drafts = 5090, #acc drafts = 3673, #gen tokens = 10151, #acc tokens = 6851, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.068, 1206.127, 6.216 ms
1.24.269.083 I slot release: id 2 | task 350 | stop processing: n_tokens = 156, truncated = 0
1.24.275.601 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.24.275.670 I slot launch_slot_: id 11 | task 440 | processing task, is_child = 0
1.24.278.013 I srv operator(): Chat format: peg-native
1.24.281.273 I slot operator(): id 11 | task 440 | Checking checkpoint with [20, 20] against 24...
1.24.285.050 W slot operator(): id 11 | task 440 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.24.285.080 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.394.149 I slot print_timing: id 6 | task 379 | prompt eval time = 117.11 ms / 4 tokens ( 29.28 ms per token, 34.16 tokens per second)
1.24.394.151 I slot print_timing: id 6 | task 379 | eval time = 5856.19 ms / 125 tokens ( 46.85 ms per token, 21.34 tokens per second)
1.24.394.151 I slot print_timing: id 6 | task 379 | total time = 5973.29 ms / 129 tokens
1.24.394.152 I slot print_timing: id 6 | task 379 | graphs reused = 1
1.24.394.155 I slot print_timing: id 6 | task 379 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.24.394.169 I statistics draft-mtp: #calls(b,g,a) = 106 334 5096, #gen drafts = 5104, #acc drafts = 3688, #gen tokens = 10179, #acc tokens = 6878, #mean acc len = 2.35, #acc rate/pos = (0.724, 0.626), dur(b,g,a) = 0.069, 1211.689, 6.245 ms
1.24.394.201 I slot release: id 6 | task 379 | stop processing: n_tokens = 152, truncated = 0
1.24.397.575 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.24.397.643 I slot launch_slot_: id 2 | task 442 | processing task, is_child = 0
1.24.402.543 I slot operator(): id 2 | task 442 | Checking checkpoint with [24, 24] against 28...
1.24.402.630 I srv operator(): Chat format: peg-native
1.24.406.344 W slot operator(): id 2 | task 442 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.24.406.373 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.519.514 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.24.519.582 I slot launch_slot_: id 6 | task 444 | processing task, is_child = 0
1.24.524.053 I slot operator(): id 6 | task 444 | Checking checkpoint with [23, 23] against 27...
1.24.527.923 W slot operator(): id 6 | task 444 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.24.527.955 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.761.906 I slot print_timing: id 0 | task 375 | n_decoded = 100, tg = 15.18 t/s, tg_3s = 15.18 t/s
1.25.005.957 I slot print_timing: id 7 | task 401 | n_decoded = 102, tg = 23.76 t/s, tg_3s = 23.76 t/s
1.25.123.445 I slot print_timing: id 13 | task 360 | prompt eval time = 117.04 ms / 4 tokens ( 29.26 ms per token, 34.18 tokens per second)
1.25.123.448 I slot print_timing: id 13 | task 360 | eval time = 8396.44 ms / 128 tokens ( 65.60 ms per token, 15.24 tokens per second)
1.25.123.449 I slot print_timing: id 13 | task 360 | total time = 8513.48 ms / 132 tokens
1.25.123.449 I slot print_timing: id 13 | task 360 | graphs reused = 1
1.25.123.452 I slot print_timing: id 13 | task 360 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.25.123.465 I statistics draft-mtp: #calls(b,g,a) = 108 340 5181, #gen drafts = 5196, #acc drafts = 3748, #gen tokens = 10362, #acc tokens = 6991, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.070, 1233.712, 6.342 ms
1.25.123.491 I slot release: id 13 | task 360 | stop processing: n_tokens = 156, truncated = 0
1.25.132.542 I srv operator(): Chat format: peg-native
1.25.243.305 I slot print_timing: id 9 | task 389 | prompt eval time = 122.15 ms / 4 tokens ( 30.54 ms per token, 32.75 tokens per second)
1.25.243.307 I slot print_timing: id 9 | task 389 | eval time = 5860.59 ms / 125 tokens ( 46.88 ms per token, 21.33 tokens per second)
1.25.243.308 I slot print_timing: id 9 | task 389 | total time = 5982.74 ms / 129 tokens
1.25.243.309 I slot print_timing: id 9 | task 389 | graphs reused = 1
1.25.243.312 I slot print_timing: id 9 | task 389 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.25.243.324 I statistics draft-mtp: #calls(b,g,a) = 108 341 5206, #gen drafts = 5211, #acc drafts = 3765, #gen tokens = 10392, #acc tokens = 7021, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.070, 1237.597, 6.375 ms
1.25.243.356 I slot release: id 9 | task 389 | stop processing: n_tokens = 152, truncated = 0
1.25.245.841 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.020
1.25.245.912 I slot launch_slot_: id 9 | task 451 | processing task, is_child = 0
1.25.250.836 I slot operator(): id 9 | task 451 | Checking checkpoint with [23, 23] against 3...
1.25.250.839 W slot operator(): id 9 | task 451 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.25.250.842 W slot operator(): id 9 | task 451 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.25.251.424 I srv operator(): Chat format: peg-native
1.25.252.423 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.375.482 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.25.375.549 I slot launch_slot_: id 13 | task 453 | processing task, is_child = 0
1.25.394.689 I slot create_check: id 9 | task 451 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.25.394.696 I slot operator(): id 13 | task 453 | Checking checkpoint with [24, 24] against 28...
1.25.398.192 W slot operator(): id 13 | task 453 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.25.398.229 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.635.568 I slot print_timing: id 7 | task 401 | prompt eval time = 116.36 ms / 4 tokens ( 29.09 ms per token, 34.37 tokens per second)
1.25.635.571 I slot print_timing: id 7 | task 401 | eval time = 4923.09 ms / 114 tokens ( 43.19 ms per token, 23.16 tokens per second)
1.25.635.571 I slot print_timing: id 7 | task 401 | total time = 5039.46 ms / 118 tokens
1.25.635.572 I slot print_timing: id 7 | task 401 | graphs reused = 1
1.25.635.575 I slot print_timing: id 7 | task 401 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.25.635.591 I statistics draft-mtp: #calls(b,g,a) = 110 344 5247, #gen drafts = 5255, #acc drafts = 3793, #gen tokens = 10480, #acc tokens = 7075, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.071, 1250.381, 6.429 ms
1.25.635.621 I slot release: id 7 | task 401 | stop processing: n_tokens = 140, truncated = 0
1.25.643.978 I srv operator(): Chat format: peg-native
1.25.756.135 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.25.756.202 I slot launch_slot_: id 7 | task 457 | processing task, is_child = 0
1.25.761.265 I slot operator(): id 7 | task 457 | Checking checkpoint with [20, 20] against 3...
1.25.761.268 W slot operator(): id 7 | task 457 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.25.761.271 W slot operator(): id 7 | task 457 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.25.762.858 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.890.607 I slot print_timing: id 10 | task 406 | n_decoded = 101, tg = 20.99 t/s, tg_3s = 20.99 t/s
1.25.911.505 I slot create_check: id 7 | task 457 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.26.030.176 I slot print_timing: id 12 | task 388 | n_decoded = 100, tg = 15.04 t/s, tg_3s = 15.04 t/s
1.26.031.572 I slot print_timing: id 15 | task 412 | n_decoded = 102, tg = 23.43 t/s, tg_3s = 23.43 t/s
1.26.625.262 I slot print_timing: id 0 | task 375 | prompt eval time = 116.13 ms / 4 tokens ( 29.03 ms per token, 34.44 tokens per second)
1.26.625.269 I slot print_timing: id 0 | task 375 | eval time = 8451.54 ms / 128 tokens ( 66.03 ms per token, 15.15 tokens per second)
1.26.625.269 I slot print_timing: id 0 | task 375 | total time = 8567.67 ms / 132 tokens
1.26.625.270 I slot print_timing: id 0 | task 375 | graphs reused = 1
1.26.625.273 I slot print_timing: id 0 | task 375 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.26.625.288 I statistics draft-mtp: #calls(b,g,a) = 111 352 5364, #gen drafts = 5379, #acc drafts = 3878, #gen tokens = 10727, #acc tokens = 7232, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.072, 1280.185, 6.571 ms
1.26.625.314 I slot release: id 0 | task 375 | stop processing: n_tokens = 156, truncated = 0
1.26.632.041 I slot print_timing: id 15 | task 412 | prompt eval time = 115.02 ms / 4 tokens ( 28.75 ms per token, 34.78 tokens per second)
1.26.632.044 I slot print_timing: id 15 | task 412 | eval time = 4953.38 ms / 114 tokens ( 43.45 ms per token, 23.01 tokens per second)
1.26.632.044 I slot print_timing: id 15 | task 412 | total time = 5068.39 ms / 118 tokens
1.26.632.045 I slot print_timing: id 15 | task 412 | graphs reused = 1
1.26.632.047 I slot print_timing: id 15 | task 412 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.26.632.055 I statistics draft-mtp: #calls(b,g,a) = 111 352 5379, #gen drafts = 5379, #acc drafts = 3888, #gen tokens = 10727, #acc tokens = 7252, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.072, 1280.185, 6.589 ms
1.26.632.081 I slot release: id 15 | task 412 | stop processing: n_tokens = 140, truncated = 0
1.26.634.544 I srv operator(): Chat format: peg-native
1.26.641.132 I srv operator(): Chat format: peg-native
1.26.739.118 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.26.739.188 I slot launch_slot_: id 15 | task 466 | processing task, is_child = 0
1.26.739.191 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.26.739.210 I slot launch_slot_: id 0 | task 467 | processing task, is_child = 0
1.26.742.056 I slot operator(): id 0 | task 467 | Checking checkpoint with [24, 24] against 28...
1.26.745.932 W slot operator(): id 0 | task 467 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.26.745.966 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.745.991 I slot operator(): id 15 | task 466 | Checking checkpoint with [20, 20] against 24...
1.26.749.521 W slot operator(): id 15 | task 466 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.26.749.553 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.986.681 I slot print_timing: id 8 | task 418 | n_decoded = 101, tg = 20.96 t/s, tg_3s = 20.96 t/s
1.26.987.779 I slot print_timing: id 10 | task 406 | prompt eval time = 116.78 ms / 4 tokens ( 29.19 ms per token, 34.25 tokens per second)
1.26.987.781 I slot print_timing: id 10 | task 406 | eval time = 5910.06 ms / 125 tokens ( 47.28 ms per token, 21.15 tokens per second)
1.26.987.781 I slot print_timing: id 10 | task 406 | total time = 6026.84 ms / 129 tokens
1.26.987.782 I slot print_timing: id 10 | task 406 | graphs reused = 1
1.26.987.786 I slot print_timing: id 10 | task 406 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.26.987.802 I statistics draft-mtp: #calls(b,g,a) = 113 355 5418, #gen drafts = 5423, #acc drafts = 3918, #gen tokens = 10815, #acc tokens = 7307, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.074, 1288.809, 6.634 ms
1.26.987.833 I slot release: id 10 | task 406 | stop processing: n_tokens = 152, truncated = 0
1.26.996.467 I srv operator(): Chat format: peg-native
1.27.105.983 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.27.106.054 I slot launch_slot_: id 10 | task 471 | processing task, is_child = 0
1.27.110.602 I slot operator(): id 10 | task 471 | Checking checkpoint with [23, 23] against 27...
1.27.114.433 W slot operator(): id 10 | task 471 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.27.114.469 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.354.219 I slot print_timing: id 14 | task 426 | n_decoded = 102, tg = 23.48 t/s, tg_3s = 23.48 t/s
1.27.590.953 I slot print_timing: id 3 | task 404 | n_decoded = 100, tg = 15.06 t/s, tg_3s = 15.06 t/s
1.27.830.123 I slot print_timing: id 12 | task 388 | prompt eval time = 118.49 ms / 4 tokens ( 29.62 ms per token, 33.76 tokens per second)
1.27.830.127 I slot print_timing: id 12 | task 388 | eval time = 8447.17 ms / 128 tokens ( 65.99 ms per token, 15.15 tokens per second)
1.27.830.127 I slot print_timing: id 12 | task 388 | total time = 8565.66 ms / 132 tokens
1.27.830.128 I slot print_timing: id 12 | task 388 | graphs reused = 1
1.27.830.131 I slot print_timing: id 12 | task 388 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.27.830.147 I statistics draft-mtp: #calls(b,g,a) = 114 362 5517, #gen drafts = 5532, #acc drafts = 3989, #gen tokens = 11032, #acc tokens = 7439, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.074, 1313.516, 6.753 ms
1.27.830.179 I slot release: id 12 | task 388 | stop processing: n_tokens = 156, truncated = 0
1.27.839.095 I srv operator(): Chat format: peg-native
1.27.951.973 I slot print_timing: id 14 | task 426 | prompt eval time = 111.27 ms / 4 tokens ( 27.82 ms per token, 35.95 tokens per second)
1.27.951.976 I slot print_timing: id 14 | task 426 | eval time = 4942.36 ms / 114 tokens ( 43.35 ms per token, 23.07 tokens per second)
1.27.951.976 I slot print_timing: id 14 | task 426 | total time = 5053.63 ms / 118 tokens
1.27.951.977 I slot print_timing: id 14 | task 426 | graphs reused = 1
1.27.951.980 I slot print_timing: id 14 | task 426 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.27.951.994 I statistics draft-mtp: #calls(b,g,a) = 114 363 5546, #gen drafts = 5547, #acc drafts = 4011, #gen tokens = 11062, #acc tokens = 7479, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.074, 1317.481, 6.787 ms
1.27.952.021 I slot release: id 14 | task 426 | stop processing: n_tokens = 140, truncated = 0
1.27.952.574 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.27.952.643 I slot launch_slot_: id 14 | task 479 | processing task, is_child = 0
1.27.957.198 I slot operator(): id 14 | task 479 | Checking checkpoint with [20, 20] against 24...
1.27.960.395 I srv operator(): Chat format: peg-native
1.27.961.091 W slot operator(): id 14 | task 479 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.27.961.122 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.071.356 I slot print_timing: id 8 | task 418 | prompt eval time = 117.14 ms / 4 tokens ( 29.28 ms per token, 34.15 tokens per second)
1.28.071.359 I slot print_timing: id 8 | task 418 | eval time = 5904.40 ms / 125 tokens ( 47.24 ms per token, 21.17 tokens per second)
1.28.071.360 I slot print_timing: id 8 | task 418 | total time = 6021.54 ms / 129 tokens
1.28.071.361 I slot print_timing: id 8 | task 418 | graphs reused = 1
1.28.071.364 I slot print_timing: id 8 | task 418 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.28.071.375 I statistics draft-mtp: #calls(b,g,a) = 115 364 5556, #gen drafts = 5561, #acc drafts = 4017, #gen tokens = 11090, #acc tokens = 7490, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.074, 1321.993, 6.801 ms
1.28.071.404 I slot release: id 8 | task 418 | stop processing: n_tokens = 152, truncated = 0
1.28.074.140 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.28.074.210 I slot launch_slot_: id 12 | task 481 | processing task, is_child = 0
1.28.079.580 I slot operator(): id 12 | task 481 | Checking checkpoint with [24, 24] against 28...
1.28.079.721 I srv operator(): Chat format: peg-native
1.28.083.365 W slot operator(): id 12 | task 481 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.28.083.401 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.196.708 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.28.196.778 I slot launch_slot_: id 8 | task 483 | processing task, is_child = 0
1.28.201.748 I slot operator(): id 8 | task 483 | Checking checkpoint with [23, 23] against 27...
1.28.205.598 W slot operator(): id 8 | task 483 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.28.205.628 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.440.165 I slot print_timing: id 1 | task 432 | n_decoded = 101, tg = 21.18 t/s, tg_3s = 21.18 t/s
1.28.561.840 I slot print_timing: id 4 | task 415 | n_decoded = 100, tg = 15.06 t/s, tg_3s = 15.06 t/s
1.28.685.894 I slot print_timing: id 11 | task 440 | n_decoded = 102, tg = 23.75 t/s, tg_3s = 23.75 t/s
1.29.288.549 I slot print_timing: id 11 | task 440 | prompt eval time = 110.24 ms / 4 tokens ( 27.56 ms per token, 36.29 tokens per second)
1.29.288.552 I slot print_timing: id 11 | task 440 | eval time = 4897.02 ms / 114 tokens ( 42.96 ms per token, 23.28 tokens per second)
1.29.288.552 I slot print_timing: id 11 | task 440 | total time = 5007.25 ms / 118 tokens
1.29.288.553 I slot print_timing: id 11 | task 440 | graphs reused = 1
1.29.288.556 I slot print_timing: id 11 | task 440 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.29.288.569 I statistics draft-mtp: #calls(b,g,a) = 117 374 5714, #gen drafts = 5718, #acc drafts = 4132, #gen tokens = 11403, #acc tokens = 7706, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.076, 1355.985, 6.973 ms
1.29.288.596 I slot release: id 11 | task 440 | stop processing: n_tokens = 140, truncated = 0
1.29.297.092 I srv operator(): Chat format: peg-native
1.29.399.411 I slot print_timing: id 3 | task 404 | prompt eval time = 110.12 ms / 4 tokens ( 27.53 ms per token, 36.33 tokens per second)
1.29.399.413 I slot print_timing: id 3 | task 404 | eval time = 8449.05 ms / 128 tokens ( 66.01 ms per token, 15.15 tokens per second)
1.29.399.414 I slot print_timing: id 3 | task 404 | total time = 8559.17 ms / 132 tokens
1.29.399.415 I slot print_timing: id 3 | task 404 | graphs reused = 1
1.29.399.418 I slot print_timing: id 3 | task 404 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.29.399.434 I statistics draft-mtp: #calls(b,g,a) = 117 375 5718, #gen drafts = 5732, #acc drafts = 4135, #gen tokens = 11431, #acc tokens = 7712, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.076, 1361.509, 6.977 ms
1.29.399.460 I slot release: id 3 | task 404 | stop processing: n_tokens = 156, truncated = 0
1.29.401.942 I slot print_timing: id 6 | task 444 | n_decoded = 101, tg = 21.21 t/s, tg_3s = 21.21 t/s
1.29.405.831 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.29.405.899 I slot launch_slot_: id 11 | task 494 | processing task, is_child = 0
1.29.408.495 I srv operator(): Chat format: peg-native
1.29.411.496 I slot operator(): id 11 | task 494 | Checking checkpoint with [20, 20] against 24...
1.29.415.319 W slot operator(): id 11 | task 494 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.29.415.354 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.523.209 I slot print_timing: id 1 | task 432 | prompt eval time = 261.96 ms / 28 tokens ( 9.36 ms per token, 106.89 tokens per second)
1.29.523.212 I slot print_timing: id 1 | task 432 | eval time = 5852.25 ms / 125 tokens ( 46.82 ms per token, 21.36 tokens per second)
1.29.523.212 I slot print_timing: id 1 | task 432 | total time = 6114.22 ms / 153 tokens
1.29.523.213 I slot print_timing: id 1 | task 432 | graphs reused = 1
1.29.523.216 I slot print_timing: id 1 | task 432 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.29.523.226 I statistics draft-mtp: #calls(b,g,a) = 118 376 5734, #gen drafts = 5746, #acc drafts = 4147, #gen tokens = 11459, #acc tokens = 7735, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.626), dur(b,g,a) = 0.077, 1367.060, 6.996 ms
1.29.523.254 I slot release: id 1 | task 432 | stop processing: n_tokens = 152, truncated = 0
1.29.528.605 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.29.528.673 I slot launch_slot_: id 3 | task 496 | processing task, is_child = 0
1.29.532.132 I srv operator(): Chat format: peg-native
1.29.533.355 I slot operator(): id 3 | task 496 | Checking checkpoint with [24, 24] against 28...
1.29.537.096 W slot operator(): id 3 | task 496 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.29.537.128 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.650.430 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.29.650.496 I slot launch_slot_: id 1 | task 498 | processing task, is_child = 0
1.29.654.320 I slot operator(): id 1 | task 498 | Checking checkpoint with [23, 23] against 27...
1.29.658.190 W slot operator(): id 1 | task 498 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.29.658.214 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.774.635 I slot print_timing: id 9 | task 451 | n_decoded = 102, tg = 23.93 t/s, tg_3s = 23.93 t/s
1.29.893.178 I slot print_timing: id 5 | task 428 | n_decoded = 100, tg = 15.13 t/s, tg_3s = 15.13 t/s
1.30.372.210 I slot print_timing: id 4 | task 415 | prompt eval time = 116.62 ms / 4 tokens ( 29.15 ms per token, 34.30 tokens per second)
1.30.372.213 I slot print_timing: id 4 | task 415 | eval time = 8450.01 ms / 128 tokens ( 66.02 ms per token, 15.15 tokens per second)
1.30.372.214 I slot print_timing: id 4 | task 415 | total time = 8566.63 ms / 132 tokens
1.30.372.215 I slot print_timing: id 4 | task 415 | graphs reused = 1
1.30.372.218 I slot print_timing: id 4 | task 415 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.30.372.231 I statistics draft-mtp: #calls(b,g,a) = 120 383 5839, #gen drafts = 5854, #acc drafts = 4221, #gen tokens = 11674, #acc tokens = 7870, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.078, 1391.596, 7.120 ms
1.30.372.260 I slot release: id 4 | task 415 | stop processing: n_tokens = 156, truncated = 0
1.30.376.039 I slot print_timing: id 9 | task 451 | prompt eval time = 261.69 ms / 25 tokens ( 10.47 ms per token, 95.53 tokens per second)
1.30.376.041 I slot print_timing: id 9 | task 451 | eval time = 4863.49 ms / 114 tokens ( 42.66 ms per token, 23.44 tokens per second)
1.30.376.042 I slot print_timing: id 9 | task 451 | total time = 5125.19 ms / 139 tokens
1.30.376.042 I slot print_timing: id 9 | task 451 | graphs reused = 1
1.30.376.045 I slot print_timing: id 9 | task 451 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.30.376.053 I statistics draft-mtp: #calls(b,g,a) = 120 383 5848, #gen drafts = 5854, #acc drafts = 4226, #gen tokens = 11674, #acc tokens = 7880, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.078, 1391.596, 7.130 ms
1.30.376.080 I slot release: id 9 | task 451 | stop processing: n_tokens = 140, truncated = 0
1.30.381.330 I srv operator(): Chat format: peg-native
1.30.384.002 I srv operator(): Chat format: peg-native
1.30.485.866 I slot print_timing: id 6 | task 444 | prompt eval time = 117.05 ms / 4 tokens ( 29.26 ms per token, 34.17 tokens per second)
1.30.485.869 I slot print_timing: id 6 | task 444 | eval time = 5844.73 ms / 125 tokens ( 46.76 ms per token, 21.39 tokens per second)
1.30.485.870 I slot print_timing: id 6 | task 444 | total time = 5961.78 ms / 129 tokens
1.30.485.871 I slot print_timing: id 6 | task 444 | graphs reused = 1
1.30.485.874 I slot print_timing: id 6 | task 444 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.30.485.891 I statistics draft-mtp: #calls(b,g,a) = 120 384 5860, #gen drafts = 5868, #acc drafts = 4235, #gen tokens = 11702, #acc tokens = 7898, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.078, 1397.202, 7.144 ms
1.30.485.919 I slot release: id 6 | task 444 | stop processing: n_tokens = 152, truncated = 0
1.30.489.682 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.30.489.754 I slot launch_slot_: id 9 | task 506 | processing task, is_child = 0
1.30.489.757 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.30.489.774 I slot launch_slot_: id 4 | task 507 | processing task, is_child = 0
1.30.494.459 I srv operator(): Chat format: peg-native
1.30.495.862 I slot operator(): id 4 | task 507 | Checking checkpoint with [24, 24] against 28...
1.30.499.635 W slot operator(): id 4 | task 507 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.30.499.667 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.499.694 I slot operator(): id 9 | task 506 | Checking checkpoint with [20, 20] against 24...
1.30.503.185 W slot operator(): id 9 | task 506 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.30.503.222 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.617.681 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.30.617.748 I slot launch_slot_: id 6 | task 509 | processing task, is_child = 0
1.30.622.198 I slot operator(): id 6 | task 509 | Checking checkpoint with [23, 23] against 27...
1.30.626.072 W slot operator(): id 6 | task 509 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.30.626.109 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.742.226 I slot print_timing: id 7 | task 457 | n_decoded = 101, tg = 21.41 t/s, tg_3s = 21.41 t/s
1.31.102.048 I slot print_timing: id 2 | task 442 | n_decoded = 100, tg = 15.18 t/s, tg_3s = 15.18 t/s
1.31.107.747 I slot print_timing: id 15 | task 466 | n_decoded = 102, tg = 24.03 t/s, tg_3s = 24.03 t/s
1.31.704.679 I slot print_timing: id 5 | task 428 | prompt eval time = 263.68 ms / 29 tokens ( 9.09 ms per token, 109.98 tokens per second)
1.31.704.683 I slot print_timing: id 5 | task 428 | eval time = 8421.40 ms / 128 tokens ( 65.79 ms per token, 15.20 tokens per second)
1.31.704.684 I slot print_timing: id 5 | task 428 | total time = 8685.07 ms / 157 tokens
1.31.704.685 I slot print_timing: id 5 | task 428 | graphs reused = 1
1.31.704.689 I slot print_timing: id 5 | task 428 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.31.704.708 I statistics draft-mtp: #calls(b,g,a) = 123 394 6008, #gen drafts = 6023, #acc drafts = 4342, #gen tokens = 12011, #acc tokens = 8097, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.080, 1432.677, 7.324 ms
1.31.704.735 I slot release: id 5 | task 428 | stop processing: n_tokens = 156, truncated = 0
1.31.710.862 I slot print_timing: id 15 | task 466 | prompt eval time = 117.33 ms / 4 tokens ( 29.33 ms per token, 34.09 tokens per second)
1.31.710.864 I slot print_timing: id 15 | task 466 | eval time = 4847.52 ms / 114 tokens ( 42.52 ms per token, 23.52 tokens per second)
1.31.710.865 I slot print_timing: id 15 | task 466 | total time = 4964.85 ms / 118 tokens
1.31.710.865 I slot print_timing: id 15 | task 466 | graphs reused = 1
1.31.710.868 I slot print_timing: id 15 | task 466 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.31.710.875 I statistics draft-mtp: #calls(b,g,a) = 123 394 6023, #gen drafts = 6023, #acc drafts = 4350, #gen tokens = 12011, #acc tokens = 8112, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.080, 1432.677, 7.342 ms
1.31.710.897 I slot release: id 15 | task 466 | stop processing: n_tokens = 140, truncated = 0
1.31.714.145 I srv operator(): Chat format: peg-native
1.31.719.546 I srv operator(): Chat format: peg-native
1.31.816.447 I slot print_timing: id 7 | task 457 | prompt eval time = 263.15 ms / 28 tokens ( 9.40 ms per token, 106.40 tokens per second)
1.31.816.449 I slot print_timing: id 7 | task 457 | eval time = 5791.99 ms / 125 tokens ( 46.34 ms per token, 21.58 tokens per second)
1.31.816.450 I slot print_timing: id 7 | task 457 | total time = 6055.14 ms / 153 tokens
1.31.816.451 I slot print_timing: id 7 | task 457 | graphs reused = 1
1.31.816.454 I slot print_timing: id 7 | task 457 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.31.816.468 I statistics draft-mtp: #calls(b,g,a) = 123 395 6030, #gen drafts = 6037, #acc drafts = 4355, #gen tokens = 12039, #acc tokens = 8122, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.080, 1437.119, 7.350 ms
1.31.816.498 I slot release: id 7 | task 457 | stop processing: n_tokens = 152, truncated = 0
1.31.819.742 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.31.819.815 I slot launch_slot_: id 15 | task 520 | processing task, is_child = 0
1.31.819.822 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.31.819.841 I slot launch_slot_: id 5 | task 521 | processing task, is_child = 0
1.31.825.122 I slot operator(): id 5 | task 521 | Checking checkpoint with [24, 24] against 28...
1.31.825.178 I srv operator(): Chat format: peg-native
1.31.828.955 W slot operator(): id 5 | task 521 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.31.828.990 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.31.829.007 I slot operator(): id 15 | task 520 | Checking checkpoint with [20, 20] against 24...
1.31.832.533 W slot operator(): id 15 | task 520 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.31.832.557 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.31.944.950 I slot print_timing: id 10 | task 471 | n_decoded = 101, tg = 21.41 t/s, tg_3s = 21.41 t/s
1.31.946.877 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.184
1.31.946.945 I slot launch_slot_: id 7 | task 523 | processing task, is_child = 0
1.31.952.017 I slot operator(): id 7 | task 523 | Checking checkpoint with [23, 23] against 27...
1.31.955.927 W slot operator(): id 7 | task 523 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.31.955.958 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.32.076.822 I slot print_timing: id 13 | task 453 | n_decoded = 100, tg = 15.23 t/s, tg_3s = 15.23 t/s
1.32.318.067 I slot print_timing: id 14 | task 479 | n_decoded = 102, tg = 24.00 t/s, tg_3s = 24.00 t/s
1.32.913.591 I slot print_timing: id 2 | task 442 | prompt eval time = 110.50 ms / 4 tokens ( 27.62 ms per token, 36.20 tokens per second)
1.32.913.594 I slot print_timing: id 2 | task 442 | eval time = 8400.52 ms / 128 tokens ( 65.63 ms per token, 15.24 tokens per second)
1.32.913.594 I slot print_timing: id 2 | task 442 | total time = 8511.02 ms / 132 tokens
1.32.913.595 I slot print_timing: id 2 | task 442 | graphs reused = 1
1.32.913.598 I slot print_timing: id 2 | task 442 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.32.913.611 I statistics draft-mtp: #calls(b,g,a) = 126 404 6161, #gen drafts = 6176, #acc drafts = 4451, #gen tokens = 12316, #acc tokens = 8302, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.081, 1469.285, 7.513 ms
1.32.913.641 I slot release: id 2 | task 442 | stop processing: n_tokens = 156, truncated = 0
1.32.920.361 I slot print_timing: id 14 | task 479 | prompt eval time = 110.47 ms / 4 tokens ( 27.62 ms per token, 36.21 tokens per second)
1.32.920.363 I slot print_timing: id 14 | task 479 | eval time = 4852.67 ms / 114 tokens ( 42.57 ms per token, 23.49 tokens per second)
1.32.920.363 I slot print_timing: id 14 | task 479 | total time = 4963.15 ms / 118 tokens
1.32.920.364 I slot print_timing: id 14 | task 479 | graphs reused = 1
1.32.920.366 I slot print_timing: id 14 | task 479 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.32.920.374 I statistics draft-mtp: #calls(b,g,a) = 126 404 6175, #gen drafts = 6176, #acc drafts = 4463, #gen tokens = 12316, #acc tokens = 8324, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.081, 1469.285, 7.532 ms
1.32.920.396 I slot release: id 14 | task 479 | stop processing: n_tokens = 140, truncated = 0
1.32.922.442 I srv operator(): Chat format: peg-native
1.32.928.741 I srv operator(): Chat format: peg-native
1.33.027.944 I slot print_timing: id 8 | task 483 | n_decoded = 101, tg = 21.45 t/s, tg_3s = 21.45 t/s
1.33.029.025 I slot print_timing: id 10 | task 471 | prompt eval time = 116.83 ms / 4 tokens ( 29.21 ms per token, 34.24 tokens per second)
1.33.029.027 I slot print_timing: id 10 | task 471 | eval time = 5801.56 ms / 125 tokens ( 46.41 ms per token, 21.55 tokens per second)
1.33.029.027 I slot print_timing: id 10 | task 471 | total time = 5918.38 ms / 129 tokens
1.33.029.028 I slot print_timing: id 10 | task 471 | graphs reused = 1
1.33.029.031 I slot print_timing: id 10 | task 471 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.33.029.045 I statistics draft-mtp: #calls(b,g,a) = 126 405 6186, #gen drafts = 6190, #acc drafts = 4472, #gen tokens = 12344, #acc tokens = 8340, #mean acc len = 2.35, #acc rate/pos = (0.723, 0.625), dur(b,g,a) = 0.081, 1474.002, 7.546 ms
1.33.029.071 I slot release: id 10 | task 471 | stop processing: n_tokens = 152, truncated = 0
1.33.030.958 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.33.031.024 I slot launch_slot_: id 14 | task 533 | processing task, is_child = 0
1.33.031.026 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.33.031.044 I slot launch_slot_: id 2 | task 534 | processing task, is_child = 0
1.33.037.003 I slot operator(): id 2 | task 534 | Checking checkpoint with [24, 24] against 28...
1.33.040.797 W slot operator(): id 2 | task 534 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.33.040.823 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.33.040.845 I slot operator(): id 14 | task 533 | Checking checkpoint with [20, 20] against 24...
1.33.044.338 W slot operator(): id 14 | task 533 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.33.044.369 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.33.268.394 I slot print_timing: id 0 | task 467 | n_decoded = 100, tg = 15.61 t/s, tg_3s = 15.61 t/s
1.33.732.926 I slot print_timing: id 11 | task 494 | n_decoded = 102, tg = 24.22 t/s, tg_3s = 24.22 t/s
1.33.842.309 I slot print_timing: id 13 | task 453 | prompt eval time = 118.08 ms / 4 tokens ( 29.52 ms per token, 33.87 tokens per second)
1.33.842.312 I slot print_timing: id 13 | task 453 | eval time = 8329.51 ms / 128 tokens ( 65.07 ms per token, 15.37 tokens per second)
1.33.842.312 I slot print_timing: id 13 | task 453 | total time = 8447.59 ms / 132 tokens
1.33.842.313 I slot print_timing: id 13 | task 453 | graphs reused = 1
1.33.842.316 I slot print_timing: id 13 | task 453 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.33.842.330 I statistics draft-mtp: #calls(b,g,a) = 128 412 6278, #gen drafts = 6292, #acc drafts = 4535, #gen tokens = 12547, #acc tokens = 8457, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.082, 1506.152, 7.660 ms
1.33.842.356 I slot release: id 13 | task 453 | stop processing: n_tokens = 156, truncated = 0
1.34.064.505 I slot print_timing: id 8 | task 483 | prompt eval time = 116.81 ms / 4 tokens ( 29.20 ms per token, 34.24 tokens per second)
1.34.064.509 I slot print_timing: id 8 | task 483 | eval time = 5745.90 ms / 125 tokens ( 45.97 ms per token, 21.75 tokens per second)
1.34.064.509 I slot print_timing: id 8 | task 483 | total time = 5862.72 ms / 129 tokens
1.34.064.510 I slot print_timing: id 8 | task 483 | graphs reused = 1
1.34.064.513 I slot print_timing: id 8 | task 483 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.34.064.524 I statistics draft-mtp: #calls(b,g,a) = 128 414 6315, #gen drafts = 6320, #acc drafts = 4557, #gen tokens = 12603, #acc tokens = 8499, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.082, 1515.523, 7.705 ms
1.34.064.550 I slot release: id 8 | task 483 | stop processing: n_tokens = 152, truncated = 0
1.34.272.961 I slot print_timing: id 11 | task 494 | prompt eval time = 110.55 ms / 4 tokens ( 27.64 ms per token, 36.18 tokens per second)
1.34.272.964 I slot print_timing: id 11 | task 494 | eval time = 4750.89 ms / 114 tokens ( 41.67 ms per token, 24.00 tokens per second)
1.34.272.965 I slot print_timing: id 11 | task 494 | total time = 4861.45 ms / 118 tokens
1.34.272.966 I slot print_timing: id 11 | task 494 | graphs reused = 1
1.34.272.969 I slot print_timing: id 11 | task 494 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.34.272.983 I statistics draft-mtp: #calls(b,g,a) = 128 416 6343, #gen drafts = 6346, #acc drafts = 4581, #gen tokens = 12655, #acc tokens = 8545, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.625), dur(b,g,a) = 0.082, 1527.250, 7.739 ms
1.34.273.004 I slot release: id 11 | task 494 | stop processing: n_tokens = 140, truncated = 0
1.34.365.626 I slot print_timing: id 1 | task 498 | n_decoded = 101, tg = 21.98 t/s, tg_3s = 21.98 t/s
1.34.559.814 I slot print_timing: id 12 | task 481 | n_decoded = 100, tg = 15.70 t/s, tg_3s = 15.70 t/s
1.34.654.009 I slot print_timing: id 9 | task 506 | n_decoded = 102, tg = 25.23 t/s, tg_3s = 25.23 t/s
1.34.838.248 I slot print_timing: id 0 | task 467 | prompt eval time = 121.02 ms / 4 tokens ( 30.25 ms per token, 33.05 tokens per second)
1.34.838.250 I slot print_timing: id 0 | task 467 | eval time = 7975.15 ms / 128 tokens ( 62.31 ms per token, 16.05 tokens per second)
1.34.838.251 I slot print_timing: id 0 | task 467 | total time = 8096.17 ms / 132 tokens
1.34.838.252 I slot print_timing: id 0 | task 467 | graphs reused = 1
1.34.838.254 I slot print_timing: id 0 | task 467 | draft acceptance = 0.44361 ( 59 accepted / 133 generated), mean acceptance length = 1.88, acceptance rate per position = (0.493, 0.388)
1.34.838.266 I statistics draft-mtp: #calls(b,g,a) = 128 422 6406, #gen drafts = 6417, #acc drafts = 4623, #gen tokens = 12796, #acc tokens = 8623, #mean acc len = 2.35, #acc rate/pos = (0.722, 0.624), dur(b,g,a) = 0.082, 1561.035, 7.805 ms
1.34.838.289 I slot release: id 0 | task 467 | stop processing: n_tokens = 156, truncated = 0
1.35.105.499 I slot print_timing: id 6 | task 509 | n_decoded = 101, tg = 23.13 t/s, tg_3s = 23.13 t/s
1.35.106.296 I slot print_timing: id 9 | task 506 | prompt eval time = 112.03 ms / 4 tokens ( 28.01 ms per token, 35.71 tokens per second)
1.35.106.298 I slot print_timing: id 9 | task 506 | eval time = 4494.56 ms / 114 tokens ( 39.43 ms per token, 25.36 tokens per second)
1.35.106.298 I slot print_timing: id 9 | task 506 | total time = 4606.59 ms / 118 tokens
1.35.106.299 I slot print_timing: id 9 | task 506 | graphs reused = 1
1.35.106.303 I slot print_timing: id 9 | task 506 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.35.106.317 I statistics draft-mtp: #calls(b,g,a) = 128 425 6447, #gen drafts = 6450, #acc drafts = 4650, #gen tokens = 12862, #acc tokens = 8672, #mean acc len = 2.35, #acc rate/pos = (0.721, 0.624), dur(b,g,a) = 0.082, 1577.364, 7.859 ms
1.35.106.340 I slot release: id 9 | task 506 | stop processing: n_tokens = 140, truncated = 0
1.35.183.670 I slot print_timing: id 1 | task 498 | prompt eval time = 116.37 ms / 4 tokens ( 29.09 ms per token, 34.37 tokens per second)
1.35.183.673 I slot print_timing: id 1 | task 498 | eval time = 5412.95 ms / 125 tokens ( 43.30 ms per token, 23.09 tokens per second)
1.35.183.673 I slot print_timing: id 1 | task 498 | total time = 5529.33 ms / 129 tokens
1.35.183.674 I slot print_timing: id 1 | task 498 | graphs reused = 1
1.35.183.677 I slot print_timing: id 1 | task 498 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.35.183.689 I statistics draft-mtp: #calls(b,g,a) = 128 426 6451, #gen drafts = 6460, #acc drafts = 4654, #gen tokens = 12882, #acc tokens = 8680, #mean acc len = 2.35, #acc rate/pos = (0.721, 0.624), dur(b,g,a) = 0.082, 1581.901, 7.865 ms
1.35.183.710 I slot release: id 1 | task 498 | stop processing: n_tokens = 152, truncated = 0
1.35.545.244 I slot print_timing: id 3 | task 496 | n_decoded = 100, tg = 16.95 t/s, tg_3s = 16.95 t/s
1.35.548.371 I slot print_timing: id 15 | task 520 | n_decoded = 102, tg = 28.27 t/s, tg_3s = 28.27 t/s
1.35.754.818 I slot print_timing: id 12 | task 481 | prompt eval time = 110.77 ms / 4 tokens ( 27.69 ms per token, 36.11 tokens per second)
1.35.754.821 I slot print_timing: id 12 | task 481 | eval time = 7564.45 ms / 128 tokens ( 59.10 ms per token, 16.92 tokens per second)
1.35.754.821 I slot print_timing: id 12 | task 481 | total time = 7675.22 ms / 132 tokens
1.35.754.822 I slot print_timing: id 12 | task 481 | graphs reused = 1
1.35.754.825 I slot print_timing: id 12 | task 481 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.35.754.835 I statistics draft-mtp: #calls(b,g,a) = 128 434 6523, #gen drafts = 6531, #acc drafts = 4702, #gen tokens = 13023, #acc tokens = 8770, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.624), dur(b,g,a) = 0.082, 1615.297, 7.948 ms
1.35.754.855 I slot release: id 12 | task 481 | stop processing: n_tokens = 156, truncated = 0
1.35.757.422 I slot print_timing: id 6 | task 509 | prompt eval time = 116.83 ms / 4 tokens ( 29.21 ms per token, 34.24 tokens per second)
1.35.757.425 I slot print_timing: id 6 | task 509 | eval time = 5018.39 ms / 125 tokens ( 40.15 ms per token, 24.91 tokens per second)
1.35.757.425 I slot print_timing: id 6 | task 509 | total time = 5135.21 ms / 129 tokens
1.35.757.425 I slot print_timing: id 6 | task 509 | graphs reused = 1
1.35.757.428 I slot print_timing: id 6 | task 509 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.35.757.433 I statistics draft-mtp: #calls(b,g,a) = 128 434 6528, #gen drafts = 6531, #acc drafts = 4707, #gen tokens = 13023, #acc tokens = 8779, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.624), dur(b,g,a) = 0.082, 1615.297, 7.954 ms
1.35.757.449 I slot release: id 6 | task 509 | stop processing: n_tokens = 152, truncated = 0
1.35.871.843 I slot print_timing: id 7 | task 523 | n_decoded = 101, tg = 26.57 t/s, tg_3s = 26.57 t/s
1.35.872.873 I slot print_timing: id 15 | task 520 | prompt eval time = 111.54 ms / 4 tokens ( 27.88 ms per token, 35.86 tokens per second)
1.35.872.875 I slot print_timing: id 15 | task 520 | eval time = 3932.32 ms / 114 tokens ( 34.49 ms per token, 28.99 tokens per second)
1.35.872.875 I slot print_timing: id 15 | task 520 | total time = 4043.86 ms / 118 tokens
1.35.872.877 I slot print_timing: id 15 | task 520 | graphs reused = 1
1.35.872.880 I slot print_timing: id 15 | task 520 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.35.872.891 I statistics draft-mtp: #calls(b,g,a) = 128 436 6545, #gen drafts = 6545, #acc drafts = 4718, #gen tokens = 13051, #acc tokens = 8798, #mean acc len = 2.34, #acc rate/pos = (0.721, 0.623), dur(b,g,a) = 0.082, 1623.248, 7.971 ms
1.35.872.909 I slot release: id 15 | task 520 | stop processing: n_tokens = 140, truncated = 0
1.36.021.326 I slot print_timing: id 4 | task 507 | n_decoded = 100, tg = 18.48 t/s, tg_3s = 18.48 t/s
1.36.172.217 I slot print_timing: id 14 | task 533 | n_decoded = 105, tg = 34.78 t/s, tg_3s = 34.78 t/s
1.36.320.583 I slot print_timing: id 7 | task 523 | prompt eval time = 117.83 ms / 4 tokens ( 29.46 ms per token, 33.95 tokens per second)
1.36.320.586 I slot print_timing: id 7 | task 523 | eval time = 4250.71 ms / 125 tokens ( 34.01 ms per token, 29.41 tokens per second)
1.36.320.587 I slot print_timing: id 7 | task 523 | total time = 4368.54 ms / 129 tokens
1.36.320.588 I slot print_timing: id 7 | task 523 | graphs reused = 1
1.36.320.591 I slot print_timing: id 7 | task 523 | draft acceptance = 0.79167 ( 76 accepted / 96 generated), mean acceptance length = 2.58, acceptance rate per position = (0.833, 0.750)
1.36.320.608 I statistics draft-mtp: #calls(b,g,a) = 128 445 6598, #gen drafts = 6599, #acc drafts = 4752, #gen tokens = 13158, #acc tokens = 8862, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.623), dur(b,g,a) = 0.082, 1657.212, 8.036 ms
1.36.320.631 I slot release: id 7 | task 523 | stop processing: n_tokens = 152, truncated = 0
1.36.358.642 I slot print_timing: id 3 | task 496 | prompt eval time = 110.63 ms / 4 tokens ( 27.66 ms per token, 36.16 tokens per second)
1.36.358.645 I slot print_timing: id 3 | task 496 | eval time = 6714.64 ms / 128 tokens ( 52.46 ms per token, 19.06 tokens per second)
1.36.358.646 I slot print_timing: id 3 | task 496 | total time = 6825.27 ms / 132 tokens
1.36.358.647 I slot print_timing: id 3 | task 496 | graphs reused = 1
1.36.358.649 I slot print_timing: id 3 | task 496 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.36.358.662 I statistics draft-mtp: #calls(b,g,a) = 128 446 6599, #gen drafts = 6603, #acc drafts = 4753, #gen tokens = 13166, #acc tokens = 8863, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.623), dur(b,g,a) = 0.082, 1660.462, 8.037 ms
1.36.358.682 I slot release: id 3 | task 496 | stop processing: n_tokens = 156, truncated = 0
1.36.360.356 I slot print_timing: id 14 | task 533 | prompt eval time = 112.29 ms / 4 tokens ( 28.07 ms per token, 35.62 tokens per second)
1.36.360.359 I slot print_timing: id 14 | task 533 | eval time = 3207.21 ms / 114 tokens ( 28.13 ms per token, 35.54 tokens per second)
1.36.360.360 I slot print_timing: id 14 | task 533 | total time = 3319.50 ms / 118 tokens
1.36.360.360 I slot print_timing: id 14 | task 533 | graphs reused = 1
1.36.360.362 I slot print_timing: id 14 | task 533 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.36.360.369 I statistics draft-mtp: #calls(b,g,a) = 128 446 6603, #gen drafts = 6603, #acc drafts = 4755, #gen tokens = 13166, #acc tokens = 8867, #mean acc len = 2.34, #acc rate/pos = (0.720, 0.623), dur(b,g,a) = 0.082, 1660.462, 8.043 ms
1.36.360.383 I slot release: id 14 | task 533 | stop processing: n_tokens = 140, truncated = 0
1.36.462.554 I slot print_timing: id 5 | task 521 | n_decoded = 100, tg = 22.11 t/s, tg_3s = 22.11 t/s
1.36.559.439 I slot print_timing: id 4 | task 507 | prompt eval time = 115.61 ms / 4 tokens ( 28.90 ms per token, 34.60 tokens per second)
1.36.559.442 I slot print_timing: id 4 | task 507 | eval time = 5947.95 ms / 128 tokens ( 46.47 ms per token, 21.52 tokens per second)
1.36.559.442 I slot print_timing: id 4 | task 507 | total time = 6063.56 ms / 132 tokens
1.36.559.443 I slot print_timing: id 4 | task 507 | graphs reused = 1
1.36.559.446 I slot print_timing: id 4 | task 507 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.36.559.459 I statistics draft-mtp: #calls(b,g,a) = 128 454 6624, #gen drafts = 6626, #acc drafts = 4765, #gen tokens = 13211, #acc tokens = 8885, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.622), dur(b,g,a) = 0.082, 1677.618, 8.070 ms
1.36.559.479 I slot release: id 4 | task 507 | stop processing: n_tokens = 156, truncated = 0
1.36.667.987 I slot print_timing: id 2 | task 534 | n_decoded = 100, tg = 28.45 t/s, tg_3s = 28.45 t/s
1.36.751.953 I slot print_timing: id 5 | task 521 | prompt eval time = 115.13 ms / 4 tokens ( 28.78 ms per token, 34.74 tokens per second)
1.36.751.957 I slot print_timing: id 5 | task 521 | eval time = 4811.69 ms / 128 tokens ( 37.59 ms per token, 26.60 tokens per second)
1.36.751.958 I slot print_timing: id 5 | task 521 | total time = 4926.82 ms / 132 tokens
1.36.751.959 I slot print_timing: id 5 | task 521 | graphs reused = 1
1.36.751.962 I slot print_timing: id 5 | task 521 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.36.751.976 I statistics draft-mtp: #calls(b,g,a) = 128 465 6646, #gen drafts = 6647, #acc drafts = 4776, #gen tokens = 13252, #acc tokens = 8905, #mean acc len = 2.34, #acc rate/pos = (0.719, 0.621), dur(b,g,a) = 0.082, 1698.120, 8.093 ms
1.36.751.993 I slot release: id 5 | task 521 | stop processing: n_tokens = 156, truncated = 0
1.36.836.425 I slot print_timing: id 2 | task 534 | prompt eval time = 115.88 ms / 4 tokens ( 28.97 ms per token, 34.52 tokens per second)
1.36.836.428 I slot print_timing: id 2 | task 534 | eval time = 3683.53 ms / 128 tokens ( 28.78 ms per token, 34.75 tokens per second)
1.36.836.428 I slot print_timing: id 2 | task 534 | total time = 3799.41 ms / 132 tokens
1.36.836.429 I slot print_timing: id 2 | task 534 | graphs reused = 8
1.36.836.432 I slot print_timing: id 2 | task 534 | draft acceptance = 0.42963 ( 58 accepted / 135 generated), mean acceptance length = 1.85, acceptance rate per position = (0.471, 0.382)
1.36.836.445 I statistics draft-mtp: #calls(b,g,a) = 128 474 6656, #gen drafts = 6656, #acc drafts = 4781, #gen tokens = 13269, #acc tokens = 8915, #mean acc len = 2.34, #acc rate/pos = (0.718, 0.621), dur(b,g,a) = 0.082, 1707.151, 8.103 ms
1.36.836.461 I slot release: id 2 | task 534 | stop processing: n_tokens = 156, truncated = 0
1.36.836.472 I srv update_slots: all slots are idle