0.00.141.810 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.141.813 I device_info:
0.00.227.847 I - CUDA0 : NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (97244 MiB, 96677 MiB free)
0.00.227.860 I - CPU : AMD Ryzen Threadripper 9960X 24-Cores (257066 MiB, 257066 MiB free)
0.00.227.950 I system_info: n_threads = 24 (n_threads_batch = 24) / 48 | CUDA : ARCHS = 1200 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | BLACKWELL_NATIVE_FP4 = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 |
0.00.228.035 I srv init: using 47 threads for HTTP server
0.00.228.253 I srv start: binding port with default address family
0.00.229.426 I srv llama_server: loading model
0.00.229.470 I srv load_model: loading model '/home/ripper/ornith-35b-lab/artifacts/quant/ornith-1.0-35b-IQ4_XS.gguf'
0.00.513.968 I srv load_model: [spec] estimated memory usage of draft model is 2246.29 MiB
0.00.513.999 I common_init_result: fitting params to device memory ...
0.00.514.000 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.02.550.505 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.566.541 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.02.600.276 I srv load_model: loading draft model '/home/ripper/ornith-35b-lab/artifacts/mtp/ornith-1.0-35b-mtp-chaincorr-h2r9fw125-h1r2-h3r6ce375-lr5e7-IQ4_XS.gguf'
0.02.909.708 W llama_context: n_ctx_seq (512) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.02.918.933 I srv load_model: initializing slots, n_slots = 16
0.02.927.438 I common_context_can_seq_rm: the context supports bounded partial sequence removal
0.02.934.088 I common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
0.02.934.092 I common_speculative_impl_draft_mtp: - n_max=2, n_min=0, p_min=0.00, n_embd=2048, backend_sampling=1
0.02.934.093 I common_speculative_impl_draft_mtp: - gpu_layers=99, cache_k=f16, cache_v=f16, ctx_tgt=yes, ctx_dft=yes, devices=[default]
0.02.934.257 I common_speculative_impl_draft_mtp: - draft sampler top_k=1 top_p=1.000 temp=1.000
0.02.934.284 I common_speculative_impl_draft_mtp: - fast backend MTP sampling enabled
0.02.934.293 I common_speculative_impl_draft_mtp: - backend draft sampler top_k=1 top_p=1.000 temp=1.000 contexts=1
0.02.953.707 I srv load_model: speculative decoding context initialized
0.02.953.710 I slot load_model: id 0 | task -1 | new slot, n_ctx = 512
0.02.953.719 I slot load_model: id 1 | task -1 | new slot, n_ctx = 512
0.02.953.719 I slot load_model: id 2 | task -1 | new slot, n_ctx = 512
0.02.953.719 I slot load_model: id 3 | task -1 | new slot, n_ctx = 512
0.02.953.720 I slot load_model: id 4 | task -1 | new slot, n_ctx = 512
0.02.953.720 I slot load_model: id 5 | task -1 | new slot, n_ctx = 512
0.02.953.720 I slot load_model: id 6 | task -1 | new slot, n_ctx = 512
0.02.953.720 I slot load_model: id 7 | task -1 | new slot, n_ctx = 512
0.02.953.720 I slot load_model: id 8 | task -1 | new slot, n_ctx = 512
0.02.953.721 I slot load_model: id 9 | task -1 | new slot, n_ctx = 512
0.02.953.721 I slot load_model: id 10 | task -1 | new slot, n_ctx = 512
0.02.953.721 I slot load_model: id 11 | task -1 | new slot, n_ctx = 512
0.02.953.721 I slot load_model: id 12 | task -1 | new slot, n_ctx = 512
0.02.953.721 I slot load_model: id 13 | task -1 | new slot, n_ctx = 512
0.02.953.722 I slot load_model: id 14 | task -1 | new slot, n_ctx = 512
0.02.953.722 I slot load_model: id 15 | task -1 | new slot, n_ctx = 512
0.02.953.775 I srv load_model: prompt cache is enabled, size limit: 8192 MiB
0.02.953.776 I srv load_model: use `--cache-ram 0` to disable the prompt cache
0.02.953.777 I srv load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.02.953.777 I srv load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.02.953.794 I srv init: idle slots will be saved to prompt cache upon starting a new task
0.02.962.459 I init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
0.02.968.839 I srv init: init: chat template, thinking = 0
0.02.968.883 I srv llama_server: model loaded
0.02.968.888 I srv llama_server: server is listening on http://127.0.0.1:18238
0.02.968.894 I srv update_slots: all slots are idle
0.38.834.459 I srv operator(): Chat format: peg-native
0.38.834.608 I slot get_availabl: id 15 | task -1 | selected slot by LRU, t_last = -1
0.38.834.609 I srv get_availabl: updating prompt cache
0.38.834.616 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.834.622 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.834.624 I srv get_availabl: prompt cache update took 0.01 ms
0.38.834.683 I slot launch_slot_: id 15 | task 0 | processing task, is_child = 0
0.38.834.684 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.834.684 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.834.685 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.834.685 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.834.685 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.834.685 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.834.685 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.834.686 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.834.687 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.834.687 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.38.834.718 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.835.263 I srv operator(): Chat format: peg-native
0.38.835.382 I srv operator(): Chat format: peg-native
0.38.835.385 I srv operator(): Chat format: peg-native
0.38.835.390 I srv operator(): Chat format: peg-native
0.38.835.393 I srv operator(): Chat format: peg-native
0.38.835.431 I srv operator(): Chat format: peg-native
0.38.835.788 I srv operator(): Chat format: peg-native
0.38.836.053 I srv operator(): Chat format: peg-native
0.38.836.223 I srv operator(): Chat format: peg-native
0.38.838.360 I srv operator(): Chat format: peg-native
0.38.839.224 I srv operator(): Chat format: peg-native
0.38.839.448 I srv operator(): Chat format: peg-native
0.38.840.202 I srv operator(): Chat format: peg-native
0.38.840.737 I srv operator(): Chat format: peg-native
0.38.848.843 I srv operator(): Chat format: peg-native
0.38.928.788 I slot get_availabl: id 14 | task -1 | selected slot by LRU, t_last = -1
0.38.928.791 I srv get_availabl: updating prompt cache
0.38.928.796 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.799 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.800 I srv get_availabl: prompt cache update took 0.01 ms
0.38.928.864 I slot launch_slot_: id 14 | task 4 | processing task, is_child = 0
0.38.928.866 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.866 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.866 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.866 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.866 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.866 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.928.867 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.38.928.869 I slot get_availabl: id 13 | task -1 | selected slot by LRU, t_last = -1
0.38.928.869 I srv get_availabl: updating prompt cache
0.38.928.869 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.870 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.870 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.884 I slot launch_slot_: id 13 | task 3 | processing task, is_child = 0
0.38.928.884 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.884 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.884 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.884 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.928.885 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.38.928.887 I slot get_availabl: id 12 | task -1 | selected slot by LRU, t_last = -1
0.38.928.887 I srv get_availabl: updating prompt cache
0.38.928.887 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.887 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.888 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.902 I slot launch_slot_: id 12 | task 6 | processing task, is_child = 0
0.38.928.902 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.902 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.903 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.903 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.903 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.928.903 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.928.903 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.38.928.904 I slot get_availabl: id 11 | task -1 | selected slot by LRU, t_last = -1
0.38.928.904 I srv get_availabl: updating prompt cache
0.38.928.905 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.905 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.905 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.916 I slot launch_slot_: id 11 | task 5 | processing task, is_child = 0
0.38.928.916 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.916 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.916 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.916 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.928.917 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.38.928.919 I slot get_availabl: id 10 | task -1 | selected slot by LRU, t_last = -1
0.38.928.919 I srv get_availabl: updating prompt cache
0.38.928.919 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.920 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.920 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.932 I slot launch_slot_: id 10 | task 7 | processing task, is_child = 0
0.38.928.932 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.932 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.933 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.38.928.935 I slot get_availabl: id 9 | task -1 | selected slot by LRU, t_last = -1
0.38.928.935 I srv get_availabl: updating prompt cache
0.38.928.935 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.936 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.936 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.947 I slot launch_slot_: id 9 | task 8 | processing task, is_child = 0
0.38.928.947 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.947 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.947 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.947 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.947 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.948 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.948 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.948 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.948 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.38.928.950 I slot get_availabl: id 8 | task -1 | selected slot by LRU, t_last = -1
0.38.928.950 I srv get_availabl: updating prompt cache
0.38.928.950 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.951 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.951 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.963 I slot launch_slot_: id 8 | task 2 | processing task, is_child = 0
0.38.928.963 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.963 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.963 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.964 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.964 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.964 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.964 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.964 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.38.928.965 I slot get_availabl: id 7 | task -1 | selected slot by LRU, t_last = -1
0.38.928.965 I srv get_availabl: updating prompt cache
0.38.928.965 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.966 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.966 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.977 I slot launch_slot_: id 7 | task 9 | processing task, is_child = 0
0.38.928.978 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.978 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.38.928.980 I slot get_availabl: id 6 | task -1 | selected slot by LRU, t_last = -1
0.38.928.980 I srv get_availabl: updating prompt cache
0.38.928.980 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.980 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.980 I srv get_availabl: prompt cache update took 0.00 ms
0.38.928.993 I slot launch_slot_: id 6 | task 10 | processing task, is_child = 0
0.38.928.993 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.928.993 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.928.993 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.928.994 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.928.994 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.928.994 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.38.928.995 I slot get_availabl: id 5 | task -1 | selected slot by LRU, t_last = -1
0.38.928.995 I srv get_availabl: updating prompt cache
0.38.928.995 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.928.996 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.928.996 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.006 I slot launch_slot_: id 5 | task 11 | processing task, is_child = 0
0.38.929.007 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.929.007 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.929.007 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.929.007 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.929.007 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.38.929.009 I slot get_availabl: id 4 | task -1 | selected slot by LRU, t_last = -1
0.38.929.009 I srv get_availabl: updating prompt cache
0.38.929.009 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.929.009 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.929.009 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.021 I slot launch_slot_: id 4 | task 12 | processing task, is_child = 0
0.38.929.022 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.929.022 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.929.022 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.929.022 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.38.929.023 I slot get_availabl: id 3 | task -1 | selected slot by LRU, t_last = -1
0.38.929.023 I srv get_availabl: updating prompt cache
0.38.929.023 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.929.024 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.929.024 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.034 I slot launch_slot_: id 3 | task 13 | processing task, is_child = 0
0.38.929.035 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.929.035 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.929.035 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.38.929.036 I slot get_availabl: id 2 | task -1 | selected slot by LRU, t_last = -1
0.38.929.036 I srv get_availabl: updating prompt cache
0.38.929.036 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.929.037 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.929.037 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.048 I slot launch_slot_: id 2 | task 14 | processing task, is_child = 0
0.38.929.048 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.929.049 I slot process_sing: id 1 | task -1 | saving idle slot to prompt cache
0.38.929.050 I slot get_availabl: id 1 | task -1 | selected slot by LRU, t_last = -1
0.38.929.050 I srv get_availabl: updating prompt cache
0.38.929.050 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.929.050 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.929.050 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.061 I slot launch_slot_: id 1 | task 15 | processing task, is_child = 0
0.38.929.061 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.38.929.063 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
0.38.929.063 I srv get_availabl: updating prompt cache
0.38.929.063 I srv load: - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.38.929.063 I srv update: - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 8192 tokens, 8589934592 est)
0.38.929.063 I srv get_availabl: prompt cache update took 0.00 ms
0.38.929.074 I slot launch_slot_: id 0 | task 16 | processing task, is_child = 0
0.38.929.130 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.164 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.178 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.201 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.206 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.230 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.247 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.252 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.255 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.271 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.282 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.299 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.314 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.320 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.929.324 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.38.943.711 I slot create_check: id 15 | task 0 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.246.458 I slot create_check: id 0 | task 16 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.259.855 I slot create_check: id 1 | task 15 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.273.394 I slot create_check: id 2 | task 14 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.286.896 I slot create_check: id 3 | task 13 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.300.994 I slot create_check: id 4 | task 12 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.314.425 I slot create_check: id 5 | task 11 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.328.118 I slot create_check: id 6 | task 10 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.341.830 I slot create_check: id 7 | task 9 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.355.312 I slot create_check: id 8 | task 2 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.368.883 I slot create_check: id 9 | task 8 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.39.382.428 I slot create_check: id 10 | task 7 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.395.869 I slot create_check: id 11 | task 5 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.409.378 I slot create_check: id 12 | task 6 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.39.422.863 I slot create_check: id 13 | task 3 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.39.436.597 I slot create_check: id 14 | task 4 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.43.723.545 I slot print_timing: id 15 | task 0 | n_decoded = 102, tg = 22.71 t/s, tg_3s = 22.71 t/s
0.43.837.580 I slot print_timing: id 0 | task 16 | n_decoded = 102, tg = 23.91 t/s, tg_3s = 23.91 t/s
0.43.841.555 I slot print_timing: id 7 | task 9 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.43.843.651 I slot print_timing: id 11 | task 5 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.43.844.246 I slot print_timing: id 12 | task 6 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.43.845.264 I slot print_timing: id 14 | task 4 | n_decoded = 102, tg = 23.92 t/s, tg_3s = 23.92 t/s
0.44.330.470 I slot print_timing: id 15 | task 0 | prompt eval time = 396.78 ms / 25 tokens ( 15.87 ms per token, 63.01 tokens per second)
0.44.330.473 I slot print_timing: id 15 | task 0 | eval time = 5098.96 ms / 114 tokens ( 44.73 ms per token, 22.36 tokens per second)
0.44.330.473 I slot print_timing: id 15 | task 0 | total time = 5495.74 ms / 139 tokens
0.44.330.479 I slot print_timing: id 15 | task 0 | graphs reused = 1
0.44.330.482 I slot print_timing: id 15 | task 0 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.330.502 I statistics draft-mtp: #calls(b,g,a) = 16 40 625, #gen drafts = 625, #acc drafts = 455, #gen tokens = 1250, #acc tokens = 845, #mean acc len = 2.35, #acc rate/pos = (0.728, 0.624), dur(b,g,a) = 0.008, 111.627, 0.808 ms
0.44.330.542 I slot release: id 15 | task 0 | stop processing: n_tokens = 140, truncated = 0
0.44.339.627 I srv operator(): Chat format: peg-native
0.44.439.487 I slot print_timing: id 0 | task 16 | prompt eval time = 642.95 ms / 25 tokens ( 25.72 ms per token, 38.88 tokens per second)
0.44.439.489 I slot print_timing: id 0 | task 16 | eval time = 4867.42 ms / 114 tokens ( 42.70 ms per token, 23.42 tokens per second)
0.44.439.490 I slot print_timing: id 0 | task 16 | total time = 5510.37 ms / 139 tokens
0.44.439.491 I slot print_timing: id 0 | task 16 | graphs reused = 1
0.44.439.493 I slot print_timing: id 0 | task 16 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.439.508 I statistics draft-mtp: #calls(b,g,a) = 16 41 626, #gen drafts = 640, #acc drafts = 456, #gen tokens = 1280, #acc tokens = 847, #mean acc len = 2.35, #acc rate/pos = (0.728, 0.625), dur(b,g,a) = 0.008, 116.268, 0.811 ms
0.44.439.541 I slot release: id 0 | task 16 | stop processing: n_tokens = 140, truncated = 0
0.44.443.756 I slot print_timing: id 7 | task 9 | prompt eval time = 647.38 ms / 25 tokens ( 25.90 ms per token, 38.62 tokens per second)
0.44.443.758 I slot print_timing: id 7 | task 9 | eval time = 4867.14 ms / 114 tokens ( 42.69 ms per token, 23.42 tokens per second)
0.44.443.759 I slot print_timing: id 7 | task 9 | total time = 5514.51 ms / 139 tokens
0.44.443.760 I slot print_timing: id 7 | task 9 | graphs reused = 1
0.44.443.768 I slot print_timing: id 7 | task 9 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.443.777 I statistics draft-mtp: #calls(b,g,a) = 16 41 633, #gen drafts = 640, #acc drafts = 463, #gen tokens = 1280, #acc tokens = 861, #mean acc len = 2.36, #acc rate/pos = (0.731, 0.629), dur(b,g,a) = 0.008, 116.268, 0.820 ms
0.44.443.802 I slot release: id 7 | task 9 | stop processing: n_tokens = 140, truncated = 0
0.44.446.161 I slot print_timing: id 11 | task 5 | prompt eval time = 649.73 ms / 25 tokens ( 25.99 ms per token, 38.48 tokens per second)
0.44.446.164 I slot print_timing: id 11 | task 5 | eval time = 4867.14 ms / 114 tokens ( 42.69 ms per token, 23.42 tokens per second)
0.44.446.165 I slot print_timing: id 11 | task 5 | total time = 5516.87 ms / 139 tokens
0.44.446.165 I slot print_timing: id 11 | task 5 | graphs reused = 1
0.44.446.168 I slot print_timing: id 11 | task 5 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.446.175 I statistics draft-mtp: #calls(b,g,a) = 16 41 637, #gen drafts = 640, #acc drafts = 467, #gen tokens = 1280, #acc tokens = 869, #mean acc len = 2.36, #acc rate/pos = (0.733, 0.631), dur(b,g,a) = 0.008, 116.268, 0.825 ms
0.44.446.195 I slot release: id 11 | task 5 | stop processing: n_tokens = 140, truncated = 0
0.44.446.777 I slot print_timing: id 12 | task 6 | prompt eval time = 650.28 ms / 25 tokens ( 26.01 ms per token, 38.44 tokens per second)
0.44.446.780 I slot print_timing: id 12 | task 6 | eval time = 4867.19 ms / 114 tokens ( 42.69 ms per token, 23.42 tokens per second)
0.44.446.780 I slot print_timing: id 12 | task 6 | total time = 5517.47 ms / 139 tokens
0.44.446.781 I slot print_timing: id 12 | task 6 | graphs reused = 1
0.44.446.783 I slot print_timing: id 12 | task 6 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.446.789 I statistics draft-mtp: #calls(b,g,a) = 16 41 638, #gen drafts = 640, #acc drafts = 468, #gen tokens = 1280, #acc tokens = 871, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.632), dur(b,g,a) = 0.008, 116.268, 0.826 ms
0.44.446.805 I slot release: id 12 | task 6 | stop processing: n_tokens = 140, truncated = 0
0.44.447.934 I srv operator(): Chat format: peg-native
0.44.447.977 I slot print_timing: id 14 | task 4 | prompt eval time = 651.46 ms / 25 tokens ( 26.06 ms per token, 38.38 tokens per second)
0.44.447.979 I slot print_timing: id 14 | task 4 | eval time = 4867.20 ms / 114 tokens ( 42.69 ms per token, 23.42 tokens per second)
0.44.447.979 I slot print_timing: id 14 | task 4 | total time = 5518.65 ms / 139 tokens
0.44.447.980 I slot print_timing: id 14 | task 4 | graphs reused = 1
0.44.447.982 I slot print_timing: id 14 | task 4 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.44.447.987 I statistics draft-mtp: #calls(b,g,a) = 16 41 640, #gen drafts = 640, #acc drafts = 470, #gen tokens = 1280, #acc tokens = 875, #mean acc len = 2.37, #acc rate/pos = (0.734, 0.633), dur(b,g,a) = 0.008, 116.268, 0.828 ms
0.44.448.001 I slot release: id 14 | task 4 | stop processing: n_tokens = 140, truncated = 0
0.44.448.013 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.448.015 I srv get_availabl: updating prompt cache
0.44.448.061 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.452.467 I srv operator(): Chat format: peg-native
0.44.455.483 I srv operator(): Chat format: peg-native
0.44.456.108 I srv operator(): Chat format: peg-native
0.44.456.392 I srv operator(): Chat format: peg-native
0.44.476.260 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.476.266 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.266 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.267 I srv get_availabl: prompt cache update took 28.25 ms
0.44.476.342 I slot launch_slot_: id 0 | task 59 | processing task, is_child = 0
0.44.476.345 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.44.476.374 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.376 I srv alloc: - prompt is already in the cache, skipping
0.44.476.377 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.476.392 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.393 I srv alloc: - prompt is already in the cache, skipping
0.44.476.394 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.44.476.409 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.411 I srv alloc: - prompt is already in the cache, skipping
0.44.476.411 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.44.476.428 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.429 I srv alloc: - prompt is already in the cache, skipping
0.44.476.430 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.44.476.445 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.447 I srv alloc: - prompt is already in the cache, skipping
0.44.476.450 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.476.452 I srv get_availabl: updating prompt cache
0.44.476.466 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.467 I srv alloc: - prompt is already in the cache, skipping
0.44.476.467 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.476.468 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.468 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.470 I srv get_availabl: prompt cache update took 0.02 ms
0.44.476.490 I slot launch_slot_: id 7 | task 60 | processing task, is_child = 0
0.44.476.491 I slot process_sing: id 11 | task -1 | saving idle slot to prompt cache
0.44.476.505 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.506 I srv alloc: - prompt is already in the cache, skipping
0.44.476.506 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.44.476.521 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.523 I srv alloc: - prompt is already in the cache, skipping
0.44.476.523 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.44.476.537 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.538 I srv alloc: - prompt is already in the cache, skipping
0.44.476.538 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.44.476.552 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.553 I srv alloc: - prompt is already in the cache, skipping
0.44.476.556 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.476.556 I srv get_availabl: updating prompt cache
0.44.476.570 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.571 I srv alloc: - prompt is already in the cache, skipping
0.44.476.571 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.476.572 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.572 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.573 I srv get_availabl: prompt cache update took 0.02 ms
0.44.476.589 I slot launch_slot_: id 11 | task 61 | processing task, is_child = 0
0.44.476.591 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.44.476.607 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.608 I srv alloc: - prompt is already in the cache, skipping
0.44.476.608 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.44.476.622 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.623 I srv alloc: - prompt is already in the cache, skipping
0.44.476.624 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.44.476.637 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.638 I srv alloc: - prompt is already in the cache, skipping
0.44.476.640 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.44.476.641 I srv get_availabl: updating prompt cache
0.44.476.655 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.656 I srv alloc: - prompt is already in the cache, skipping
0.44.476.657 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.44.476.657 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.657 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.659 I srv get_availabl: prompt cache update took 0.02 ms
0.44.476.673 I slot launch_slot_: id 12 | task 62 | processing task, is_child = 0
0.44.476.674 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.44.476.690 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.691 I srv alloc: - prompt is already in the cache, skipping
0.44.476.691 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.44.476.705 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.706 I srv alloc: - prompt is already in the cache, skipping
0.44.476.708 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.44.476.709 I srv get_availabl: updating prompt cache
0.44.476.722 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.723 I srv alloc: - prompt is already in the cache, skipping
0.44.476.724 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.44.476.724 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.724 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.725 I srv get_availabl: prompt cache update took 0.01 ms
0.44.476.740 I slot launch_slot_: id 14 | task 63 | processing task, is_child = 0
0.44.476.741 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.44.476.756 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.757 I srv alloc: - prompt is already in the cache, skipping
0.44.476.759 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.44.476.759 I srv get_availabl: updating prompt cache
0.44.476.785 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.44.476.786 I srv alloc: - prompt is already in the cache, skipping
0.44.476.787 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.44.476.788 I srv update: - cache state: 1 prompts, 129.625 MiB (limits: 8192.000 MiB, 8192 tokens, 8847 est)
0.44.476.788 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.44.476.790 I srv get_availabl: prompt cache update took 0.03 ms
0.44.476.809 I slot launch_slot_: id 15 | task 64 | processing task, is_child = 0
0.44.483.831 I slot operator(): id 0 | task 59 | Checking checkpoint with [20, 20] against 3...
0.44.483.834 W slot operator(): id 0 | task 59 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.483.838 W slot operator(): id 0 | task 59 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.485.501 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.485.528 I slot operator(): id 7 | task 60 | Checking checkpoint with [20, 20] against 3...
0.44.485.530 W slot operator(): id 7 | task 60 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.485.532 W slot operator(): id 7 | task 60 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.487.072 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.487.078 I slot operator(): id 11 | task 61 | Checking checkpoint with [20, 20] against 24...
0.44.490.947 W slot operator(): id 11 | task 61 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.490.986 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.490.996 I slot operator(): id 12 | task 62 | Checking checkpoint with [20, 20] against 3...
0.44.490.997 W slot operator(): id 12 | task 62 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.490.998 W slot operator(): id 12 | task 62 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.492.560 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.492.575 I slot operator(): id 14 | task 63 | Checking checkpoint with [20, 20] against 3...
0.44.492.577 W slot operator(): id 14 | task 63 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.44.492.580 W slot operator(): id 14 | task 63 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.44.494.130 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.494.145 I slot operator(): id 15 | task 64 | Checking checkpoint with [20, 20] against 24...
0.44.497.880 W slot operator(): id 15 | task 64 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.44.497.914 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.44.687.069 I slot create_check: id 0 | task 59 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.700.311 I slot create_check: id 7 | task 60 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.44.713.845 I slot create_check: id 12 | task 62 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.44.727.963 I slot create_check: id 14 | task 63 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.44.965.171 I slot print_timing: id 2 | task 14 | n_decoded = 101, tg = 18.73 t/s, tg_3s = 18.73 t/s
0.44.965.757 I slot print_timing: id 3 | task 13 | n_decoded = 101, tg = 18.73 t/s, tg_3s = 18.73 t/s
0.44.966.619 I slot print_timing: id 5 | task 11 | n_decoded = 101, tg = 18.73 t/s, tg_3s = 18.73 t/s
0.44.967.197 I slot print_timing: id 6 | task 10 | n_decoded = 101, tg = 18.73 t/s, tg_3s = 18.73 t/s
0.44.968.674 I slot print_timing: id 9 | task 8 | n_decoded = 101, tg = 18.74 t/s, tg_3s = 18.74 t/s
0.46.165.504 I slot print_timing: id 2 | task 14 | prompt eval time = 644.44 ms / 28 tokens ( 23.02 ms per token, 43.45 tokens per second)
0.46.165.507 I slot print_timing: id 2 | task 14 | eval time = 6591.88 ms / 125 tokens ( 52.74 ms per token, 18.96 tokens per second)
0.46.165.507 I slot print_timing: id 2 | task 14 | total time = 7236.32 ms / 153 tokens
0.46.165.508 I slot print_timing: id 2 | task 14 | graphs reused = 1
0.46.165.511 I slot print_timing: id 2 | task 14 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.46.165.523 I statistics draft-mtp: #calls(b,g,a) = 22 54 825, #gen drafts = 838, #acc drafts = 575, #gen tokens = 1676, #acc tokens = 1074, #mean acc len = 2.30, #acc rate/pos = (0.697, 0.605), dur(b,g,a) = 0.010, 159.545, 1.071 ms
0.46.165.548 I slot release: id 2 | task 14 | stop processing: n_tokens = 154, truncated = 0
0.46.166.139 I slot print_timing: id 3 | task 13 | prompt eval time = 645.08 ms / 28 tokens ( 23.04 ms per token, 43.41 tokens per second)
0.46.166.141 I slot print_timing: id 3 | task 13 | eval time = 6591.88 ms / 125 tokens ( 52.74 ms per token, 18.96 tokens per second)
0.46.166.142 I slot print_timing: id 3 | task 13 | total time = 7236.96 ms / 153 tokens
0.46.166.142 I slot print_timing: id 3 | task 13 | graphs reused = 1
0.46.166.145 I slot print_timing: id 3 | task 13 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.46.166.151 I statistics draft-mtp: #calls(b,g,a) = 22 54 826, #gen drafts = 838, #acc drafts = 576, #gen tokens = 1676, #acc tokens = 1076, #mean acc len = 2.30, #acc rate/pos = (0.697, 0.605), dur(b,g,a) = 0.010, 159.545, 1.072 ms
0.46.166.167 I slot release: id 3 | task 13 | stop processing: n_tokens = 154, truncated = 0
0.46.167.015 I slot print_timing: id 5 | task 11 | prompt eval time = 646.21 ms / 28 tokens ( 23.08 ms per token, 43.33 tokens per second)
0.46.167.017 I slot print_timing: id 5 | task 11 | eval time = 6591.60 ms / 125 tokens ( 52.73 ms per token, 18.96 tokens per second)
0.46.167.018 I slot print_timing: id 5 | task 11 | total time = 7237.81 ms / 153 tokens
0.46.167.019 I slot print_timing: id 5 | task 11 | graphs reused = 1
0.46.167.021 I slot print_timing: id 5 | task 11 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.46.167.030 I statistics draft-mtp: #calls(b,g,a) = 22 54 828, #gen drafts = 838, #acc drafts = 577, #gen tokens = 1676, #acc tokens = 1078, #mean acc len = 2.30, #acc rate/pos = (0.697, 0.605), dur(b,g,a) = 0.010, 159.545, 1.074 ms
0.46.167.048 I slot release: id 5 | task 11 | stop processing: n_tokens = 154, truncated = 0
0.46.167.633 I slot print_timing: id 6 | task 10 | prompt eval time = 646.83 ms / 28 tokens ( 23.10 ms per token, 43.29 tokens per second)
0.46.167.634 I slot print_timing: id 6 | task 10 | eval time = 6591.58 ms / 125 tokens ( 52.73 ms per token, 18.96 tokens per second)
0.46.167.635 I slot print_timing: id 6 | task 10 | total time = 7238.41 ms / 153 tokens
0.46.167.635 I slot print_timing: id 6 | task 10 | graphs reused = 1
0.46.167.637 I slot print_timing: id 6 | task 10 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.46.167.641 I statistics draft-mtp: #calls(b,g,a) = 22 54 829, #gen drafts = 838, #acc drafts = 578, #gen tokens = 1676, #acc tokens = 1080, #mean acc len = 2.30, #acc rate/pos = (0.697, 0.606), dur(b,g,a) = 0.010, 159.545, 1.075 ms
0.46.167.655 I slot release: id 6 | task 10 | stop processing: n_tokens = 154, truncated = 0
0.46.169.090 I slot print_timing: id 9 | task 8 | prompt eval time = 648.57 ms / 28 tokens ( 23.16 ms per token, 43.17 tokens per second)
0.46.169.092 I slot print_timing: id 9 | task 8 | eval time = 6591.27 ms / 125 tokens ( 52.73 ms per token, 18.96 tokens per second)
0.46.169.092 I slot print_timing: id 9 | task 8 | total time = 7239.84 ms / 153 tokens
0.46.169.093 I slot print_timing: id 9 | task 8 | graphs reused = 1
0.46.169.095 I slot print_timing: id 9 | task 8 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.46.169.100 I statistics draft-mtp: #calls(b,g,a) = 22 54 832, #gen drafts = 838, #acc drafts = 580, #gen tokens = 1676, #acc tokens = 1084, #mean acc len = 2.30, #acc rate/pos = (0.697, 0.606), dur(b,g,a) = 0.010, 159.545, 1.078 ms
0.46.169.115 I slot release: id 9 | task 8 | stop processing: n_tokens = 154, truncated = 0
0.46.175.452 I srv operator(): Chat format: peg-native
0.46.176.410 I srv operator(): Chat format: peg-native
0.46.176.812 I srv operator(): Chat format: peg-native
0.46.177.129 I srv operator(): Chat format: peg-native
0.46.177.675 I srv operator(): Chat format: peg-native
0.46.260.616 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
0.46.260.619 I srv get_availabl: updating prompt cache
0.46.260.663 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.264 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
0.46.288.270 I srv update: - cache state: 2 prompts, 259.657 MiB (limits: 8192.000 MiB, 8192 tokens, 9275 est)
0.46.288.271 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.46.288.271 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.46.288.272 I srv get_availabl: prompt cache update took 27.65 ms
0.46.288.356 I slot launch_slot_: id 2 | task 79 | processing task, is_child = 0
0.46.288.358 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
0.46.288.378 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.380 I srv alloc: - prompt is already in the cache, skipping
0.46.288.381 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.46.288.397 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.399 I srv alloc: - prompt is already in the cache, skipping
0.46.288.399 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.46.288.415 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.416 I srv alloc: - prompt is already in the cache, skipping
0.46.288.416 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.46.288.433 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.434 I srv alloc: - prompt is already in the cache, skipping
0.46.288.438 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.46.288.438 I srv get_availabl: updating prompt cache
0.46.288.452 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.453 I srv alloc: - prompt is already in the cache, skipping
0.46.288.454 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.46.288.454 I srv update: - cache state: 2 prompts, 259.657 MiB (limits: 8192.000 MiB, 8192 tokens, 9275 est)
0.46.288.456 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.46.288.457 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.46.288.457 I srv get_availabl: prompt cache update took 0.02 ms
0.46.288.475 I slot launch_slot_: id 3 | task 80 | processing task, is_child = 0
0.46.288.476 I slot process_sing: id 5 | task -1 | saving idle slot to prompt cache
0.46.288.490 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.491 I srv alloc: - prompt is already in the cache, skipping
0.46.288.491 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.46.288.505 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.506 I srv alloc: - prompt is already in the cache, skipping
0.46.288.507 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.46.288.521 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.522 I srv alloc: - prompt is already in the cache, skipping
0.46.288.524 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.46.288.525 I srv get_availabl: updating prompt cache
0.46.288.539 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.541 I srv alloc: - prompt is already in the cache, skipping
0.46.288.541 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.46.288.542 I srv update: - cache state: 2 prompts, 259.657 MiB (limits: 8192.000 MiB, 8192 tokens, 9275 est)
0.46.288.542 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.46.288.543 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.46.288.543 I srv get_availabl: prompt cache update took 0.02 ms
0.46.288.558 I slot launch_slot_: id 5 | task 81 | processing task, is_child = 0
0.46.288.559 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.46.288.573 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.574 I srv alloc: - prompt is already in the cache, skipping
0.46.288.575 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.46.288.589 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.590 I srv alloc: - prompt is already in the cache, skipping
0.46.288.591 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
0.46.288.593 I srv get_availabl: updating prompt cache
0.46.288.607 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.608 I srv alloc: - prompt is already in the cache, skipping
0.46.288.608 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
0.46.288.609 I srv update: - cache state: 2 prompts, 259.657 MiB (limits: 8192.000 MiB, 8192 tokens, 9275 est)
0.46.288.609 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.46.288.609 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.46.288.611 I srv get_availabl: prompt cache update took 0.02 ms
0.46.288.624 I slot launch_slot_: id 6 | task 82 | processing task, is_child = 0
0.46.288.626 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
0.46.288.640 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.641 I srv alloc: - prompt is already in the cache, skipping
0.46.288.643 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.46.288.644 I srv get_availabl: updating prompt cache
0.46.288.658 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.46.288.659 I srv alloc: - prompt is already in the cache, skipping
0.46.288.659 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.46.288.660 I srv update: - cache state: 2 prompts, 259.657 MiB (limits: 8192.000 MiB, 8192 tokens, 9275 est)
0.46.288.660 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.46.288.661 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.46.288.662 I srv get_availabl: prompt cache update took 0.02 ms
0.46.288.676 I slot launch_slot_: id 9 | task 83 | processing task, is_child = 0
0.46.293.953 I slot operator(): id 2 | task 79 | Checking checkpoint with [23, 23] against 3...
0.46.293.956 W slot operator(): id 2 | task 79 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.46.293.959 W slot operator(): id 2 | task 79 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.46.295.568 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.295.589 I slot operator(): id 3 | task 80 | Checking checkpoint with [23, 23] against 27...
0.46.299.282 W slot operator(): id 3 | task 80 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.46.299.320 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.299.337 I slot operator(): id 5 | task 81 | Checking checkpoint with [23, 23] against 3...
0.46.299.339 W slot operator(): id 5 | task 81 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.46.299.340 W slot operator(): id 5 | task 81 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.46.300.966 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.300.983 I slot operator(): id 6 | task 82 | Checking checkpoint with [23, 23] against 3...
0.46.300.984 W slot operator(): id 6 | task 82 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.46.300.987 W slot operator(): id 6 | task 82 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.46.302.428 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.302.452 I slot operator(): id 9 | task 83 | Checking checkpoint with [23, 23] against 27...
0.46.306.213 W slot operator(): id 9 | task 83 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.46.306.240 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.46.480.666 I slot create_check: id 2 | task 79 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.46.494.166 I slot create_check: id 5 | task 81 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.46.507.606 I slot create_check: id 6 | task 82 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.46.865.507 I slot print_timing: id 1 | task 15 | n_decoded = 100, tg = 13.71 t/s, tg_3s = 13.71 t/s
0.46.866.979 I slot print_timing: id 4 | task 12 | n_decoded = 100, tg = 13.71 t/s, tg_3s = 13.71 t/s
0.46.869.357 I slot print_timing: id 8 | task 2 | n_decoded = 100, tg = 13.71 t/s, tg_3s = 13.71 t/s
0.46.870.222 I slot print_timing: id 10 | task 7 | n_decoded = 100, tg = 13.71 t/s, tg_3s = 13.71 t/s
0.46.871.710 I slot print_timing: id 13 | task 3 | n_decoded = 100, tg = 13.71 t/s, tg_3s = 13.71 t/s
0.49.022.897 I slot print_timing: id 10 | task 7 | prompt eval time = 649.12 ms / 29 tokens ( 22.38 ms per token, 44.68 tokens per second)
0.49.022.902 I slot print_timing: id 10 | task 7 | eval time = 9444.49 ms / 128 tokens ( 73.79 ms per token, 13.55 tokens per second)
0.49.022.903 I slot print_timing: id 10 | task 7 | total time = 10093.61 ms / 157 tokens
0.49.022.904 I slot print_timing: id 10 | task 7 | graphs reused = 1
0.49.022.906 I slot print_timing: id 10 | task 7 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.49.022.919 I statistics draft-mtp: #calls(b,g,a) = 27 77 1177, #gen drafts = 1191, #acc drafts = 777, #gen tokens = 2377, #acc tokens = 1440, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.563), dur(b,g,a) = 0.012, 230.294, 1.504 ms
0.49.022.945 I slot release: id 10 | task 7 | stop processing: n_tokens = 156, truncated = 0
0.49.023.182 I slot print_timing: id 13 | task 3 | prompt eval time = 650.84 ms / 29 tokens ( 22.44 ms per token, 44.56 tokens per second)
0.49.023.184 I slot print_timing: id 13 | task 3 | eval time = 9443.03 ms / 128 tokens ( 73.77 ms per token, 13.55 tokens per second)
0.49.023.184 I slot print_timing: id 13 | task 3 | total time = 10093.88 ms / 157 tokens
0.49.023.184 I slot print_timing: id 13 | task 3 | graphs reused = 1
0.49.023.185 I slot print_timing: id 13 | task 3 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.49.023.188 I statistics draft-mtp: #calls(b,g,a) = 27 77 1177, #gen drafts = 1191, #acc drafts = 777, #gen tokens = 2377, #acc tokens = 1440, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.563), dur(b,g,a) = 0.012, 230.294, 1.504 ms
0.49.023.212 I slot release: id 13 | task 3 | stop processing: n_tokens = 156, truncated = 0
0.49.028.253 I slot print_timing: id 11 | task 61 | n_decoded = 102, tg = 23.36 t/s, tg_3s = 23.36 t/s
0.49.030.014 I slot print_timing: id 15 | task 64 | n_decoded = 102, tg = 23.35 t/s, tg_3s = 23.35 t/s
0.49.032.831 I srv operator(): Chat format: peg-native
0.49.033.222 I srv operator(): Chat format: peg-native
0.49.127.581 I slot print_timing: id 1 | task 15 | prompt eval time = 643.80 ms / 29 tokens ( 22.20 ms per token, 45.04 tokens per second)
0.49.127.583 I slot print_timing: id 1 | task 15 | eval time = 9554.60 ms / 128 tokens ( 74.65 ms per token, 13.40 tokens per second)
0.49.127.584 I slot print_timing: id 1 | task 15 | total time = 10198.41 ms / 157 tokens
0.49.127.585 I slot print_timing: id 1 | task 15 | graphs reused = 1
0.49.127.587 I slot print_timing: id 1 | task 15 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
0.49.127.597 I statistics draft-mtp: #calls(b,g,a) = 27 78 1191, #gen drafts = 1202, #acc drafts = 786, #gen tokens = 2399, #acc tokens = 1458, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.564), dur(b,g,a) = 0.012, 236.699, 1.521 ms
0.49.127.619 I slot release: id 1 | task 15 | stop processing: n_tokens = 156, truncated = 0
0.49.127.816 I slot print_timing: id 4 | task 12 | prompt eval time = 645.66 ms / 29 tokens ( 22.26 ms per token, 44.92 tokens per second)
0.49.127.818 I slot print_timing: id 4 | task 12 | eval time = 9552.98 ms / 128 tokens ( 74.63 ms per token, 13.40 tokens per second)
0.49.127.818 I slot print_timing: id 4 | task 12 | total time = 10198.64 ms / 157 tokens
0.49.127.818 I slot print_timing: id 4 | task 12 | graphs reused = 1
0.49.127.819 I slot print_timing: id 4 | task 12 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
0.49.127.821 I statistics draft-mtp: #calls(b,g,a) = 27 78 1191, #gen drafts = 1202, #acc drafts = 786, #gen tokens = 2399, #acc tokens = 1458, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.564), dur(b,g,a) = 0.012, 236.699, 1.521 ms
0.49.127.839 I slot release: id 4 | task 12 | stop processing: n_tokens = 156, truncated = 0
0.49.128.030 I slot print_timing: id 8 | task 2 | prompt eval time = 648.01 ms / 29 tokens ( 22.35 ms per token, 44.75 tokens per second)
0.49.128.034 I slot print_timing: id 8 | task 2 | eval time = 9550.78 ms / 128 tokens ( 74.62 ms per token, 13.40 tokens per second)
0.49.128.034 I slot print_timing: id 8 | task 2 | total time = 10198.78 ms / 157 tokens
0.49.128.035 I slot print_timing: id 8 | task 2 | graphs reused = 1
0.49.128.035 I slot print_timing: id 8 | task 2 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
0.49.128.036 I statistics draft-mtp: #calls(b,g,a) = 27 78 1191, #gen drafts = 1202, #acc drafts = 786, #gen tokens = 2399, #acc tokens = 1458, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.564), dur(b,g,a) = 0.012, 236.699, 1.521 ms
0.49.128.047 I slot release: id 8 | task 2 | stop processing: n_tokens = 156, truncated = 0
0.49.134.256 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.49.134.257 I srv get_availabl: updating prompt cache
0.49.134.300 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.137.130 I srv operator(): Chat format: peg-native
0.49.137.517 I srv operator(): Chat format: peg-native
0.49.137.679 I srv operator(): Chat format: peg-native
0.49.162.049 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.49.162.054 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.162.056 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.162.056 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.162.056 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.162.057 I srv get_availabl: prompt cache update took 27.80 ms
0.49.162.135 I slot launch_slot_: id 1 | task 107 | processing task, is_child = 0
0.49.162.137 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
0.49.162.158 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.160 I srv alloc: - prompt is already in the cache, skipping
0.49.162.160 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.49.162.178 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.179 I srv alloc: - prompt is already in the cache, skipping
0.49.162.179 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.49.162.195 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.197 I srv alloc: - prompt is already in the cache, skipping
0.49.162.197 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.49.162.213 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.214 I srv alloc: - prompt is already in the cache, skipping
0.49.162.217 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.49.162.218 I srv get_availabl: updating prompt cache
0.49.162.232 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.233 I srv alloc: - prompt is already in the cache, skipping
0.49.162.234 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.49.162.235 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.162.236 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.162.236 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.162.237 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.162.237 I srv get_availabl: prompt cache update took 0.02 ms
0.49.162.256 I slot launch_slot_: id 4 | task 108 | processing task, is_child = 0
0.49.162.258 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
0.49.162.272 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.273 I srv alloc: - prompt is already in the cache, skipping
0.49.162.273 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.49.162.288 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.289 I srv alloc: - prompt is already in the cache, skipping
0.49.162.289 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.49.162.303 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.305 I srv alloc: - prompt is already in the cache, skipping
0.49.162.307 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.49.162.307 I srv get_availabl: updating prompt cache
0.49.162.321 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.322 I srv alloc: - prompt is already in the cache, skipping
0.49.162.323 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.49.162.323 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.162.324 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.162.324 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.162.324 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.162.326 I srv get_availabl: prompt cache update took 0.02 ms
0.49.162.340 I slot launch_slot_: id 8 | task 109 | processing task, is_child = 0
0.49.162.342 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.49.162.356 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.357 I srv alloc: - prompt is already in the cache, skipping
0.49.162.357 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.49.162.371 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.373 I srv alloc: - prompt is already in the cache, skipping
0.49.162.374 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.49.162.375 I srv get_availabl: updating prompt cache
0.49.162.390 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.391 I srv alloc: - prompt is already in the cache, skipping
0.49.162.391 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.49.162.392 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.162.392 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.162.392 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.162.393 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.162.394 I srv get_availabl: prompt cache update took 0.02 ms
0.49.162.408 I slot launch_slot_: id 10 | task 110 | processing task, is_child = 0
0.49.162.409 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.49.162.423 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.424 I srv alloc: - prompt is already in the cache, skipping
0.49.162.426 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.49.162.428 I srv get_availabl: updating prompt cache
0.49.162.442 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.49.162.443 I srv alloc: - prompt is already in the cache, skipping
0.49.162.443 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.49.162.444 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.162.444 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.162.444 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.162.445 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.162.445 I srv get_availabl: prompt cache update took 0.02 ms
0.49.162.460 I slot launch_slot_: id 13 | task 111 | processing task, is_child = 0
0.49.168.940 I slot operator(): id 1 | task 107 | Checking checkpoint with [24, 24] against 28...
0.49.172.542 W slot operator(): id 1 | task 107 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.49.172.590 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.172.598 I slot operator(): id 4 | task 108 | Checking checkpoint with [24, 24] against 3...
0.49.172.600 W slot operator(): id 4 | task 108 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.172.600 W slot operator(): id 4 | task 108 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.49.174.154 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.174.177 I slot operator(): id 8 | task 109 | Checking checkpoint with [24, 24] against 3...
0.49.174.178 W slot operator(): id 8 | task 109 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.174.180 W slot operator(): id 8 | task 109 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.49.175.656 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.175.679 I slot operator(): id 10 | task 110 | Checking checkpoint with [24, 24] against 3...
0.49.175.681 W slot operator(): id 10 | task 110 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.175.683 W slot operator(): id 10 | task 110 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.49.177.125 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.177.137 I slot operator(): id 13 | task 111 | Checking checkpoint with [24, 24] against 28...
0.49.180.719 W slot operator(): id 13 | task 111 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.49.180.752 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.355.427 I slot create_check: id 4 | task 108 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.49.368.741 I slot create_check: id 8 | task 109 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.49.382.343 I slot create_check: id 10 | task 110 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.49.747.293 I slot print_timing: id 11 | task 61 | prompt eval time = 174.13 ms / 4 tokens ( 43.53 ms per token, 22.97 tokens per second)
0.49.747.296 I slot print_timing: id 11 | task 61 | eval time = 5086.06 ms / 114 tokens ( 44.61 ms per token, 22.41 tokens per second)
0.49.747.297 I slot print_timing: id 11 | task 61 | total time = 5260.19 ms / 118 tokens
0.49.747.298 I slot print_timing: id 11 | task 61 | graphs reused = 1
0.49.747.302 I slot print_timing: id 11 | task 61 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.49.747.314 I statistics draft-mtp: #calls(b,g,a) = 32 82 1254, #gen drafts = 1258, #acc drafts = 828, #gen tokens = 2511, #acc tokens = 1537, #mean acc len = 2.23, #acc rate/pos = (0.660, 0.565), dur(b,g,a) = 0.015, 255.715, 1.605 ms
0.49.747.342 I slot release: id 11 | task 61 | stop processing: n_tokens = 140, truncated = 0
0.49.749.093 I slot print_timing: id 15 | task 64 | prompt eval time = 167.67 ms / 4 tokens ( 41.92 ms per token, 23.86 tokens per second)
0.49.749.096 I slot print_timing: id 15 | task 64 | eval time = 5087.26 ms / 114 tokens ( 44.63 ms per token, 22.41 tokens per second)
0.49.749.097 I slot print_timing: id 15 | task 64 | total time = 5254.93 ms / 118 tokens
0.49.749.098 I slot print_timing: id 15 | task 64 | graphs reused = 1
0.49.749.101 I slot print_timing: id 15 | task 64 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.49.749.108 I statistics draft-mtp: #calls(b,g,a) = 32 82 1258, #gen drafts = 1258, #acc drafts = 830, #gen tokens = 2511, #acc tokens = 1541, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.565), dur(b,g,a) = 0.015, 255.715, 1.610 ms
0.49.749.127 I slot release: id 15 | task 64 | stop processing: n_tokens = 140, truncated = 0
0.49.756.573 I srv operator(): Chat format: peg-native
0.49.757.520 I srv operator(): Chat format: peg-native
0.49.859.343 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.49.859.345 I srv get_availabl: updating prompt cache
0.49.859.386 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.49.859.390 I srv alloc: - prompt is already in the cache, skipping
0.49.859.391 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.49.859.393 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.859.394 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.859.395 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.859.395 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.859.395 I srv get_availabl: prompt cache update took 0.05 ms
0.49.859.461 I slot launch_slot_: id 11 | task 117 | processing task, is_child = 0
0.49.859.463 I slot process_sing: id 15 | task -1 | saving idle slot to prompt cache
0.49.859.479 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.49.859.480 I srv alloc: - prompt is already in the cache, skipping
0.49.859.482 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.49.859.482 I srv get_availabl: updating prompt cache
0.49.859.495 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.49.859.496 I srv alloc: - prompt is already in the cache, skipping
0.49.859.497 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.49.859.497 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.49.859.498 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.49.859.499 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.49.859.499 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.49.859.499 I srv get_availabl: prompt cache update took 0.02 ms
0.49.859.514 I slot launch_slot_: id 15 | task 118 | processing task, is_child = 0
0.49.863.170 I slot operator(): id 11 | task 117 | Checking checkpoint with [20, 20] against 3...
0.49.863.172 W slot operator(): id 11 | task 117 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.49.863.174 W slot operator(): id 11 | task 117 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.49.864.724 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.49.864.760 I slot operator(): id 15 | task 118 | Checking checkpoint with [20, 20] against 24...
0.49.868.780 W slot operator(): id 15 | task 118 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.49.868.823 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.50.017.660 I slot create_check: id 11 | task 117 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.50.254.002 I slot print_timing: id 7 | task 60 | n_decoded = 101, tg = 18.67 t/s, tg_3s = 18.67 t/s
0.50.257.239 I slot print_timing: id 14 | task 63 | n_decoded = 101, tg = 18.67 t/s, tg_3s = 18.67 t/s
0.50.973.988 I slot print_timing: id 5 | task 81 | n_decoded = 102, tg = 23.44 t/s, tg_3s = 23.44 t/s
0.51.457.217 I slot print_timing: id 7 | task 60 | prompt eval time = 359.16 ms / 28 tokens ( 12.83 ms per token, 77.96 tokens per second)
0.51.457.220 I slot print_timing: id 7 | task 60 | eval time = 6612.51 ms / 125 tokens ( 52.90 ms per token, 18.90 tokens per second)
0.51.457.221 I slot print_timing: id 7 | task 60 | total time = 6971.67 ms / 153 tokens
0.51.457.222 I slot print_timing: id 7 | task 60 | graphs reused = 1
0.51.457.224 I slot print_timing: id 7 | task 60 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.51.457.236 I statistics draft-mtp: #calls(b,g,a) = 34 96 1469, #gen drafts = 1477, #acc drafts = 973, #gen tokens = 2949, #acc tokens = 1803, #mean acc len = 2.23, #acc rate/pos = (0.662, 0.565), dur(b,g,a) = 0.016, 298.222, 1.863 ms
0.51.457.262 I slot release: id 7 | task 60 | stop processing: n_tokens = 154, truncated = 0
0.51.461.056 I slot print_timing: id 14 | task 63 | prompt eval time = 353.50 ms / 28 tokens ( 12.62 ms per token, 79.21 tokens per second)
0.51.461.059 I slot print_timing: id 14 | task 63 | eval time = 6614.97 ms / 125 tokens ( 52.92 ms per token, 18.90 tokens per second)
0.51.461.059 I slot print_timing: id 14 | task 63 | total time = 6968.47 ms / 153 tokens
0.51.461.060 I slot print_timing: id 14 | task 63 | graphs reused = 1
0.51.461.063 I slot print_timing: id 14 | task 63 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.51.461.069 I statistics draft-mtp: #calls(b,g,a) = 34 96 1476, #gen drafts = 1477, #acc drafts = 979, #gen tokens = 2949, #acc tokens = 1815, #mean acc len = 2.23, #acc rate/pos = (0.663, 0.566), dur(b,g,a) = 0.016, 298.222, 1.872 ms
0.51.461.094 I slot release: id 14 | task 63 | stop processing: n_tokens = 154, truncated = 0
0.51.465.940 I srv operator(): Chat format: peg-native
0.51.469.174 I srv operator(): Chat format: peg-native
0.51.571.516 I slot print_timing: id 5 | task 81 | prompt eval time = 323.87 ms / 25 tokens ( 12.95 ms per token, 77.19 tokens per second)
0.51.571.519 I slot print_timing: id 5 | task 81 | eval time = 4948.29 ms / 114 tokens ( 43.41 ms per token, 23.04 tokens per second)
0.51.571.520 I slot print_timing: id 5 | task 81 | total time = 5272.16 ms / 139 tokens
0.51.571.520 I slot print_timing: id 5 | task 81 | graphs reused = 1
0.51.571.524 I slot print_timing: id 5 | task 81 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.51.571.535 I statistics draft-mtp: #calls(b,g,a) = 34 97 1483, #gen drafts = 1491, #acc drafts = 983, #gen tokens = 2977, #acc tokens = 1823, #mean acc len = 2.23, #acc rate/pos = (0.663, 0.566), dur(b,g,a) = 0.016, 307.187, 1.881 ms
0.51.571.561 I slot release: id 5 | task 81 | stop processing: n_tokens = 140, truncated = 0
0.51.575.415 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.51.575.417 I srv get_availabl: updating prompt cache
0.51.575.472 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.51.575.474 I srv alloc: - prompt is already in the cache, skipping
0.51.575.475 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.51.575.479 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.51.575.481 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.51.575.483 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.51.575.485 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.51.575.485 I srv get_availabl: prompt cache update took 0.07 ms
0.51.575.549 I slot launch_slot_: id 5 | task 133 | processing task, is_child = 0
0.51.575.551 I slot process_sing: id 7 | task -1 | saving idle slot to prompt cache
0.51.575.568 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.51.575.569 I srv alloc: - prompt is already in the cache, skipping
0.51.575.570 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.51.575.586 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.51.575.587 I srv alloc: - prompt is already in the cache, skipping
0.51.575.589 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.51.575.589 I srv get_availabl: updating prompt cache
0.51.575.603 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.51.575.604 I srv alloc: - prompt is already in the cache, skipping
0.51.575.604 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.51.575.605 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.51.575.606 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.51.575.607 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.51.575.607 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.51.575.607 I srv get_availabl: prompt cache update took 0.02 ms
0.51.575.622 I slot launch_slot_: id 7 | task 134 | processing task, is_child = 0
0.51.575.623 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.51.575.636 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.51.575.638 I srv alloc: - prompt is already in the cache, skipping
0.51.580.921 I srv operator(): Chat format: peg-native
0.51.581.336 I slot operator(): id 5 | task 133 | Checking checkpoint with [20, 20] against 3...
0.51.581.338 W slot operator(): id 5 | task 133 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.51.581.340 W slot operator(): id 5 | task 133 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.51.582.946 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.582.969 I slot operator(): id 7 | task 134 | Checking checkpoint with [23, 23] against 27...
0.51.587.117 W slot operator(): id 7 | task 134 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.51.587.146 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.710.646 I slot print_timing: id 9 | task 83 | n_decoded = 101, tg = 19.23 t/s, tg_3s = 19.23 t/s
0.51.713.642 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.51.713.645 I srv get_availabl: updating prompt cache
0.51.713.694 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.51.713.696 I srv alloc: - prompt is already in the cache, skipping
0.51.713.699 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.51.713.701 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.51.713.703 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.51.713.705 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.51.713.705 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.51.713.705 I srv get_availabl: prompt cache update took 0.06 ms
0.51.713.781 I slot launch_slot_: id 14 | task 136 | processing task, is_child = 0
0.51.732.508 I slot create_check: id 5 | task 133 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.51.732.515 I slot operator(): id 14 | task 136 | Checking checkpoint with [23, 23] against 3...
0.51.732.516 W slot operator(): id 14 | task 136 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.51.732.517 W slot operator(): id 14 | task 136 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.51.734.261 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.51.882.180 I slot create_check: id 14 | task 136 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.51.996.263 I slot print_timing: id 3 | task 80 | n_decoded = 101, tg = 18.24 t/s, tg_3s = 18.24 t/s
0.52.000.622 I slot print_timing: id 12 | task 62 | n_decoded = 100, tg = 13.98 t/s, tg_3s = 13.98 t/s
0.52.115.086 I slot print_timing: id 0 | task 59 | n_decoded = 100, tg = 13.75 t/s, tg_3s = 13.75 t/s
0.52.963.039 I slot print_timing: id 9 | task 83 | prompt eval time = 155.37 ms / 4 tokens ( 38.84 ms per token, 25.74 tokens per second)
0.52.963.045 I slot print_timing: id 9 | task 83 | eval time = 6505.19 ms / 125 tokens ( 52.04 ms per token, 19.22 tokens per second)
0.52.963.045 I slot print_timing: id 9 | task 83 | total time = 6660.57 ms / 129 tokens
0.52.963.047 I slot print_timing: id 9 | task 83 | graphs reused = 1
0.52.963.049 I slot print_timing: id 9 | task 83 | draft acceptance = 0.71154 ( 74 accepted / 104 generated), mean acceptance length = 2.42, acceptance rate per position = (0.750, 0.673)
0.52.963.061 I statistics draft-mtp: #calls(b,g,a) = 37 108 1655, #gen drafts = 1661, #acc drafts = 1102, #gen tokens = 3317, #acc tokens = 2046, #mean acc len = 2.24, #acc rate/pos = (0.666, 0.570), dur(b,g,a) = 0.018, 343.595, 2.100 ms
0.52.963.085 I slot release: id 9 | task 83 | stop processing: n_tokens = 154, truncated = 0
0.52.971.750 I srv operator(): Chat format: peg-native
0.53.080.367 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
0.53.080.370 I srv get_availabl: updating prompt cache
0.53.080.413 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.53.080.417 I srv alloc: - prompt is already in the cache, skipping
0.53.080.418 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
0.53.080.420 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.53.080.421 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.53.080.422 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.53.080.422 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.53.080.422 I srv get_availabl: prompt cache update took 0.05 ms
0.53.080.488 I slot launch_slot_: id 9 | task 148 | processing task, is_child = 0
0.53.084.892 I slot operator(): id 9 | task 148 | Checking checkpoint with [23, 23] against 3...
0.53.084.895 W slot operator(): id 9 | task 148 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.53.084.897 W slot operator(): id 9 | task 148 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.53.086.414 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.211.006 I slot print_timing: id 3 | task 80 | prompt eval time = 161.89 ms / 4 tokens ( 40.47 ms per token, 24.71 tokens per second)
0.53.211.013 I slot print_timing: id 3 | task 80 | eval time = 6753.51 ms / 125 tokens ( 54.03 ms per token, 18.51 tokens per second)
0.53.211.014 I slot print_timing: id 3 | task 80 | total time = 6915.40 ms / 129 tokens
0.53.211.015 I slot print_timing: id 3 | task 80 | graphs reused = 1
0.53.211.018 I slot print_timing: id 3 | task 80 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
0.53.211.032 I statistics draft-mtp: #calls(b,g,a) = 37 110 1680, #gen drafts = 1691, #acc drafts = 1115, #gen tokens = 3377, #acc tokens = 2069, #mean acc len = 2.23, #acc rate/pos = (0.664, 0.568), dur(b,g,a) = 0.018, 352.050, 2.132 ms
0.53.211.064 I slot release: id 3 | task 80 | stop processing: n_tokens = 154, truncated = 0
0.53.219.632 I srv operator(): Chat format: peg-native
0.53.234.871 I slot create_check: id 9 | task 148 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.53.347.506 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.53.347.509 I srv get_availabl: updating prompt cache
0.53.347.552 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.53.347.554 I srv alloc: - prompt is already in the cache, skipping
0.53.347.555 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.53.347.558 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.53.347.559 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.53.347.559 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.53.347.560 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.53.347.560 I srv get_availabl: prompt cache update took 0.05 ms
0.53.347.628 I slot launch_slot_: id 3 | task 151 | processing task, is_child = 0
0.53.351.395 I slot operator(): id 3 | task 151 | Checking checkpoint with [23, 23] against 27...
0.53.355.387 W slot operator(): id 3 | task 151 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.53.355.417 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.53.710.388 I slot print_timing: id 2 | task 79 | n_decoded = 100, tg = 14.11 t/s, tg_3s = 14.11 t/s
0.53.712.290 I slot print_timing: id 6 | task 82 | n_decoded = 100, tg = 14.11 t/s, tg_3s = 14.11 t/s
0.53.831.842 I slot print_timing: id 4 | task 108 | n_decoded = 102, tg = 23.54 t/s, tg_3s = 23.54 t/s
0.53.834.707 I slot print_timing: id 10 | task 110 | n_decoded = 102, tg = 23.53 t/s, tg_3s = 23.53 t/s
0.54.310.200 I slot print_timing: id 12 | task 62 | prompt eval time = 354.44 ms / 29 tokens ( 12.22 ms per token, 81.82 tokens per second)
0.54.310.206 I slot print_timing: id 12 | task 62 | eval time = 9464.74 ms / 128 tokens ( 73.94 ms per token, 13.52 tokens per second)
0.54.310.207 I slot print_timing: id 12 | task 62 | total time = 9819.17 ms / 157 tokens
0.54.310.208 I slot print_timing: id 12 | task 62 | graphs reused = 1
0.54.310.211 I slot print_timing: id 12 | task 62 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.54.310.223 I statistics draft-mtp: #calls(b,g,a) = 39 119 1816, #gen drafts = 1831, #acc drafts = 1198, #gen tokens = 3655, #acc tokens = 2219, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.562), dur(b,g,a) = 0.020, 381.270, 2.304 ms
0.54.310.253 I slot release: id 12 | task 62 | stop processing: n_tokens = 156, truncated = 0
0.54.317.292 I slot print_timing: id 15 | task 118 | n_decoded = 102, tg = 23.59 t/s, tg_3s = 23.59 t/s
0.54.319.483 I srv operator(): Chat format: peg-native
0.54.423.372 I slot print_timing: id 0 | task 59 | prompt eval time = 360.14 ms / 29 tokens ( 12.42 ms per token, 80.52 tokens per second)
0.54.423.376 I slot print_timing: id 0 | task 59 | eval time = 9579.37 ms / 128 tokens ( 74.84 ms per token, 13.36 tokens per second)
0.54.423.377 I slot print_timing: id 0 | task 59 | total time = 9939.51 ms / 157 tokens
0.54.423.377 I slot print_timing: id 0 | task 59 | graphs reused = 1
0.54.423.381 I slot print_timing: id 0 | task 59 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
0.54.423.395 I statistics draft-mtp: #calls(b,g,a) = 39 120 1831, #gen drafts = 1845, #acc drafts = 1209, #gen tokens = 3683, #acc tokens = 2236, #mean acc len = 2.22, #acc rate/pos = (0.660, 0.561), dur(b,g,a) = 0.020, 384.938, 2.327 ms
0.54.423.423 I slot release: id 0 | task 59 | stop processing: n_tokens = 156, truncated = 0
0.54.425.923 I slot print_timing: id 4 | task 108 | prompt eval time = 325.80 ms / 25 tokens ( 13.03 ms per token, 76.73 tokens per second)
0.54.425.925 I slot print_timing: id 4 | task 108 | eval time = 4927.51 ms / 114 tokens ( 43.22 ms per token, 23.14 tokens per second)
0.54.425.926 I slot print_timing: id 4 | task 108 | total time = 5253.31 ms / 139 tokens
0.54.425.927 I slot print_timing: id 4 | task 108 | graphs reused = 1
0.54.425.930 I slot print_timing: id 4 | task 108 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.54.425.938 I statistics draft-mtp: #calls(b,g,a) = 39 120 1835, #gen drafts = 1845, #acc drafts = 1213, #gen tokens = 3683, #acc tokens = 2243, #mean acc len = 2.22, #acc rate/pos = (0.661, 0.561), dur(b,g,a) = 0.020, 384.938, 2.334 ms
0.54.425.958 I slot release: id 4 | task 108 | stop processing: n_tokens = 140, truncated = 0
0.54.429.369 I slot print_timing: id 10 | task 110 | prompt eval time = 323.93 ms / 25 tokens ( 12.96 ms per token, 77.18 tokens per second)
0.54.429.372 I slot print_timing: id 10 | task 110 | eval time = 4929.74 ms / 114 tokens ( 43.24 ms per token, 23.12 tokens per second)
0.54.429.372 I slot print_timing: id 10 | task 110 | total time = 5253.67 ms / 139 tokens
0.54.429.374 I slot print_timing: id 10 | task 110 | graphs reused = 1
0.54.429.377 I slot print_timing: id 10 | task 110 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.54.429.387 I statistics draft-mtp: #calls(b,g,a) = 39 120 1841, #gen drafts = 1845, #acc drafts = 1218, #gen tokens = 3683, #acc tokens = 2253, #mean acc len = 2.22, #acc rate/pos = (0.662, 0.562), dur(b,g,a) = 0.020, 384.938, 2.342 ms
0.54.429.412 I slot release: id 10 | task 110 | stop processing: n_tokens = 140, truncated = 0
0.54.431.572 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.54.431.574 I srv get_availabl: updating prompt cache
0.54.431.616 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.54.431.617 I srv alloc: - prompt is already in the cache, skipping
0.54.431.618 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.54.431.622 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.54.431.624 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.54.431.624 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.54.431.624 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.54.431.625 I srv get_availabl: prompt cache update took 0.05 ms
0.54.431.687 I slot launch_slot_: id 4 | task 161 | processing task, is_child = 0
0.54.431.688 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
0.54.431.705 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.431.706 I srv alloc: - prompt is already in the cache, skipping
0.54.431.706 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.54.431.722 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.54.431.724 I srv alloc: - prompt is already in the cache, skipping
0.54.431.724 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.54.431.740 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.431.741 I srv alloc: - prompt is already in the cache, skipping
0.54.432.371 I srv operator(): Chat format: peg-native
0.54.434.902 I srv operator(): Chat format: peg-native
0.54.437.212 I slot operator(): id 4 | task 161 | Checking checkpoint with [20, 20] against 24...
0.54.438.294 I srv operator(): Chat format: peg-native
0.54.441.009 W slot operator(): id 4 | task 161 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.54.441.069 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.540.152 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.54.540.155 I srv get_availabl: updating prompt cache
0.54.540.196 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.540.199 I srv alloc: - prompt is already in the cache, skipping
0.54.540.200 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.54.540.203 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.54.540.204 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.54.540.205 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.54.540.206 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.54.540.206 I srv get_availabl: prompt cache update took 0.05 ms
0.54.540.272 I slot launch_slot_: id 0 | task 163 | processing task, is_child = 0
0.54.540.274 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
0.54.540.290 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.54.540.292 I srv alloc: - prompt is already in the cache, skipping
0.54.540.292 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.54.540.307 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.540.308 I srv alloc: - prompt is already in the cache, skipping
0.54.540.310 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.54.540.310 I srv get_availabl: updating prompt cache
0.54.540.323 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.54.540.325 I srv alloc: - prompt is already in the cache, skipping
0.54.540.325 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.54.540.326 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.54.540.326 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.54.540.326 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.54.540.326 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.54.540.327 I srv get_availabl: prompt cache update took 0.02 ms
0.54.540.346 I slot launch_slot_: id 10 | task 164 | processing task, is_child = 0
0.54.540.347 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
0.54.540.361 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.540.362 I srv alloc: - prompt is already in the cache, skipping
0.54.540.364 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.54.540.365 I srv get_availabl: updating prompt cache
0.54.540.378 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.54.540.379 I srv alloc: - prompt is already in the cache, skipping
0.54.540.380 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.54.540.380 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.54.540.380 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.54.540.381 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.54.540.381 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.54.540.382 I srv get_availabl: prompt cache update took 0.02 ms
0.54.540.398 I slot launch_slot_: id 12 | task 165 | processing task, is_child = 0
0.54.544.808 I slot operator(): id 0 | task 163 | Checking checkpoint with [24, 24] against 28...
0.54.548.686 W slot operator(): id 0 | task 163 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.54.548.718 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.548.741 I slot operator(): id 10 | task 164 | Checking checkpoint with [20, 20] against 3...
0.54.548.743 W slot operator(): id 10 | task 164 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.54.548.744 W slot operator(): id 10 | task 164 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.54.550.623 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.550.650 I slot operator(): id 12 | task 165 | Checking checkpoint with [24, 24] against 3...
0.54.550.651 W slot operator(): id 12 | task 165 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.54.550.653 W slot operator(): id 12 | task 165 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.54.552.825 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.54.712.220 I slot create_check: id 10 | task 164 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.54.725.660 I slot create_check: id 12 | task 165 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.54.843.682 I slot print_timing: id 8 | task 109 | n_decoded = 101, tg = 18.90 t/s, tg_3s = 18.90 t/s
0.54.965.377 I slot print_timing: id 15 | task 118 | prompt eval time = 127.94 ms / 4 tokens ( 31.99 ms per token, 31.26 tokens per second)
0.54.965.380 I slot print_timing: id 15 | task 118 | eval time = 4972.65 ms / 114 tokens ( 43.62 ms per token, 22.93 tokens per second)
0.54.965.380 I slot print_timing: id 15 | task 118 | total time = 5100.59 ms / 118 tokens
0.54.965.381 I slot print_timing: id 15 | task 118 | graphs reused = 1
0.54.965.384 I slot print_timing: id 15 | task 118 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.54.965.396 I statistics draft-mtp: #calls(b,g,a) = 43 124 1900, #gen drafts = 1900, #acc drafts = 1251, #gen tokens = 3793, #acc tokens = 2313, #mean acc len = 2.22, #acc rate/pos = (0.658, 0.559), dur(b,g,a) = 0.022, 402.055, 2.424 ms
0.54.965.421 I slot release: id 15 | task 118 | stop processing: n_tokens = 140, truncated = 0
0.54.974.083 I srv operator(): Chat format: peg-native
0.55.078.655 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.55.078.657 I srv get_availabl: updating prompt cache
0.55.078.699 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.55.078.701 I srv alloc: - prompt is already in the cache, skipping
0.55.078.702 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.55.078.705 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.55.078.706 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.55.078.706 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.55.078.707 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.55.078.707 I srv get_availabl: prompt cache update took 0.05 ms
0.55.078.779 I slot launch_slot_: id 15 | task 170 | processing task, is_child = 0
0.55.081.560 I slot operator(): id 15 | task 170 | Checking checkpoint with [20, 20] against 3...
0.55.081.563 W slot operator(): id 15 | task 170 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.55.081.565 W slot operator(): id 15 | task 170 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.55.083.633 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.55.229.889 I slot create_check: id 15 | task 170 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.55.588.517 I slot print_timing: id 11 | task 117 | n_decoded = 101, tg = 18.50 t/s, tg_3s = 18.50 t/s
0.56.059.064 I slot print_timing: id 2 | task 79 | prompt eval time = 328.97 ms / 29 tokens ( 11.34 ms per token, 88.15 tokens per second)
0.56.059.069 I slot print_timing: id 2 | task 79 | eval time = 9436.11 ms / 128 tokens ( 73.72 ms per token, 13.56 tokens per second)
0.56.059.070 I slot print_timing: id 2 | task 79 | total time = 9765.08 ms / 157 tokens
0.56.059.071 I slot print_timing: id 2 | task 79 | graphs reused = 1
0.56.059.075 I slot print_timing: id 2 | task 79 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.56.059.091 I statistics draft-mtp: #calls(b,g,a) = 44 133 2025, #gen drafts = 2039, #acc drafts = 1320, #gen tokens = 4069, #acc tokens = 2441, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.554), dur(b,g,a) = 0.023, 428.794, 2.577 ms
0.56.059.119 I slot release: id 2 | task 79 | stop processing: n_tokens = 156, truncated = 0
0.56.059.327 I slot print_timing: id 6 | task 82 | prompt eval time = 322.54 ms / 29 tokens ( 11.12 ms per token, 89.91 tokens per second)
0.56.059.330 I slot print_timing: id 6 | task 82 | eval time = 9435.79 ms / 128 tokens ( 73.72 ms per token, 13.57 tokens per second)
0.56.059.331 I slot print_timing: id 6 | task 82 | total time = 9758.33 ms / 157 tokens
0.56.059.331 I slot print_timing: id 6 | task 82 | graphs reused = 1
0.56.059.333 I slot print_timing: id 6 | task 82 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.56.059.335 I statistics draft-mtp: #calls(b,g,a) = 44 133 2025, #gen drafts = 2039, #acc drafts = 1320, #gen tokens = 4069, #acc tokens = 2441, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.554), dur(b,g,a) = 0.023, 428.794, 2.577 ms
0.56.059.347 I slot release: id 6 | task 82 | stop processing: n_tokens = 156, truncated = 0
0.56.062.974 I slot print_timing: id 8 | task 109 | prompt eval time = 324.87 ms / 28 tokens ( 11.60 ms per token, 86.19 tokens per second)
0.56.062.977 I slot print_timing: id 8 | task 109 | eval time = 6563.91 ms / 125 tokens ( 52.51 ms per token, 19.04 tokens per second)
0.56.062.977 I slot print_timing: id 8 | task 109 | total time = 6888.78 ms / 153 tokens
0.56.062.978 I slot print_timing: id 8 | task 109 | graphs reused = 1
0.56.062.980 I slot print_timing: id 8 | task 109 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
0.56.062.986 I statistics draft-mtp: #calls(b,g,a) = 44 133 2032, #gen drafts = 2039, #acc drafts = 1325, #gen tokens = 4069, #acc tokens = 2451, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.554), dur(b,g,a) = 0.023, 428.794, 2.588 ms
0.56.063.008 I slot release: id 8 | task 109 | stop processing: n_tokens = 154, truncated = 0
0.56.069.253 I srv operator(): Chat format: peg-native
0.56.069.260 I srv operator(): Chat format: peg-native
0.56.072.520 I srv operator(): Chat format: peg-native
0.56.169.803 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.56.169.805 I srv get_availabl: updating prompt cache
0.56.169.849 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.56.169.851 I srv alloc: - prompt is already in the cache, skipping
0.56.169.853 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.56.169.856 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.56.169.857 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.56.169.857 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.56.169.858 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.56.169.858 I srv get_availabl: prompt cache update took 0.05 ms
0.56.169.926 I slot launch_slot_: id 8 | task 180 | processing task, is_child = 0
0.56.169.927 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
0.56.169.945 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.56.169.946 I srv alloc: - prompt is already in the cache, skipping
0.56.169.946 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.56.169.962 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.56.169.963 I srv alloc: - prompt is already in the cache, skipping
0.56.169.965 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.56.169.965 I srv get_availabl: updating prompt cache
0.56.169.979 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.56.169.980 I srv alloc: - prompt is already in the cache, skipping
0.56.169.980 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.56.169.981 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.56.169.983 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.56.169.983 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.56.169.983 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.56.169.983 I srv get_availabl: prompt cache update took 0.02 ms
0.56.170.001 I slot launch_slot_: id 2 | task 181 | processing task, is_child = 0
0.56.170.002 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
0.56.170.016 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.56.170.017 I srv alloc: - prompt is already in the cache, skipping
0.56.170.018 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
0.56.170.019 I srv get_availabl: updating prompt cache
0.56.170.032 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.56.170.033 I srv alloc: - prompt is already in the cache, skipping
0.56.170.034 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
0.56.170.034 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.56.170.035 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.56.170.036 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.56.170.036 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.56.170.036 I srv get_availabl: prompt cache update took 0.02 ms
0.56.170.052 I slot launch_slot_: id 6 | task 182 | processing task, is_child = 0
0.56.175.672 I slot operator(): id 2 | task 181 | Checking checkpoint with [24, 24] against 3...
0.56.175.675 W slot operator(): id 2 | task 181 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.56.175.678 W slot operator(): id 2 | task 181 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.56.177.183 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.177.224 I slot operator(): id 6 | task 182 | Checking checkpoint with [24, 24] against 28...
0.56.181.223 W slot operator(): id 6 | task 182 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
0.56.181.248 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.181.283 I slot operator(): id 8 | task 180 | Checking checkpoint with [23, 23] against 27...
0.56.184.874 W slot operator(): id 8 | task 180 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.56.184.900 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.56.316.755 I slot print_timing: id 14 | task 136 | n_decoded = 102, tg = 23.60 t/s, tg_3s = 23.60 t/s
0.56.335.868 I slot create_check: id 2 | task 181 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.56.577.299 I slot print_timing: id 13 | task 111 | n_decoded = 100, tg = 13.80 t/s, tg_3s = 13.80 t/s
0.56.692.505 I slot print_timing: id 1 | task 107 | n_decoded = 100, tg = 13.58 t/s, tg_3s = 13.58 t/s
0.56.818.911 I slot print_timing: id 11 | task 117 | prompt eval time = 266.43 ms / 28 tokens ( 9.52 ms per token, 105.09 tokens per second)
0.56.818.914 I slot print_timing: id 11 | task 117 | eval time = 6689.29 ms / 125 tokens ( 53.51 ms per token, 18.69 tokens per second)
0.56.818.914 I slot print_timing: id 11 | task 117 | total time = 6955.72 ms / 153 tokens
0.56.818.915 I slot print_timing: id 11 | task 117 | graphs reused = 1
0.56.818.918 I slot print_timing: id 11 | task 117 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
0.56.818.930 I statistics draft-mtp: #calls(b,g,a) = 47 139 2124, #gen drafts = 2128, #acc drafts = 1386, #gen tokens = 4247, #acc tokens = 2565, #mean acc len = 2.21, #acc rate/pos = (0.653, 0.555), dur(b,g,a) = 0.025, 453.610, 2.717 ms
0.56.818.956 I slot release: id 11 | task 117 | stop processing: n_tokens = 154, truncated = 0
0.56.828.077 I srv operator(): Chat format: peg-native
0.56.936.570 I slot print_timing: id 14 | task 136 | prompt eval time = 261.84 ms / 25 tokens ( 10.47 ms per token, 95.48 tokens per second)
0.56.936.576 I slot print_timing: id 14 | task 136 | eval time = 4942.19 ms / 114 tokens ( 43.35 ms per token, 23.07 tokens per second)
0.56.936.577 I slot print_timing: id 14 | task 136 | total time = 5204.03 ms / 139 tokens
0.56.936.578 I slot print_timing: id 14 | task 136 | graphs reused = 1
0.56.936.581 I slot print_timing: id 14 | task 136 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.56.936.592 I statistics draft-mtp: #calls(b,g,a) = 47 140 2142, #gen drafts = 2143, #acc drafts = 1399, #gen tokens = 4277, #acc tokens = 2589, #mean acc len = 2.21, #acc rate/pos = (0.653, 0.556), dur(b,g,a) = 0.025, 457.656, 2.740 ms
0.56.936.616 I slot release: id 14 | task 136 | stop processing: n_tokens = 140, truncated = 0
0.56.937.078 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
0.56.937.081 I srv get_availabl: updating prompt cache
0.56.937.122 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.56.937.124 I srv alloc: - prompt is already in the cache, skipping
0.56.937.125 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
0.56.937.128 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.56.937.130 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.56.937.131 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.56.937.131 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.56.937.131 I srv get_availabl: prompt cache update took 0.05 ms
0.56.937.196 I slot launch_slot_: id 11 | task 189 | processing task, is_child = 0
0.56.937.199 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
0.56.937.214 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.56.937.216 I srv alloc: - prompt is already in the cache, skipping
0.56.941.594 I slot operator(): id 11 | task 189 | Checking checkpoint with [23, 23] against 27...
0.56.944.610 I srv operator(): Chat format: peg-native
0.56.945.578 W slot operator(): id 11 | task 189 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
0.56.945.620 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.057.942 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
0.57.057.944 I srv get_availabl: updating prompt cache
0.57.057.985 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.57.057.987 I srv alloc: - prompt is already in the cache, skipping
0.57.057.988 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
0.57.057.990 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.57.057.991 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.57.057.992 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.57.057.992 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.57.057.993 I srv get_availabl: prompt cache update took 0.05 ms
0.57.058.058 I slot launch_slot_: id 14 | task 191 | processing task, is_child = 0
0.57.061.598 I slot operator(): id 14 | task 191 | Checking checkpoint with [20, 20] against 24...
0.57.065.522 W slot operator(): id 14 | task 191 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
0.57.065.556 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.57.181.768 I slot print_timing: id 7 | task 134 | n_decoded = 101, tg = 18.45 t/s, tg_3s = 18.45 t/s
0.58.385.458 I slot print_timing: id 7 | task 134 | prompt eval time = 123.35 ms / 4 tokens ( 30.84 ms per token, 32.43 tokens per second)
0.58.385.461 I slot print_timing: id 7 | task 134 | eval time = 6679.12 ms / 125 tokens ( 53.43 ms per token, 18.72 tokens per second)
0.58.385.462 I slot print_timing: id 7 | task 134 | total time = 6802.46 ms / 129 tokens
0.58.385.463 I slot print_timing: id 7 | task 134 | graphs reused = 1
0.58.385.466 I slot print_timing: id 7 | task 134 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
0.58.385.479 I statistics draft-mtp: #calls(b,g,a) = 49 152 2324, #gen drafts = 2332, #acc drafts = 1516, #gen tokens = 4655, #acc tokens = 2803, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.554), dur(b,g,a) = 0.026, 492.915, 2.959 ms
0.58.385.506 I slot release: id 7 | task 134 | stop processing: n_tokens = 154, truncated = 0
0.58.394.202 I srv operator(): Chat format: peg-native
0.58.504.980 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
0.58.504.982 I srv get_availabl: updating prompt cache
0.58.505.025 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
0.58.505.028 I srv alloc: - prompt is already in the cache, skipping
0.58.505.029 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
0.58.505.031 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.58.505.032 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.58.505.033 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.58.505.033 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.58.505.033 I srv get_availabl: prompt cache update took 0.05 ms
0.58.505.098 I slot launch_slot_: id 7 | task 204 | processing task, is_child = 0
0.58.509.768 I slot operator(): id 7 | task 204 | Checking checkpoint with [23, 23] against 3...
0.58.509.771 W slot operator(): id 7 | task 204 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.509.774 W slot operator(): id 7 | task 204 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
0.58.511.218 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.58.659.798 I slot create_check: id 7 | task 204 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.58.770.254 I slot print_timing: id 13 | task 111 | prompt eval time = 153.10 ms / 4 tokens ( 38.28 ms per token, 26.13 tokens per second)
0.58.770.260 I slot print_timing: id 13 | task 111 | eval time = 9440.00 ms / 128 tokens ( 73.75 ms per token, 13.56 tokens per second)
0.58.770.261 I slot print_timing: id 13 | task 111 | total time = 9593.10 ms / 132 tokens
0.58.770.261 I slot print_timing: id 13 | task 111 | graphs reused = 1
0.58.770.264 I slot print_timing: id 13 | task 111 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
0.58.770.279 I statistics draft-mtp: #calls(b,g,a) = 50 155 2362, #gen drafts = 2376, #acc drafts = 1539, #gen tokens = 4741, #acc tokens = 2847, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.554), dur(b,g,a) = 0.027, 507.337, 3.003 ms
0.58.770.305 I slot release: id 13 | task 111 | stop processing: n_tokens = 156, truncated = 0
0.58.779.461 I srv operator(): Chat format: peg-native
0.58.885.632 I slot print_timing: id 1 | task 107 | prompt eval time = 161.03 ms / 4 tokens ( 40.26 ms per token, 24.84 tokens per second)
0.58.885.638 I slot print_timing: id 1 | task 107 | eval time = 9555.63 ms / 128 tokens ( 74.65 ms per token, 13.40 tokens per second)
0.58.885.639 I slot print_timing: id 1 | task 107 | total time = 9716.66 ms / 132 tokens
0.58.885.640 I slot print_timing: id 1 | task 107 | graphs reused = 1
0.58.885.643 I slot print_timing: id 1 | task 107 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.382, 0.276)
0.58.885.663 I statistics draft-mtp: #calls(b,g,a) = 50 156 2376, #gen drafts = 2390, #acc drafts = 1549, #gen tokens = 4769, #acc tokens = 2864, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.553), dur(b,g,a) = 0.027, 511.817, 3.020 ms
0.58.885.694 I slot release: id 1 | task 107 | stop processing: n_tokens = 156, truncated = 0
0.58.887.615 I slot print_timing: id 3 | task 151 | n_decoded = 101, tg = 18.64 t/s, tg_3s = 18.64 t/s
0.58.888.190 I slot print_timing: id 4 | task 161 | n_decoded = 102, tg = 23.43 t/s, tg_3s = 23.43 t/s
0.58.888.824 I slot print_timing: id 5 | task 133 | n_decoded = 100, tg = 14.22 t/s, tg_3s = 14.22 t/s
0.58.892.708 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
0.58.892.712 I srv get_availabl: updating prompt cache
0.58.892.754 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.58.892.756 I srv alloc: - prompt is already in the cache, skipping
0.58.892.758 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.107
0.58.892.762 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.58.892.767 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.58.892.767 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.58.892.767 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.58.892.768 I srv get_availabl: prompt cache update took 0.06 ms
0.58.892.832 I slot launch_slot_: id 1 | task 208 | processing task, is_child = 0
0.58.892.833 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
0.58.892.850 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.58.892.851 I srv alloc: - prompt is already in the cache, skipping
0.58.894.526 I srv operator(): Chat format: peg-native
0.58.897.343 I slot operator(): id 1 | task 208 | Checking checkpoint with [24, 24] against 3...
0.58.897.346 W slot operator(): id 1 | task 208 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.58.897.349 W slot operator(): id 1 | task 208 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.58.898.840 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.020.941 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
0.59.020.944 I srv get_availabl: updating prompt cache
0.59.020.986 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
0.59.020.988 I srv alloc: - prompt is already in the cache, skipping
0.59.020.989 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
0.59.020.993 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.59.020.995 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.59.020.995 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.59.020.995 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.59.020.996 I srv get_availabl: prompt cache update took 0.05 ms
0.59.021.061 I slot launch_slot_: id 13 | task 210 | processing task, is_child = 0
0.59.039.136 I slot create_check: id 1 | task 208 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
0.59.039.145 I slot operator(): id 13 | task 210 | Checking checkpoint with [24, 24] against 3...
0.59.039.145 W slot operator(): id 13 | task 210 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.59.039.146 W slot operator(): id 13 | task 210 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
0.59.040.623 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.169.562 I slot print_timing: id 12 | task 165 | n_decoded = 102, tg = 23.55 t/s, tg_3s = 23.55 t/s
0.59.187.730 I slot create_check: id 13 | task 210 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
0.59.543.969 I slot print_timing: id 4 | task 161 | prompt eval time = 97.49 ms / 4 tokens ( 24.37 ms per token, 41.03 tokens per second)
0.59.543.975 I slot print_timing: id 4 | task 161 | eval time = 5009.25 ms / 114 tokens ( 43.94 ms per token, 22.76 tokens per second)
0.59.543.976 I slot print_timing: id 4 | task 161 | total time = 5106.74 ms / 118 tokens
0.59.543.977 I slot print_timing: id 4 | task 161 | graphs reused = 1
0.59.543.981 I slot print_timing: id 4 | task 161 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.59.543.996 I statistics draft-mtp: #calls(b,g,a) = 52 161 2454, #gen drafts = 2465, #acc drafts = 1601, #gen tokens = 4919, #acc tokens = 2962, #mean acc len = 2.21, #acc rate/pos = (0.652, 0.555), dur(b,g,a) = 0.029, 530.002, 3.129 ms
0.59.544.030 I slot release: id 4 | task 161 | stop processing: n_tokens = 140, truncated = 0
0.59.553.179 I srv operator(): Chat format: peg-native
0.59.665.665 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
0.59.665.667 I srv get_availabl: updating prompt cache
0.59.665.712 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.59.665.714 I srv alloc: - prompt is already in the cache, skipping
0.59.665.715 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
0.59.665.718 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.59.665.719 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.59.665.719 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.59.665.721 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.59.665.721 I srv get_availabl: prompt cache update took 0.05 ms
0.59.665.797 I slot launch_slot_: id 4 | task 216 | processing task, is_child = 0
0.59.669.890 I slot operator(): id 4 | task 216 | Checking checkpoint with [20, 20] against 3...
0.59.669.893 W slot operator(): id 4 | task 216 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.59.669.896 W slot operator(): id 4 | task 216 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.59.671.352 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
0.59.800.695 I slot print_timing: id 12 | task 165 | prompt eval time = 288.56 ms / 25 tokens ( 11.54 ms per token, 86.64 tokens per second)
0.59.800.698 I slot print_timing: id 12 | task 165 | eval time = 4961.46 ms / 114 tokens ( 43.52 ms per token, 22.98 tokens per second)
0.59.800.699 I slot print_timing: id 12 | task 165 | total time = 5250.02 ms / 139 tokens
0.59.800.699 I slot print_timing: id 12 | task 165 | graphs reused = 1
0.59.800.702 I slot print_timing: id 12 | task 165 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
0.59.800.714 I statistics draft-mtp: #calls(b,g,a) = 52 163 2492, #gen drafts = 2495, #acc drafts = 1630, #gen tokens = 4979, #acc tokens = 3014, #mean acc len = 2.21, #acc rate/pos = (0.654, 0.555), dur(b,g,a) = 0.029, 538.153, 3.181 ms
0.59.800.740 I slot release: id 12 | task 165 | stop processing: n_tokens = 140, truncated = 0
0.59.810.198 I srv operator(): Chat format: peg-native
0.59.821.155 I slot create_check: id 4 | task 216 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
0.59.934.208 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
0.59.934.211 I srv get_availabl: updating prompt cache
0.59.934.253 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
0.59.934.256 I srv alloc: - prompt is already in the cache, skipping
0.59.934.256 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
0.59.934.259 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
0.59.934.260 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
0.59.934.262 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
0.59.934.263 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
0.59.934.263 I srv get_availabl: prompt cache update took 0.05 ms
0.59.934.329 I slot launch_slot_: id 12 | task 219 | processing task, is_child = 0
0.59.938.088 I slot operator(): id 12 | task 219 | Checking checkpoint with [20, 20] against 3...
0.59.938.091 W slot operator(): id 12 | task 219 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
0.59.938.094 W slot operator(): id 12 | task 219 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
0.59.939.583 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.085.337 I slot create_check: id 12 | task 219 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.00.199.827 I slot print_timing: id 3 | task 151 | prompt eval time = 116.89 ms / 4 tokens ( 29.22 ms per token, 34.22 tokens per second)
1.00.199.830 I slot print_timing: id 3 | task 151 | eval time = 6731.52 ms / 125 tokens ( 53.85 ms per token, 18.57 tokens per second)
1.00.199.830 I slot print_timing: id 3 | task 151 | total time = 6848.41 ms / 129 tokens
1.00.199.831 I slot print_timing: id 3 | task 151 | graphs reused = 1
1.00.199.834 I slot print_timing: id 3 | task 151 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.00.199.845 I statistics draft-mtp: #calls(b,g,a) = 54 166 2528, #gen drafts = 2539, #acc drafts = 1652, #gen tokens = 5067, #acc tokens = 3052, #mean acc len = 2.21, #acc rate/pos = (0.653, 0.554), dur(b,g,a) = 0.030, 551.183, 3.234 ms
1.00.199.872 I slot release: id 3 | task 151 | stop processing: n_tokens = 154, truncated = 0
1.00.202.885 I slot print_timing: id 10 | task 164 | n_decoded = 101, tg = 18.83 t/s, tg_3s = 18.83 t/s
1.00.208.958 I srv operator(): Chat format: peg-native
1.00.318.946 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.00.318.949 I srv get_availabl: updating prompt cache
1.00.318.991 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.00.318.993 I srv alloc: - prompt is already in the cache, skipping
1.00.318.994 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.00.318.997 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.00.318.998 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.00.318.999 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.00.319.000 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.00.319.000 I srv get_availabl: prompt cache update took 0.05 ms
1.00.319.062 I slot launch_slot_: id 3 | task 223 | processing task, is_child = 0
1.00.322.858 I slot operator(): id 3 | task 223 | Checking checkpoint with [23, 23] against 3...
1.00.322.860 W slot operator(): id 3 | task 223 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.00.322.863 W slot operator(): id 3 | task 223 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.00.324.433 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.00.450.726 I slot print_timing: id 9 | task 148 | n_decoded = 100, tg = 14.07 t/s, tg_3s = 14.07 t/s
1.00.471.255 I slot create_check: id 3 | task 223 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.00.826.077 I slot print_timing: id 2 | task 181 | n_decoded = 102, tg = 23.30 t/s, tg_3s = 23.30 t/s
1.01.306.137 I slot print_timing: id 5 | task 133 | prompt eval time = 276.60 ms / 29 tokens ( 9.54 ms per token, 104.84 tokens per second)
1.01.306.140 I slot print_timing: id 5 | task 133 | eval time = 9448.17 ms / 128 tokens ( 73.81 ms per token, 13.55 tokens per second)
1.01.306.140 I slot print_timing: id 5 | task 133 | total time = 9724.77 ms / 157 tokens
1.01.306.141 I slot print_timing: id 5 | task 133 | graphs reused = 1
1.01.306.144 I slot print_timing: id 5 | task 133 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.01.306.157 I statistics draft-mtp: #calls(b,g,a) = 55 175 2664, #gen drafts = 2679, #acc drafts = 1734, #gen tokens = 5346, #acc tokens = 3203, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.551), dur(b,g,a) = 0.030, 580.459, 3.404 ms
1.01.306.182 I slot release: id 5 | task 133 | stop processing: n_tokens = 156, truncated = 0
1.01.316.379 I srv operator(): Chat format: peg-native
1.01.423.458 I slot print_timing: id 2 | task 181 | prompt eval time = 273.38 ms / 25 tokens ( 10.94 ms per token, 91.45 tokens per second)
1.01.423.462 I slot print_timing: id 2 | task 181 | eval time = 4974.38 ms / 114 tokens ( 43.63 ms per token, 22.92 tokens per second)
1.01.423.462 I slot print_timing: id 2 | task 181 | total time = 5247.77 ms / 139 tokens
1.01.423.463 I slot print_timing: id 2 | task 181 | graphs reused = 1
1.01.423.466 I slot print_timing: id 2 | task 181 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.01.423.478 I statistics draft-mtp: #calls(b,g,a) = 55 176 2682, #gen drafts = 2694, #acc drafts = 1746, #gen tokens = 5376, #acc tokens = 3225, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.551), dur(b,g,a) = 0.030, 584.935, 3.428 ms
1.01.423.507 I slot release: id 2 | task 181 | stop processing: n_tokens = 140, truncated = 0
1.01.427.062 I slot print_timing: id 10 | task 164 | prompt eval time = 290.19 ms / 28 tokens ( 10.36 ms per token, 96.49 tokens per second)
1.01.427.065 I slot print_timing: id 10 | task 164 | eval time = 6588.11 ms / 125 tokens ( 52.70 ms per token, 18.97 tokens per second)
1.01.427.066 I slot print_timing: id 10 | task 164 | total time = 6878.30 ms / 153 tokens
1.01.427.067 I slot print_timing: id 10 | task 164 | graphs reused = 1
1.01.427.069 I slot print_timing: id 10 | task 164 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.01.427.077 I statistics draft-mtp: #calls(b,g,a) = 55 176 2689, #gen drafts = 2694, #acc drafts = 1751, #gen tokens = 5376, #acc tokens = 3235, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.552), dur(b,g,a) = 0.030, 584.935, 3.435 ms
1.01.427.100 I slot release: id 10 | task 164 | stop processing: n_tokens = 154, truncated = 0
1.01.431.028 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.01.431.030 I srv get_availabl: updating prompt cache
1.01.431.085 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.01.431.090 I srv alloc: - prompt is already in the cache, skipping
1.01.431.091 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.01.431.097 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.01.431.098 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.01.431.099 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.01.431.100 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.01.431.100 I srv get_availabl: prompt cache update took 0.07 ms
1.01.431.185 I slot launch_slot_: id 5 | task 233 | processing task, is_child = 0
1.01.431.186 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.01.431.217 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.01.431.219 I srv alloc: - prompt is already in the cache, skipping
1.01.431.219 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.01.431.245 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.01.431.247 I srv alloc: - prompt is already in the cache, skipping
1.01.433.314 I srv operator(): Chat format: peg-native
1.01.435.876 I srv operator(): Chat format: peg-native
1.01.437.549 I slot operator(): id 5 | task 233 | Checking checkpoint with [24, 24] against 28...
1.01.441.278 W slot operator(): id 5 | task 233 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.01.441.344 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.547.452 I slot print_timing: id 14 | task 191 | n_decoded = 102, tg = 23.34 t/s, tg_3s = 23.34 t/s
1.01.548.105 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.01.548.107 I srv get_availabl: updating prompt cache
1.01.548.149 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.01.548.152 I srv alloc: - prompt is already in the cache, skipping
1.01.548.152 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.01.548.155 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.01.548.156 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.01.548.156 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.01.548.157 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.01.548.157 I srv get_availabl: prompt cache update took 0.05 ms
1.01.548.222 I slot launch_slot_: id 10 | task 235 | processing task, is_child = 0
1.01.548.224 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.01.548.240 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.01.548.241 I srv alloc: - prompt is already in the cache, skipping
1.01.548.243 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.01.548.243 I srv get_availabl: updating prompt cache
1.01.548.256 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.01.548.257 I srv alloc: - prompt is already in the cache, skipping
1.01.548.257 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.01.548.258 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.01.548.258 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.01.548.258 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.01.548.259 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.01.548.259 I srv get_availabl: prompt cache update took 0.01 ms
1.01.548.278 I slot launch_slot_: id 2 | task 236 | processing task, is_child = 0
1.01.553.409 I slot operator(): id 2 | task 236 | Checking checkpoint with [20, 20] against 24...
1.01.557.428 W slot operator(): id 2 | task 236 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.01.557.472 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.557.489 I slot operator(): id 10 | task 235 | Checking checkpoint with [23, 23] against 27...
1.01.561.045 W slot operator(): id 10 | task 235 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.01.561.082 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.01.796.858 I slot print_timing: id 0 | task 163 | n_decoded = 100, tg = 14.07 t/s, tg_3s = 14.07 t/s
1.01.800.699 I slot print_timing: id 8 | task 180 | n_decoded = 101, tg = 18.40 t/s, tg_3s = 18.40 t/s
1.02.164.461 I slot print_timing: id 14 | task 191 | prompt eval time = 116.18 ms / 4 tokens ( 29.05 ms per token, 34.43 tokens per second)
1.02.164.464 I slot print_timing: id 14 | task 191 | eval time = 4986.66 ms / 114 tokens ( 43.74 ms per token, 22.86 tokens per second)
1.02.164.464 I slot print_timing: id 14 | task 191 | total time = 5102.84 ms / 118 tokens
1.02.164.465 I slot print_timing: id 14 | task 191 | graphs reused = 1
1.02.164.468 I slot print_timing: id 14 | task 191 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.02.164.479 I statistics draft-mtp: #calls(b,g,a) = 58 182 2784, #gen drafts = 2785, #acc drafts = 1813, #gen tokens = 5558, #acc tokens = 3350, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.552), dur(b,g,a) = 0.032, 607.302, 3.555 ms
1.02.164.505 I slot release: id 14 | task 191 | stop processing: n_tokens = 140, truncated = 0
1.02.173.071 I srv operator(): Chat format: peg-native
1.02.280.590 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.02.280.593 I srv get_availabl: updating prompt cache
1.02.280.646 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.02.280.648 I srv alloc: - prompt is already in the cache, skipping
1.02.280.650 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.02.280.654 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.02.280.655 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.02.280.656 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.02.280.657 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.02.280.658 I srv get_availabl: prompt cache update took 0.06 ms
1.02.280.722 I slot launch_slot_: id 14 | task 243 | processing task, is_child = 0
1.02.284.260 I slot operator(): id 14 | task 243 | Checking checkpoint with [20, 20] against 3...
1.02.284.263 W slot operator(): id 14 | task 243 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.02.284.264 W slot operator(): id 14 | task 243 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.02.286.747 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.02.414.934 I slot print_timing: id 11 | task 189 | n_decoded = 101, tg = 18.83 t/s, tg_3s = 18.83 t/s
1.02.416.733 I slot print_timing: id 15 | task 170 | n_decoded = 100, tg = 14.13 t/s, tg_3s = 14.13 t/s
1.02.435.238 I slot create_check: id 14 | task 243 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.02.788.098 I slot print_timing: id 9 | task 148 | prompt eval time = 256.46 ms / 29 tokens ( 8.84 ms per token, 113.08 tokens per second)
1.02.788.106 I slot print_timing: id 9 | task 148 | eval time = 9446.72 ms / 128 tokens ( 73.80 ms per token, 13.55 tokens per second)
1.02.788.107 I slot print_timing: id 9 | task 148 | total time = 9703.18 ms / 157 tokens
1.02.788.108 I slot print_timing: id 9 | task 148 | graphs reused = 1
1.02.788.111 I slot print_timing: id 9 | task 148 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.02.788.124 I statistics draft-mtp: #calls(b,g,a) = 59 187 2846, #gen drafts = 2861, #acc drafts = 1853, #gen tokens = 5709, #acc tokens = 3422, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.551), dur(b,g,a) = 0.033, 624.842, 3.628 ms
1.02.788.150 I slot release: id 9 | task 148 | stop processing: n_tokens = 156, truncated = 0
1.02.796.703 I srv operator(): Chat format: peg-native
1.02.911.209 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
1.02.911.211 I srv get_availabl: updating prompt cache
1.02.911.255 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.02.911.257 I srv alloc: - prompt is already in the cache, skipping
1.02.911.258 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.107
1.02.911.261 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.02.911.262 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.02.911.264 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.02.911.264 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.02.911.264 I srv get_availabl: prompt cache update took 0.05 ms
1.02.911.332 I slot launch_slot_: id 9 | task 249 | processing task, is_child = 0
1.02.915.477 I slot operator(): id 9 | task 249 | Checking checkpoint with [24, 24] against 3...
1.02.915.479 W slot operator(): id 9 | task 249 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.02.915.483 W slot operator(): id 9 | task 249 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.02.917.063 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.043.944 I slot print_timing: id 8 | task 180 | prompt eval time = 129.33 ms / 4 tokens ( 32.33 ms per token, 30.93 tokens per second)
1.03.043.949 I slot print_timing: id 8 | task 180 | eval time = 6733.31 ms / 125 tokens ( 53.87 ms per token, 18.56 tokens per second)
1.03.043.950 I slot print_timing: id 8 | task 180 | total time = 6862.64 ms / 129 tokens
1.03.043.951 I slot print_timing: id 8 | task 180 | graphs reused = 1
1.03.043.953 I slot print_timing: id 8 | task 180 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.03.043.964 I statistics draft-mtp: #calls(b,g,a) = 59 189 2885, #gen drafts = 2891, #acc drafts = 1878, #gen tokens = 5769, #acc tokens = 3469, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.551), dur(b,g,a) = 0.033, 633.515, 3.681 ms
1.03.043.989 I slot release: id 8 | task 180 | stop processing: n_tokens = 154, truncated = 0
1.03.053.162 I srv operator(): Chat format: peg-native
1.03.065.879 I slot create_check: id 9 | task 249 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.03.179.508 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.03.179.511 I srv get_availabl: updating prompt cache
1.03.179.552 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.03.179.554 I srv alloc: - prompt is already in the cache, skipping
1.03.179.555 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.03.179.559 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.03.179.560 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.03.179.561 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.03.179.562 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.03.179.562 I srv get_availabl: prompt cache update took 0.05 ms
1.03.179.629 I slot launch_slot_: id 8 | task 252 | processing task, is_child = 0
1.03.184.323 I slot operator(): id 8 | task 252 | Checking checkpoint with [23, 23] against 3...
1.03.184.326 W slot operator(): id 8 | task 252 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.03.184.328 W slot operator(): id 8 | task 252 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.03.185.703 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.310.996 I slot print_timing: id 6 | task 182 | n_decoded = 100, tg = 14.28 t/s, tg_3s = 14.28 t/s
1.03.332.879 I slot create_check: id 8 | task 252 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.03.691.409 I slot print_timing: id 11 | task 189 | prompt eval time = 109.53 ms / 4 tokens ( 27.38 ms per token, 36.52 tokens per second)
1.03.691.415 I slot print_timing: id 11 | task 189 | eval time = 6640.27 ms / 125 tokens ( 53.12 ms per token, 18.82 tokens per second)
1.03.691.415 I slot print_timing: id 11 | task 189 | total time = 6749.80 ms / 129 tokens
1.03.691.417 I slot print_timing: id 11 | task 189 | graphs reused = 1
1.03.691.419 I slot print_timing: id 11 | task 189 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.03.691.433 I statistics draft-mtp: #calls(b,g,a) = 61 194 2963, #gen drafts = 2967, #acc drafts = 1930, #gen tokens = 5921, #acc tokens = 3565, #mean acc len = 2.20, #acc rate/pos = (0.651, 0.552), dur(b,g,a) = 0.035, 652.980, 3.780 ms
1.03.691.458 I slot release: id 11 | task 189 | stop processing: n_tokens = 154, truncated = 0
1.03.692.676 I slot print_timing: id 13 | task 210 | n_decoded = 102, tg = 23.22 t/s, tg_3s = 23.22 t/s
1.03.699.813 I srv operator(): Chat format: peg-native
1.03.808.263 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
1.03.808.265 I srv get_availabl: updating prompt cache
1.03.808.305 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.03.808.308 I srv alloc: - prompt is already in the cache, skipping
1.03.808.309 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
1.03.808.311 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.03.808.312 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.03.808.314 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.03.808.314 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.03.808.315 I srv get_availabl: prompt cache update took 0.05 ms
1.03.808.377 I slot launch_slot_: id 11 | task 258 | processing task, is_child = 0
1.03.812.417 I slot operator(): id 11 | task 258 | Checking checkpoint with [23, 23] against 3...
1.03.812.420 W slot operator(): id 11 | task 258 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.03.812.423 W slot operator(): id 11 | task 258 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.03.813.890 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.03.960.997 I slot create_check: id 11 | task 258 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.04.191.127 I slot print_timing: id 0 | task 163 | prompt eval time = 142.98 ms / 4 tokens ( 35.75 ms per token, 27.98 tokens per second)
1.04.191.132 I slot print_timing: id 0 | task 163 | eval time = 9503.31 ms / 128 tokens ( 74.24 ms per token, 13.47 tokens per second)
1.04.191.133 I slot print_timing: id 0 | task 163 | total time = 9646.29 ms / 132 tokens
1.04.191.134 I slot print_timing: id 0 | task 163 | graphs reused = 1
1.04.191.136 I slot print_timing: id 0 | task 163 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.04.191.149 I statistics draft-mtp: #calls(b,g,a) = 62 198 3012, #gen drafts = 3027, #acc drafts = 1956, #gen tokens = 6040, #acc tokens = 3613, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.550), dur(b,g,a) = 0.036, 667.714, 3.842 ms
1.04.191.174 I slot release: id 0 | task 163 | stop processing: n_tokens = 156, truncated = 0
1.04.200.089 I srv operator(): Chat format: peg-native
1.04.311.261 I slot print_timing: id 13 | task 210 | prompt eval time = 260.52 ms / 25 tokens ( 10.42 ms per token, 95.96 tokens per second)
1.04.311.264 I slot print_timing: id 13 | task 210 | eval time = 5011.58 ms / 114 tokens ( 43.96 ms per token, 22.75 tokens per second)
1.04.311.265 I slot print_timing: id 13 | task 210 | total time = 5272.09 ms / 139 tokens
1.04.311.266 I slot print_timing: id 13 | task 210 | graphs reused = 1
1.04.311.269 I slot print_timing: id 13 | task 210 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.04.311.283 I statistics draft-mtp: #calls(b,g,a) = 62 199 3040, #gen drafts = 3042, #acc drafts = 1977, #gen tokens = 6070, #acc tokens = 3650, #mean acc len = 2.20, #acc rate/pos = (0.650, 0.550), dur(b,g,a) = 0.036, 670.425, 3.882 ms
1.04.311.307 I slot release: id 13 | task 210 | stop processing: n_tokens = 140, truncated = 0
1.04.312.555 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.019
1.04.312.557 I srv get_availabl: updating prompt cache
1.04.312.601 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.312.603 I srv alloc: - prompt is already in the cache, skipping
1.04.312.604 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.107
1.04.312.606 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.04.312.607 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.04.312.607 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.04.312.608 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.04.312.609 I srv get_availabl: prompt cache update took 0.05 ms
1.04.312.675 I slot launch_slot_: id 0 | task 263 | processing task, is_child = 0
1.04.312.678 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
1.04.312.694 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.04.312.696 I srv alloc: - prompt is already in the cache, skipping
1.04.316.275 I slot operator(): id 0 | task 263 | Checking checkpoint with [24, 24] against 3...
1.04.316.278 W slot operator(): id 0 | task 263 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.04.316.280 W slot operator(): id 0 | task 263 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.04.317.761 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.320.158 I srv operator(): Chat format: peg-native
1.04.439.310 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.04.439.313 I srv get_availabl: updating prompt cache
1.04.439.354 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.04.439.356 I srv alloc: - prompt is already in the cache, skipping
1.04.439.357 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.04.439.360 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.04.439.362 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.04.439.362 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.04.439.363 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.04.439.364 I srv get_availabl: prompt cache update took 0.05 ms
1.04.439.429 I slot launch_slot_: id 13 | task 265 | processing task, is_child = 0
1.04.456.267 I slot create_check: id 0 | task 263 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.04.456.274 I slot operator(): id 13 | task 265 | Checking checkpoint with [20, 20] against 24...
1.04.459.865 W slot operator(): id 13 | task 265 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.04.459.914 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.04.692.882 I slot print_timing: id 1 | task 208 | n_decoded = 101, tg = 18.27 t/s, tg_3s = 18.27 t/s
1.04.810.942 I slot print_timing: id 15 | task 170 | prompt eval time = 259.81 ms / 29 tokens ( 8.96 ms per token, 111.62 tokens per second)
1.04.810.946 I slot print_timing: id 15 | task 170 | eval time = 9469.54 ms / 128 tokens ( 73.98 ms per token, 13.52 tokens per second)
1.04.810.946 I slot print_timing: id 15 | task 170 | total time = 9729.35 ms / 157 tokens
1.04.810.947 I slot print_timing: id 15 | task 170 | graphs reused = 1
1.04.810.950 I slot print_timing: id 15 | task 170 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.04.810.966 I statistics draft-mtp: #calls(b,g,a) = 64 203 3086, #gen drafts = 3101, #acc drafts = 2004, #gen tokens = 6187, #acc tokens = 3702, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.550), dur(b,g,a) = 0.037, 682.968, 3.937 ms
1.04.810.993 I slot release: id 15 | task 170 | stop processing: n_tokens = 156, truncated = 0
1.04.820.088 I srv operator(): Chat format: peg-native
1.04.925.739 I slot print_timing: id 3 | task 223 | n_decoded = 102, tg = 23.49 t/s, tg_3s = 23.49 t/s
1.04.930.783 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.04.930.785 I srv get_availabl: updating prompt cache
1.04.930.827 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.04.930.830 I srv alloc: - prompt is already in the cache, skipping
1.04.930.831 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.04.930.833 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.04.930.834 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.04.930.835 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.04.930.836 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.04.930.837 I srv get_availabl: prompt cache update took 0.05 ms
1.04.930.902 I slot launch_slot_: id 15 | task 270 | processing task, is_child = 0
1.04.933.664 I slot operator(): id 15 | task 270 | Checking checkpoint with [24, 24] against 28...
1.04.937.618 W slot operator(): id 15 | task 270 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.04.937.655 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.533.705 I slot print_timing: id 3 | task 223 | prompt eval time = 260.42 ms / 25 tokens ( 10.42 ms per token, 96.00 tokens per second)
1.05.533.712 I slot print_timing: id 3 | task 223 | eval time = 4950.40 ms / 114 tokens ( 43.42 ms per token, 23.03 tokens per second)
1.05.533.712 I slot print_timing: id 3 | task 223 | total time = 5210.83 ms / 139 tokens
1.05.533.714 I slot print_timing: id 3 | task 223 | graphs reused = 1
1.05.533.717 I slot print_timing: id 3 | task 223 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.05.533.730 I statistics draft-mtp: #calls(b,g,a) = 65 209 3183, #gen drafts = 3195, #acc drafts = 2065, #gen tokens = 6374, #acc tokens = 3814, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.037, 699.777, 4.056 ms
1.05.533.753 I slot release: id 3 | task 223 | stop processing: n_tokens = 140, truncated = 0
1.05.537.417 I slot print_timing: id 12 | task 219 | n_decoded = 101, tg = 18.92 t/s, tg_3s = 18.92 t/s
1.05.542.660 I srv operator(): Chat format: peg-native
1.05.645.853 I slot print_timing: id 6 | task 182 | prompt eval time = 133.11 ms / 4 tokens ( 33.28 ms per token, 30.05 tokens per second)
1.05.645.858 I slot print_timing: id 6 | task 182 | eval time = 9335.49 ms / 128 tokens ( 72.93 ms per token, 13.71 tokens per second)
1.05.645.858 I slot print_timing: id 6 | task 182 | total time = 9468.60 ms / 132 tokens
1.05.645.859 I slot print_timing: id 6 | task 182 | graphs reused = 1
1.05.645.862 I slot print_timing: id 6 | task 182 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.05.645.874 I statistics draft-mtp: #calls(b,g,a) = 65 210 3195, #gen drafts = 3209, #acc drafts = 2070, #gen tokens = 6402, #acc tokens = 3823, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.037, 704.359, 4.070 ms
1.05.645.900 I slot release: id 6 | task 182 | stop processing: n_tokens = 156, truncated = 0
1.05.652.961 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.05.652.963 I srv get_availabl: updating prompt cache
1.05.653.017 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.05.653.020 I srv alloc: - prompt is already in the cache, skipping
1.05.653.021 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.05.653.026 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.05.653.027 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.05.653.028 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.05.653.030 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.05.653.031 I srv get_availabl: prompt cache update took 0.07 ms
1.05.653.095 I slot launch_slot_: id 3 | task 277 | processing task, is_child = 0
1.05.653.097 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
1.05.653.116 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.05.653.117 I srv alloc: - prompt is already in the cache, skipping
1.05.654.631 I srv operator(): Chat format: peg-native
1.05.657.704 I slot operator(): id 3 | task 277 | Checking checkpoint with [20, 20] against 3...
1.05.657.707 W slot operator(): id 3 | task 277 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.05.657.709 W slot operator(): id 3 | task 277 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.05.659.304 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.782.721 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.05.782.723 I srv get_availabl: updating prompt cache
1.05.782.782 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.05.782.785 I srv alloc: - prompt is already in the cache, skipping
1.05.782.787 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.05.782.790 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.05.782.791 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.05.782.793 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.05.782.794 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.05.782.794 I srv get_availabl: prompt cache update took 0.07 ms
1.05.782.862 I slot launch_slot_: id 6 | task 279 | processing task, is_child = 0
1.05.801.114 I slot create_check: id 3 | task 277 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.05.801.121 I slot operator(): id 6 | task 279 | Checking checkpoint with [24, 24] against 3...
1.05.801.122 W slot operator(): id 6 | task 279 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.05.801.122 W slot operator(): id 6 | task 279 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.05.802.607 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.05.927.704 I slot print_timing: id 1 | task 208 | prompt eval time = 266.55 ms / 28 tokens ( 9.52 ms per token, 105.05 tokens per second)
1.05.927.710 I slot print_timing: id 1 | task 208 | eval time = 6763.79 ms / 125 tokens ( 54.11 ms per token, 18.48 tokens per second)
1.05.927.710 I slot print_timing: id 1 | task 208 | total time = 7030.34 ms / 153 tokens
1.05.927.711 I slot print_timing: id 1 | task 208 | graphs reused = 1
1.05.927.715 I slot print_timing: id 1 | task 208 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.05.927.731 I statistics draft-mtp: #calls(b,g,a) = 66 212 3225, #gen drafts = 3237, #acc drafts = 2093, #gen tokens = 6458, #acc tokens = 3867, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.550), dur(b,g,a) = 0.038, 713.535, 4.106 ms
1.05.927.757 I slot release: id 1 | task 208 | stop processing: n_tokens = 154, truncated = 0
1.05.929.377 I slot print_timing: id 7 | task 204 | n_decoded = 100, tg = 13.97 t/s, tg_3s = 13.97 t/s
1.05.935.797 I srv operator(): Chat format: peg-native
1.05.952.078 I slot create_check: id 6 | task 279 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.06.059.219 I slot print_timing: id 2 | task 236 | n_decoded = 102, tg = 23.27 t/s, tg_3s = 23.27 t/s
1.06.064.817 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
1.06.064.819 I srv get_availabl: updating prompt cache
1.06.064.873 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.06.064.876 I srv alloc: - prompt is already in the cache, skipping
1.06.064.878 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
1.06.064.881 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.06.064.883 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.06.064.884 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.06.064.884 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.06.064.885 I srv get_availabl: prompt cache update took 0.07 ms
1.06.064.952 I slot launch_slot_: id 1 | task 282 | processing task, is_child = 0
1.06.068.596 I slot operator(): id 1 | task 282 | Checking checkpoint with [23, 23] against 3...
1.06.068.599 W slot operator(): id 1 | task 282 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.06.068.602 W slot operator(): id 1 | task 282 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.06.070.107 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.217.499 I slot create_check: id 1 | task 282 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.06.693.569 I slot print_timing: id 2 | task 236 | prompt eval time = 121.61 ms / 4 tokens ( 30.40 ms per token, 32.89 tokens per second)
1.06.693.573 I slot print_timing: id 2 | task 236 | eval time = 5018.53 ms / 114 tokens ( 44.02 ms per token, 22.72 tokens per second)
1.06.693.574 I slot print_timing: id 2 | task 236 | total time = 5140.14 ms / 118 tokens
1.06.693.574 I slot print_timing: id 2 | task 236 | graphs reused = 1
1.06.693.577 I slot print_timing: id 2 | task 236 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.06.693.589 I statistics draft-mtp: #calls(b,g,a) = 68 218 3316, #gen drafts = 3329, #acc drafts = 2155, #gen tokens = 6642, #acc tokens = 3980, #mean acc len = 2.20, #acc rate/pos = (0.650, 0.550), dur(b,g,a) = 0.039, 734.305, 4.217 ms
1.06.693.615 I slot release: id 2 | task 236 | stop processing: n_tokens = 140, truncated = 0
1.06.703.889 I srv operator(): Chat format: peg-native
1.06.813.996 I slot print_timing: id 12 | task 219 | prompt eval time = 259.72 ms / 28 tokens ( 9.28 ms per token, 107.81 tokens per second)
1.06.814.000 I slot print_timing: id 12 | task 219 | eval time = 6616.17 ms / 125 tokens ( 52.93 ms per token, 18.89 tokens per second)
1.06.814.000 I slot print_timing: id 12 | task 219 | total time = 6875.89 ms / 153 tokens
1.06.814.001 I slot print_timing: id 12 | task 219 | graphs reused = 1
1.06.814.004 I slot print_timing: id 12 | task 219 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.06.814.016 I statistics draft-mtp: #calls(b,g,a) = 68 219 3341, #gen drafts = 3344, #acc drafts = 2173, #gen tokens = 6672, #acc tokens = 4014, #mean acc len = 2.20, #acc rate/pos = (0.650, 0.551), dur(b,g,a) = 0.039, 738.018, 4.246 ms
1.06.814.045 I slot release: id 12 | task 219 | stop processing: n_tokens = 154, truncated = 0
1.06.815.257 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.06.815.259 I srv get_availabl: updating prompt cache
1.06.815.300 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.06.815.302 I srv alloc: - prompt is already in the cache, skipping
1.06.815.303 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.06.815.306 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.06.815.307 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.06.815.308 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.06.815.309 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.06.815.309 I srv get_availabl: prompt cache update took 0.05 ms
1.06.815.383 I slot launch_slot_: id 12 | task 289 | processing task, is_child = 0
1.06.815.385 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.06.815.410 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.06.815.412 I srv alloc: - prompt is already in the cache, skipping
1.06.820.478 I slot operator(): id 12 | task 289 | Checking checkpoint with [23, 23] against 27...
1.06.824.798 W slot operator(): id 12 | task 289 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.06.824.841 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.06.825.597 I srv operator(): Chat format: peg-native
1.06.937.338 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.06.937.340 I srv get_availabl: updating prompt cache
1.06.937.381 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.06.937.383 I srv alloc: - prompt is already in the cache, skipping
1.06.937.384 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.06.937.387 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.06.937.388 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.06.937.389 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.06.937.389 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.06.937.391 I srv get_availabl: prompt cache update took 0.05 ms
1.06.937.456 I slot launch_slot_: id 2 | task 291 | processing task, is_child = 0
1.06.941.189 I slot operator(): id 2 | task 291 | Checking checkpoint with [20, 20] against 24...
1.06.945.061 W slot operator(): id 2 | task 291 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.06.945.094 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.07.059.897 I slot print_timing: id 4 | task 216 | n_decoded = 100, tg = 14.02 t/s, tg_3s = 14.02 t/s
1.07.062.523 I slot print_timing: id 10 | task 235 | n_decoded = 101, tg = 18.75 t/s, tg_3s = 18.75 t/s
1.07.784.265 I slot print_timing: id 8 | task 252 | n_decoded = 102, tg = 23.51 t/s, tg_3s = 23.51 t/s
1.08.262.900 I slot print_timing: id 7 | task 204 | prompt eval time = 260.25 ms / 29 tokens ( 8.97 ms per token, 111.43 tokens per second)
1.08.262.907 I slot print_timing: id 7 | task 204 | eval time = 9492.85 ms / 128 tokens ( 74.16 ms per token, 13.48 tokens per second)
1.08.262.907 I slot print_timing: id 7 | task 204 | total time = 9753.11 ms / 157 tokens
1.08.262.908 I slot print_timing: id 7 | task 204 | graphs reused = 1
1.08.262.912 I slot print_timing: id 7 | task 204 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.08.262.926 I statistics draft-mtp: #calls(b,g,a) = 70 231 3517, #gen drafts = 3532, #acc drafts = 2283, #gen tokens = 7047, #acc tokens = 4215, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.041, 776.465, 4.454 ms
1.08.262.951 I slot release: id 7 | task 204 | stop processing: n_tokens = 156, truncated = 0
1.08.267.874 I slot print_timing: id 10 | task 235 | prompt eval time = 117.80 ms / 4 tokens ( 29.45 ms per token, 33.96 tokens per second)
1.08.267.876 I slot print_timing: id 10 | task 235 | eval time = 6592.57 ms / 125 tokens ( 52.74 ms per token, 18.96 tokens per second)
1.08.267.877 I slot print_timing: id 10 | task 235 | total time = 6710.37 ms / 129 tokens
1.08.267.877 I slot print_timing: id 10 | task 235 | graphs reused = 1
1.08.267.880 I slot print_timing: id 10 | task 235 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.08.267.888 I statistics draft-mtp: #calls(b,g,a) = 70 231 3527, #gen drafts = 3532, #acc drafts = 2290, #gen tokens = 7047, #acc tokens = 4228, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.041, 776.465, 4.468 ms
1.08.267.909 I slot release: id 10 | task 235 | stop processing: n_tokens = 154, truncated = 0
1.08.271.855 I srv operator(): Chat format: peg-native
1.08.275.979 I srv operator(): Chat format: peg-native
1.08.376.611 I slot print_timing: id 8 | task 252 | prompt eval time = 260.94 ms / 25 tokens ( 10.44 ms per token, 95.81 tokens per second)
1.08.376.614 I slot print_timing: id 8 | task 252 | eval time = 4931.33 ms / 114 tokens ( 43.26 ms per token, 23.12 tokens per second)
1.08.376.615 I slot print_timing: id 8 | task 252 | total time = 5192.27 ms / 139 tokens
1.08.376.616 I slot print_timing: id 8 | task 252 | graphs reused = 1
1.08.376.619 I slot print_timing: id 8 | task 252 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.08.376.630 I statistics draft-mtp: #calls(b,g,a) = 70 232 3540, #gen drafts = 3546, #acc drafts = 2298, #gen tokens = 7075, #acc tokens = 4243, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.041, 781.612, 4.485 ms
1.08.376.654 I slot release: id 8 | task 252 | stop processing: n_tokens = 140, truncated = 0
1.08.379.782 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.08.379.784 I srv get_availabl: updating prompt cache
1.08.379.827 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.08.379.831 I srv alloc: - prompt is already in the cache, skipping
1.08.379.832 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.08.379.834 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.08.379.835 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.08.379.836 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.08.379.836 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.08.379.836 I srv get_availabl: prompt cache update took 0.05 ms
1.08.379.903 I slot launch_slot_: id 7 | task 304 | processing task, is_child = 0
1.08.379.904 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
1.08.379.919 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.08.379.921 I srv alloc: - prompt is already in the cache, skipping
1.08.379.921 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.08.379.936 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.08.379.938 I srv alloc: - prompt is already in the cache, skipping
1.08.379.940 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.08.379.941 I srv get_availabl: updating prompt cache
1.08.379.955 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.08.379.956 I srv alloc: - prompt is already in the cache, skipping
1.08.379.956 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.08.379.957 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.08.379.957 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.08.379.957 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.08.379.958 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.08.379.958 I srv get_availabl: prompt cache update took 0.02 ms
1.08.379.975 I slot launch_slot_: id 10 | task 305 | processing task, is_child = 0
1.08.379.976 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
1.08.379.989 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.08.379.990 I srv alloc: - prompt is already in the cache, skipping
1.08.385.012 I slot operator(): id 7 | task 304 | Checking checkpoint with [24, 24] against 28...
1.08.388.168 I srv operator(): Chat format: peg-native
1.08.389.058 W slot operator(): id 7 | task 304 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.08.389.099 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.389.117 I slot operator(): id 10 | task 305 | Checking checkpoint with [23, 23] against 27...
1.08.392.746 W slot operator(): id 10 | task 305 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.08.392.782 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.503.720 I slot print_timing: id 5 | task 233 | n_decoded = 100, tg = 14.36 t/s, tg_3s = 14.36 t/s
1.08.504.939 I slot print_timing: id 9 | task 249 | n_decoded = 101, tg = 18.94 t/s, tg_3s = 18.94 t/s
1.08.507.352 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.08.507.354 I srv get_availabl: updating prompt cache
1.08.507.398 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.08.507.399 I srv alloc: - prompt is already in the cache, skipping
1.08.507.400 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.08.507.403 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.08.507.404 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.08.507.405 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.08.507.405 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.08.507.405 I srv get_availabl: prompt cache update took 0.05 ms
1.08.507.470 I slot launch_slot_: id 8 | task 307 | processing task, is_child = 0
1.08.512.253 I slot operator(): id 8 | task 307 | Checking checkpoint with [20, 20] against 24...
1.08.516.212 W slot operator(): id 8 | task 307 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.08.516.247 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.08.876.262 I slot print_timing: id 13 | task 265 | n_decoded = 102, tg = 23.71 t/s, tg_3s = 23.71 t/s
1.09.352.890 I slot print_timing: id 4 | task 216 | prompt eval time = 257.49 ms / 29 tokens ( 8.88 ms per token, 112.63 tokens per second)
1.09.352.897 I slot print_timing: id 4 | task 216 | eval time = 9425.48 ms / 128 tokens ( 73.64 ms per token, 13.58 tokens per second)
1.09.352.897 I slot print_timing: id 4 | task 216 | total time = 9682.97 ms / 157 tokens
1.09.352.898 I slot print_timing: id 4 | task 216 | graphs reused = 1
1.09.352.902 I slot print_timing: id 4 | task 216 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.09.352.915 I statistics draft-mtp: #calls(b,g,a) = 73 240 3654, #gen drafts = 3669, #acc drafts = 2373, #gen tokens = 7320, #acc tokens = 4383, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.550), dur(b,g,a) = 0.043, 809.503, 4.624 ms
1.09.352.940 I slot release: id 4 | task 216 | stop processing: n_tokens = 156, truncated = 0
1.09.362.227 I srv operator(): Chat format: peg-native
1.09.474.232 I slot print_timing: id 13 | task 265 | prompt eval time = 117.18 ms / 4 tokens ( 29.30 ms per token, 34.14 tokens per second)
1.09.474.237 I slot print_timing: id 13 | task 265 | eval time = 4900.76 ms / 114 tokens ( 42.99 ms per token, 23.26 tokens per second)
1.09.474.238 I slot print_timing: id 13 | task 265 | total time = 5017.94 ms / 118 tokens
1.09.474.239 I slot print_timing: id 13 | task 265 | graphs reused = 1
1.09.474.242 I slot print_timing: id 13 | task 265 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.09.474.252 I statistics draft-mtp: #calls(b,g,a) = 73 241 3682, #gen drafts = 3684, #acc drafts = 2392, #gen tokens = 7350, #acc tokens = 4416, #mean acc len = 2.20, #acc rate/pos = (0.650, 0.550), dur(b,g,a) = 0.043, 813.601, 4.665 ms
1.09.474.278 I slot release: id 13 | task 265 | stop processing: n_tokens = 140, truncated = 0
1.09.475.363 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.09.475.365 I srv get_availabl: updating prompt cache
1.09.475.409 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.09.475.411 I srv alloc: - prompt is already in the cache, skipping
1.09.475.412 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.09.475.415 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.09.475.416 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.09.475.417 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.09.475.418 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.09.475.419 I srv get_availabl: prompt cache update took 0.05 ms
1.09.475.483 I slot launch_slot_: id 4 | task 316 | processing task, is_child = 0
1.09.475.485 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
1.09.475.501 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.09.475.503 I srv alloc: - prompt is already in the cache, skipping
1.09.480.919 I slot operator(): id 4 | task 316 | Checking checkpoint with [24, 24] against 28...
1.09.483.221 I srv operator(): Chat format: peg-native
1.09.484.807 W slot operator(): id 4 | task 316 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.09.484.849 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.597.615 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.09.597.618 I srv get_availabl: updating prompt cache
1.09.597.658 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.09.597.662 I srv alloc: - prompt is already in the cache, skipping
1.09.597.663 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.09.597.666 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.09.597.668 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.09.597.668 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.09.597.668 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.09.597.669 I srv get_availabl: prompt cache update took 0.05 ms
1.09.597.735 I slot launch_slot_: id 13 | task 318 | processing task, is_child = 0
1.09.601.429 I slot operator(): id 13 | task 318 | Checking checkpoint with [20, 20] against 3...
1.09.601.432 W slot operator(): id 13 | task 318 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.09.601.435 W slot operator(): id 13 | task 318 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.09.602.890 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.730.090 I slot print_timing: id 9 | task 249 | prompt eval time = 256.00 ms / 28 tokens ( 9.14 ms per token, 109.38 tokens per second)
1.09.730.093 I slot print_timing: id 9 | task 249 | eval time = 6558.60 ms / 125 tokens ( 52.47 ms per token, 19.06 tokens per second)
1.09.730.094 I slot print_timing: id 9 | task 249 | total time = 6814.59 ms / 153 tokens
1.09.730.095 I slot print_timing: id 9 | task 249 | graphs reused = 1
1.09.730.099 I slot print_timing: id 9 | task 249 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.09.730.112 I statistics draft-mtp: #calls(b,g,a) = 74 243 3708, #gen drafts = 3713, #acc drafts = 2409, #gen tokens = 7408, #acc tokens = 4449, #mean acc len = 2.20, #acc rate/pos = (0.650, 0.550), dur(b,g,a) = 0.044, 822.643, 4.696 ms
1.09.730.136 I slot release: id 9 | task 249 | stop processing: n_tokens = 154, truncated = 0
1.09.731.923 I slot print_timing: id 14 | task 243 | n_decoded = 100, tg = 13.92 t/s, tg_3s = 13.92 t/s
1.09.738.951 I srv operator(): Chat format: peg-native
1.09.750.063 I slot create_check: id 13 | task 318 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.09.862.066 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.09.862.069 I srv get_availabl: updating prompt cache
1.09.862.111 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.09.862.113 I srv alloc: - prompt is already in the cache, skipping
1.09.862.114 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.09.862.116 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.09.862.118 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.09.862.118 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.09.862.119 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.09.862.119 I srv get_availabl: prompt cache update took 0.05 ms
1.09.862.184 I slot launch_slot_: id 9 | task 321 | processing task, is_child = 0
1.09.866.391 I slot operator(): id 9 | task 321 | Checking checkpoint with [23, 23] against 3...
1.09.866.394 W slot operator(): id 9 | task 321 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.09.866.397 W slot operator(): id 9 | task 321 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.09.868.163 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.09.990.609 I slot print_timing: id 0 | task 263 | n_decoded = 101, tg = 18.64 t/s, tg_3s = 18.64 t/s
1.10.015.343 I slot create_check: id 9 | task 321 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.10.372.299 I slot print_timing: id 6 | task 279 | n_decoded = 102, tg = 23.64 t/s, tg_3s = 23.64 t/s
1.10.851.680 I slot print_timing: id 5 | task 233 | prompt eval time = 103.81 ms / 4 tokens ( 25.95 ms per token, 38.53 tokens per second)
1.10.851.687 I slot print_timing: id 5 | task 233 | eval time = 9310.29 ms / 128 tokens ( 72.74 ms per token, 13.75 tokens per second)
1.10.851.688 I slot print_timing: id 5 | task 233 | total time = 9414.10 ms / 132 tokens
1.10.851.689 I slot print_timing: id 5 | task 233 | graphs reused = 1
1.10.851.692 I slot print_timing: id 5 | task 233 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.10.851.707 I statistics draft-mtp: #calls(b,g,a) = 76 252 3837, #gen drafts = 3852, #acc drafts = 2488, #gen tokens = 7685, #acc tokens = 4593, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.046, 853.853, 4.852 ms
1.10.851.734 I slot release: id 5 | task 233 | stop processing: n_tokens = 156, truncated = 0
1.10.860.622 I srv operator(): Chat format: peg-native
1.10.970.471 I slot print_timing: id 6 | task 279 | prompt eval time = 256.77 ms / 25 tokens ( 10.27 ms per token, 97.36 tokens per second)
1.10.970.474 I slot print_timing: id 6 | task 279 | eval time = 4912.56 ms / 114 tokens ( 43.09 ms per token, 23.21 tokens per second)
1.10.970.475 I slot print_timing: id 6 | task 279 | total time = 5169.33 ms / 139 tokens
1.10.970.476 I slot print_timing: id 6 | task 279 | graphs reused = 1
1.10.970.479 I slot print_timing: id 6 | task 279 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.10.970.489 I statistics draft-mtp: #calls(b,g,a) = 76 253 3858, #gen drafts = 3867, #acc drafts = 2502, #gen tokens = 7715, #acc tokens = 4618, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.548), dur(b,g,a) = 0.046, 858.273, 4.883 ms
1.10.970.512 I slot release: id 6 | task 279 | stop processing: n_tokens = 140, truncated = 0
1.10.974.949 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.10.974.951 I srv get_availabl: updating prompt cache
1.10.974.995 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.10.974.997 I srv alloc: - prompt is already in the cache, skipping
1.10.974.998 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.10.975.001 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.10.975.002 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.10.975.004 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.10.975.004 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.10.975.004 I srv get_availabl: prompt cache update took 0.05 ms
1.10.975.069 I slot launch_slot_: id 5 | task 331 | processing task, is_child = 0
1.10.975.071 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
1.10.975.086 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.10.975.087 I srv alloc: - prompt is already in the cache, skipping
1.10.979.000 I slot operator(): id 5 | task 331 | Checking checkpoint with [24, 24] against 28...
1.10.979.470 I srv operator(): Chat format: peg-native
1.10.982.945 W slot operator(): id 5 | task 331 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.10.982.969 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.094.660 I slot print_timing: id 11 | task 258 | n_decoded = 100, tg = 14.24 t/s, tg_3s = 14.24 t/s
1.11.096.287 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.11.096.289 I srv get_availabl: updating prompt cache
1.11.096.331 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.11.096.333 I srv alloc: - prompt is already in the cache, skipping
1.11.096.334 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.11.096.336 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.11.096.337 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.11.096.338 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.11.096.338 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.11.096.339 I srv get_availabl: prompt cache update took 0.05 ms
1.11.096.405 I slot launch_slot_: id 6 | task 333 | processing task, is_child = 0
1.11.100.602 I slot operator(): id 6 | task 333 | Checking checkpoint with [20, 20] against 3...
1.11.100.605 W slot operator(): id 6 | task 333 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.11.100.608 W slot operator(): id 6 | task 333 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.11.102.264 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.225.157 I slot print_timing: id 0 | task 263 | prompt eval time = 256.90 ms / 28 tokens ( 9.17 ms per token, 108.99 tokens per second)
1.11.225.164 I slot print_timing: id 0 | task 263 | eval time = 6651.95 ms / 125 tokens ( 53.22 ms per token, 18.79 tokens per second)
1.11.225.165 I slot print_timing: id 0 | task 263 | total time = 6908.85 ms / 153 tokens
1.11.225.166 I slot print_timing: id 0 | task 263 | graphs reused = 1
1.11.225.169 I slot print_timing: id 0 | task 263 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.11.225.184 I statistics draft-mtp: #calls(b,g,a) = 77 255 3882, #gen drafts = 3896, #acc drafts = 2519, #gen tokens = 7773, #acc tokens = 4650, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.046, 866.317, 4.914 ms
1.11.225.209 I slot release: id 0 | task 263 | stop processing: n_tokens = 154, truncated = 0
1.11.226.585 I slot print_timing: id 3 | task 277 | n_decoded = 101, tg = 19.06 t/s, tg_3s = 19.06 t/s
1.11.233.961 I srv operator(): Chat format: peg-native
1.11.249.992 I slot create_check: id 6 | task 333 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.11.356.906 I slot print_timing: id 2 | task 291 | n_decoded = 102, tg = 23.73 t/s, tg_3s = 23.73 t/s
1.11.361.863 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.11.361.865 I srv get_availabl: updating prompt cache
1.11.361.905 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.11.361.908 I srv alloc: - prompt is already in the cache, skipping
1.11.361.909 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.11.361.912 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.11.361.913 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.11.361.913 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.11.361.914 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.11.361.914 I srv get_availabl: prompt cache update took 0.05 ms
1.11.361.981 I slot launch_slot_: id 0 | task 336 | processing task, is_child = 0
1.11.364.697 I slot operator(): id 0 | task 336 | Checking checkpoint with [23, 23] against 3...
1.11.364.699 W slot operator(): id 0 | task 336 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.11.364.702 W slot operator(): id 0 | task 336 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.11.366.269 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.11.511.002 I slot create_check: id 0 | task 336 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.11.986.805 I slot print_timing: id 2 | task 291 | prompt eval time = 116.54 ms / 4 tokens ( 29.13 ms per token, 34.32 tokens per second)
1.11.986.810 I slot print_timing: id 2 | task 291 | eval time = 4929.06 ms / 114 tokens ( 43.24 ms per token, 23.13 tokens per second)
1.11.986.810 I slot print_timing: id 2 | task 291 | total time = 5045.60 ms / 118 tokens
1.11.986.811 I slot print_timing: id 2 | task 291 | graphs reused = 1
1.11.986.813 I slot print_timing: id 2 | task 291 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.11.986.825 I statistics draft-mtp: #calls(b,g,a) = 79 261 3975, #gen drafts = 3988, #acc drafts = 2578, #gen tokens = 7956, #acc tokens = 4759, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.048, 884.141, 5.030 ms
1.11.986.850 I slot release: id 2 | task 291 | stop processing: n_tokens = 140, truncated = 0
1.11.993.382 I slot print_timing: id 15 | task 270 | n_decoded = 100, tg = 14.40 t/s, tg_3s = 14.40 t/s
1.11.996.055 I srv operator(): Chat format: peg-native
1.12.101.213 I slot print_timing: id 14 | task 243 | prompt eval time = 263.06 ms / 29 tokens ( 9.07 ms per token, 110.24 tokens per second)
1.12.101.219 I slot print_timing: id 14 | task 243 | eval time = 9553.86 ms / 128 tokens ( 74.64 ms per token, 13.40 tokens per second)
1.12.101.220 I slot print_timing: id 14 | task 243 | total time = 9816.92 ms / 157 tokens
1.12.101.221 I slot print_timing: id 14 | task 243 | graphs reused = 1
1.12.101.224 I slot print_timing: id 14 | task 243 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.382, 0.276)
1.12.101.242 I statistics draft-mtp: #calls(b,g,a) = 79 262 3988, #gen drafts = 4002, #acc drafts = 2588, #gen tokens = 7984, #acc tokens = 4777, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.048, 888.672, 5.047 ms
1.12.101.267 I slot release: id 14 | task 243 | stop processing: n_tokens = 156, truncated = 0
1.12.108.172 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.12.108.176 I srv get_availabl: updating prompt cache
1.12.108.218 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.12.108.220 I srv alloc: - prompt is already in the cache, skipping
1.12.108.221 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.12.108.223 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.12.108.224 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.12.108.225 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.12.108.227 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.12.108.227 I srv get_availabl: prompt cache update took 0.05 ms
1.12.108.291 I slot launch_slot_: id 14 | task 343 | processing task, is_child = 0
1.12.108.292 I slot process_sing: id 2 | task -1 | saving idle slot to prompt cache
1.12.108.310 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.12.108.311 I srv alloc: - prompt is already in the cache, skipping
1.12.111.046 I srv operator(): Chat format: peg-native
1.12.112.808 I slot operator(): id 14 | task 343 | Checking checkpoint with [24, 24] against 28...
1.12.116.730 W slot operator(): id 14 | task 343 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.12.116.780 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.228.480 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.12.228.482 I srv get_availabl: updating prompt cache
1.12.228.525 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.12.228.527 I srv alloc: - prompt is already in the cache, skipping
1.12.228.528 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.12.228.531 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.12.228.532 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.12.228.532 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.12.228.534 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.12.228.534 I srv get_availabl: prompt cache update took 0.05 ms
1.12.228.599 I slot launch_slot_: id 2 | task 345 | processing task, is_child = 0
1.12.232.308 I slot operator(): id 2 | task 345 | Checking checkpoint with [20, 20] against 3...
1.12.232.310 W slot operator(): id 2 | task 345 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.12.232.312 W slot operator(): id 2 | task 345 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.12.233.993 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.361.963 I slot print_timing: id 12 | task 289 | n_decoded = 101, tg = 18.60 t/s, tg_3s = 18.60 t/s
1.12.381.536 I slot create_check: id 2 | task 345 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.12.495.542 I slot print_timing: id 3 | task 277 | prompt eval time = 268.69 ms / 28 tokens ( 9.60 ms per token, 104.21 tokens per second)
1.12.495.549 I slot print_timing: id 3 | task 277 | eval time = 6569.12 ms / 125 tokens ( 52.55 ms per token, 19.03 tokens per second)
1.12.495.549 I slot print_timing: id 3 | task 277 | total time = 6837.82 ms / 153 tokens
1.12.495.550 I slot print_timing: id 3 | task 277 | graphs reused = 1
1.12.495.553 I slot print_timing: id 3 | task 277 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.12.495.565 I statistics draft-mtp: #calls(b,g,a) = 81 265 4034, #gen drafts = 4046, #acc drafts = 2617, #gen tokens = 8072, #acc tokens = 4830, #mean acc len = 2.20, #acc rate/pos = (0.649, 0.549), dur(b,g,a) = 0.050, 900.510, 5.103 ms
1.12.495.589 I slot release: id 3 | task 277 | stop processing: n_tokens = 154, truncated = 0
1.12.504.588 I srv operator(): Chat format: peg-native
1.12.615.339 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.12.615.341 I srv get_availabl: updating prompt cache
1.12.615.385 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.12.615.388 I srv alloc: - prompt is already in the cache, skipping
1.12.615.389 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.12.615.393 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.12.615.395 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.12.615.395 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.12.615.396 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.12.615.397 I srv get_availabl: prompt cache update took 0.06 ms
1.12.615.474 I slot launch_slot_: id 3 | task 349 | processing task, is_child = 0
1.12.619.287 I slot operator(): id 3 | task 349 | Checking checkpoint with [23, 23] against 3...
1.12.619.290 W slot operator(): id 3 | task 349 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.12.619.292 W slot operator(): id 3 | task 349 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.12.620.891 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.12.768.192 I slot create_check: id 3 | task 349 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.13.005.361 I slot print_timing: id 8 | task 307 | n_decoded = 102, tg = 23.30 t/s, tg_3s = 23.30 t/s
1.13.364.546 I slot print_timing: id 1 | task 282 | n_decoded = 100, tg = 14.22 t/s, tg_3s = 14.22 t/s
1.13.483.864 I slot print_timing: id 11 | task 258 | prompt eval time = 260.53 ms / 29 tokens ( 8.98 ms per token, 111.31 tokens per second)
1.13.483.872 I slot print_timing: id 11 | task 258 | eval time = 9410.89 ms / 128 tokens ( 73.52 ms per token, 13.60 tokens per second)
1.13.483.872 I slot print_timing: id 11 | task 258 | total time = 9671.42 ms / 157 tokens
1.13.483.873 I slot print_timing: id 11 | task 258 | graphs reused = 1
1.13.483.877 I slot print_timing: id 11 | task 258 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.13.483.892 I statistics draft-mtp: #calls(b,g,a) = 82 273 4155, #gen drafts = 4170, #acc drafts = 2693, #gen tokens = 8319, #acc tokens = 4971, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.051, 926.966, 5.248 ms
1.13.483.920 I slot release: id 11 | task 258 | stop processing: n_tokens = 156, truncated = 0
1.13.493.011 I srv operator(): Chat format: peg-native
1.13.603.453 I slot print_timing: id 8 | task 307 | prompt eval time = 116.19 ms / 4 tokens ( 29.05 ms per token, 34.43 tokens per second)
1.13.603.456 I slot print_timing: id 8 | task 307 | eval time = 4974.99 ms / 114 tokens ( 43.64 ms per token, 22.91 tokens per second)
1.13.603.457 I slot print_timing: id 8 | task 307 | total time = 5091.18 ms / 118 tokens
1.13.603.458 I slot print_timing: id 8 | task 307 | graphs reused = 1
1.13.603.461 I slot print_timing: id 8 | task 307 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.13.603.472 I statistics draft-mtp: #calls(b,g,a) = 82 274 4179, #gen drafts = 4185, #acc drafts = 2709, #gen tokens = 8349, #acc tokens = 5000, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.051, 930.947, 5.276 ms
1.13.603.499 I slot release: id 8 | task 307 | stop processing: n_tokens = 140, truncated = 0
1.13.605.140 I slot print_timing: id 12 | task 289 | prompt eval time = 110.73 ms / 4 tokens ( 27.68 ms per token, 36.12 tokens per second)
1.13.605.143 I slot print_timing: id 12 | task 289 | eval time = 6673.91 ms / 125 tokens ( 53.39 ms per token, 18.73 tokens per second)
1.13.605.144 I slot print_timing: id 12 | task 289 | total time = 6784.65 ms / 129 tokens
1.13.605.144 I slot print_timing: id 12 | task 289 | graphs reused = 1
1.13.605.148 I slot print_timing: id 12 | task 289 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.13.605.156 I statistics draft-mtp: #calls(b,g,a) = 82 274 4182, #gen drafts = 4185, #acc drafts = 2712, #gen tokens = 8349, #acc tokens = 5005, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.051, 930.947, 5.279 ms
1.13.605.176 I slot release: id 12 | task 289 | stop processing: n_tokens = 154, truncated = 0
1.13.606.506 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.13.606.508 I srv get_availabl: updating prompt cache
1.13.606.566 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.13.606.568 I srv alloc: - prompt is already in the cache, skipping
1.13.606.569 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.13.606.573 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.13.606.574 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.13.606.575 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.13.606.576 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.13.606.576 I srv get_availabl: prompt cache update took 0.07 ms
1.13.606.650 I slot launch_slot_: id 11 | task 358 | processing task, is_child = 0
1.13.606.652 I slot process_sing: id 8 | task -1 | saving idle slot to prompt cache
1.13.606.678 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.13.606.679 I srv alloc: - prompt is already in the cache, skipping
1.13.606.679 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.13.606.704 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.13.606.705 I srv alloc: - prompt is already in the cache, skipping
1.13.612.111 I slot operator(): id 11 | task 358 | Checking checkpoint with [24, 24] against 28...
1.13.613.471 I srv operator(): Chat format: peg-native
1.13.616.587 I srv operator(): Chat format: peg-native
1.13.616.604 W slot operator(): id 11 | task 358 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.13.616.645 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.723.197 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.13.723.200 I srv get_availabl: updating prompt cache
1.13.723.242 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.13.723.244 I srv alloc: - prompt is already in the cache, skipping
1.13.723.245 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.13.723.248 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.13.723.249 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.13.723.249 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.13.723.250 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.13.723.250 I srv get_availabl: prompt cache update took 0.05 ms
1.13.723.314 I slot launch_slot_: id 8 | task 360 | processing task, is_child = 0
1.13.723.315 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.13.723.331 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.13.723.332 I srv alloc: - prompt is already in the cache, skipping
1.13.723.334 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.13.723.334 I srv get_availabl: updating prompt cache
1.13.723.347 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.13.723.348 I srv alloc: - prompt is already in the cache, skipping
1.13.723.348 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.13.723.349 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.13.723.349 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.13.723.350 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.13.723.350 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.13.723.350 I srv get_availabl: prompt cache update took 0.02 ms
1.13.723.368 I slot launch_slot_: id 12 | task 361 | processing task, is_child = 0
1.13.728.492 I slot operator(): id 8 | task 360 | Checking checkpoint with [20, 20] against 24...
1.13.732.405 W slot operator(): id 8 | task 360 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.13.732.444 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.732.459 I slot operator(): id 12 | task 361 | Checking checkpoint with [23, 23] against 27...
1.13.735.997 W slot operator(): id 12 | task 361 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.13.736.033 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.13.975.954 I slot print_timing: id 10 | task 305 | n_decoded = 101, tg = 18.45 t/s, tg_3s = 18.45 t/s
1.14.329.997 I slot print_timing: id 15 | task 270 | prompt eval time = 115.75 ms / 4 tokens ( 28.94 ms per token, 34.56 tokens per second)
1.14.330.003 I slot print_timing: id 15 | task 270 | eval time = 9280.55 ms / 128 tokens ( 72.50 ms per token, 13.79 tokens per second)
1.14.330.004 I slot print_timing: id 15 | task 270 | total time = 9396.30 ms / 132 tokens
1.14.330.005 I slot print_timing: id 15 | task 270 | graphs reused = 1
1.14.330.012 I slot print_timing: id 15 | task 270 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.14.330.025 I statistics draft-mtp: #calls(b,g,a) = 85 280 4260, #gen drafts = 4275, #acc drafts = 2761, #gen tokens = 8528, #acc tokens = 5095, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.053, 952.531, 5.378 ms
1.14.330.048 I slot release: id 15 | task 270 | stop processing: n_tokens = 156, truncated = 0
1.14.339.222 I srv operator(): Chat format: peg-native
1.14.449.071 I slot print_timing: id 9 | task 321 | n_decoded = 102, tg = 23.61 t/s, tg_3s = 23.61 t/s
1.14.451.431 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.14.451.433 I srv get_availabl: updating prompt cache
1.14.451.473 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.14.451.475 I srv alloc: - prompt is already in the cache, skipping
1.14.451.477 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.14.451.480 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.14.451.481 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.14.451.481 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.14.451.482 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.14.451.482 I srv get_availabl: prompt cache update took 0.05 ms
1.14.451.547 I slot launch_slot_: id 15 | task 368 | processing task, is_child = 0
1.14.454.274 I slot operator(): id 15 | task 368 | Checking checkpoint with [24, 24] against 28...
1.14.458.337 W slot operator(): id 15 | task 368 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.14.458.375 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.057.002 I slot print_timing: id 9 | task 321 | prompt eval time = 261.86 ms / 25 tokens ( 10.47 ms per token, 95.47 tokens per second)
1.15.057.005 I slot print_timing: id 9 | task 321 | eval time = 4928.73 ms / 114 tokens ( 43.23 ms per token, 23.13 tokens per second)
1.15.057.006 I slot print_timing: id 9 | task 321 | total time = 5190.59 ms / 139 tokens
1.15.057.007 I slot print_timing: id 9 | task 321 | graphs reused = 1
1.15.057.009 I slot print_timing: id 9 | task 321 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.15.057.022 I statistics draft-mtp: #calls(b,g,a) = 86 286 4363, #gen drafts = 4369, #acc drafts = 2827, #gen tokens = 8716, #acc tokens = 5220, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.054, 968.867, 5.500 ms
1.15.057.047 I slot release: id 9 | task 321 | stop processing: n_tokens = 140, truncated = 0
1.15.066.357 I srv operator(): Chat format: peg-native
1.15.173.795 I slot print_timing: id 10 | task 305 | prompt eval time = 111.42 ms / 4 tokens ( 27.85 ms per token, 35.90 tokens per second)
1.15.173.798 I slot print_timing: id 10 | task 305 | eval time = 6673.24 ms / 125 tokens ( 53.39 ms per token, 18.73 tokens per second)
1.15.173.799 I slot print_timing: id 10 | task 305 | total time = 6784.66 ms / 129 tokens
1.15.173.800 I slot print_timing: id 10 | task 305 | graphs reused = 1
1.15.173.805 I slot print_timing: id 10 | task 305 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.15.173.815 I statistics draft-mtp: #calls(b,g,a) = 86 287 4379, #gen drafts = 4384, #acc drafts = 2838, #gen tokens = 8746, #acc tokens = 5240, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.054, 972.976, 5.526 ms
1.15.173.838 I slot release: id 10 | task 305 | stop processing: n_tokens = 154, truncated = 0
1.15.175.627 I slot print_timing: id 13 | task 318 | n_decoded = 101, tg = 18.99 t/s, tg_3s = 18.99 t/s
1.15.176.539 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.15.176.540 I srv get_availabl: updating prompt cache
1.15.176.583 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.15.176.585 I srv alloc: - prompt is already in the cache, skipping
1.15.176.586 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.15.176.589 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.15.176.590 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.15.176.591 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.15.176.591 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.15.176.591 I srv get_availabl: prompt cache update took 0.05 ms
1.15.176.655 I slot launch_slot_: id 10 | task 375 | processing task, is_child = 0
1.15.176.656 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
1.15.176.672 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.15.176.672 I srv alloc: - prompt is already in the cache, skipping
1.15.180.643 I slot operator(): id 10 | task 375 | Checking checkpoint with [23, 23] against 27...
1.15.182.934 I srv operator(): Chat format: peg-native
1.15.184.554 W slot operator(): id 10 | task 375 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.15.184.595 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.297.607 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.15.297.610 I srv get_availabl: updating prompt cache
1.15.297.655 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.15.297.656 I srv alloc: - prompt is already in the cache, skipping
1.15.297.657 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.15.297.660 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.15.297.661 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.15.297.662 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.15.297.662 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.15.297.663 I srv get_availabl: prompt cache update took 0.05 ms
1.15.297.728 I slot launch_slot_: id 9 | task 377 | processing task, is_child = 0
1.15.301.980 I slot operator(): id 9 | task 377 | Checking checkpoint with [20, 20] against 24...
1.15.305.850 W slot operator(): id 9 | task 377 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.15.305.886 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.422.726 I slot print_timing: id 7 | task 304 | n_decoded = 100, tg = 14.45 t/s, tg_3s = 14.45 t/s
1.15.659.816 I slot print_timing: id 1 | task 282 | prompt eval time = 261.25 ms / 29 tokens ( 9.01 ms per token, 111.00 tokens per second)
1.15.659.821 I slot print_timing: id 1 | task 282 | eval time = 9329.93 ms / 128 tokens ( 72.89 ms per token, 13.72 tokens per second)
1.15.659.821 I slot print_timing: id 1 | task 282 | total time = 9591.18 ms / 157 tokens
1.15.659.822 I slot print_timing: id 1 | task 282 | graphs reused = 1
1.15.659.826 I slot print_timing: id 1 | task 282 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.15.659.840 I statistics draft-mtp: #calls(b,g,a) = 88 291 4429, #gen drafts = 4444, #acc drafts = 2872, #gen tokens = 8865, #acc tokens = 5304, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.056, 987.577, 5.589 ms
1.15.659.867 I slot release: id 1 | task 282 | stop processing: n_tokens = 156, truncated = 0
1.15.668.934 I srv operator(): Chat format: peg-native
1.15.780.816 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.15.780.818 I srv get_availabl: updating prompt cache
1.15.780.864 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.15.780.867 I srv alloc: - prompt is already in the cache, skipping
1.15.780.868 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.15.780.871 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.15.780.873 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.15.780.874 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.15.780.874 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.15.780.875 I srv get_availabl: prompt cache update took 0.06 ms
1.15.780.939 I slot launch_slot_: id 1 | task 382 | processing task, is_child = 0
1.15.784.564 I slot operator(): id 1 | task 382 | Checking checkpoint with [24, 24] against 28...
1.15.788.654 W slot operator(): id 1 | task 382 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.15.788.688 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.15.901.586 I slot print_timing: id 0 | task 336 | n_decoded = 102, tg = 23.84 t/s, tg_3s = 23.84 t/s
1.16.390.378 I slot print_timing: id 13 | task 318 | prompt eval time = 254.26 ms / 28 tokens ( 9.08 ms per token, 110.12 tokens per second)
1.16.390.381 I slot print_timing: id 13 | task 318 | eval time = 6534.67 ms / 125 tokens ( 52.28 ms per token, 19.13 tokens per second)
1.16.390.382 I slot print_timing: id 13 | task 318 | total time = 6788.93 ms / 153 tokens
1.16.390.383 I slot print_timing: id 13 | task 318 | graphs reused = 1
1.16.390.386 I slot print_timing: id 13 | task 318 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.16.390.403 I statistics draft-mtp: #calls(b,g,a) = 89 297 4536, #gen drafts = 4538, #acc drafts = 2941, #gen tokens = 9053, #acc tokens = 5433, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.056, 1005.872, 5.719 ms
1.16.390.430 I slot release: id 13 | task 318 | stop processing: n_tokens = 154, truncated = 0
1.16.398.940 I srv operator(): Chat format: peg-native
1.16.500.533 I slot print_timing: id 0 | task 336 | prompt eval time = 258.48 ms / 25 tokens ( 10.34 ms per token, 96.72 tokens per second)
1.16.500.536 I slot print_timing: id 0 | task 336 | eval time = 4877.31 ms / 114 tokens ( 42.78 ms per token, 23.37 tokens per second)
1.16.500.537 I slot print_timing: id 0 | task 336 | total time = 5135.79 ms / 139 tokens
1.16.500.538 I slot print_timing: id 0 | task 336 | graphs reused = 1
1.16.500.542 I slot print_timing: id 0 | task 336 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.16.500.559 I statistics draft-mtp: #calls(b,g,a) = 89 298 4539, #gen drafts = 4553, #acc drafts = 2943, #gen tokens = 9083, #acc tokens = 5437, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.056, 1009.507, 5.724 ms
1.16.500.588 I slot release: id 0 | task 336 | stop processing: n_tokens = 140, truncated = 0
1.16.507.952 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.16.507.955 I srv get_availabl: updating prompt cache
1.16.508.002 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.16.508.005 I srv alloc: - prompt is already in the cache, skipping
1.16.508.006 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.16.508.010 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.16.508.012 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.16.508.013 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.16.508.013 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.16.508.015 I srv get_availabl: prompt cache update took 0.06 ms
1.16.508.080 I slot launch_slot_: id 13 | task 389 | processing task, is_child = 0
1.16.508.081 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.16.508.099 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.16.508.101 I srv alloc: - prompt is already in the cache, skipping
1.16.510.290 I srv operator(): Chat format: peg-native
1.16.511.707 I slot operator(): id 13 | task 389 | Checking checkpoint with [23, 23] against 27...
1.16.515.696 W slot operator(): id 13 | task 389 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.16.515.749 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.16.624.356 I slot print_timing: id 4 | task 316 | n_decoded = 100, tg = 14.22 t/s, tg_3s = 14.22 t/s
1.16.625.663 I slot print_timing: id 6 | task 333 | n_decoded = 101, tg = 19.16 t/s, tg_3s = 19.16 t/s
1.16.629.434 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.16.629.435 I srv get_availabl: updating prompt cache
1.16.629.476 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.16.629.477 I srv alloc: - prompt is already in the cache, skipping
1.16.629.478 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.16.629.481 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.16.629.482 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.16.629.482 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.16.629.483 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.16.629.484 I srv get_availabl: prompt cache update took 0.05 ms
1.16.629.549 I slot launch_slot_: id 0 | task 391 | processing task, is_child = 0
1.16.632.302 I slot operator(): id 0 | task 391 | Checking checkpoint with [20, 20] against 24...
1.16.636.300 W slot operator(): id 0 | task 391 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.16.636.338 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.111.807 I slot print_timing: id 3 | task 349 | n_decoded = 102, tg = 24.11 t/s, tg_3s = 24.11 t/s
1.17.712.815 I slot print_timing: id 7 | task 304 | prompt eval time = 115.14 ms / 4 tokens ( 28.78 ms per token, 34.74 tokens per second)
1.17.712.819 I slot print_timing: id 7 | task 304 | eval time = 9212.64 ms / 128 tokens ( 71.97 ms per token, 13.89 tokens per second)
1.17.712.820 I slot print_timing: id 7 | task 304 | total time = 9327.77 ms / 132 tokens
1.17.712.821 I slot print_timing: id 7 | task 304 | graphs reused = 1
1.17.712.824 I slot print_timing: id 7 | task 304 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.17.712.838 I statistics draft-mtp: #calls(b,g,a) = 91 308 4694, #gen drafts = 4709, #acc drafts = 3039, #gen tokens = 9394, #acc tokens = 5613, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.057, 1039.825, 5.899 ms
1.17.712.865 I slot release: id 7 | task 304 | stop processing: n_tokens = 156, truncated = 0
1.17.714.507 I slot print_timing: id 2 | task 345 | n_decoded = 101, tg = 19.35 t/s, tg_3s = 19.35 t/s
1.17.715.110 I slot print_timing: id 3 | task 349 | prompt eval time = 261.16 ms / 25 tokens ( 10.45 ms per token, 95.73 tokens per second)
1.17.715.111 I slot print_timing: id 3 | task 349 | eval time = 4834.65 ms / 114 tokens ( 42.41 ms per token, 23.58 tokens per second)
1.17.715.112 I slot print_timing: id 3 | task 349 | total time = 5095.81 ms / 139 tokens
1.17.715.112 I slot print_timing: id 3 | task 349 | graphs reused = 1
1.17.715.114 I slot print_timing: id 3 | task 349 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.17.715.122 I statistics draft-mtp: #calls(b,g,a) = 91 308 4698, #gen drafts = 4709, #acc drafts = 3042, #gen tokens = 9394, #acc tokens = 5619, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.057, 1039.825, 5.903 ms
1.17.715.145 I slot release: id 3 | task 349 | stop processing: n_tokens = 140, truncated = 0
1.17.722.050 I srv operator(): Chat format: peg-native
1.17.724.300 I srv operator(): Chat format: peg-native
1.17.826.145 I slot print_timing: id 6 | task 333 | prompt eval time = 254.96 ms / 28 tokens ( 9.11 ms per token, 109.82 tokens per second)
1.17.826.148 I slot print_timing: id 6 | task 333 | eval time = 6470.56 ms / 125 tokens ( 51.76 ms per token, 19.32 tokens per second)
1.17.826.149 I slot print_timing: id 6 | task 333 | total time = 6725.52 ms / 153 tokens
1.17.826.150 I slot print_timing: id 6 | task 333 | graphs reused = 1
1.17.826.154 I slot print_timing: id 6 | task 333 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.17.826.165 I statistics draft-mtp: #calls(b,g,a) = 91 309 4715, #gen drafts = 4723, #acc drafts = 3052, #gen tokens = 9422, #acc tokens = 5639, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.057, 1044.863, 5.923 ms
1.17.826.189 I slot release: id 6 | task 333 | stop processing: n_tokens = 154, truncated = 0
1.17.830.904 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.17.830.905 I srv get_availabl: updating prompt cache
1.17.830.948 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.17.830.949 I srv alloc: - prompt is already in the cache, skipping
1.17.830.950 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.17.830.954 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.17.830.955 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.17.830.955 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.17.830.956 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.17.830.957 I srv get_availabl: prompt cache update took 0.05 ms
1.17.831.022 I slot launch_slot_: id 7 | task 402 | processing task, is_child = 0
1.17.831.023 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.17.831.045 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.17.831.046 I srv alloc: - prompt is already in the cache, skipping
1.17.831.046 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
1.17.831.063 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.17.831.064 I srv alloc: - prompt is already in the cache, skipping
1.17.831.066 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.17.831.066 I srv get_availabl: updating prompt cache
1.17.831.086 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.17.831.086 I srv alloc: - prompt is already in the cache, skipping
1.17.831.087 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.17.831.087 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.17.831.088 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.17.831.088 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.17.831.089 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.17.831.089 I srv get_availabl: prompt cache update took 0.02 ms
1.17.831.110 I slot launch_slot_: id 6 | task 403 | processing task, is_child = 0
1.17.831.111 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.17.831.127 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.17.831.128 I srv alloc: - prompt is already in the cache, skipping
1.17.834.843 I srv operator(): Chat format: peg-native
1.17.836.099 I slot operator(): id 6 | task 403 | Checking checkpoint with [23, 23] against 27...
1.17.840.094 W slot operator(): id 6 | task 403 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.17.840.136 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.840.154 I slot operator(): id 7 | task 402 | Checking checkpoint with [24, 24] against 28...
1.17.843.802 W slot operator(): id 7 | task 402 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.17.843.839 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.17.958.670 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.17.958.672 I srv get_availabl: updating prompt cache
1.17.958.719 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.17.958.721 I srv alloc: - prompt is already in the cache, skipping
1.17.958.722 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.17.958.725 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.17.958.727 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.17.958.727 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.17.958.728 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.17.958.729 I srv get_availabl: prompt cache update took 0.06 ms
1.17.958.802 I slot launch_slot_: id 3 | task 405 | processing task, is_child = 0
1.17.962.622 I slot operator(): id 3 | task 405 | Checking checkpoint with [20, 20] against 24...
1.17.966.547 W slot operator(): id 3 | task 405 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.17.966.586 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.081.408 I slot print_timing: id 5 | task 331 | n_decoded = 100, tg = 14.30 t/s, tg_3s = 14.30 t/s
1.18.083.216 I slot print_timing: id 8 | task 360 | n_decoded = 102, tg = 24.10 t/s, tg_3s = 24.10 t/s
1.18.686.522 I slot print_timing: id 8 | task 360 | prompt eval time = 121.79 ms / 4 tokens ( 30.45 ms per token, 32.84 tokens per second)
1.18.686.525 I slot print_timing: id 8 | task 360 | eval time = 4836.22 ms / 114 tokens ( 42.42 ms per token, 23.57 tokens per second)
1.18.686.526 I slot print_timing: id 8 | task 360 | total time = 4958.01 ms / 118 tokens
1.18.686.527 I slot print_timing: id 8 | task 360 | graphs reused = 1
1.18.686.530 I slot print_timing: id 8 | task 360 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.18.686.542 I statistics draft-mtp: #calls(b,g,a) = 94 316 4824, #gen drafts = 4831, #acc drafts = 3124, #gen tokens = 9638, #acc tokens = 5772, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.057, 1067.424, 6.045 ms
1.18.686.568 I slot release: id 8 | task 360 | stop processing: n_tokens = 140, truncated = 0
1.18.695.249 I srv operator(): Chat format: peg-native
1.18.805.202 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.18.805.205 I srv get_availabl: updating prompt cache
1.18.805.247 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.18.805.249 I srv alloc: - prompt is already in the cache, skipping
1.18.805.250 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.18.805.254 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.18.805.256 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.18.805.257 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.18.805.257 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.18.805.258 I srv get_availabl: prompt cache update took 0.05 ms
1.18.805.324 I slot launch_slot_: id 8 | task 413 | processing task, is_child = 0
1.18.810.557 I slot operator(): id 8 | task 413 | Checking checkpoint with [20, 20] against 3...
1.18.810.559 W slot operator(): id 8 | task 413 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.18.810.561 W slot operator(): id 8 | task 413 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.18.812.473 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.18.934.721 I slot print_timing: id 4 | task 316 | prompt eval time = 110.09 ms / 4 tokens ( 27.52 ms per token, 36.33 tokens per second)
1.18.934.724 I slot print_timing: id 4 | task 316 | eval time = 9343.68 ms / 128 tokens ( 73.00 ms per token, 13.70 tokens per second)
1.18.934.725 I slot print_timing: id 4 | task 316 | total time = 9453.77 ms / 132 tokens
1.18.934.726 I slot print_timing: id 4 | task 316 | graphs reused = 1
1.18.934.729 I slot print_timing: id 4 | task 316 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.18.934.745 I statistics draft-mtp: #calls(b,g,a) = 94 318 4846, #gen drafts = 4860, #acc drafts = 3138, #gen tokens = 9695, #acc tokens = 5798, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.057, 1077.277, 6.067 ms
1.18.934.770 I slot release: id 4 | task 316 | stop processing: n_tokens = 156, truncated = 0
1.18.936.694 I slot print_timing: id 2 | task 345 | prompt eval time = 261.65 ms / 28 tokens ( 9.34 ms per token, 107.01 tokens per second)
1.18.936.696 I slot print_timing: id 2 | task 345 | eval time = 6442.72 ms / 125 tokens ( 51.54 ms per token, 19.40 tokens per second)
1.18.936.696 I slot print_timing: id 2 | task 345 | total time = 6704.37 ms / 153 tokens
1.18.936.697 I slot print_timing: id 2 | task 345 | graphs reused = 1
1.18.936.699 I slot print_timing: id 2 | task 345 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.18.936.707 I statistics draft-mtp: #calls(b,g,a) = 94 318 4849, #gen drafts = 4860, #acc drafts = 3141, #gen tokens = 9695, #acc tokens = 5804, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.057, 1077.277, 6.071 ms
1.18.936.731 I slot release: id 2 | task 345 | stop processing: n_tokens = 154, truncated = 0
1.18.944.330 I srv operator(): Chat format: peg-native
1.18.945.329 I srv operator(): Chat format: peg-native
1.18.961.658 I slot create_check: id 8 | task 413 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.19.065.944 I slot print_timing: id 12 | task 361 | n_decoded = 101, tg = 19.37 t/s, tg_3s = 19.37 t/s
1.19.067.135 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.19.067.136 I srv get_availabl: updating prompt cache
1.19.067.181 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.19.067.183 I srv alloc: - prompt is already in the cache, skipping
1.19.067.184 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.19.067.187 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.19.067.188 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.19.067.188 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.19.067.188 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.19.067.189 I srv get_availabl: prompt cache update took 0.05 ms
1.19.067.256 I slot launch_slot_: id 2 | task 416 | processing task, is_child = 0
1.19.067.257 I slot process_sing: id 4 | task -1 | saving idle slot to prompt cache
1.19.067.276 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.19.067.277 I srv alloc: - prompt is already in the cache, skipping
1.19.067.279 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.19.067.279 I srv get_availabl: updating prompt cache
1.19.067.294 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.19.067.295 I srv alloc: - prompt is already in the cache, skipping
1.19.067.295 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.19.067.296 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.19.067.296 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.19.067.297 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.19.067.297 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.19.067.297 I srv get_availabl: prompt cache update took 0.02 ms
1.19.067.318 I slot launch_slot_: id 4 | task 417 | processing task, is_child = 0
1.19.071.838 I slot operator(): id 2 | task 416 | Checking checkpoint with [23, 23] against 27...
1.19.075.711 W slot operator(): id 2 | task 416 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.19.075.752 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.075.776 I slot operator(): id 4 | task 417 | Checking checkpoint with [24, 24] against 3...
1.19.075.776 W slot operator(): id 4 | task 417 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.19.075.777 W slot operator(): id 4 | task 417 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.19.077.311 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.19.206.934 I slot print_timing: id 14 | task 343 | n_decoded = 100, tg = 14.32 t/s, tg_3s = 14.32 t/s
1.19.224.578 I slot create_check: id 4 | task 417 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.19.702.903 I slot print_timing: id 9 | task 377 | n_decoded = 102, tg = 23.81 t/s, tg_3s = 23.81 t/s
1.20.307.308 I slot print_timing: id 9 | task 377 | prompt eval time = 117.28 ms / 4 tokens ( 29.32 ms per token, 34.11 tokens per second)
1.20.307.311 I slot print_timing: id 9 | task 377 | eval time = 4888.03 ms / 114 tokens ( 42.88 ms per token, 23.32 tokens per second)
1.20.307.312 I slot print_timing: id 9 | task 377 | total time = 5005.31 ms / 118 tokens
1.20.307.313 I slot print_timing: id 9 | task 377 | graphs reused = 1
1.20.307.316 I slot print_timing: id 9 | task 377 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.20.307.330 I statistics draft-mtp: #calls(b,g,a) = 97 329 5024, #gen drafts = 5030, #acc drafts = 3253, #gen tokens = 10034, #acc tokens = 6009, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.059, 1113.807, 6.280 ms
1.20.307.356 I slot release: id 9 | task 377 | stop processing: n_tokens = 140, truncated = 0
1.20.309.179 I slot print_timing: id 12 | task 361 | prompt eval time = 118.09 ms / 4 tokens ( 29.52 ms per token, 33.87 tokens per second)
1.20.309.181 I slot print_timing: id 12 | task 361 | eval time = 6458.61 ms / 125 tokens ( 51.67 ms per token, 19.35 tokens per second)
1.20.309.182 I slot print_timing: id 12 | task 361 | total time = 6576.70 ms / 129 tokens
1.20.309.182 I slot print_timing: id 12 | task 361 | graphs reused = 1
1.20.309.185 I slot print_timing: id 12 | task 361 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.20.309.194 I statistics draft-mtp: #calls(b,g,a) = 97 329 5027, #gen drafts = 5030, #acc drafts = 3256, #gen tokens = 10034, #acc tokens = 6015, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.549), dur(b,g,a) = 0.059, 1113.807, 6.284 ms
1.20.309.216 I slot release: id 12 | task 361 | stop processing: n_tokens = 154, truncated = 0
1.20.316.091 I srv operator(): Chat format: peg-native
1.20.318.140 I srv operator(): Chat format: peg-native
1.20.412.097 I slot print_timing: id 5 | task 331 | prompt eval time = 110.30 ms / 4 tokens ( 27.58 ms per token, 36.26 tokens per second)
1.20.412.101 I slot print_timing: id 5 | task 331 | eval time = 9322.77 ms / 128 tokens ( 72.83 ms per token, 13.73 tokens per second)
1.20.412.101 I slot print_timing: id 5 | task 331 | total time = 9433.07 ms / 132 tokens
1.20.412.103 I slot print_timing: id 5 | task 331 | graphs reused = 1
1.20.412.106 I slot print_timing: id 5 | task 331 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.20.412.120 I statistics draft-mtp: #calls(b,g,a) = 97 330 5030, #gen drafts = 5043, #acc drafts = 3256, #gen tokens = 10060, #acc tokens = 6015, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.059, 1119.371, 6.287 ms
1.20.412.146 I slot release: id 5 | task 331 | stop processing: n_tokens = 156, truncated = 0
1.20.418.479 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.20.418.481 I srv get_availabl: updating prompt cache
1.20.418.524 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.20.418.526 I srv alloc: - prompt is already in the cache, skipping
1.20.418.527 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.20.418.530 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.20.418.531 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.20.418.532 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.20.418.532 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.20.418.533 I srv get_availabl: prompt cache update took 0.05 ms
1.20.418.600 I slot launch_slot_: id 5 | task 429 | processing task, is_child = 0
1.20.418.601 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
1.20.418.616 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.20.418.617 I srv alloc: - prompt is already in the cache, skipping
1.20.418.618 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.20.418.633 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.20.418.634 I srv alloc: - prompt is already in the cache, skipping
1.20.418.635 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.20.418.636 I srv get_availabl: updating prompt cache
1.20.418.649 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.20.418.650 I srv alloc: - prompt is already in the cache, skipping
1.20.418.650 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.20.418.650 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.20.418.651 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.20.418.651 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.20.418.651 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.20.418.652 I srv get_availabl: prompt cache update took 0.01 ms
1.20.418.671 I slot launch_slot_: id 12 | task 430 | processing task, is_child = 0
1.20.418.671 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
1.20.418.686 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.20.418.687 I srv alloc: - prompt is already in the cache, skipping
1.20.421.075 I srv operator(): Chat format: peg-native
1.20.424.242 I slot operator(): id 5 | task 429 | Checking checkpoint with [24, 24] against 28...
1.20.428.323 W slot operator(): id 5 | task 429 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.20.428.365 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.428.384 I slot operator(): id 12 | task 430 | Checking checkpoint with [23, 23] against 27...
1.20.431.976 W slot operator(): id 12 | task 430 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.20.432.017 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.545.118 I slot print_timing: id 11 | task 358 | n_decoded = 100, tg = 14.64 t/s, tg_3s = 14.64 t/s
1.20.546.001 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.20.546.002 I srv get_availabl: updating prompt cache
1.20.546.045 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.20.546.047 I srv alloc: - prompt is already in the cache, skipping
1.20.546.047 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.20.546.050 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.20.546.051 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.20.546.052 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.20.546.053 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.20.546.053 I srv get_availabl: prompt cache update took 0.05 ms
1.20.546.118 I slot launch_slot_: id 9 | task 432 | processing task, is_child = 0
1.20.550.278 I slot operator(): id 9 | task 432 | Checking checkpoint with [20, 20] against 24...
1.20.554.201 W slot operator(): id 9 | task 432 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.20.554.236 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.20.671.963 I slot print_timing: id 10 | task 375 | n_decoded = 101, tg = 18.77 t/s, tg_3s = 18.77 t/s
1.21.028.955 I slot print_timing: id 0 | task 391 | n_decoded = 102, tg = 23.83 t/s, tg_3s = 23.83 t/s
1.21.399.229 I slot print_timing: id 15 | task 368 | n_decoded = 100, tg = 14.64 t/s, tg_3s = 14.64 t/s
1.21.511.918 I slot print_timing: id 14 | task 343 | prompt eval time = 109.66 ms / 4 tokens ( 27.41 ms per token, 36.48 tokens per second)
1.21.511.921 I slot print_timing: id 14 | task 343 | eval time = 9289.42 ms / 128 tokens ( 72.57 ms per token, 13.78 tokens per second)
1.21.511.922 I slot print_timing: id 14 | task 343 | total time = 9399.08 ms / 132 tokens
1.21.511.923 I slot print_timing: id 14 | task 343 | graphs reused = 1
1.21.511.926 I slot print_timing: id 14 | task 343 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.21.511.941 I statistics draft-mtp: #calls(b,g,a) = 100 339 5167, #gen drafts = 5182, #acc drafts = 3344, #gen tokens = 10337, #acc tokens = 6178, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.060, 1149.312, 6.452 ms
1.21.511.967 I slot release: id 14 | task 343 | stop processing: n_tokens = 156, truncated = 0
1.21.520.804 I srv operator(): Chat format: peg-native
1.21.628.368 I slot print_timing: id 0 | task 391 | prompt eval time = 116.10 ms / 4 tokens ( 29.02 ms per token, 34.45 tokens per second)
1.21.628.371 I slot print_timing: id 0 | task 391 | eval time = 4879.93 ms / 114 tokens ( 42.81 ms per token, 23.36 tokens per second)
1.21.628.371 I slot print_timing: id 0 | task 391 | total time = 4996.03 ms / 118 tokens
1.21.628.372 I slot print_timing: id 0 | task 391 | graphs reused = 1
1.21.628.381 I slot print_timing: id 0 | task 391 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.21.628.397 I statistics draft-mtp: #calls(b,g,a) = 100 340 5183, #gen drafts = 5197, #acc drafts = 3356, #gen tokens = 10367, #acc tokens = 6198, #mean acc len = 2.20, #acc rate/pos = (0.648, 0.548), dur(b,g,a) = 0.060, 1152.967, 6.473 ms
1.21.628.420 I slot release: id 0 | task 391 | stop processing: n_tokens = 140, truncated = 0
1.21.635.272 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.21.635.273 I srv get_availabl: updating prompt cache
1.21.635.315 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.21.635.317 I srv alloc: - prompt is already in the cache, skipping
1.21.635.318 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.21.635.320 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.21.635.321 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.21.635.322 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.21.635.322 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.21.635.323 I srv get_availabl: prompt cache update took 0.05 ms
1.21.635.386 I slot launch_slot_: id 14 | task 442 | processing task, is_child = 0
1.21.635.387 I slot process_sing: id 0 | task -1 | saving idle slot to prompt cache
1.21.635.404 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.21.635.405 I srv alloc: - prompt is already in the cache, skipping
1.21.637.792 I srv operator(): Chat format: peg-native
1.21.638.942 I slot operator(): id 14 | task 442 | Checking checkpoint with [24, 24] against 28...
1.21.642.863 W slot operator(): id 14 | task 442 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.21.642.901 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.755.184 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.21.755.186 I srv get_availabl: updating prompt cache
1.21.755.227 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.21.755.230 I srv alloc: - prompt is already in the cache, skipping
1.21.755.231 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.21.755.234 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.21.755.234 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.21.755.235 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.21.755.235 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.21.755.236 I srv get_availabl: prompt cache update took 0.05 ms
1.21.755.302 I slot launch_slot_: id 0 | task 444 | processing task, is_child = 0
1.21.758.008 I slot operator(): id 0 | task 444 | Checking checkpoint with [20, 20] against 3...
1.21.758.010 W slot operator(): id 0 | task 444 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.21.758.013 W slot operator(): id 0 | task 444 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.21.759.972 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.21.886.197 I slot print_timing: id 10 | task 375 | prompt eval time = 110.09 ms / 4 tokens ( 27.52 ms per token, 36.33 tokens per second)
1.21.886.201 I slot print_timing: id 10 | task 375 | eval time = 6595.44 ms / 125 tokens ( 52.76 ms per token, 18.95 tokens per second)
1.21.886.202 I slot print_timing: id 10 | task 375 | total time = 6705.53 ms / 129 tokens
1.21.886.203 I slot print_timing: id 10 | task 375 | graphs reused = 1
1.21.886.206 I slot print_timing: id 10 | task 375 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.21.886.217 I statistics draft-mtp: #calls(b,g,a) = 101 342 5221, #gen drafts = 5226, #acc drafts = 3379, #gen tokens = 10425, #acc tokens = 6242, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.060, 1159.147, 6.522 ms
1.21.886.247 I slot release: id 10 | task 375 | stop processing: n_tokens = 154, truncated = 0
1.21.887.773 I slot print_timing: id 13 | task 389 | n_decoded = 101, tg = 19.18 t/s, tg_3s = 19.18 t/s
1.21.895.448 I srv operator(): Chat format: peg-native
1.21.906.819 I slot create_check: id 0 | task 444 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.22.019.112 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.22.019.114 I srv get_availabl: updating prompt cache
1.22.019.159 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.22.019.161 I srv alloc: - prompt is already in the cache, skipping
1.22.019.162 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.22.019.165 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.22.019.166 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.22.019.167 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.22.019.167 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.22.019.167 I srv get_availabl: prompt cache update took 0.05 ms
1.22.019.244 I slot launch_slot_: id 10 | task 447 | processing task, is_child = 0
1.22.023.330 I slot operator(): id 10 | task 447 | Checking checkpoint with [23, 23] against 3...
1.22.023.332 W slot operator(): id 10 | task 447 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.22.023.336 W slot operator(): id 10 | task 447 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.22.024.931 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.22.171.919 I slot create_check: id 10 | task 447 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.22.407.107 I slot print_timing: id 3 | task 405 | n_decoded = 102, tg = 23.57 t/s, tg_3s = 23.57 t/s
1.22.769.640 I slot print_timing: id 1 | task 382 | n_decoded = 100, tg = 14.56 t/s, tg_3s = 14.56 t/s
1.22.888.377 I slot print_timing: id 11 | task 358 | prompt eval time = 104.42 ms / 4 tokens ( 26.10 ms per token, 38.31 tokens per second)
1.22.888.381 I slot print_timing: id 11 | task 358 | eval time = 9171.82 ms / 128 tokens ( 71.65 ms per token, 13.96 tokens per second)
1.22.888.382 I slot print_timing: id 11 | task 358 | total time = 9276.24 ms / 132 tokens
1.22.888.383 I slot print_timing: id 11 | task 358 | graphs reused = 1
1.22.888.386 I slot print_timing: id 11 | task 358 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.22.888.402 I statistics draft-mtp: #calls(b,g,a) = 103 350 5334, #gen drafts = 5349, #acc drafts = 3450, #gen tokens = 10670, #acc tokens = 6373, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.061, 1186.737, 6.675 ms
1.22.888.432 I slot release: id 11 | task 358 | stop processing: n_tokens = 156, truncated = 0
1.22.897.518 I srv operator(): Chat format: peg-native
1.23.005.293 I slot print_timing: id 3 | task 405 | prompt eval time = 116.48 ms / 4 tokens ( 29.12 ms per token, 34.34 tokens per second)
1.23.005.296 I slot print_timing: id 3 | task 405 | eval time = 4926.17 ms / 114 tokens ( 43.21 ms per token, 23.14 tokens per second)
1.23.005.297 I slot print_timing: id 3 | task 405 | total time = 5042.65 ms / 118 tokens
1.23.005.298 I slot print_timing: id 3 | task 405 | graphs reused = 1
1.23.005.300 I slot print_timing: id 3 | task 405 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.23.005.312 I statistics draft-mtp: #calls(b,g,a) = 103 351 5353, #gen drafts = 5364, #acc drafts = 3462, #gen tokens = 10700, #acc tokens = 6393, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.061, 1190.816, 6.700 ms
1.23.005.338 I slot release: id 3 | task 405 | stop processing: n_tokens = 140, truncated = 0
1.23.010.994 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.23.010.995 I srv get_availabl: updating prompt cache
1.23.011.041 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.23.011.043 I srv alloc: - prompt is already in the cache, skipping
1.23.011.044 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.23.011.047 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.23.011.049 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.23.011.049 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.23.011.050 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.23.011.051 I srv get_availabl: prompt cache update took 0.06 ms
1.23.011.129 I slot launch_slot_: id 11 | task 456 | processing task, is_child = 0
1.23.011.130 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.23.011.154 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.23.011.155 I srv alloc: - prompt is already in the cache, skipping
1.23.014.029 I srv operator(): Chat format: peg-native
1.23.016.409 I slot operator(): id 11 | task 456 | Checking checkpoint with [24, 24] against 28...
1.23.020.283 W slot operator(): id 11 | task 456 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.23.020.321 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.132.803 I slot print_timing: id 13 | task 389 | prompt eval time = 110.38 ms / 4 tokens ( 27.59 ms per token, 36.24 tokens per second)
1.23.132.806 I slot print_timing: id 13 | task 389 | eval time = 6510.70 ms / 125 tokens ( 52.09 ms per token, 19.20 tokens per second)
1.23.132.807 I slot print_timing: id 13 | task 389 | total time = 6621.07 ms / 129 tokens
1.23.132.808 I slot print_timing: id 13 | task 389 | graphs reused = 1
1.23.132.810 I slot print_timing: id 13 | task 389 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.23.132.821 I statistics draft-mtp: #calls(b,g,a) = 104 352 5376, #gen drafts = 5378, #acc drafts = 3478, #gen tokens = 10728, #acc tokens = 6425, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.062, 1196.036, 6.728 ms
1.23.132.847 I slot release: id 13 | task 389 | stop processing: n_tokens = 154, truncated = 0
1.23.133.447 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.23.133.449 I srv get_availabl: updating prompt cache
1.23.133.490 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.23.133.492 I srv alloc: - prompt is already in the cache, skipping
1.23.133.493 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.23.133.496 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.23.133.497 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.23.133.498 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.23.133.498 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.23.133.499 I srv get_availabl: prompt cache update took 0.05 ms
1.23.133.569 I slot launch_slot_: id 13 | task 458 | processing task, is_child = 0
1.23.133.570 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.23.133.592 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.23.133.593 I srv alloc: - prompt is already in the cache, skipping
1.23.138.288 I slot operator(): id 13 | task 458 | Checking checkpoint with [23, 23] against 27...
1.23.141.301 I srv operator(): Chat format: peg-native
1.23.142.131 W slot operator(): id 13 | task 458 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.23.142.170 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.255.934 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.23.255.937 I srv get_availabl: updating prompt cache
1.23.255.982 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.23.255.985 I srv alloc: - prompt is already in the cache, skipping
1.23.255.986 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.23.255.989 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.23.255.990 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.23.255.990 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.23.255.991 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.23.255.991 I srv get_availabl: prompt cache update took 0.05 ms
1.23.256.069 I slot launch_slot_: id 3 | task 460 | processing task, is_child = 0
1.23.259.906 I slot operator(): id 3 | task 460 | Checking checkpoint with [20, 20] against 24...
1.23.263.880 W slot operator(): id 3 | task 460 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.23.263.912 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.23.379.708 I slot print_timing: id 6 | task 403 | n_decoded = 101, tg = 18.61 t/s, tg_3s = 18.61 t/s
1.23.619.406 I slot print_timing: id 4 | task 417 | n_decoded = 102, tg = 23.82 t/s, tg_3s = 23.82 t/s
1.23.735.342 I slot print_timing: id 15 | task 368 | prompt eval time = 115.97 ms / 4 tokens ( 28.99 ms per token, 34.49 tokens per second)
1.23.735.346 I slot print_timing: id 15 | task 368 | eval time = 9165.08 ms / 128 tokens ( 71.60 ms per token, 13.97 tokens per second)
1.23.735.346 I slot print_timing: id 15 | task 368 | total time = 9281.04 ms / 132 tokens
1.23.735.347 I slot print_timing: id 15 | task 368 | graphs reused = 1
1.23.735.351 I slot print_timing: id 15 | task 368 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.23.735.366 I statistics draft-mtp: #calls(b,g,a) = 106 357 5439, #gen drafts = 5454, #acc drafts = 3519, #gen tokens = 10879, #acc tokens = 6500, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.064, 1212.715, 6.811 ms
1.23.735.401 I slot release: id 15 | task 368 | stop processing: n_tokens = 156, truncated = 0
1.23.744.735 I srv operator(): Chat format: peg-native
1.23.856.402 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.23.856.404 I srv get_availabl: updating prompt cache
1.23.856.445 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.23.856.447 I srv alloc: - prompt is already in the cache, skipping
1.23.856.448 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.23.856.451 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.23.856.452 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.23.856.452 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.23.856.452 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.23.856.453 I srv get_availabl: prompt cache update took 0.05 ms
1.23.856.519 I slot launch_slot_: id 15 | task 466 | processing task, is_child = 0
1.23.859.220 I slot operator(): id 15 | task 466 | Checking checkpoint with [24, 24] against 28...
1.23.863.129 W slot operator(): id 15 | task 466 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.23.863.165 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.217.906 I slot print_timing: id 4 | task 417 | prompt eval time = 260.73 ms / 25 tokens ( 10.43 ms per token, 95.89 tokens per second)
1.24.217.909 I slot print_timing: id 4 | task 417 | eval time = 4881.38 ms / 114 tokens ( 42.82 ms per token, 23.35 tokens per second)
1.24.217.910 I slot print_timing: id 4 | task 417 | total time = 5142.11 ms / 139 tokens
1.24.217.911 I slot print_timing: id 4 | task 417 | graphs reused = 1
1.24.217.914 I slot print_timing: id 4 | task 417 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.24.217.929 I statistics draft-mtp: #calls(b,g,a) = 107 361 5505, #gen drafts = 5516, #acc drafts = 3561, #gen tokens = 11003, #acc tokens = 6579, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.065, 1223.619, 6.887 ms
1.24.217.955 I slot release: id 4 | task 417 | stop processing: n_tokens = 140, truncated = 0
1.24.225.969 I srv operator(): Chat format: peg-native
1.24.338.435 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.24.338.437 I srv get_availabl: updating prompt cache
1.24.338.480 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.24.338.482 I srv alloc: - prompt is already in the cache, skipping
1.24.338.483 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.24.338.485 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.24.338.486 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.24.338.487 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.24.338.487 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.24.338.488 I srv get_availabl: prompt cache update took 0.05 ms
1.24.338.562 I slot launch_slot_: id 4 | task 471 | processing task, is_child = 0
1.24.342.668 I slot operator(): id 4 | task 471 | Checking checkpoint with [20, 20] against 3...
1.24.342.670 W slot operator(): id 4 | task 471 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.24.342.673 W slot operator(): id 4 | task 471 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.24.344.228 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.468.373 I slot print_timing: id 2 | task 416 | n_decoded = 101, tg = 19.17 t/s, tg_3s = 19.17 t/s
1.24.491.068 I slot create_check: id 4 | task 471 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.24.607.532 I slot print_timing: id 6 | task 403 | prompt eval time = 116.48 ms / 4 tokens ( 29.12 ms per token, 34.34 tokens per second)
1.24.607.534 I slot print_timing: id 6 | task 403 | eval time = 6654.93 ms / 125 tokens ( 53.24 ms per token, 18.78 tokens per second)
1.24.607.535 I slot print_timing: id 6 | task 403 | total time = 6771.41 ms / 129 tokens
1.24.607.536 I slot print_timing: id 6 | task 403 | graphs reused = 1
1.24.607.538 I slot print_timing: id 6 | task 403 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.24.607.551 I statistics draft-mtp: #calls(b,g,a) = 108 364 5552, #gen drafts = 5561, #acc drafts = 3590, #gen tokens = 11093, #acc tokens = 6633, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.066, 1235.875, 6.948 ms
1.24.607.576 I slot release: id 6 | task 403 | stop processing: n_tokens = 154, truncated = 0
1.24.616.400 I srv operator(): Chat format: peg-native
1.24.727.467 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.24.727.470 I srv get_availabl: updating prompt cache
1.24.727.511 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.24.727.513 I srv alloc: - prompt is already in the cache, skipping
1.24.727.514 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.24.727.518 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.24.727.519 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.24.727.520 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.24.727.520 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.24.727.521 I srv get_availabl: prompt cache update took 0.05 ms
1.24.727.595 I slot launch_slot_: id 6 | task 475 | processing task, is_child = 0
1.24.732.249 I slot operator(): id 6 | task 475 | Checking checkpoint with [23, 23] against 3...
1.24.732.251 W slot operator(): id 6 | task 475 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.24.732.254 W slot operator(): id 6 | task 475 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.24.734.055 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.24.859.628 I slot print_timing: id 7 | task 402 | n_decoded = 100, tg = 14.48 t/s, tg_3s = 14.48 t/s
1.24.881.938 I slot create_check: id 6 | task 475 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.24.998.269 I slot print_timing: id 9 | task 432 | n_decoded = 102, tg = 23.55 t/s, tg_3s = 23.55 t/s
1.25.113.989 I slot print_timing: id 1 | task 382 | prompt eval time = 116.32 ms / 4 tokens ( 29.08 ms per token, 34.39 tokens per second)
1.25.113.992 I slot print_timing: id 1 | task 382 | eval time = 9213.07 ms / 128 tokens ( 71.98 ms per token, 13.89 tokens per second)
1.25.113.993 I slot print_timing: id 1 | task 382 | total time = 9329.40 ms / 132 tokens
1.25.113.994 I slot print_timing: id 1 | task 382 | graphs reused = 1
1.25.113.997 I slot print_timing: id 1 | task 382 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.25.114.010 I statistics draft-mtp: #calls(b,g,a) = 109 368 5606, #gen drafts = 5621, #acc drafts = 3626, #gen tokens = 11212, #acc tokens = 6699, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.067, 1252.447, 7.017 ms
1.25.114.049 I slot release: id 1 | task 382 | stop processing: n_tokens = 156, truncated = 0
1.25.123.196 I srv operator(): Chat format: peg-native
1.25.235.980 I slot get_availabl: id 1 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.25.235.982 I srv get_availabl: updating prompt cache
1.25.236.023 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.25.236.025 I srv alloc: - prompt is already in the cache, skipping
1.25.236.026 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.25.236.029 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.25.236.029 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.25.236.030 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.25.236.031 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.25.236.032 I srv get_availabl: prompt cache update took 0.05 ms
1.25.236.104 I slot launch_slot_: id 1 | task 480 | processing task, is_child = 0
1.25.239.809 I slot operator(): id 1 | task 480 | Checking checkpoint with [24, 24] against 28...
1.25.243.737 W slot operator(): id 1 | task 480 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.25.243.781 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.603.968 I slot print_timing: id 9 | task 432 | prompt eval time = 116.51 ms / 4 tokens ( 29.13 ms per token, 34.33 tokens per second)
1.25.603.972 I slot print_timing: id 9 | task 432 | eval time = 4937.16 ms / 114 tokens ( 43.31 ms per token, 23.09 tokens per second)
1.25.603.972 I slot print_timing: id 9 | task 432 | total time = 5053.67 ms / 118 tokens
1.25.603.974 I slot print_timing: id 9 | task 432 | graphs reused = 1
1.25.603.977 I slot print_timing: id 9 | task 432 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.25.603.988 I statistics draft-mtp: #calls(b,g,a) = 110 372 5677, #gen drafts = 5683, #acc drafts = 3675, #gen tokens = 11336, #acc tokens = 6790, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.067, 1265.339, 7.110 ms
1.25.604.014 I slot release: id 9 | task 432 | stop processing: n_tokens = 140, truncated = 0
1.25.612.195 I srv operator(): Chat format: peg-native
1.25.717.134 I slot print_timing: id 2 | task 416 | prompt eval time = 128.81 ms / 4 tokens ( 32.20 ms per token, 31.05 tokens per second)
1.25.717.137 I slot print_timing: id 2 | task 416 | eval time = 6516.47 ms / 125 tokens ( 52.13 ms per token, 19.18 tokens per second)
1.25.717.137 I slot print_timing: id 2 | task 416 | total time = 6645.28 ms / 129 tokens
1.25.717.139 I slot print_timing: id 2 | task 416 | graphs reused = 1
1.25.717.143 I slot print_timing: id 2 | task 416 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.25.717.158 I statistics draft-mtp: #calls(b,g,a) = 110 373 5686, #gen drafts = 5698, #acc drafts = 3681, #gen tokens = 11366, #acc tokens = 6802, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.067, 1269.496, 7.121 ms
1.25.717.183 I slot release: id 2 | task 416 | stop processing: n_tokens = 154, truncated = 0
1.25.722.468 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.25.722.470 I srv get_availabl: updating prompt cache
1.25.722.515 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.25.722.517 I srv alloc: - prompt is already in the cache, skipping
1.25.722.518 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.25.722.520 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.25.722.521 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.25.722.521 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.25.722.522 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.25.722.523 I srv get_availabl: prompt cache update took 0.05 ms
1.25.722.587 I slot launch_slot_: id 2 | task 485 | processing task, is_child = 0
1.25.722.588 I slot process_sing: id 9 | task -1 | saving idle slot to prompt cache
1.25.722.607 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.25.722.607 I srv alloc: - prompt is already in the cache, skipping
1.25.726.091 I srv operator(): Chat format: peg-native
1.25.727.712 I slot operator(): id 2 | task 485 | Checking checkpoint with [23, 23] against 27...
1.25.731.581 W slot operator(): id 2 | task 485 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.25.731.621 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.25.843.555 I slot print_timing: id 12 | task 430 | n_decoded = 101, tg = 19.04 t/s, tg_3s = 19.04 t/s
1.25.845.112 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.25.845.113 I srv get_availabl: updating prompt cache
1.25.845.155 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.25.845.156 I srv alloc: - prompt is already in the cache, skipping
1.25.845.157 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.25.845.160 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.25.845.161 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.25.845.162 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.25.845.163 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.25.845.163 I srv get_availabl: prompt cache update took 0.05 ms
1.25.845.233 I slot launch_slot_: id 9 | task 487 | processing task, is_child = 0
1.25.849.404 I slot operator(): id 9 | task 487 | Checking checkpoint with [20, 20] against 24...
1.25.853.326 W slot operator(): id 9 | task 487 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.25.853.366 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.26.089.572 I slot print_timing: id 8 | task 413 | n_decoded = 100, tg = 14.23 t/s, tg_3s = 14.23 t/s
1.26.573.089 I slot print_timing: id 10 | task 447 | n_decoded = 102, tg = 23.79 t/s, tg_3s = 23.79 t/s
1.27.058.220 I slot print_timing: id 12 | task 430 | prompt eval time = 111.74 ms / 4 tokens ( 27.93 ms per token, 35.80 tokens per second)
1.27.058.223 I slot print_timing: id 12 | task 430 | eval time = 6518.07 ms / 125 tokens ( 52.14 ms per token, 19.18 tokens per second)
1.27.058.223 I slot print_timing: id 12 | task 430 | total time = 6629.81 ms / 129 tokens
1.27.058.224 I slot print_timing: id 12 | task 430 | graphs reused = 1
1.27.058.229 I slot print_timing: id 12 | task 430 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.27.058.242 I statistics draft-mtp: #calls(b,g,a) = 112 384 5868, #gen drafts = 5871, #acc drafts = 3795, #gen tokens = 11711, #acc tokens = 7012, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.068, 1303.833, 7.346 ms
1.27.058.268 I slot release: id 12 | task 430 | stop processing: n_tokens = 154, truncated = 0
1.27.067.328 I srv operator(): Chat format: peg-native
1.27.167.354 I slot print_timing: id 7 | task 402 | prompt eval time = 112.70 ms / 4 tokens ( 28.18 ms per token, 35.49 tokens per second)
1.27.167.358 I slot print_timing: id 7 | task 402 | eval time = 9214.46 ms / 128 tokens ( 71.99 ms per token, 13.89 tokens per second)
1.27.167.358 I slot print_timing: id 7 | task 402 | total time = 9327.17 ms / 132 tokens
1.27.167.359 I slot print_timing: id 7 | task 402 | graphs reused = 1
1.27.167.362 I slot print_timing: id 7 | task 402 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.27.167.375 I statistics draft-mtp: #calls(b,g,a) = 112 385 5871, #gen drafts = 5885, #acc drafts = 3796, #gen tokens = 11739, #acc tokens = 7014, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.068, 1309.177, 7.351 ms
1.27.167.399 I slot release: id 7 | task 402 | stop processing: n_tokens = 156, truncated = 0
1.27.172.523 I slot print_timing: id 10 | task 447 | prompt eval time = 261.35 ms / 25 tokens ( 10.45 ms per token, 95.66 tokens per second)
1.27.172.525 I slot print_timing: id 10 | task 447 | eval time = 4887.82 ms / 114 tokens ( 42.88 ms per token, 23.32 tokens per second)
1.27.172.525 I slot print_timing: id 10 | task 447 | total time = 5149.18 ms / 139 tokens
1.27.172.526 I slot print_timing: id 10 | task 447 | graphs reused = 1
1.27.172.529 I slot print_timing: id 10 | task 447 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.27.172.539 I statistics draft-mtp: #calls(b,g,a) = 112 385 5881, #gen drafts = 5885, #acc drafts = 3803, #gen tokens = 11739, #acc tokens = 7028, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.068, 1309.177, 7.362 ms
1.27.172.561 I slot release: id 10 | task 447 | stop processing: n_tokens = 140, truncated = 0
1.27.174.351 I slot get_availabl: id 7 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.27.174.352 I srv get_availabl: updating prompt cache
1.27.174.394 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.27.174.396 I srv alloc: - prompt is already in the cache, skipping
1.27.174.397 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.27.174.399 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.27.174.400 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.27.174.401 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.27.174.401 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.27.174.401 I srv get_availabl: prompt cache update took 0.05 ms
1.27.174.465 I slot launch_slot_: id 7 | task 499 | processing task, is_child = 0
1.27.174.467 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.27.174.485 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.27.174.486 I srv alloc: - prompt is already in the cache, skipping
1.27.174.486 I slot process_sing: id 12 | task -1 | saving idle slot to prompt cache
1.27.174.507 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.27.174.507 I srv alloc: - prompt is already in the cache, skipping
1.27.176.367 I srv operator(): Chat format: peg-native
1.27.180.260 I slot operator(): id 7 | task 499 | Checking checkpoint with [24, 24] against 28...
1.27.180.997 I srv operator(): Chat format: peg-native
1.27.184.127 W slot operator(): id 7 | task 499 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.27.184.166 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.290.328 I slot get_availabl: id 12 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.27.290.330 I srv get_availabl: updating prompt cache
1.27.290.382 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.27.290.384 I srv alloc: - prompt is already in the cache, skipping
1.27.290.386 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.27.290.389 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.27.290.390 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.27.290.391 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.27.290.391 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.27.290.392 I srv get_availabl: prompt cache update took 0.06 ms
1.27.290.462 I slot launch_slot_: id 12 | task 501 | processing task, is_child = 0
1.27.290.464 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.27.290.485 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.27.290.486 I srv alloc: - prompt is already in the cache, skipping
1.27.290.488 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.27.290.489 I srv get_availabl: updating prompt cache
1.27.290.504 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.27.290.504 I srv alloc: - prompt is already in the cache, skipping
1.27.290.504 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.27.290.506 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.27.290.506 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.27.290.506 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.27.290.507 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.27.290.508 I srv get_availabl: prompt cache update took 0.02 ms
1.27.290.532 I slot launch_slot_: id 10 | task 502 | processing task, is_child = 0
1.27.295.121 I slot operator(): id 10 | task 502 | Checking checkpoint with [20, 20] against 24...
1.27.299.069 W slot operator(): id 10 | task 502 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.27.299.099 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.299.130 I slot operator(): id 12 | task 501 | Checking checkpoint with [23, 23] against 27...
1.27.302.720 W slot operator(): id 12 | task 501 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.27.302.748 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.27.417.814 I slot print_timing: id 0 | task 444 | n_decoded = 101, tg = 18.69 t/s, tg_3s = 18.69 t/s
1.27.539.278 I slot print_timing: id 5 | task 429 | n_decoded = 100, tg = 14.29 t/s, tg_3s = 14.29 t/s
1.27.658.571 I slot print_timing: id 3 | task 460 | n_decoded = 102, tg = 23.82 t/s, tg_3s = 23.82 t/s
1.28.262.588 I slot print_timing: id 3 | task 460 | prompt eval time = 116.51 ms / 4 tokens ( 29.13 ms per token, 34.33 tokens per second)
1.28.262.591 I slot print_timing: id 3 | task 460 | eval time = 4886.15 ms / 114 tokens ( 42.86 ms per token, 23.33 tokens per second)
1.28.262.591 I slot print_timing: id 3 | task 460 | total time = 5002.66 ms / 118 tokens
1.28.262.592 I slot print_timing: id 3 | task 460 | graphs reused = 1
1.28.262.595 I slot print_timing: id 3 | task 460 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.28.262.608 I statistics draft-mtp: #calls(b,g,a) = 115 394 6012, #gen drafts = 6024, #acc drafts = 3889, #gen tokens = 12016, #acc tokens = 7187, #mean acc len = 2.20, #acc rate/pos = (0.647, 0.549), dur(b,g,a) = 0.069, 1338.823, 7.517 ms
1.28.262.632 I slot release: id 3 | task 460 | stop processing: n_tokens = 140, truncated = 0
1.28.271.898 I srv operator(): Chat format: peg-native
1.28.375.835 I slot print_timing: id 8 | task 413 | prompt eval time = 250.50 ms / 29 tokens ( 8.64 ms per token, 115.77 tokens per second)
1.28.375.838 I slot print_timing: id 8 | task 413 | eval time = 9314.75 ms / 128 tokens ( 72.77 ms per token, 13.74 tokens per second)
1.28.375.838 I slot print_timing: id 8 | task 413 | total time = 9565.24 ms / 157 tokens
1.28.375.839 I slot print_timing: id 8 | task 413 | graphs reused = 1
1.28.375.842 I slot print_timing: id 8 | task 413 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.28.375.856 I statistics draft-mtp: #calls(b,g,a) = 115 395 6024, #gen drafts = 6038, #acc drafts = 3894, #gen tokens = 12044, #acc tokens = 7197, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.069, 1344.073, 7.529 ms
1.28.375.880 I slot release: id 8 | task 413 | stop processing: n_tokens = 156, truncated = 0
1.28.382.324 I slot get_availabl: id 8 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.28.382.326 I srv get_availabl: updating prompt cache
1.28.382.377 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.28.382.380 I srv alloc: - prompt is already in the cache, skipping
1.28.382.382 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.28.382.386 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.28.382.387 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.28.382.388 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.28.382.388 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.28.382.389 I srv get_availabl: prompt cache update took 0.06 ms
1.28.382.455 I slot launch_slot_: id 8 | task 512 | processing task, is_child = 0
1.28.382.456 I slot process_sing: id 3 | task -1 | saving idle slot to prompt cache
1.28.382.474 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.28.382.475 I srv alloc: - prompt is already in the cache, skipping
1.28.384.535 I srv operator(): Chat format: peg-native
1.28.387.719 I slot operator(): id 8 | task 512 | Checking checkpoint with [24, 24] against 28...
1.28.391.623 W slot operator(): id 8 | task 512 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.28.391.675 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.503.804 I slot print_timing: id 13 | task 458 | n_decoded = 101, tg = 19.22 t/s, tg_3s = 19.22 t/s
1.28.504.704 I slot get_availabl: id 3 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.28.504.705 I srv get_availabl: updating prompt cache
1.28.504.750 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.28.504.752 I srv alloc: - prompt is already in the cache, skipping
1.28.504.753 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.28.504.756 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.28.504.757 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.28.504.758 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.28.504.759 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.28.504.759 I srv get_availabl: prompt cache update took 0.05 ms
1.28.504.829 I slot launch_slot_: id 3 | task 514 | processing task, is_child = 0
1.28.508.598 I slot operator(): id 3 | task 514 | Checking checkpoint with [20, 20] against 3...
1.28.508.600 W slot operator(): id 3 | task 514 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.28.508.603 W slot operator(): id 3 | task 514 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.28.510.401 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.633.019 I slot print_timing: id 0 | task 444 | prompt eval time = 254.69 ms / 28 tokens ( 9.10 ms per token, 109.94 tokens per second)
1.28.633.023 I slot print_timing: id 0 | task 444 | eval time = 6620.29 ms / 125 tokens ( 52.96 ms per token, 18.88 tokens per second)
1.28.633.024 I slot print_timing: id 0 | task 444 | total time = 6874.98 ms / 153 tokens
1.28.633.025 I slot print_timing: id 0 | task 444 | graphs reused = 1
1.28.633.028 I slot print_timing: id 0 | task 444 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.28.633.046 I statistics draft-mtp: #calls(b,g,a) = 116 397 6053, #gen drafts = 6067, #acc drafts = 3914, #gen tokens = 12102, #acc tokens = 7232, #mean acc len = 2.19, #acc rate/pos = (0.647, 0.548), dur(b,g,a) = 0.070, 1353.021, 7.565 ms
1.28.633.072 I slot release: id 0 | task 444 | stop processing: n_tokens = 154, truncated = 0
1.28.641.846 I srv operator(): Chat format: peg-native
1.28.656.684 I slot create_check: id 3 | task 514 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.28.768.643 I slot print_timing: id 14 | task 442 | n_decoded = 100, tg = 14.24 t/s, tg_3s = 14.24 t/s
1.28.768.951 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.28.768.952 I srv get_availabl: updating prompt cache
1.28.768.996 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.28.768.998 I srv alloc: - prompt is already in the cache, skipping
1.28.768.999 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.28.769.003 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.28.769.003 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.28.769.004 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.28.769.005 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.28.769.006 I srv get_availabl: prompt cache update took 0.05 ms
1.28.769.072 I slot launch_slot_: id 0 | task 517 | processing task, is_child = 0
1.28.771.831 I slot operator(): id 0 | task 517 | Checking checkpoint with [23, 23] against 3...
1.28.771.833 W slot operator(): id 0 | task 517 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.28.771.836 W slot operator(): id 0 | task 517 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.28.773.694 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.28.919.360 I slot create_check: id 0 | task 517 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.29.276.006 I slot print_timing: id 6 | task 475 | n_decoded = 102, tg = 23.82 t/s, tg_3s = 23.82 t/s
1.29.761.204 I slot print_timing: id 13 | task 458 | prompt eval time = 110.24 ms / 4 tokens ( 27.56 ms per token, 36.29 tokens per second)
1.29.761.208 I slot print_timing: id 13 | task 458 | eval time = 6512.66 ms / 125 tokens ( 52.10 ms per token, 19.19 tokens per second)
1.29.761.208 I slot print_timing: id 13 | task 458 | total time = 6622.90 ms / 129 tokens
1.29.761.209 I slot print_timing: id 13 | task 458 | graphs reused = 1
1.29.761.213 I slot print_timing: id 13 | task 458 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.29.761.226 I statistics draft-mtp: #calls(b,g,a) = 118 406 6205, #gen drafts = 6207, #acc drafts = 4008, #gen tokens = 12381, #acc tokens = 7406, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.072, 1378.781, 7.751 ms
1.29.761.253 I slot release: id 13 | task 458 | stop processing: n_tokens = 154, truncated = 0
1.29.769.790 I srv operator(): Chat format: peg-native
1.29.870.376 I slot print_timing: id 5 | task 429 | prompt eval time = 115.61 ms / 4 tokens ( 28.90 ms per token, 34.60 tokens per second)
1.29.870.380 I slot print_timing: id 5 | task 429 | eval time = 9330.50 ms / 128 tokens ( 72.89 ms per token, 13.72 tokens per second)
1.29.870.381 I slot print_timing: id 5 | task 429 | total time = 9446.10 ms / 132 tokens
1.29.870.382 I slot print_timing: id 5 | task 429 | graphs reused = 1
1.29.870.385 I slot print_timing: id 5 | task 429 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.29.870.399 I statistics draft-mtp: #calls(b,g,a) = 118 407 6207, #gen drafts = 6221, #acc drafts = 4008, #gen tokens = 12409, #acc tokens = 7406, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.072, 1383.932, 7.754 ms
1.29.870.422 I slot release: id 5 | task 429 | stop processing: n_tokens = 156, truncated = 0
1.29.873.476 I slot print_timing: id 6 | task 475 | prompt eval time = 262.01 ms / 25 tokens ( 10.48 ms per token, 95.42 tokens per second)
1.29.873.478 I slot print_timing: id 6 | task 475 | eval time = 4879.20 ms / 114 tokens ( 42.80 ms per token, 23.36 tokens per second)
1.29.873.478 I slot print_timing: id 6 | task 475 | total time = 5141.21 ms / 139 tokens
1.29.873.479 I slot print_timing: id 6 | task 475 | graphs reused = 1
1.29.873.482 I slot print_timing: id 6 | task 475 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.29.873.489 I statistics draft-mtp: #calls(b,g,a) = 118 407 6213, #gen drafts = 6221, #acc drafts = 4013, #gen tokens = 12409, #acc tokens = 7414, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.072, 1383.932, 7.761 ms
1.29.873.515 I slot release: id 6 | task 475 | stop processing: n_tokens = 140, truncated = 0
1.29.877.193 I slot get_availabl: id 5 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.29.877.194 I srv get_availabl: updating prompt cache
1.29.877.241 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.29.877.243 I srv alloc: - prompt is already in the cache, skipping
1.29.877.244 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.29.877.248 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.29.877.249 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.29.877.250 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.29.877.251 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.29.877.251 I srv get_availabl: prompt cache update took 0.06 ms
1.29.877.318 I slot launch_slot_: id 5 | task 527 | processing task, is_child = 0
1.29.877.319 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
1.29.877.337 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.29.877.338 I srv alloc: - prompt is already in the cache, skipping
1.29.877.338 I slot process_sing: id 13 | task -1 | saving idle slot to prompt cache
1.29.877.356 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.29.877.357 I srv alloc: - prompt is already in the cache, skipping
1.29.879.819 I srv operator(): Chat format: peg-native
1.29.881.590 I srv operator(): Chat format: peg-native
1.29.882.333 I slot operator(): id 5 | task 527 | Checking checkpoint with [24, 24] against 28...
1.29.886.203 W slot operator(): id 5 | task 527 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.29.886.246 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.29.989.247 I slot print_timing: id 4 | task 471 | n_decoded = 101, tg = 18.76 t/s, tg_3s = 18.76 t/s
1.29.993.245 I slot get_availabl: id 13 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.29.993.247 I srv get_availabl: updating prompt cache
1.29.993.292 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.29.993.294 I srv alloc: - prompt is already in the cache, skipping
1.29.993.295 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.29.993.298 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.29.993.299 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.29.993.300 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.29.993.301 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.29.993.302 I srv get_availabl: prompt cache update took 0.05 ms
1.29.993.368 I slot launch_slot_: id 13 | task 529 | processing task, is_child = 0
1.29.993.370 I slot process_sing: id 6 | task -1 | saving idle slot to prompt cache
1.29.993.387 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.29.993.388 I srv alloc: - prompt is already in the cache, skipping
1.29.993.391 I slot get_availabl: id 6 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.179
1.29.993.391 I srv get_availabl: updating prompt cache
1.29.993.405 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.29.993.406 I srv alloc: - prompt is already in the cache, skipping
1.29.993.406 I srv load: - looking for better prompt, base f_keep = 0.179, sim = 1.000
1.29.993.407 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.29.993.408 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.29.993.409 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.29.993.410 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.29.993.410 I srv get_availabl: prompt cache update took 0.02 ms
1.29.993.433 I slot launch_slot_: id 6 | task 530 | processing task, is_child = 0
1.29.998.598 I slot operator(): id 6 | task 530 | Checking checkpoint with [20, 20] against 24...
1.30.002.554 W slot operator(): id 6 | task 530 | restored context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_past = 21, size = 62.978 MiB)
1.30.002.596 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.002.613 I slot operator(): id 13 | task 529 | Checking checkpoint with [23, 23] against 27...
1.30.006.242 W slot operator(): id 13 | task 529 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.30.006.281 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.30.244.981 I slot print_timing: id 9 | task 487 | n_decoded = 102, tg = 23.84 t/s, tg_3s = 23.84 t/s
1.30.246.213 I slot print_timing: id 11 | task 456 | n_decoded = 100, tg = 14.05 t/s, tg_3s = 14.05 t/s
1.30.848.951 I slot print_timing: id 9 | task 487 | prompt eval time = 116.50 ms / 4 tokens ( 29.13 ms per token, 34.33 tokens per second)
1.30.848.955 I slot print_timing: id 9 | task 487 | eval time = 4883.02 ms / 114 tokens ( 42.83 ms per token, 23.35 tokens per second)
1.30.848.956 I slot print_timing: id 9 | task 487 | total time = 4999.53 ms / 118 tokens
1.30.848.957 I slot print_timing: id 9 | task 487 | graphs reused = 1
1.30.848.960 I slot print_timing: id 9 | task 487 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.30.848.975 I statistics draft-mtp: #calls(b,g,a) = 121 415 6338, #gen drafts = 6344, #acc drafts = 4094, #gen tokens = 12655, #acc tokens = 7567, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.075, 1410.623, 7.911 ms
1.30.849.004 I slot release: id 9 | task 487 | stop processing: n_tokens = 140, truncated = 0
1.30.851.840 I slot print_timing: id 15 | task 466 | n_decoded = 100, tg = 14.54 t/s, tg_3s = 14.54 t/s
1.30.858.239 I srv operator(): Chat format: peg-native
1.30.969.325 I slot get_availabl: id 9 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.021
1.30.969.327 I srv get_availabl: updating prompt cache
1.30.969.372 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.30.969.374 I srv alloc: - prompt is already in the cache, skipping
1.30.969.375 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.103
1.30.969.378 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.30.969.379 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.30.969.380 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.30.969.380 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.30.969.380 I srv get_availabl: prompt cache update took 0.05 ms
1.30.969.446 I slot launch_slot_: id 9 | task 539 | processing task, is_child = 0
1.30.974.353 I slot operator(): id 9 | task 539 | Checking checkpoint with [20, 20] against 3...
1.30.974.356 W slot operator(): id 9 | task 539 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.30.974.358 W slot operator(): id 9 | task 539 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.30.976.195 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.31.097.426 I slot print_timing: id 14 | task 442 | prompt eval time = 109.44 ms / 4 tokens ( 27.36 ms per token, 36.55 tokens per second)
1.31.097.429 I slot print_timing: id 14 | task 442 | eval time = 9349.01 ms / 128 tokens ( 73.04 ms per token, 13.69 tokens per second)
1.31.097.430 I slot print_timing: id 14 | task 442 | total time = 9458.45 ms / 132 tokens
1.31.097.431 I slot print_timing: id 14 | task 442 | graphs reused = 1
1.31.097.434 I slot print_timing: id 14 | task 442 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.31.097.449 I statistics draft-mtp: #calls(b,g,a) = 121 417 6359, #gen drafts = 6373, #acc drafts = 4110, #gen tokens = 12712, #acc tokens = 7593, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.075, 1419.680, 7.937 ms
1.31.097.480 I slot release: id 14 | task 442 | stop processing: n_tokens = 156, truncated = 0
1.31.099.082 I slot print_timing: id 2 | task 485 | n_decoded = 101, tg = 19.20 t/s, tg_3s = 19.20 t/s
1.31.106.493 I srv operator(): Chat format: peg-native
1.31.122.766 I slot create_check: id 9 | task 539 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.31.231.577 I slot print_timing: id 4 | task 471 | prompt eval time = 261.54 ms / 28 tokens ( 9.34 ms per token, 107.06 tokens per second)
1.31.231.580 I slot print_timing: id 4 | task 471 | eval time = 6627.35 ms / 125 tokens ( 53.02 ms per token, 18.86 tokens per second)
1.31.231.580 I slot print_timing: id 4 | task 471 | total time = 6888.89 ms / 153 tokens
1.31.231.581 I slot print_timing: id 4 | task 471 | graphs reused = 1
1.31.231.584 I slot print_timing: id 4 | task 471 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.31.231.596 I statistics draft-mtp: #calls(b,g,a) = 122 418 6378, #gen drafts = 6387, #acc drafts = 4122, #gen tokens = 12740, #acc tokens = 7615, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.075, 1424.506, 7.964 ms
1.31.231.623 I slot release: id 4 | task 471 | stop processing: n_tokens = 154, truncated = 0
1.31.235.532 I slot get_availabl: id 4 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.182
1.31.235.534 I srv get_availabl: updating prompt cache
1.31.235.579 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.31.235.580 I srv alloc: - prompt is already in the cache, skipping
1.31.235.581 I srv load: - looking for better prompt, base f_keep = 0.182, sim = 1.000
1.31.235.584 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.31.235.585 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.31.235.587 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.31.235.587 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.31.235.588 I srv get_availabl: prompt cache update took 0.05 ms
1.31.235.655 I slot launch_slot_: id 4 | task 542 | processing task, is_child = 0
1.31.235.656 I slot process_sing: id 14 | task -1 | saving idle slot to prompt cache
1.31.235.673 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.31.235.674 I srv alloc: - prompt is already in the cache, skipping
1.31.240.583 I slot operator(): id 4 | task 542 | Checking checkpoint with [23, 23] against 27...
1.31.240.870 I srv operator(): Chat format: peg-native
1.31.244.385 W slot operator(): id 4 | task 542 | restored context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_past = 24, size = 63.001 MiB)
1.31.244.434 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.31.357.359 I slot get_availabl: id 14 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.31.357.362 I srv get_availabl: updating prompt cache
1.31.357.403 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.31.357.405 I srv alloc: - prompt is already in the cache, skipping
1.31.357.407 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.31.357.409 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.31.357.411 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.31.357.411 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.31.357.411 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.31.357.412 I srv get_availabl: prompt cache update took 0.05 ms
1.31.357.483 I slot launch_slot_: id 14 | task 544 | processing task, is_child = 0
1.31.361.126 I slot operator(): id 14 | task 544 | Checking checkpoint with [24, 24] against 3...
1.31.361.128 W slot operator(): id 14 | task 544 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.31.361.131 W slot operator(): id 14 | task 544 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.31.363.407 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.31.510.416 I slot create_check: id 14 | task 544 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.31.749.004 I slot print_timing: id 10 | task 502 | n_decoded = 102, tg = 23.55 t/s, tg_3s = 23.54 t/s
1.32.228.774 I slot print_timing: id 1 | task 480 | n_decoded = 100, tg = 14.55 t/s, tg_3s = 14.55 t/s
1.32.349.359 I slot print_timing: id 2 | task 485 | prompt eval time = 110.63 ms / 4 tokens ( 27.66 ms per token, 36.16 tokens per second)
1.32.349.362 I slot print_timing: id 2 | task 485 | eval time = 6510.99 ms / 125 tokens ( 52.09 ms per token, 19.20 tokens per second)
1.32.349.363 I slot print_timing: id 2 | task 485 | total time = 6621.62 ms / 129 tokens
1.32.349.363 I slot print_timing: id 2 | task 485 | graphs reused = 1
1.32.349.366 I slot print_timing: id 2 | task 485 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.32.349.382 I statistics draft-mtp: #calls(b,g,a) = 124 427 6514, #gen drafts = 6527, #acc drafts = 4210, #gen tokens = 13020, #acc tokens = 7778, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.077, 1453.390, 8.128 ms
1.32.349.407 I slot release: id 2 | task 485 | stop processing: n_tokens = 154, truncated = 0
1.32.352.820 I slot print_timing: id 10 | task 502 | prompt eval time = 121.72 ms / 4 tokens ( 30.43 ms per token, 32.86 tokens per second)
1.32.352.822 I slot print_timing: id 10 | task 502 | eval time = 4935.97 ms / 114 tokens ( 43.30 ms per token, 23.10 tokens per second)
1.32.352.822 I slot print_timing: id 10 | task 502 | total time = 5057.68 ms / 118 tokens
1.32.352.823 I slot print_timing: id 10 | task 502 | graphs reused = 1
1.32.352.826 I slot print_timing: id 10 | task 502 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.32.352.832 I statistics draft-mtp: #calls(b,g,a) = 124 427 6522, #gen drafts = 6527, #acc drafts = 4214, #gen tokens = 13020, #acc tokens = 7785, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.548), dur(b,g,a) = 0.077, 1453.390, 8.135 ms
1.32.352.855 I slot release: id 10 | task 502 | stop processing: n_tokens = 140, truncated = 0
1.32.360.047 I srv operator(): Chat format: peg-native
1.32.363.424 I srv operator(): Chat format: peg-native
1.32.463.517 I slot get_availabl: id 2 | task -1 | selected slot by LCP similarity, sim_best = 0.103 (> 0.100 thold), f_keep = 0.019
1.32.463.519 I srv get_availabl: updating prompt cache
1.32.463.563 W srv prompt_save: - saving prompt with length 154, total state size = 67.030 MiB (draft: 1.206 MiB)
1.32.463.566 I srv alloc: - prompt is already in the cache, skipping
1.32.463.567 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.103
1.32.463.571 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.32.463.572 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.32.463.572 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.32.463.573 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.32.463.574 I srv get_availabl: prompt cache update took 0.05 ms
1.32.463.644 I slot launch_slot_: id 2 | task 554 | processing task, is_child = 0
1.32.463.645 I slot process_sing: id 10 | task -1 | saving idle slot to prompt cache
1.32.463.664 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.32.463.665 I srv alloc: - prompt is already in the cache, skipping
1.32.463.666 I slot get_availabl: id 10 | task -1 | selected slot by LCP similarity, sim_best = 0.107 (> 0.100 thold), f_keep = 0.021
1.32.463.666 I srv get_availabl: updating prompt cache
1.32.463.680 W srv prompt_save: - saving prompt with length 140, total state size = 66.647 MiB (draft: 1.097 MiB)
1.32.463.681 I srv alloc: - prompt is already in the cache, skipping
1.32.463.682 I srv load: - looking for better prompt, base f_keep = 0.021, sim = 0.107
1.32.463.682 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.32.463.683 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.32.463.683 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.32.463.684 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.32.463.685 I srv get_availabl: prompt cache update took 0.02 ms
1.32.463.704 I slot launch_slot_: id 10 | task 555 | processing task, is_child = 0
1.32.468.794 I slot operator(): id 2 | task 554 | Checking checkpoint with [23, 23] against 3...
1.32.468.796 W slot operator(): id 2 | task 554 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.32.468.799 W slot operator(): id 2 | task 554 | erased invalidated context checkpoint (pos_min = 23, pos_max = 23, n_tokens = 24, n_swa = 0, pos_next = 0, size = 63.001 MiB)
1.32.470.534 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.32.470.562 I slot operator(): id 10 | task 555 | Checking checkpoint with [20, 20] against 3...
1.32.470.562 W slot operator(): id 10 | task 555 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.32.470.564 W slot operator(): id 10 | task 555 | erased invalidated context checkpoint (pos_min = 20, pos_max = 20, n_tokens = 21, n_swa = 0, pos_next = 0, size = 62.978 MiB)
1.32.472.823 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.32.605.917 I slot print_timing: id 11 | task 456 | prompt eval time = 110.27 ms / 4 tokens ( 27.57 ms per token, 36.27 tokens per second)
1.32.605.920 I slot print_timing: id 11 | task 456 | eval time = 9479.21 ms / 128 tokens ( 74.06 ms per token, 13.50 tokens per second)
1.32.605.921 I slot print_timing: id 11 | task 456 | total time = 9589.48 ms / 132 tokens
1.32.605.922 I slot print_timing: id 11 | task 456 | graphs reused = 1
1.32.605.925 I slot print_timing: id 11 | task 456 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
1.32.605.942 I statistics draft-mtp: #calls(b,g,a) = 124 429 6541, #gen drafts = 6554, #acc drafts = 4223, #gen tokens = 13073, #acc tokens = 7802, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.077, 1463.581, 8.157 ms
1.32.605.967 I slot release: id 11 | task 456 | stop processing: n_tokens = 156, truncated = 0
1.32.614.266 I srv operator(): Chat format: peg-native
1.32.631.762 I slot create_check: id 2 | task 554 | created context checkpoint 1 of 32 (pos_min = 24, pos_max = 24, n_tokens = 25, size = 63.009 MiB)
1.32.645.341 I slot create_check: id 10 | task 555 | created context checkpoint 1 of 32 (pos_min = 23, pos_max = 23, n_tokens = 24, size = 63.001 MiB)
1.32.759.059 I slot get_availabl: id 11 | task -1 | selected slot by LCP similarity, sim_best = 0.120 (> 0.100 thold), f_keep = 0.019
1.32.759.061 I srv get_availabl: updating prompt cache
1.32.759.103 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.32.759.105 I srv alloc: - prompt is already in the cache, skipping
1.32.759.106 I srv load: - looking for better prompt, base f_keep = 0.019, sim = 0.120
1.32.759.109 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.32.759.110 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.32.759.111 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.32.759.111 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.32.759.111 I srv get_availabl: prompt cache update took 0.05 ms
1.32.759.182 I slot launch_slot_: id 11 | task 558 | processing task, is_child = 0
1.32.763.258 I slot operator(): id 11 | task 558 | Checking checkpoint with [24, 24] against 3...
1.32.763.260 W slot operator(): id 11 | task 558 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
1.32.763.263 W slot operator(): id 11 | task 558 | erased invalidated context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_swa = 0, pos_next = 0, size = 63.009 MiB)
1.32.764.858 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.32.892.361 I slot print_timing: id 12 | task 501 | n_decoded = 101, tg = 18.45 t/s, tg_3s = 18.45 t/s
1.32.911.294 I slot create_check: id 11 | task 558 | created context checkpoint 1 of 32 (pos_min = 20, pos_max = 20, n_tokens = 21, size = 62.978 MiB)
1.33.262.356 I slot print_timing: id 15 | task 466 | prompt eval time = 115.61 ms / 4 tokens ( 28.90 ms per token, 34.60 tokens per second)
1.33.262.360 I slot print_timing: id 15 | task 466 | eval time = 9287.50 ms / 128 tokens ( 72.56 ms per token, 13.78 tokens per second)
1.33.262.361 I slot print_timing: id 15 | task 466 | total time = 9403.11 ms / 132 tokens
1.33.262.362 I slot print_timing: id 15 | task 466 | graphs reused = 1
1.33.262.365 I slot print_timing: id 15 | task 466 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.33.262.377 I statistics draft-mtp: #calls(b,g,a) = 127 434 6613, #gen drafts = 6628, #acc drafts = 4269, #gen tokens = 13220, #acc tokens = 7886, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.079, 1482.293, 8.252 ms
1.33.262.399 I slot release: id 15 | task 466 | stop processing: n_tokens = 156, truncated = 0
1.33.271.971 I srv operator(): Chat format: peg-native
1.33.376.447 I slot print_timing: id 0 | task 517 | n_decoded = 102, tg = 23.48 t/s, tg_3s = 23.48 t/s
1.33.383.488 I slot get_availabl: id 15 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 0.186
1.33.383.491 I srv get_availabl: updating prompt cache
1.33.383.538 W srv prompt_save: - saving prompt with length 156, total state size = 67.085 MiB (draft: 1.222 MiB)
1.33.383.540 I srv alloc: - prompt is already in the cache, skipping
1.33.383.542 I srv load: - looking for better prompt, base f_keep = 0.186, sim = 1.000
1.33.383.546 I srv update: - cache state: 3 prompts, 389.751 MiB (limits: 8192.000 MiB, 8192 tokens, 9458 est)
1.33.383.547 I srv update: - prompt 0x57e5f7f2d970: 140 tokens, checkpoints: 1, 129.625 MiB
1.33.383.548 I srv update: - prompt 0x57e5eb13f090: 154 tokens, checkpoints: 1, 130.032 MiB
1.33.383.548 I srv update: - prompt 0x57e5f963fdd0: 156 tokens, checkpoints: 1, 130.094 MiB
1.33.383.549 I srv get_availabl: prompt cache update took 0.06 ms
1.33.383.617 I slot launch_slot_: id 15 | task 564 | processing task, is_child = 0
1.33.386.370 I slot operator(): id 15 | task 564 | Checking checkpoint with [24, 24] against 28...
1.33.390.331 W slot operator(): id 15 | task 564 | restored context checkpoint (pos_min = 24, pos_max = 24, n_tokens = 25, n_past = 25, size = 63.009 MiB)
1.33.390.370 I srv stream_sessi: stream_session_attach_pipe: conv_id= (empty=1)
1.33.988.135 I slot print_timing: id 0 | task 517 | prompt eval time = 259.59 ms / 25 tokens ( 10.38 ms per token, 96.30 tokens per second)
1.33.988.138 I slot print_timing: id 0 | task 517 | eval time = 4956.68 ms / 114 tokens ( 43.48 ms per token, 23.00 tokens per second)
1.33.988.139 I slot print_timing: id 0 | task 517 | total time = 5216.28 ms / 139 tokens
1.33.988.140 I slot print_timing: id 0 | task 517 | graphs reused = 1
1.33.988.143 I slot print_timing: id 0 | task 517 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.33.988.156 I statistics draft-mtp: #calls(b,g,a) = 128 440 6707, #gen drafts = 6722, #acc drafts = 4331, #gen tokens = 13408, #acc tokens = 8002, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.080, 1498.836, 8.368 ms
1.33.988.180 I slot release: id 0 | task 517 | stop processing: n_tokens = 140, truncated = 0
1.34.102.650 I slot print_timing: id 3 | task 514 | n_decoded = 101, tg = 18.91 t/s, tg_3s = 18.91 t/s
1.34.107.009 I slot print_timing: id 12 | task 501 | prompt eval time = 117.98 ms / 4 tokens ( 29.50 ms per token, 33.90 tokens per second)
1.34.107.012 I slot print_timing: id 12 | task 501 | eval time = 6689.88 ms / 125 tokens ( 53.52 ms per token, 18.68 tokens per second)
1.34.107.013 I slot print_timing: id 12 | task 501 | total time = 6807.86 ms / 129 tokens
1.34.107.014 I slot print_timing: id 12 | task 501 | graphs reused = 1
1.34.107.017 I slot print_timing: id 12 | task 501 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.34.107.032 I statistics draft-mtp: #calls(b,g,a) = 128 441 6734, #gen drafts = 6737, #acc drafts = 4347, #gen tokens = 13438, #acc tokens = 8033, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.080, 1501.515, 8.402 ms
1.34.107.057 I slot release: id 12 | task 501 | stop processing: n_tokens = 154, truncated = 0
1.34.212.515 I slot print_timing: id 7 | task 499 | n_decoded = 100, tg = 14.43 t/s, tg_3s = 14.43 t/s
1.34.426.635 I slot print_timing: id 6 | task 530 | n_decoded = 102, tg = 23.68 t/s, tg_3s = 23.68 t/s
1.34.529.757 I slot print_timing: id 1 | task 480 | prompt eval time = 116.93 ms / 4 tokens ( 29.23 ms per token, 34.21 tokens per second)
1.34.529.760 I slot print_timing: id 1 | task 480 | eval time = 9172.99 ms / 128 tokens ( 71.66 ms per token, 13.95 tokens per second)
1.34.529.760 I slot print_timing: id 1 | task 480 | total time = 9289.92 ms / 132 tokens
1.34.529.761 I slot print_timing: id 1 | task 480 | graphs reused = 1
1.34.529.771 I slot print_timing: id 1 | task 480 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.34.529.785 I statistics draft-mtp: #calls(b,g,a) = 128 445 6779, #gen drafts = 6792, #acc drafts = 4376, #gen tokens = 13547, #acc tokens = 8084, #mean acc len = 2.19, #acc rate/pos = (0.646, 0.547), dur(b,g,a) = 0.080, 1516.026, 8.457 ms
1.34.529.808 I slot release: id 1 | task 480 | stop processing: n_tokens = 156, truncated = 0
1.34.935.137 I slot print_timing: id 6 | task 530 | prompt eval time = 121.37 ms / 4 tokens ( 30.34 ms per token, 32.96 tokens per second)
1.34.935.142 I slot print_timing: id 6 | task 530 | eval time = 4815.15 ms / 114 tokens ( 42.24 ms per token, 23.68 tokens per second)
1.34.935.142 I slot print_timing: id 6 | task 530 | total time = 4936.52 ms / 118 tokens
1.34.935.143 I slot print_timing: id 6 | task 530 | graphs reused = 1
1.34.935.147 I slot print_timing: id 6 | task 530 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.34.935.162 I statistics draft-mtp: #calls(b,g,a) = 128 449 6836, #gen drafts = 6844, #acc drafts = 4412, #gen tokens = 13651, #acc tokens = 8153, #mean acc len = 2.19, #acc rate/pos = (0.645, 0.547), dur(b,g,a) = 0.080, 1530.428, 8.532 ms
1.34.935.188 I slot release: id 6 | task 530 | stop processing: n_tokens = 140, truncated = 0
1.35.124.534 I slot print_timing: id 3 | task 514 | prompt eval time = 253.67 ms / 28 tokens ( 9.06 ms per token, 110.38 tokens per second)
1.35.124.537 I slot print_timing: id 3 | task 514 | eval time = 6362.25 ms / 125 tokens ( 50.90 ms per token, 19.65 tokens per second)
1.35.124.538 I slot print_timing: id 3 | task 514 | total time = 6615.92 ms / 153 tokens
1.35.124.539 I slot print_timing: id 3 | task 514 | graphs reused = 1
1.35.124.542 I slot print_timing: id 3 | task 514 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.35.124.557 I statistics draft-mtp: #calls(b,g,a) = 128 451 6858, #gen drafts = 6868, #acc drafts = 4426, #gen tokens = 13699, #acc tokens = 8178, #mean acc len = 2.19, #acc rate/pos = (0.645, 0.547), dur(b,g,a) = 0.080, 1540.217, 8.565 ms
1.35.124.581 I slot release: id 3 | task 514 | stop processing: n_tokens = 154, truncated = 0
1.35.215.798 I slot print_timing: id 13 | task 529 | n_decoded = 101, tg = 19.82 t/s, tg_3s = 19.82 t/s
1.35.301.883 I slot print_timing: id 8 | task 512 | n_decoded = 100, tg = 14.70 t/s, tg_3s = 14.70 t/s
1.35.568.227 I slot print_timing: id 14 | task 544 | n_decoded = 102, tg = 25.85 t/s, tg_3s = 25.85 t/s
1.36.000.076 I slot print_timing: id 7 | task 499 | prompt eval time = 103.56 ms / 4 tokens ( 25.89 ms per token, 38.62 tokens per second)
1.36.000.081 I slot print_timing: id 7 | task 499 | eval time = 8716.23 ms / 128 tokens ( 68.10 ms per token, 14.69 tokens per second)
1.36.000.081 I slot print_timing: id 7 | task 499 | total time = 8819.79 ms / 132 tokens
1.36.000.082 I slot print_timing: id 7 | task 499 | graphs reused = 1
1.36.000.085 I slot print_timing: id 7 | task 499 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.36.000.097 I statistics draft-mtp: #calls(b,g,a) = 128 461 6967, #gen drafts = 6977, #acc drafts = 4489, #gen tokens = 13916, #acc tokens = 8293, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.546), dur(b,g,a) = 0.080, 1592.507, 8.714 ms
1.36.000.119 I slot release: id 7 | task 499 | stop processing: n_tokens = 156, truncated = 0
1.36.004.155 I slot print_timing: id 14 | task 544 | prompt eval time = 261.61 ms / 25 tokens ( 10.46 ms per token, 95.56 tokens per second)
1.36.004.157 I slot print_timing: id 14 | task 544 | eval time = 4381.41 ms / 114 tokens ( 38.43 ms per token, 26.02 tokens per second)
1.36.004.158 I slot print_timing: id 14 | task 544 | total time = 4643.02 ms / 139 tokens
1.36.004.159 I slot print_timing: id 14 | task 544 | graphs reused = 1
1.36.004.162 I slot print_timing: id 14 | task 544 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.36.004.171 I statistics draft-mtp: #calls(b,g,a) = 128 461 6976, #gen drafts = 6977, #acc drafts = 4495, #gen tokens = 13916, #acc tokens = 8302, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.546), dur(b,g,a) = 0.080, 1592.507, 8.726 ms
1.36.004.192 I slot release: id 14 | task 544 | stop processing: n_tokens = 140, truncated = 0
1.36.075.378 I slot print_timing: id 4 | task 542 | n_decoded = 101, tg = 21.38 t/s, tg_3s = 21.38 t/s
1.36.077.969 I slot print_timing: id 13 | task 529 | prompt eval time = 117.62 ms / 4 tokens ( 29.41 ms per token, 34.01 tokens per second)
1.36.077.972 I slot print_timing: id 13 | task 529 | eval time = 5957.72 ms / 125 tokens ( 47.66 ms per token, 20.98 tokens per second)
1.36.077.972 I slot print_timing: id 13 | task 529 | total time = 6075.34 ms / 129 tokens
1.36.077.974 I slot print_timing: id 13 | task 529 | graphs reused = 1
1.36.077.976 I slot print_timing: id 13 | task 529 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.36.077.988 I statistics draft-mtp: #calls(b,g,a) = 128 462 6985, #gen drafts = 6986, #acc drafts = 4500, #gen tokens = 13934, #acc tokens = 8312, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.546), dur(b,g,a) = 0.080, 1598.207, 8.739 ms
1.36.078.007 I slot release: id 13 | task 529 | stop processing: n_tokens = 154, truncated = 0
1.36.206.489 I slot print_timing: id 5 | task 527 | n_decoded = 100, tg = 16.08 t/s, tg_3s = 16.08 t/s
1.36.405.256 I slot print_timing: id 11 | task 558 | n_decoded = 102, tg = 30.17 t/s, tg_3s = 30.17 t/s
1.36.726.092 I slot print_timing: id 8 | task 512 | prompt eval time = 110.06 ms / 4 tokens ( 27.52 ms per token, 36.34 tokens per second)
1.36.726.094 I slot print_timing: id 8 | task 512 | eval time = 8228.29 ms / 128 tokens ( 64.28 ms per token, 15.56 tokens per second)
1.36.726.095 I slot print_timing: id 8 | task 512 | total time = 8338.35 ms / 132 tokens
1.36.726.096 I slot print_timing: id 8 | task 512 | graphs reused = 1
1.36.726.099 I slot print_timing: id 8 | task 512 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.373, 0.307)
1.36.726.113 I statistics draft-mtp: #calls(b,g,a) = 128 472 7058, #gen drafts = 7065, #acc drafts = 4542, #gen tokens = 14091, #acc tokens = 8388, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.545), dur(b,g,a) = 0.080, 1645.510, 8.831 ms
1.36.726.131 I slot release: id 8 | task 512 | stop processing: n_tokens = 156, truncated = 0
1.36.727.409 I slot print_timing: id 4 | task 542 | prompt eval time = 110.43 ms / 4 tokens ( 27.61 ms per token, 36.22 tokens per second)
1.36.727.411 I slot print_timing: id 4 | task 542 | eval time = 5376.38 ms / 125 tokens ( 43.01 ms per token, 23.25 tokens per second)
1.36.727.411 I slot print_timing: id 4 | task 542 | total time = 5486.81 ms / 129 tokens
1.36.727.411 I slot print_timing: id 4 | task 542 | graphs reused = 1
1.36.727.413 I slot print_timing: id 4 | task 542 | draft acceptance = 0.68868 ( 73 accepted / 106 generated), mean acceptance length = 2.38, acceptance rate per position = (0.736, 0.642)
1.36.727.420 I statistics draft-mtp: #calls(b,g,a) = 128 472 7060, #gen drafts = 7065, #acc drafts = 4544, #gen tokens = 14091, #acc tokens = 8392, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.545), dur(b,g,a) = 0.080, 1645.510, 8.835 ms
1.36.727.434 I slot release: id 4 | task 542 | stop processing: n_tokens = 154, truncated = 0
1.36.729.165 I slot print_timing: id 11 | task 558 | prompt eval time = 260.75 ms / 25 tokens ( 10.43 ms per token, 95.88 tokens per second)
1.36.729.167 I slot print_timing: id 11 | task 558 | eval time = 3705.16 ms / 114 tokens ( 32.50 ms per token, 30.77 tokens per second)
1.36.729.167 I slot print_timing: id 11 | task 558 | total time = 3965.90 ms / 139 tokens
1.36.729.168 I slot print_timing: id 11 | task 558 | graphs reused = 1
1.36.729.171 I slot print_timing: id 11 | task 558 | draft acceptance = 0.93750 ( 75 accepted / 80 generated), mean acceptance length = 2.88, acceptance rate per position = (1.000, 0.875)
1.36.729.177 I statistics draft-mtp: #calls(b,g,a) = 128 472 7064, #gen drafts = 7065, #acc drafts = 4546, #gen tokens = 14091, #acc tokens = 8396, #mean acc len = 2.19, #acc rate/pos = (0.644, 0.545), dur(b,g,a) = 0.080, 1645.510, 8.840 ms
1.36.729.193 I slot release: id 11 | task 558 | stop processing: n_tokens = 140, truncated = 0
1.36.813.822 I slot print_timing: id 10 | task 555 | n_decoded = 101, tg = 24.87 t/s, tg_3s = 24.87 t/s
1.36.897.146 I slot print_timing: id 9 | task 539 | n_decoded = 100, tg = 17.64 t/s, tg_3s = 17.64 t/s
1.37.189.879 I slot print_timing: id 5 | task 527 | prompt eval time = 104.07 ms / 4 tokens ( 26.02 ms per token, 38.44 tokens per second)
1.37.189.883 I slot print_timing: id 5 | task 527 | eval time = 7203.46 ms / 128 tokens ( 56.28 ms per token, 17.77 tokens per second)
1.37.189.883 I slot print_timing: id 5 | task 527 | total time = 7307.53 ms / 132 tokens
1.37.189.884 I slot print_timing: id 5 | task 527 | graphs reused = 1
1.37.189.887 I slot print_timing: id 5 | task 527 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.37.189.901 I statistics draft-mtp: #calls(b,g,a) = 128 483 7115, #gen drafts = 7119, #acc drafts = 4567, #gen tokens = 14198, #acc tokens = 8434, #mean acc len = 2.19, #acc rate/pos = (0.642, 0.543), dur(b,g,a) = 0.080, 1688.633, 8.909 ms
1.37.189.918 I slot release: id 5 | task 527 | stop processing: n_tokens = 156, truncated = 0
1.37.225.651 I slot print_timing: id 10 | task 555 | prompt eval time = 282.24 ms / 28 tokens ( 10.08 ms per token, 99.21 tokens per second)
1.37.225.654 I slot print_timing: id 10 | task 555 | eval time = 4472.84 ms / 125 tokens ( 35.78 ms per token, 27.95 tokens per second)
1.37.225.654 I slot print_timing: id 10 | task 555 | total time = 4755.08 ms / 153 tokens
1.37.225.655 I slot print_timing: id 10 | task 555 | graphs reused = 1
1.37.225.657 I slot print_timing: id 10 | task 555 | draft acceptance = 0.66667 ( 72 accepted / 108 generated), mean acceptance length = 2.33, acceptance rate per position = (0.722, 0.611)
1.37.225.668 I statistics draft-mtp: #calls(b,g,a) = 128 484 7122, #gen drafts = 7123, #acc drafts = 4572, #gen tokens = 14206, #acc tokens = 8442, #mean acc len = 2.19, #acc rate/pos = (0.642, 0.543), dur(b,g,a) = 0.080, 1691.781, 8.920 ms
1.37.225.684 I slot release: id 10 | task 555 | stop processing: n_tokens = 154, truncated = 0
1.37.304.245 I slot print_timing: id 2 | task 554 | n_decoded = 100, tg = 21.97 t/s, tg_3s = 21.97 t/s
1.37.436.471 I slot print_timing: id 15 | task 564 | n_decoded = 100, tg = 25.42 t/s, tg_3s = 25.42 t/s
1.37.509.414 I slot print_timing: id 9 | task 539 | prompt eval time = 254.47 ms / 29 tokens ( 8.77 ms per token, 113.96 tokens per second)
1.37.509.416 I slot print_timing: id 9 | task 539 | eval time = 6280.59 ms / 128 tokens ( 49.07 ms per token, 20.38 tokens per second)
1.37.509.417 I slot print_timing: id 9 | task 539 | total time = 6535.05 ms / 157 tokens
1.37.509.418 I slot print_timing: id 9 | task 539 | graphs reused = 1
1.37.509.420 I slot print_timing: id 9 | task 539 | draft acceptance = 0.33113 ( 50 accepted / 151 generated), mean acceptance length = 1.66, acceptance rate per position = (0.368, 0.289)
1.37.509.431 I statistics draft-mtp: #calls(b,g,a) = 128 495 7153, #gen drafts = 7155, #acc drafts = 4586, #gen tokens = 14269, #acc tokens = 8465, #mean acc len = 2.18, #acc rate/pos = (0.641, 0.542), dur(b,g,a) = 0.080, 1721.660, 8.957 ms
1.37.509.448 I slot release: id 9 | task 539 | stop processing: n_tokens = 156, truncated = 0
1.37.697.657 I slot print_timing: id 2 | task 554 | prompt eval time = 283.73 ms / 29 tokens ( 9.78 ms per token, 102.21 tokens per second)
1.37.697.659 I slot print_timing: id 2 | task 554 | eval time = 4945.13 ms / 128 tokens ( 38.63 ms per token, 25.88 tokens per second)
1.37.697.659 I slot print_timing: id 2 | task 554 | total time = 5228.86 ms / 157 tokens
1.37.697.660 I slot print_timing: id 2 | task 554 | graphs reused = 1
1.37.697.663 I slot print_timing: id 2 | task 554 | draft acceptance = 0.34228 ( 51 accepted / 149 generated), mean acceptance length = 1.68, acceptance rate per position = (0.387, 0.293)
1.37.697.673 I statistics draft-mtp: #calls(b,g,a) = 128 506 7175, #gen drafts = 7176, #acc drafts = 4591, #gen tokens = 14310, #acc tokens = 8473, #mean acc len = 2.18, #acc rate/pos = (0.640, 0.541), dur(b,g,a) = 0.080, 1741.207, 8.978 ms
1.37.697.687 I slot release: id 2 | task 554 | stop processing: n_tokens = 156, truncated = 0
1.37.740.743 I slot print_timing: id 15 | task 564 | prompt eval time = 116.52 ms / 4 tokens ( 29.13 ms per token, 34.33 tokens per second)
1.37.740.746 I slot print_timing: id 15 | task 564 | eval time = 4237.85 ms / 128 tokens ( 33.11 ms per token, 30.20 tokens per second)
1.37.740.746 I slot print_timing: id 15 | task 564 | total time = 4354.37 ms / 132 tokens
1.37.740.747 I slot print_timing: id 15 | task 564 | graphs reused = 4
1.37.740.750 I slot print_timing: id 15 | task 564 | draft acceptance = 0.35374 ( 52 accepted / 147 generated), mean acceptance length = 1.70, acceptance rate per position = (0.392, 0.311)
1.37.740.760 I statistics draft-mtp: #calls(b,g,a) = 128 510 7180, #gen drafts = 7180, #acc drafts = 4592, #gen tokens = 14317, #acc tokens = 8475, #mean acc len = 2.18, #acc rate/pos = (0.640, 0.541), dur(b,g,a) = 0.080, 1745.092, 8.985 ms
1.37.740.776 I slot release: id 15 | task 564 | stop processing: n_tokens = 156, truncated = 0
1.37.740.786 I srv update_slots: all slots are idle