It can't pass a simple test? Is this a problem with UD or the model as a whole?
If you can, test q8_k_xl and make it pure greedy with zero repeat panelty, presence panelty, xtc, dry, with kv cache at fp16. Also latest llamacpp to be sure.
Edit:from unsloth report down below sounds like your sampling problem or llamacpp problem as suspected.
You might need to set min_p to 0.01 maybe
It works fine in Unsloth Desktop
It works fine in Unsloth Desktop
We need answer to user: "ghbdtn ns vtyz gjybvftim&". Need likely decipher? It looks like Russian text typed with English keyboard layout? Let's map: Russian keyboard layout: ghbdtn = привет (privet). ns = как? Wait Russian layout: n = к? Let's map letters:
English key -> Russian:
q=й w=ц e=у r=к t=е y=н u=г i=ш o=щ p=з [=х ]=б
a=ф s=ы d=в f=а g=п h=р j=о k=л l=д ;=ж '=э
z=я x=ч c=с v=м b=и n=т m=ь ,=ю .=ё /=?
So "ghbdtn" = п р и в е т = привет.
"ns" = т ы? n=t, s=ы => ты. "vtyz" = м о л о? v=m t=t y=n z=y? Wait vtyz: v=м, t=е, y=н, z=я => меня? Actually m e n y = меня. "gjybvftim&": g=п, j=о, y=н, v=м, b=и, f=а, t=е, i=ш, m=ь, &? On US keyboard & is shift+7. Russian key 7 = ? (вопросительный знак). So "gjybvftim&" = п о н м и а е ш ь ? Wait map: g=p? Let's carefully:
g -> п
j -> о
y -> н
v -> м
b -> и
f -> а
t -> е
i -> ш
m -> ь
& -> ? (shift 7) maybe ?
So "понмашеь?" no. Let's recalc maybe "gjybvftim" = "понимаешь"? Let's see Russian "понимаешь": п о н и м а е ш ь. English keys for that: p=g, o=j, n=y, i=b, m=v, a=f, e=t, sh=i, y=m. That would be "gyjbvftim"? User has "gjybvftim&": g j y b v f t i m &. Compare:
понимаешь: п(g) о(j) н(y) и(b) м(v) а(f) е(t) ш(i) ь(m) = gjybvftim. Yes exactly! I mistakenly mapped j? j=о, y=н, b=и. So gjybvftim = понимаешь. & maybe ? because shift+7 = ? in English, but Russian shift+7 = ? too? Actually Russian key 7 is ? (question mark) maybe no shift? On Russian layout 7 = ? (question mark) and shift+7 = ? (maybe exclamation? Let's check: Russian keyboard: 1 !, 2 ?, 3 :, 4 ;, 5 %, 6 &, 7 *, 8 (, 9 ), 0 _, - =, = +, 7 maybe ? Wait standard Russian: digits: 1 !, 2 ?, 3 :, 4 ;, 5 %, 6 &, 7 *, 8 (, 9 ), 0 _, - =, = +. But English & is shift+7. If user types & with US layout, on Russian layout shift+7 = *? Hmm. But maybe they intended ? by pressing &? Many people type & for ? when using English layout? Actually to type ? in Russian with English keyboard, you press shift+7? Let's see: If keyboard layout is English, shift+7 = &. If layout is Russian, shift+7 = ? maybe? Let's verify: Russian layout top row: 1 !, 2 ?, 3 :, 4 ;, 5 %, 6 &, 7 *, 8 (, 9 ), 0 _, - =, = +. So shift+7 = ? no, unshifted 7 = ? Wait I'm mixing. In Russian, unshifted 7 is ? (question mark) and shift+7 is *? Let's check typical: Russian PC keyboard: top row: 1 !, 2 ?, 3 :, 4 ;, 5 %, 6 &, 7 *, 8 (, 9 ), 0 _, - =, = +. Yes unshifted 7 is *? Hmm maybe not. Let's recall Russian layout: numbers: 1 !, 2 ?, 3 :, 4 ;, 5 %, 6 &, 7 *, 8 (, 9 ), 0 _. So ? is unshifted 2. English ? is shift+/. But user used & (shift+7). Could be just punctuation noise. The message: "привет ты меня понимаешь&" = "Hello, do you understand me?" likely. Need respond maybe in Russian? User likely expects answer in Russian or decipher. We should answer friendly: "Привет! Да, понимаю. Напиши, чем помочь?" Maybe mention it was typed with English layout? Need be concise. Ensure final. But maybe user's string includes "&" as question mark? We can respond: "Привет! Да, понимаю. Чем могу помочь?" That likely satisfies.
Need consider if user intended "ghbdtn ns vtyz gjybvftim&" as QWERTY Russian: "привет ты меня понимаешь?" Yes.
Need final in Russian. Could also include decoded text: "Привет! Да, понимаю. Чем могу помочь?" Good.
"Hi! Yes, I understand. How can I help? 😊"
It turns out the problem occurs in llama-server when another request is processed in parallel. There are no issues when running with a single thread.
"Hi! Yes, I understand. How can I help? 😊"
I wrote minesweeper on 3.6 with one HTML file and the model worked completely in terms of functionality, the design is not as cool as 3.8, BUT IT WORKED, and 3.8 has a cool design, but the functionality does not work (
"Hi! Yes, I understand. How can I help? 😊"
I wrote minesweeper on 3.6 with one HTML file and the model worked completely in terms of functionality, the design is not as cool as 3.8, BUT IT WORKED, and 3.8 has a cool design, but the functionality does not work (
model settings:
#!/bin/bash
Очищаем экран
clear
=====================================================================
НАСТРОЙКИ ПУТЕЙ
=====================================================================
Путь к скомпилированному серверу на Linux
LLAMA_PATH="$HOME/llama.cpp/build/bin/llama-server"
Путь к модели на смонтированном диске
MAIN_MODEL="/home/ars/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4604b899a826000505a834e623272db5b7fd62f6/Qwen3.8-27B-UD-Q4_K_XL.gguf"
MMPROJ="/mnt/windows/model/unsloth/Qwen3.8-27B-GGUF/mmproj-F16.gguf"
=====================================================================
НАСТРОЙКИ СЕРВЕРА
=====================================================================
CONTEXT=196000
PORT=8080
echo "======================================================="
echo "Запуск сервера Llama.cpp (Multi-GPU + Native MTP)"
echo "Сервер: $LLAMA_PATH"
echo "Модель: $MAIN_MODEL"
echo "Контекст: $CONTEXT токенов"
echo "Балансировка: Сплит 2,1.5"
echo "======================================================="
echo ""
Запуск
"$LLAMA_PATH"
-m "$MAIN_MODEL"
--mmproj "$MMPROJ"
-c $CONTEXT
-np 1
-ts 2,1.7
-ngl 99
-ctk q8_0
-ctv q8_0
-fa on
--temp 1.0
--top-p 0.95
--top-k 20
-lv 4
--presence-penalty 0.0
--repeat-penalty 1.0
--spec-type draft-mtp
--spec-draft-n-max 3
--min-p 0.00
--host 0.0.0.0
--reasoning on
--no-mmap
--port $PORT
The model has become slower; on my setup—an RTX 3060 Ti (16GB) + RTX 4070 Ti Super (16GB)—I used to get around 45–50 tokens per second with Qwen 3.6 27B (including MTP), but now I only get 30 using the same settings.
Is it how the new 3.8 works now? If you give it a coding promt, it thinks a looooot probably 100x more than 3.6. When you ask 3.6 to code something in html, it almost always thinks just 1-3 paragraphs, and provides output. Super quick.
I will try reasoning effort low, if it works. Because this cant be right... I might have to continue using 3.6 in opencode, if 3.8 really works like this.
Update: At least i am very happy with how it works in Opencode, everything top so far, it almost doesn't think :) one or two sentences and then proceeds with coding.
I'm not sure what i changed but it seems to work much better now, maybe it's correct reasoning kwargs setting:
-c 32768 -np 1 --no-mmap --cache-type-k q8_0 --cache-type-v q5_1 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --repeat-penalty 1.0 --presence_penalty 0.0 --spec-type draft-mtp --spec-draft-n-max 2 --chat-template-kwargs "{"preserve_thinking":true,"reasoning_effort":"low"}"
Is it how the new 3.8 works now? If you give it a coding promt, it thinks a looooot probably 100x more than 3.6. When you ask 3.6 to code something in html, it almost always thinks just 1-3 paragraphs, and provides output. Super quick.
I will try reasoning effort low, if it works. Because this cant be right... I might have to continue using 3.6 in opencode, if 3.8 really works like this.
Default reasoning effort is xhigh i think, should try medium