card: --fit confirmed as the cause, not just the suspect
Browse files
README.md
CHANGED
|
@@ -149,11 +149,13 @@ Measured resident VRAM, `Q6_K`, `--ctx-size 4096`, `--n-gpu-layers 99`, flash-at
|
|
| 149 |
> round trip pays the host-boundary cost again — partial offload hurts MTP roughly twice as
|
| 150 |
> much as it hurts plain decode.
|
| 151 |
>
|
| 152 |
-
> **The
|
| 153 |
-
>
|
| 154 |
-
>
|
| 155 |
-
>
|
| 156 |
-
>
|
|
|
|
|
|
|
| 157 |
>
|
| 158 |
> **Diagnose it:** check the `offloaded X/Y layers to GPU` line at startup. Anything but
|
| 159 |
> `34/34` explains the slowdown by itself.
|
|
|
|
| 149 |
> round trip pays the host-boundary cost again — partial offload hurts MTP roughly twice as
|
| 150 |
> much as it hurts plain decode.
|
| 151 |
>
|
| 152 |
+
> **The cause is `--fit`, which is ON by default.** It silently adjusts any argument you left
|
| 153 |
+
> *unset* — including `-ngl` — to fit device memory with a 1024 MiB margin. Enabling MTP raises
|
| 154 |
+
> the estimate, so `--fit` quietly shaves the layers that MTP needs. A machine reporting
|
| 155 |
+
> comfortable free VRAM is **not** evidence against this: `--fit` keeps usage under budget
|
| 156 |
+
> precisely by moving layers to the host. Confirmed in the field — this was diagnosed from a
|
| 157 |
+
> user report on a 16 GB card sitting at 12.6 GB used, and `-fit off` with an explicit
|
| 158 |
+
> `-ngl 99` fixed it.
|
| 159 |
>
|
| 160 |
> **Diagnose it:** check the `offloaded X/Y layers to GPU` line at startup. Anything but
|
| 161 |
> `34/34` explains the slowdown by itself.
|