artificial-citizen commited on
Commit
5957ee5
·
verified ·
1 Parent(s): 98ac344

card: --fit confirmed as the cause, not just the suspect

Browse files
Files changed (1) hide show
  1. README.md +7 -5
README.md CHANGED
@@ -149,11 +149,13 @@ Measured resident VRAM, `Q6_K`, `--ctx-size 4096`, `--n-gpu-layers 99`, flash-at
149
  > round trip pays the host-boundary cost again — partial offload hurts MTP roughly twice as
150
  > much as it hurts plain decode.
151
  >
152
- > **The most likely cause is `--fit`, which is ON by default.** It silently adjusts any
153
- > argument you left *unset* — including `-ngl` — to fit device memory with a 1024 MiB margin.
154
- > Enabling MTP raises the estimate, so `--fit` can quietly shave the layers that MTP needs.
155
- > A machine reporting comfortable free VRAM is **not** evidence against this: `--fit` keeps
156
- > usage under budget precisely by moving layers to the host.
 
 
157
  >
158
  > **Diagnose it:** check the `offloaded X/Y layers to GPU` line at startup. Anything but
159
  > `34/34` explains the slowdown by itself.
 
149
  > round trip pays the host-boundary cost again — partial offload hurts MTP roughly twice as
150
  > much as it hurts plain decode.
151
  >
152
+ > **The cause is `--fit`, which is ON by default.** It silently adjusts any argument you left
153
+ > *unset* — including `-ngl` — to fit device memory with a 1024 MiB margin. Enabling MTP raises
154
+ > the estimate, so `--fit` quietly shaves the layers that MTP needs. A machine reporting
155
+ > comfortable free VRAM is **not** evidence against this: `--fit` keeps usage under budget
156
+ > precisely by moving layers to the host. Confirmed in the field — this was diagnosed from a
157
+ > user report on a 16 GB card sitting at 12.6 GB used, and `-fit off` with an explicit
158
+ > `-ngl 99` fixed it.
159
  >
160
  > **Diagnose it:** check the `offloaded X/Y layers to GPU` line at startup. Anything but
161
  > `34/34` explains the slowdown by itself.