gemma-4-26B-A4B-it-GGUF-UD-Q4_K_XL

Sleeping

filip-krolop commited on Apr 21

Commit

fe1795b

verified ·

1 Parent(s): e93b5d3

Update Dockerfile

Files changed (1) hide show

Dockerfile CHANGED Viewed

@@ -20,5 +20,7 @@ CMD ["-m", "/app/gemma-4-31B-it-IQ4_NL.gguf", \
      "-t", "2", \
      "--cache-type-k", "q8_0", \
      "--cache-type-v", "iq4_nl", \
-     "-c", "16000", \
-     "-n", "19456"]

      "-t", "2", \
      "--cache-type-k", "q8_0", \
      "--cache-type-v", "iq4_nl", \
+     "-c", "1024", \
+     "-n", "1024", \
+     "--n-parallel", "1", \
+     "--no-kv-offload"]