Spaces:

apathy-exe
/

Xing4.0-29B-A4B

Running on Zero

apathy-exe commited on about 11 hours ago

Commit

249b2e0

verified ·

1 Parent(s): 629f378

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -107,6 +107,11 @@ def start_llama_server():
         "-t", "16",
         "-tb", "16",
         "-fa", "on",
         "--spec-type", "draft-mtp",
         "--spec-draft-n-max", "2",
         "--spec-draft-n-min", "1",

         "-t", "16",
         "-tb", "16",
         "-fa", "on",
+        "--load-mode", "mmap",
+        "--no-warmup",
+        "--no-repack",
+        "-lzm", "on",
+        "--fit", "off",
         "--spec-type", "draft-mtp",
         "--spec-draft-n-max", "2",
         "--spec-draft-n-min", "1",