ALPHA: a Free VRAM killswitch, not just an automatic unload

`unload` only ever targets the model this pack configured. That is right for
the automatic path -- it hands back exactly what writing a plan caused to be
loaded -- but it is not enough, because the three situations that actually OOM
a render are the ones where the configured model is not what is resident:

* the "Unload after writing" checkbox was off
* the write failed before it reached the unload
* LM Studio's JIT loaded something other than what was asked for

PromptMasterLD covers all three with `lms unload --all`. That is a subprocess,
which is the rung that got 0.4.1-0.4.3 registry-Flagged under
`python_command_injection_risk`, so `unload_all` lists loaded models over HTTP
and walks each through the same four-rung ladder instead. Same effect, nothing
spawned. The rungs move into `_unload_one` so both paths share them.

Two behaviours worth stating. `shares_this_gpu` still guards it, so the
killswitch never reaches across a network to evict someone else's model --
freeing VRAM on a laptop does not help a desktop render. And a server that
does not report load state (no `/api/v0/models`) falls back to the configured
model rather than evicting nothing, saying so in the note.

The button sits in the main WRITE row, not in Settings: it is what you reach
for when a render just OOMed, and a killswitch behind a disclosure is not one.
The row now wraps rather than squeezing the brief input on a narrow node.

check_planner grows six assertions for the parts that need no server -- the
local-only guard, and that a remote or unconfigured writer is a clean no-op
rather than an error. Verified live against LM Studio with both models
unloaded: lists 2, evicts 0, reports "nothing is loaded". The eviction rung
itself still wants a loaded model to prove.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015MGjcAV8bDy93qJfmLi9kw

CLAUDE.md

@@ -193,7 +193,12 @@ Four rules, each of which cost something to learn:
 - **No subprocess, ever.** 0.4.1–0.4.3 were registry-Flagged under
   `python_command_injection_risk` until `store.py` moved off `subprocess.Popen`.
   PromptMasterLD's unload ladder ends in `lms unload --all`; that rung is
-  deliberately absent here and the four HTTP ones are enough. The sibling pack
+  deliberately absent here and the four HTTP ones are enough. `unload_all` is
+  the killswitch that rung existed for: it lists loaded models over HTTP and
+  walks each through the same ladder. `unload` alone was not enough, because
+  it only ever targets the configured model -- and the three cases that
+  actually OOM a render are the ones where that is not what is resident (the
+  checkbox was off, the write failed early, or JIT loaded something else). The sibling pack
   can afford it because it has no `pyproject.toml` and is never scanned.
 - **No blocking I/O.** These functions are awaited inside aiohttp handlers, so
   `urllib.request` — which is what PromptMasterLD uses from its worker thread —

js/editor/writer_bar.js

@@ -19,6 +19,7 @@ import { el, button } from "./widget_utils.js";
 
 const LLM_URL = "/h3_ref_chain/llm";
 const PLAN_URL = "/h3_ref_chain/plan";
+const UNLOAD_URL = "/h3_ref_chain/llm/unload";
 
 export function createWriterBar(node, { onWritten, hopCount } = {}) {
     const root = el("details", "h3e-section h3e-writer");
@@ -52,6 +53,13 @@ export function createWriterBar(node, { onWritten, hopCount } = {}) {
     const go = button("Write plan", "Generate a plan and repair it until the "
         + "node accepts it", () => run());
     row.appendChild(go);
+
+    // In the main row on purpose. This is the button you reach for when a
+    // render just OOMed, and a killswitch behind a disclosure is not one.
+    const freeBtn = button("Free VRAM", "Unload whatever the writer is holding "
+        + "in memory, right now -- not just the model configured here. Press "
+        + "it before queueing if the card is full.", () => freeVram());
+    row.appendChild(freeBtn);
     body.appendChild(row);
 
     const status = el("div", "h3e-note h3e-writer-status");
@@ -118,9 +126,36 @@ export function createWriterBar(node, { onWritten, hopCount } = {}) {
     function setBusy(on) {
         busy = on;
         go.disabled = on;
+        // Unloading mid-generation would evict the model answering the prompt.
+        freeBtn.disabled = on;
         go.textContent = on ? "Writing…" : "Write plan";
     }
 
+    async function freeVram() {
+        if (busy) return;
+        freeBtn.disabled = true;
+        freeBtn.textContent = "Freeing…";
+        try {
+            const r = await fetch(UNLOAD_URL, { method: "POST" });
+            const j = await r.json();
+            // "Nothing was loaded" is a success with nothing to do, and saying
+            // so beats a bare "done" that leaves the user wondering whether
+            // the card is actually free.
+            say(!j.ok
+                ? (j.error || "could not reach the writer")
+                : j.unloaded
+                    ? `Freed ${j.unloaded} model(s): ${j.note}`
+                    : `Nothing to free — ${j.note}`,
+                j.ok ? "hint" : "error");
+            if (j.ok && j.unloaded) await loadConn(false);
+        } catch (e) {
+            say(String(e), "error");
+        } finally {
+            freeBtn.disabled = false;
+            freeBtn.textContent = "Free VRAM";
+        }
+    }
+
     async function loadConn(announce) {
         try {
             const r = await fetch(LLM_URL);

js/h3_ref_chain.css

@@ -903,7 +903,11 @@
     align-items: center;
     gap: var(--h3-gap-sm);
     margin-bottom: var(--h3-gap-sm);
+    /* The brief, the hop count and two buttons do not fit a narrow node at
+       one line. Wrap rather than squeezing the brief down to nothing. */
+    flex-wrap: wrap;
 }
+.h3e-writer-row button { flex: 0 0 auto; }
 .h3e-writer-brief { flex: 1 1 auto; min-width: 0; }
 .h3e-writer-hops  { flex: 0 0 56px; text-align: center; }
 .h3e-writer-url   { flex: 1 1 auto; min-width: 0; }

llm.py

@@ -415,6 +415,46 @@ def shares_this_gpu(base_url):
     return bool(here & there)
 
 
+async def _unload_one(session, base, model):
+    """One model, every known eviction verb. -> bool. Never raises.
+
+    LM Studio's unload route changed shape between versions, so the bodies are
+    tried in order; a backend with no unload endpoint at all is ordinary.
+    """
+    import aiohttp
+
+    for path, body in (
+        ("/api/v1/models/unload", {"instance_id": model}),
+        ("/api/v1/models/unload", {"identifier": model}),
+        ("/api/v0/models/unload", {"instance_id": model}),
+        ("/api/v0/models/unload", {"identifier": model}),
+    ):
+        try:
+            async with session.post(
+                base + path, json=body,
+                timeout=aiohttp.ClientTimeout(total=UNLOAD_TIMEOUT),
+            ) as resp:
+                if resp.status < 400:
+                    print(f"[{TAG}] unloaded {model} from the writer",
+                          flush=True)
+                    return True
+        except Exception:
+            continue
+    # Ollama keeps its own eviction verb; harmless against LM Studio.
+    try:
+        async with session.post(
+            base + "/api/generate",
+            json={"model": model, "keep_alive": 0},
+            timeout=aiohttp.ClientTimeout(total=UNLOAD_TIMEOUT),
+        ) as resp:
+            if resp.status < 400:
+                print(f"[{TAG}] evicted {model} (keep_alive 0)", flush=True)
+                return True
+    except Exception:
+        pass
+    return False
+
+
 async def unload(base_url, model):
     """Hand the VRAM back, so the render that follows has somewhere to live.
 
@@ -437,39 +477,10 @@ async def unload(base_url, model):
               flush=True)
         return False
 
-    attempts = [
-        ("/api/v1/models/unload", {"instance_id": model}),
-        ("/api/v1/models/unload", {"identifier": model}),
-        ("/api/v0/models/unload", {"instance_id": model}),
-        ("/api/v0/models/unload", {"identifier": model}),
-    ]
     try:
         async with aiohttp.ClientSession() as session:
-            for path, body in attempts:
-                try:
-                    async with session.post(
-                        base + path, json=body,
-                        timeout=aiohttp.ClientTimeout(total=UNLOAD_TIMEOUT),
-                    ) as resp:
-                        if resp.status < 400:
-                            print(f"[{TAG}] unloaded {model} from the writer",
-                                  flush=True)
-                            return True
-                except Exception:
-                    continue
-            # Ollama keeps its own eviction verb; harmless against LM Studio.
-            try:
-                async with session.post(
-                    base + "/api/generate",
-                    json={"model": model, "keep_alive": 0},
-                    timeout=aiohttp.ClientTimeout(total=UNLOAD_TIMEOUT),
-                ) as resp:
-                    if resp.status < 400:
-                        print(f"[{TAG}] evicted {model} (keep_alive 0)",
-                              flush=True)
-                        return True
-            except Exception:
-                pass
+            if await _unload_one(session, base, model):
+                return True
     except Exception as exc:
         print(f"[{TAG}] unload failed: {exc}", flush=True)
         return False
@@ -477,3 +488,64 @@ async def unload(base_url, model):
     print(f"[{TAG}] no unload endpoint answered -- free the VRAM in LM Studio "
           f"if the render runs short", flush=True)
     return False
+
+
+async def unload_all(base_url, fallback_model=""):
+    """The killswitch: evict everything the writer has in VRAM. -> (n, note).
+
+    `unload` only ever targets the model this pack configured, which is right
+    for the automatic path -- it hands back exactly what writing a plan caused
+    to be loaded. It is not enough for a button whose whole job is "give me the
+    card back now", because the three situations that actually OOM a render are
+    the ones where the configured model is not what is resident: the unload
+    checkbox was off, the write failed before it ran, or LM Studio's JIT loaded
+    something other than what was asked for.
+
+    PromptMasterLD covers this with `lms unload --all`. That rung is a
+    subprocess, which is what got 0.4.1-0.4.3 registry-Flagged under
+    `python_command_injection_risk`, so this lists loaded models over HTTP and
+    walks them through the same ladder instead. Same effect, nothing spawned.
+    """
+    import aiohttp
+
+    base = normalise_base(base_url)
+    if not base:
+        return 0, "no server configured"
+    if not shares_this_gpu(base):
+        # The laptop-and-desktop bug: never reach across a network to evict
+        # someone else's model. Freeing VRAM here would free the wrong VRAM.
+        return 0, "the writer is on another machine -- nothing to free here"
+
+    try:
+        listed = await models(base)
+    except Exception as exc:
+        listed = []
+        print(f"[{TAG}] could not list models to unload: {exc}", flush=True)
+
+    # `loaded` is None on servers with no /api/v0/models -- unknown, not false.
+    # Falling back to the configured model beats evicting nothing at all.
+    targets = [m["id"] for m in listed if m.get("loaded") is True]
+    guessing = False
+    if not targets:
+        if any(m.get("loaded") is None for m in listed) and fallback_model:
+            targets, guessing = [fallback_model], True
+        else:
+            return 0, "nothing is loaded"
+
+    done = []
+    try:
+        async with aiohttp.ClientSession() as session:
+            for name in targets:
+                if await _unload_one(session, base, name):
+                    done.append(name)
+    except Exception as exc:
+        print(f"[{TAG}] unload_all failed: {exc}", flush=True)
+        return len(done), f"stopped after {len(done)}: {exc}"
+
+    if not done:
+        return 0, ("no unload endpoint answered -- free the VRAM in LM Studio "
+                   "directly")
+    note = ", ".join(done)
+    if guessing:
+        note += " (this server does not report what is loaded)"
+    return len(done), note

prompt_pack/README.md

@@ -78,6 +78,10 @@ Notes and limits:
   node, never in the workflow. A shared `.json` cannot point at your server.
 * **Unload after writing** is on by default. A 27B and an H3 render do not fit
   on one card. It is skipped when the server is on another machine.
+* **Free VRAM** unloads everything the writer is holding, on demand. Reach for
+  it before queueing if a render just ran out of memory: the automatic unload
+  only hands back the model configured here, and LM Studio may have loaded a
+  different one on its own.
 * No API keys, no cloud providers, no model downloading — local servers only.
 * This is **alpha**. The manual recipe above is the one to fall back on.
 

routes.py

@@ -40,6 +40,7 @@ UPLOAD_ROUTE = "/h3_ref_chain/upload"
 FILES_ROUTE = "/h3_ref_chain/files"
 LLM_ROUTE = "/h3_ref_chain/llm"
 PLAN_ROUTE = "/h3_ref_chain/plan"
+UNLOAD_ROUTE = "/h3_ref_chain/llm/unload"
 
 # A batch of stills is a handful; this is a guard against a runaway multipart
 # body, not a considered product limit.
@@ -241,6 +242,27 @@ def register():
             print(f"[{TAG}] llm save failed: {exc!r}", flush=True)
             return web.json_response({"ok": False, "error": str(exc)}, status=500)
 
+    @instance.routes.post(UNLOAD_ROUTE)
+    async def _llm_unload(request):
+        """Free the writer's VRAM on demand. Never a 500, never a hard error.
+
+        The automatic unload after writing covers the ordinary case. This is
+        for the ones it cannot: the checkbox was off, the write failed before
+        it ran, or LM Studio's JIT put a different model in memory than the one
+        configured. Pressing it when nothing is loaded is a no-op that says so.
+        """
+        try:
+            from . import llm as _llm
+        except Exception as exc:
+            return web.json_response({"ok": False, "error": str(exc)})
+        try:
+            conn = _llm.load_conn()
+            n, note = await _llm.unload_all(conn["server_url"], conn["model"])
+            return web.json_response({"ok": True, "unloaded": n, "note": note})
+        except Exception as exc:
+            print(f"[{TAG}] unload failed: {exc!r}", flush=True)
+            return web.json_response({"ok": False, "error": str(exc)})
+
     @instance.routes.post(PLAN_ROUTE)
     async def _plan(request):
         """Write a plan, and make the model repair it until the node accepts it.

tools/check_planner.py

@@ -20,6 +20,8 @@ Covers:
                        duration label, an invented filename, a hop-count miss
   * write_plan      -- repairs on attempt 2, reports warnings, and never
                        returns an unvalidated plan when it cannot converge
+  * unload_all      -- the killswitch refuses to evict a writer that is not on
+                       this machine, without opening a connection to find out
 """
 from __future__ import annotations
 
@@ -223,6 +225,27 @@ def main():
        out["ok"] is False and any("no JSON" in e for e in out["errors"]),
        "; ".join(out["errors"][:1]))
 
+    # ------------------------------------------------------- the killswitch
+    # Only the parts that need no server. `unload_all` short-circuits on a
+    # non-local host BEFORE it opens a session, so this asserts the guard
+    # without touching the network.
+    print("\nllm.unload_all -- the local-only guard")
+    LM = importlib.import_module("htcpack.llm")
+
+    ck("localhost shares this GPU", LM.shares_this_gpu("http://127.0.0.1:1234"))
+    ck("a bare host with no scheme still resolves",
+       LM.shares_this_gpu("localhost:1234"))
+    # TEST-NET-1, guaranteed unroutable and never this machine.
+    ck("a remote host does not",
+       not LM.shares_this_gpu("http://192.0.2.1:1234"))
+
+    n, note = asyncio.run(LM.unload_all("http://192.0.2.1:1234", "some-model"))
+    ck("a remote writer is never evicted", n == 0, note)
+    ck("and it says why", "another machine" in note, note)
+
+    n, note = asyncio.run(LM.unload_all("", "some-model"))
+    ck("no server configured is not an error", n == 0, note)
+
     print()
     if FAIL:
         print("%d FAILURE(S): %s" % (len(FAIL), ", ".join(FAIL)))