chore: install llama-cpp from prebuilt Space wheel

README.md

@@ -200,7 +200,7 @@ The **Actor Engine** selector supports:
 - **Local OpenBMB:** the existing experimental local OpenBMB Transformers actor backend.
 - **Hugging Face API:** the existing remote actor backend using Hugging Face Inference Providers.
 
-The local backend libraries are included in the app runtime dependencies. Model weights are still loaded lazily: the first selected LoRA or GGUF beat downloads/loads the model and keeps the loaded backend object in memory for the app lifecycle. If a selected backend fails, times out, errors, or returns invalid JSON, runtime output goes through first-JSON extraction, Actor JSON sanitization, validation, and then falls back to the deterministic actor generator.
+The LoRA runtime libraries are included in the app runtime dependencies. The Space `requirements.txt` also includes `llama-cpp-python` through the upstream prebuilt CUDA wheel index so the GGUF backend can use GPU offload when the Space image has a compatible wheel. Model weights are still loaded lazily: the first selected LoRA or GGUF beat downloads/loads the model and keeps the loaded backend object in memory for the app lifecycle. If a selected backend fails, times out, errors, or returns invalid JSON, runtime output goes through first-JSON extraction, Actor JSON sanitization, validation, and then falls back to the deterministic actor generator.
 
 Local model defaults are set for demo reliability: LoRA uses `ACTOR_LORA_DEVICE=auto`, GGUF uses `ACTOR_GGUF_N_GPU_LAYERS=-1`, and both local actor backends default to a 120 second generation timeout.
 
@@ -220,6 +220,8 @@ ACTOR_MODEL_BACKEND=local_gguf \
 uv run python app.py
 ```
 
+`requirements.txt` uses the upstream prebuilt CUDA 12.1 wheel index for `llama-cpp-python` to avoid source builds where possible on the current GPU Space. If the Space CUDA/runtime image changes, replace `/cu121` with the matching CUDA wheel index from the `llama-cpp-python` docs, such as `/cu124`, `/cu125`, `/cu130`, or `/cu132`. For a CPU Space, replace `/cu121` with `/cpu` and set `ACTOR_GGUF_N_GPU_LAYERS=0`. If no compatible prebuilt wheel exists for the Space image, pip may still fall back to a slow native build.
+
 To force a local file instead of downloading from the Hugging Face GGUF repo:
 
 ```bash
@@ -240,7 +242,7 @@ uv run python app.py
 Limitations:
 
 - Local LoRA needs enough GPU or CPU memory for `openbmb/MiniCPM5-1B` plus the adapter.
-- Local GGUF needs enough CPU/GPU memory for the quantized file; by default it downloads the configured GGUF from Hugging Face Hub, or uses `ACTOR_GGUF_MODEL_PATH` when set. The GGUF file is not committed to git.
+- Local GGUF needs `llama-cpp-python` and enough CPU/GPU memory for the quantized file; by default it downloads the configured GGUF from Hugging Face Hub, or uses `ACTOR_GGUF_MODEL_PATH` when set. The GGUF file is not committed to git.
 - Hosted Spaces may use fallback unless a model backend is explicitly configured.
 - GGUF is prompted with ChatML, matching the best eval format used for the Q4_K_M actor model.
 
@@ -281,7 +283,7 @@ USE_ZEROGPU=true
 
 `USE_ZEROGPU` defaults to `false`, so the app still starts locally and on regular CPU Spaces without requesting ZeroGPU. When enabled and the `spaces` package is available, local OpenBMB generation runs through a `@spaces.GPU(duration=30)` function. The app does not load local models at startup.
 
-The Space requirements are resolved for Python 3.10 compatibility because ZeroGPU builds may use a Python 3.10 base image. The local project also supports Python 3.11. The runtime pins `torch==2.8.0`, which is the preferred ZeroGPU-compatible Torch version for this Space. If Hugging Face reports a different supported Torch version for ZeroGPU, update `pyproject.toml`, regenerate `requirements.txt`, and refresh `uv.lock`.
+The Space requirements are resolved for Python 3.10 compatibility because ZeroGPU builds may use a Python 3.10 base image. The local project also supports Python 3.11. The runtime pins `torch==2.8.0`, which is the preferred ZeroGPU-compatible Torch version for this Space. `llama-cpp-python` is included in `requirements.txt` through its prebuilt CUDA wheel index for GGUF support on the current GPU Space; for CPU Space hardware, switch that index to `/cpu` and set `ACTOR_GGUF_N_GPU_LAYERS=0`. If the Space image cannot use the selected wheel, pip may fall back to a slow source build. If Hugging Face reports a different supported Torch version for ZeroGPU, update `pyproject.toml`, regenerate `requirements.txt`, and refresh `uv.lock`.
 
 If ZeroGPU is not enabled, `spaces` is unavailable, local model loading fails, CUDA is unavailable inside the GPU function, or model output is invalid, the show falls back gracefully. The Hugging Face API backend remains selectable and is the recommended fallback when you have `HF_TOKEN`; deterministic mode remains the no-token safety path.
 

pyproject.toml

@@ -9,7 +9,6 @@ dependencies = [
     "edge-tts",
     "gradio==6.5.1",
     "huggingface-hub==1.18.0",
-    "llama-cpp-python==0.3.28",
     "peft==0.19.1",
     "pydantic<=2.12.5",
     "spaces==0.50.4",

requirements.txt

@@ -1,5 +1,13 @@
 # This file was autogenerated by uv via the following command:
 #    uv pip compile pyproject.toml -o requirements.txt
+#
+# Manual Space runtime addition:
+# Use llama-cpp-python's prebuilt CUDA wheel index so the Local GGUF Actor model
+# can use GPU offload on the current GPU Space without forcing a source build
+# when a compatible wheel is available. For CPU Spaces, change /cu121 to /cpu
+# and set ACTOR_GGUF_N_GPU_LAYERS=0.
+--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
+llama-cpp-python==0.3.28
 accelerate==1.13.0
     # via
     #   ai-puppet-theater (pyproject.toml)
@@ -40,8 +48,6 @@ click==8.4.1
     #   huggingface-hub
     #   typer
     #   uvicorn
-diskcache==5.6.3
-    # via llama-cpp-python
 edge-tts==7.2.8
     # via ai-puppet-theater (pyproject.toml)
 fastapi==0.136.3
@@ -102,10 +108,7 @@ idna==3.18
 jinja2==3.1.6
     # via
     #   gradio
-    #   llama-cpp-python
     #   torch
-llama-cpp-python==0.3.28
-    # via ai-puppet-theater (pyproject.toml)
 markdown-it-py==4.2.0
     # via rich
 markupsafe==3.0.3
@@ -126,7 +129,6 @@ numpy==2.2.6
     # via
     #   accelerate
     #   gradio
-    #   llama-cpp-python
     #   pandas
     #   peft
     #   transformers
@@ -244,7 +246,6 @@ typing-extensions==4.15.0
     #   gradio
     #   gradio-client
     #   huggingface-hub
-    #   llama-cpp-python
     #   pydantic
     #   pydantic-core
     #   spaces

uv.lock

@@ -36,7 +36,6 @@ dependencies = [
     { name = "edge-tts" },
     { name = "gradio" },
     { name = "huggingface-hub" },
-    { name = "llama-cpp-python" },
     { name = "peft" },
     { name = "pydantic" },
     { name = "spaces" },
@@ -56,7 +55,6 @@ requires-dist = [
     { name = "edge-tts" },
     { name = "gradio", specifier = "==6.5.1" },
     { name = "huggingface-hub", specifier = "==1.18.0" },
-    { name = "llama-cpp-python", specifier = "==0.3.28" },
     { name = "peft", specifier = "==0.19.1" },
     { name = "pydantic", specifier = "<=2.12.5" },
     { name = "spaces", specifier = "==0.50.4" },
@@ -305,15 +303,6 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
 ]
 
-[[package]]
-name = "diskcache"
-version = "5.6.3"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/3f/21/1c1ffc1a039ddcc459db43cc108658f32c57d271d7289a2794e401d0fdb6/diskcache-5.6.3.tar.gz", hash = "sha256:2c3a3fa2743d8535d832ec61c2054a1641f41775aa7c556758a109941e33e4fc", size = 67916, upload-time = "2023-08-31T06:12:00.316Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/3f/27/4570e78fc0bf5ea0ca45eb1de3818a23787af9b390c0b0a0033a1b8236f9/diskcache-5.6.3-py3-none-any.whl", hash = "sha256:5e31b2d5fbad117cc363ebaf6b689474db18a1f6438bc82358b024abd4c2ca19", size = 45550, upload-time = "2023-08-31T06:11:58.822Z" },
-]
-
 [[package]]
 name = "edge-tts"
 version = "7.2.8"
@@ -595,19 +584,6 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
 ]
 
-[[package]]
-name = "llama-cpp-python"
-version = "0.3.28"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
-    { name = "diskcache" },
-    { name = "jinja2" },
-    { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
-    { name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
-    { name = "typing-extensions" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/83/c1/ac90e90d28f405009a89f08a5727d3f0dac49309575fb2d1a5ff1d265826/llama_cpp_python-0.3.28.tar.gz", hash = "sha256:958227b394f413425d6039952096daa0b8b98328c6b99d652862aec775f1672d", size = 68296944, upload-time = "2026-06-07T23:54:28.292Z" }
-
 [[package]]
 name = "markdown-it-py"
 version = "4.2.0"