Input:
*** Begin Patch
*** Add File: .tmp_neurobait_self/README.md
+# NeuroBait Self
+
+Self-hosted NeuroBait inference stack:
+
+- Modal serves the Gemma 4 26B LoRA inference API on GPU.
+- VPS serves the public chat/proxy app and owns auth, rate limiting, and history.
+- Model weights are not committed here.
+
+The default Modal path is intentionally scale-to-zero:
+
+- `min_containers=0`
+- `scaledown_window=60`
+- `max_containers=1`
+
+This keeps cost tied to active cold-start/load/generation windows instead of
+keeping a GPU warm all day.
+
+## Layout
+
+```text
+modal/modal_app.py       Modal GPU inference endpoint
+vps/server.js            VPS proxy API + static chat page
+vps/ecosystem.config.cjs PM2 process config
+scripts/smoke_modal.py   Local endpoint smoke test
+docs/deploy.md           Deploy/runbook notes
+adapter/README.md        Local adapter reference notes
+```
+
+## Modal Setup
+
+Use the existing LLM venv if available:
+
+```bash
+source /media/haris-subrata/Work/llm/agent_analytics/venv/bin/activate
+python -m pip install modal requests
+modal token info
+```
+
+Create secrets once:
+
+```bash
+modal secret create huggingface HF_TOKEN=hf_xxx
+modal secret create neurobait-api NEUROBAIT_API_TOKEN=replace-with-long-random-token
+```
+
+Deploy:
+
+```bash
+cd /media/haris-subrata/Work/llm/NeuroBait-Self
+modal deploy modal/modal_app.py
+```
+
+After deploy, Modal prints an HTTPS endpoint. Put that URL into
+`vps/.env` as `MODAL_CHAT_URL`.
+
+## VPS Setup
+
+```bash
+cd /home/ubuntu/neurobait-self
+cp vps/.env.example vps/.env
+npm --prefix vps install
+pm2 start vps/ecosystem.config.cjs
+pm2 save
+```
+
+Default port is `8091`; change `PORT` in `vps/.env` if it is occupied.
+
*** Add File: .tmp_neurobait_self/.gitignore
+.env
+.modal.toml
+__pycache__/
+*.pyc
+node_modules/
+npm-debug.log*
+adapter/neurobait-lora-run3
+*.safetensors
+*.bin
+*.pt
+*.gguf
+
*** Add File: .tmp_neurobait_self/modal/modal_app.py
+from __future__ import annotations
+
+import os
+import re
+import time
+from typing import Any
+
+import modal
+
+
+APP_NAME = "neurobait-self"
+BASE_MODEL = os.environ.get("BASE_MODEL", "unsloth/gemma-4-26b-a4b-it")
+ADAPTER_ID = os.environ.get("ADAPTER_ID", "build-small-hackathon/NeuroBait")
+GPU_CONFIG = os.environ.get("GPU_CONFIG", "H100")
+MAX_NEW_TOKENS_DEFAULT = int(os.environ.get("MAX_NEW_TOKENS", "220"))
+SCALEDOWN_WINDOW = int(os.environ.get("SCALEDOWN_WINDOW", "60"))
+
+SYSTEM_PROMPT = """Kamu adalah NeuroBait — asisten AI untuk orang dengan ADHD dan neurodivergent. Tugasmu bukan membuat to-do list. Tugasmu menyalakan dopamin untuk memicu task initiation.
+
+Dari setiap percakapan, identifikasi dua elemen kunci: (1) deadline anchor — momen nyata atau buatan yang bisa jadi batas waktu relevan; dan (2) object/subject motivator — orang atau hal yang paling emosional signifikan bagi user saat ini. Gunakan keduanya sebagai bahan bakar Resep Engagement yang personal, bukan generik.
+
+Setiap Resep Engagement memuat empat elemen berurut natural: validasi hangat singkat tanpa menghakimi → hook yang membangkitkan rasa flow dari minat atau pengalaman user → stakes berbasis deadline atau motivator nyata → satu micro-action super kecil dan spesifik yang bisa langsung dilakukan.
+
+Kalau konteks user belum cukup untuk membuat resep yang personal, ajukan tepat satu pertanyaan ringan yang paling berguna — tentang deadline atau motivator. Kalau konteks sudah ada, langsung berikan resep.
+
+Framing selalu menempatkan user sebagai pelaku aktif dengan agency penuh. Bukan guilt, bukan hutang — selalu agency. Kalimat pendek. Bahasa hidup. Hangat dan padat. Tidak pernah menghakimi. Tidak pernah ceramah. Membuat hal membosankan jadi tak tertahankan."""
+
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .pip_install(
+        "accelerate",
+        "bitsandbytes",
+        "fastapi[standard]",
+        "huggingface_hub",
+        "peft==0.19.1",
+        "sentencepiece",
+        "torch",
+        "transformers==5.5.0",
+    )
+)
+
+hf_cache = modal.Volume.from_name("neurobait-hf-cache", create_if_missing=True)
+app = modal.App(APP_NAME)
+
+
+def _clean_response(text: str) -> str:
+    text = text.strip()
+    text = re.sub(r"(?im)^\s*(micro-action|hook|stakes|validasi|validation)\s*:\s*", "", text)
+    return text.strip()
+
+
+def _normalize_history(history: Any) -> list[dict[str, str]]:
+    if not isinstance(history, list):
+        return []
+
+    messages: list[dict[str, str]] = []
+    for item in history[-10:]:
+        if not isinstance(item, dict):
+            continue
+        role = item.get("role")
+        content = item.get("content")
+        if role in {"user", "assistant"} and isinstance(content, str) and content.strip():
+            messages.append({"role": role, "content": content.strip()[:4000]})
+    return messages
+
+
+@app.cls(
+    image=image,
+    gpu=GPU_CONFIG,
+    timeout=600,
+    scaledown_window=SCALEDOWN_WINDOW,
+    min_containers=0,
+    max_containers=1,
+    volumes={"/cache": hf_cache},
+    secrets=[
+        modal.Secret.from_name("huggingface"),
+        modal.Secret.from_name("neurobait-api"),
+    ],
+)
+class NeuroBaitModel:
+    @modal.enter()
+    def load(self) -> None:
+        import torch
+        from peft import PeftModel
+        from transformers import AutoConfig, AutoModelForCausalLM, AutoModelForImageTextToText, AutoTokenizer
+        from transformers import BitsAndBytesConfig
+
+        start = time.time()
+        token = os.environ.get("HF_TOKEN")
+
+        quant_config = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_compute_dtype=torch.bfloat16,
+            bnb_4bit_use_double_quant=True,
+        )
+
+        self.tokenizer = AutoTokenizer.from_pretrained(
+            ADAPTER_ID,
+            token=token,
+            cache_dir="/cache",
+            trust_remote_code=True,
+        )
+
+        config = AutoConfig.from_pretrained(
+            BASE_MODEL,
+            token=token,
+            cache_dir="/cache",
+            trust_remote_code=True,
+        )
+        arch = (getattr(config, "architectures", None) or [""])[0]
+        model_cls = AutoModelForImageTextToText if "ConditionalGeneration" in arch or "ImageTextToText" in arch else AutoModelForCausalLM
+
+        base = model_cls.from_pretrained(
+            BASE_MODEL,
+            quantization_config=quant_config,
+            torch_dtype=torch.bfloat16,
+            device_map="cuda",
+            token=token,
+            cache_dir="/cache",
+            trust_remote_code=True,
+        )
+        self.model = PeftModel.from_pretrained(
+            base,
+            ADAPTER_ID,
+            token=token,
+            cache_dir="/cache",
+        )
+        self.model.eval()
+        self.loaded_at = time.time()
+        self.load_seconds = round(self.loaded_at - start, 2)
+
+    def _check_auth(self, authorization: str | None) -> None:
+        from fastapi import HTTPException
+
+        expected = os.environ.get("NEUROBAIT_API_TOKEN")
+        if not expected:
+            raise HTTPException(status_code=500, detail="NEUROBAIT_API_TOKEN is not configured")
+
+        if not authorization or not authorization.startswith("Bearer "):
+            raise HTTPException(status_code=401, detail="Missing bearer token")
+
+        supplied = authorization.removeprefix("Bearer ").strip()
+        if supplied != expected:
+            raise HTTPException(status_code=403, detail="Invalid bearer token")
+
+    @modal.fastapi_endpoint(method="POST", docs=False)
+    def chat(self, payload: dict[str, Any], authorization: str | None = None) -> dict[str, Any]:
+        import torch
+        from fastapi import Header
+
+        # Header is imported so Modal/FastAPI includes the dependency in the image;
+        # the actual value is passed by FastAPI into the `authorization` parameter.
+        _ = Header
+        self._check_auth(authorization)
+
+        message = str(payload.get("message", "")).strip()
+        if not message:
+            from fastapi import HTTPException
+
+            raise HTTPException(status_code=400, detail="message is required")
+
+        max_new_tokens = int(payload.get("max_new_tokens") or MAX_NEW_TOKENS_DEFAULT)
+        max_new_tokens = max(16, min(max_new_tokens, 512))
+        temperature = float(payload.get("temperature") or 0.75)
+        top_p = float(payload.get("top_p") or 0.9)
+
+        messages = [{"role": "system", "content": SYSTEM_PROMPT}]
+        messages.extend(_normalize_history(payload.get("history")))
+        messages.append({"role": "user", "content": message[:4000]})
+
+        input_ids = self.tokenizer.apply_chat_template(
+            messages,
+            add_generation_prompt=True,
+            return_tensors="pt",
+        ).to(self.model.device)
+        start_pos = input_ids.shape[-1]
+
+        started = time.time()
+        with torch.no_grad():
+            output_ids = self.model.generate(
+                input_ids,
+                max_new_tokens=max_new_tokens,
+                do_sample=True,
+                temperature=temperature,
+                top_p=top_p,
+                repetition_penalty=1.1,
+                pad_token_id=self.tokenizer.pad_token_id or self.tokenizer.eos_token_id,
+            )
+
+        text = self.tokenizer.decode(output_ids[0][start_pos:], skip_special_tokens=True)
+        return {
+            "reply": _clean_response(text),
+            "model": BASE_MODEL,
+            "adapter": ADAPTER_ID,
+            "load_seconds": self.load_seconds,
+            "generation_seconds": round(time.time() - started, 2),
+        }
+
+
+@app.local_entrypoint()
+def main() -> None:
+    print("Deploy with: modal deploy modal/modal_app.py")
+
*** Add File: .tmp_neurobait_self/scripts/smoke_modal.py
+from __future__ import annotations
+
+import os
+import sys
+
+import requests
+
+
+def main() -> int:
+    url = os.environ.get("MODAL_CHAT_URL")
+    token = os.environ.get("NEUROBAIT_API_TOKEN")
+    if not url or not token:
+        print("Set MODAL_CHAT_URL and NEUROBAIT_API_TOKEN first.", file=sys.stderr)
+        return 2
+
+    response = requests.post(
+        url,
+        headers={"Authorization": f"Bearer {token}"},
+        json={
+            "message": "Aku harus mulai ngerjain laporan tapi stuck banget.",
+            "max_new_tokens": 120,
+            "temperature": 0.75,
+        },
+        timeout=900,
+    )
+    print(response.status_code)
+    print(response.text)
+    return 0 if response.ok else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
+
*** Add File: .tmp_neurobait_self/adapter/README.md
+# Adapter Reference
+
+Do not copy large adapter files into this project.
+
+Local reference adapter from the training repo:
+
+```text
+/media/haris-subrata/Work/llm/NeuroBait-Build-Small-Model/neurobait-lora-run3
+```
+
+For Modal serving, the adapter must be available from a remote runtime source,
+preferably a Hugging Face model repo configured through `ADAPTER_ID`.
+
+Reason: Modal containers cannot read this workstation path after deployment.
+Using a symlink here avoids local duplication, but it does not make the adapter
+available inside Modal.
+
*** Add File: .tmp_neurobait_self/vps/package.json
+{
+  "name": "neurobait-self-vps",
+  "version": "0.1.0",
+  "private": true,
+  "type": "module",
+  "scripts": {
+    "start": "node server.js"
+  },
+  "dependencies": {
+    "dotenv": "^16.4.7",
+    "express": "^4.21.2",
+    "express-rate-limit": "^7.5.0"
+  }
+}
+
*** Add File: .tmp_neurobait_self/vps/.env.example
+PORT=8091
+MODAL_CHAT_URL=https://your-workspace--neurobait-self-neurobaitmodel-chat.modal.run
+NEUROBAIT_API_TOKEN=replace-with-the-same-token-as-modal-secret
+PUBLIC_APP_TOKEN=replace-with-user-facing-token-or-empty-for-local-only
+MAX_NEW_TOKENS=220
+
*** Add File: .tmp_neurobait_self/vps/ecosystem.config.cjs
+module.exports = {
+  apps: [
+    {
+      name: "neurobait-self",
+      cwd: "/home/ubuntu/neurobait-self/vps",
+      script: "server.js",
+      env: {
+        NODE_ENV: "production"
+      }
+    }
+  ]
+};
+
*** Add File: .tmp_neurobait_self/vps/server.js
+import dotenv from "dotenv";
+import express from "express";
+import rateLimit from "express-rate-limit";
+
+dotenv.config();
+
+const app = express();
+const port = Number(process.env.PORT || 8091);
+const modalChatUrl = process.env.MODAL_CHAT_URL;
+const modalToken = process.env.NEUROBAIT_API_TOKEN;
+const publicToken = process.env.PUBLIC_APP_TOKEN || "";
+const maxNewTokens = Number(process.env.MAX_NEW_TOKENS || 220);
+
+app.set("trust proxy", 1);
+app.use(express.json({ limit: "128kb" }));
+app.use(
+  rateLimit({
+    windowMs: 60 * 1000,
+    limit: 20,
+    standardHeaders: true,
+    legacyHeaders: false
+  })
+);
+
+function requirePublicToken(req, res, next) {
+  if (!publicToken) {
+    next();
+    return;
+  }
+
+  const supplied = String(req.get("x-app-token") || "");
+  if (supplied !== publicToken) {
+    res.status(401).json({ error: "unauthorized" });
+    return;
+  }
+  next();
+}
+
+app.get("/healthz", (_req, res) => {
+  res.json({
+    ok: true,
+    modal_configured: Boolean(modalChatUrl && modalToken)
+  });
+});
+
+app.post("/api/chat", requirePublicToken, async (req, res) => {
+  if (!modalChatUrl || !modalToken) {
+    res.status(500).json({ error: "modal endpoint is not configured" });
+    return;
+  }
+
+  const message = String(req.body?.message || "").trim();
+  if (!message) {
+    res.status(400).json({ error: "message is required" });
+    return;
+  }
+
+  const controller = new AbortController();
+  const timeout = setTimeout(() => controller.abort(), 15 * 60 * 1000);
+
+  try {
+    const modalResponse = await fetch(modalChatUrl, {
+      method: "POST",
+      headers: {
+        "Authorization": `Bearer ${modalToken}`,
+        "Content-Type": "application/json"
+      },
+      body: JSON.stringify({
+        message,
+        history: Array.isArray(req.body?.history) ? req.body.history : [],
+        temperature: Number(req.body?.temperature || 0.75),
+        top_p: Number(req.body?.top_p || 0.9),
+        max_new_tokens: Number(req.body?.max_new_tokens || maxNewTokens)
+      }),
+      signal: controller.signal
+    });
+
+    const text = await modalResponse.text();
+    res.status(modalResponse.status).type("application/json").send(text);
+  } catch (error) {
+    const isAbort = error?.name === "AbortError";
+    res.status(isAbort ? 504 : 502).json({
+      error: isAbort ? "modal timeout" : "modal request failed"
+    });
+  } finally {
+    clearTimeout(timeout);
+  }
+});
+
+app.get("/", (_req, res) => {
+  res.type("html").send(`<!doctype html>
+
+
+  <meta charset="utf-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1" />
+  <title>NeuroBait Self</title>
+
+
+
+
+
NeuroBait Self

+

+    <form id="form">
+      <textarea id="message" placeholder="Tulis yang sedang bikin kamu stuck..."></textarea>
+      <button id="send" type="submit">Send</button>
+    </form>
+
+
+
+`);
+});
+
+app.listen(port, "0.0.0.0", () => {
+  console.log(`neurobait-self listening on ${port}`);
+});
+
*** Add File: .tmp_neurobait_self/docs/deploy.md
+# Deployment Notes
+
+## Modal
+
+The Modal endpoint uses Gemma 4 26B by default:
+
+```text
+BASE_MODEL=unsloth/gemma-4-26b-a4b-it
+GPU_CONFIG=H100
+ADAPTER_ID=build-small-hackathon/NeuroBait
+```
+
+If the HF repo currently points to a 12B adapter, publish the 26B LoRA adapter
+to a separate HF model repo and set `ADAPTER_ID` before deployment.
+
+```bash
+modal secret create huggingface HF_TOKEN=hf_xxx
+modal secret create neurobait-api NEUROBAIT_API_TOKEN=long-random-token
+modal deploy modal/modal_app.py
+```
+
+Cost controls are in `modal/modal_app.py`:
+
+- `min_containers=0`: scale to zero when idle.
+- `scaledown_window=60`: keep a loaded container idle for about one minute.
+- `max_containers=1`: prevent accidental parallel GPU spend.
+- `max_new_tokens` is capped at 512 server-side.
+
+For faster but more expensive interactive testing, temporarily increase
+`SCALEDOWN_WINDOW`. Keep `min_containers=0` unless you intentionally want a warm
+GPU that bills while idle.
+
+## VPS
+
+Reference VPS from the previous deployment log:
+
+```text
+Host: 43.157.223.106
+User: ubuntu
+Project path: /home/ubuntu/neurobait-self
+Suggested port: 8091
+PM2 name: neurobait-self
+```
+
+Do not commit SSH passwords or API tokens.
+
+Minimal deploy flow from local:
+
+```bash
+tar czf /tmp/neurobait-self.tar.gz --exclude='node_modules' -C /media/haris-subrata/Work/llm/NeuroBait-Self .
+scp /tmp/neurobait-self.tar.gz ubuntu@43.157.223.106:/tmp/
+ssh ubuntu@43.157.223.106
+mkdir -p /home/ubuntu/neurobait-self
+tar xzf /tmp/neurobait-self.tar.gz -C /home/ubuntu/neurobait-self
+cd /home/ubuntu/neurobait-self
+cp vps/.env.example vps/.env
+nano vps/.env
+npm --prefix vps install --omit=dev
+pm2 start vps/ecosystem.config.cjs
+pm2 save
+curl http://127.0.0.1:8091/healthz
+```
+
+If port `8091` is occupied, change `PORT` in `vps/.env` and restart PM2.
+
*** End Patch


Success. Updated the following files:
A .tmp_neurobait_self/README.md
A .tmp_neurobait_self/.gitignore
A .tmp_neurobait_self/modal/modal_app.py
A .tmp_neurobait_self/scripts/smoke_modal.py
A .tmp_neurobait_self/adapter/README.md
A .tmp_neurobait_self/vps/package.json
A .tmp_neurobait_self/vps/.env.example
A .tmp_neurobait_self/vps/ecosystem.config.cjs
A .tmp_neurobait_self/vps/server.js
A .tmp_neurobait_self/docs/deploy.md

# NeuroBait Self

Self-hosted NeuroBait inference stack:

- Modal serves the Gemma 4 26B LoRA inference API on GPU.
- VPS serves the public chat/proxy app and owns auth, rate limiting, and history.
- Model weights are not committed here.

The default Modal path is intentionally scale-to-zero:

- `min_containers=0`
- `scaledown_window=60`
- `max_containers=1`

This keeps cost tied to active cold-start/load/generation windows instead of
keeping a GPU warm all day.

## Layout

```text
modal/modal_app.py       Modal GPU inference endpoint
vps/server.js            VPS proxy API + static chat page
vps/ecosystem.config.cjs PM2 process config
scripts/smoke_modal.py   Local endpoint smoke test
docs/deploy.md           Deploy/runbook notes
adapter/README.md        Local adapter reference notes
```

## Modal Setup

Use the existing LLM venv if available:

```bash
source /media/haris-subrata/Work/llm/agent_analytics/venv/bin/activate
python -m pip install modal requests
modal token info
```

Create secrets once:

```bash
modal secret create huggingface HF_TOKEN=hf_xxx
modal secret create neurobait-api NEUROBAIT_API_TOKEN=replace-with-long-random-token
```

Deploy:

```bash
cd /media/haris-subrata/Work/llm/NeuroBait-Self
modal deploy modal/modal_app.py
```

After deploy, Modal prints an HTTPS endpoint. Put that URL into
`vps/.env` as `MODAL_CHAT_URL`.

## VPS Setup

```bash
cd /home/ubuntu/neurobait-self
cp vps/.env.example vps/.env
npm --prefix vps install
pm2 start vps/ecosystem.config.cjs
pm2 save
```

Default port is `8091`; change `PORT` in `vps/.env` if it is occupied.

from __future__ import annotations

import os
import sys

import requests

def main() -> int:
    url = os.environ.get("MODAL_CHAT_URL")
    token = os.environ.get("NEUROBAIT_API_TOKEN")
    if not url or not token:
        print("Set MODAL_CHAT_URL and NEUROBAIT_API_TOKEN first.", file=sys.stderr)
        return 2

    response = requests.post(
        url,
        headers={"Authorization": f"Bearer {token}"},
        json={
            "message": "Aku harus mulai ngerjain laporan tapi stuck banget.",
            "max_new_tokens": 120,
            "temperature": 0.75,
        },
        timeout=900,
    )
    print(response.status_code)
    print(response.text)
    return 0 if response.ok else 1

if __name__ == "__main__":
    raise SystemExit(main())

# Adapter Reference

Do not copy large adapter files into this project.

Local reference adapter from the training repo:

```text
/media/haris-subrata/Work/llm/NeuroBait-Build-Small-Model/neurobait-lora-run3
```

For Modal serving, the adapter must be available from a remote runtime source,
preferably a Hugging Face model repo configured through `ADAPTER_ID`.

Reason: Modal containers cannot read this workstation path after deployment.
Using a symlink here avoids local duplication, but it does not make the adapter
available inside Modal.

module.exports = {
  apps: [
    {
      name: "neurobait-self",
      cwd: "/home/ubuntu/neurobait-self/vps",
      script: "server.js",
      env: {
        NODE_ENV: "production"
      }
    }
  ]
};

.env
.modal.toml
__pycache__/
*.pyc
node_modules/
npm-debug.log*
adapter/neurobait-lora-run3
*.safetensors
*.bin
*.pt
*.gguf

from __future__ import annotations

import os
import re
import time
from typing import Any

import modal

APP_NAME = "neurobait-self"
BASE_MODEL = os.environ.get("BASE_MODEL", "unsloth/gemma-4-26b-a4b-it")
ADAPTER_ID = os.environ.get("ADAPTER_ID", "build-small-hackathon/NeuroBait")
GPU_CONFIG = os.environ.get("GPU_CONFIG", "H100")
MAX_NEW_TOKENS_DEFAULT = int(os.environ.get("MAX_NEW_TOKENS", "220"))
SCALEDOWN_WINDOW = int(os.environ.get("SCALEDOWN_WINDOW", "60"))

SYSTEM_PROMPT = """Kamu adalah NeuroBait — asisten AI untuk orang dengan ADHD dan neurodivergent. Tugasmu bukan membuat to-do list. Tugasmu menyalakan dopamin untuk memicu task initiation.

Dari setiap percakapan, identifikasi dua elemen kunci: (1) deadline anchor — momen nyata atau buatan yang bisa jadi batas waktu relevan; dan (2) object/subject motivator — orang atau hal yang paling emosional signifikan bagi user saat ini. Gunakan keduanya sebagai bahan bakar Resep Engagement yang personal, bukan generik.

Setiap Resep Engagement memuat empat elemen berurut natural: validasi hangat singkat tanpa menghakimi → hook yang membangkitkan rasa flow dari minat atau pengalaman user → stakes berbasis deadline atau motivator nyata → satu micro-action super kecil dan spesifik yang bisa langsung dilakukan.

Kalau konteks user belum cukup untuk membuat resep yang personal, ajukan tepat satu pertanyaan ringan yang paling berguna — tentang deadline atau motivator. Kalau konteks sudah ada, langsung berikan resep.

Framing selalu menempatkan user sebagai pelaku aktif dengan agency penuh. Bukan guilt, bukan hutang — selalu agency. Kalimat pendek. Bahasa hidup. Hangat dan padat. Tidak pernah menghakimi. Tidak pernah ceramah. Membuat hal membosankan jadi tak tertahankan."""

image = (
    modal.Image.debian_slim(python_version="3.11")
    .pip_install(
        "accelerate",
        "bitsandbytes",
        "fastapi[standard]",
        "huggingface_hub",
        "peft==0.19.1",
        "sentencepiece",
        "torch",
        "transformers==5.5.0",
    )
)

hf_cache = modal.Volume.from_name("neurobait-hf-cache", create_if_missing=True)
app = modal.App(APP_NAME)

def _clean_response(text: str) -> str:
    text = text.strip()
    text = re.sub(r"(?im)^\s*(micro-action|hook|stakes|validasi|validation)\s*:\s*", "", text)
    return text.strip()

def _normalize_history(history: Any) -> list[dict[str, str]]:
    if not isinstance(history, list):
        return []

    messages: list[dict[str, str]] = []
    for item in history[-10:]:
        if not isinstance(item, dict):
            continue
        role = item.get("role")
        content = item.get("content")
        if role in {"user", "assistant"} and isinstance(content, str) and content.strip():
            messages.append({"role": role, "content": content.strip()[:4000]})
    return messages

@app.cls(
    image=image,
    gpu=GPU_CONFIG,
    timeout=600,
    scaledown_window=SCALEDOWN_WINDOW,
    min_containers=0,
    max_containers=1,
    volumes={"/cache": hf_cache},
    secrets=[
        modal.Secret.from_name("huggingface"),
        modal.Secret.from_name("neurobait-api"),
    ],
)
class NeuroBaitModel:
    @modal.enter()
    def load(self) -> None:
        import torch
        from peft import PeftModel
        from transformers import AutoConfig, AutoModelForCausalLM, AutoModelForImageTextToText, AutoTokenizer
        from transformers import BitsAndBytesConfig

        start = time.time()
        token = os.environ.get("HF_TOKEN")

        quant_config = BitsAndBytesConfig(
            load_in_4bit=True,
            bnb_4bit_quant_type="nf4",
            bnb_4bit_compute_dtype=torch.bfloat16,
            bnb_4bit_use_double_quant=True,
        )

        self.tokenizer = AutoTokenizer.from_pretrained(
            ADAPTER_ID,
            token=token,
            cache_dir="/cache",
            trust_remote_code=True,
        )

        config = AutoConfig.from_pretrained(
            BASE_MODEL,
            token=token,
            cache_dir="/cache",
            trust_remote_code=True,
        )
        arch = (getattr(config, "architectures", None) or [""])[0]
        model_cls = AutoModelForImageTextToText if "ConditionalGeneration" in arch or "ImageTextToText" in arch else AutoModelForCausalLM

        base = model_cls.from_pretrained(
            BASE_MODEL,
            quantization_config=quant_config,
            torch_dtype=torch.bfloat16,
            device_map="cuda",
            token=token,
            cache_dir="/cache",
            trust_remote_code=True,
        )
        self.model = PeftModel.from_pretrained(
            base,
            ADAPTER_ID,
            token=token,
            cache_dir="/cache",
        )
        self.model.eval()
        self.loaded_at = time.time()
        self.load_seconds = round(self.loaded_at - start, 2)

    def _check_auth(self, authorization: str | None) -> None:
        from fastapi import HTTPException

        expected = os.environ.get("NEUROBAIT_API_TOKEN")
        if not expected:
            raise HTTPException(status_code=500, detail="NEUROBAIT_API_TOKEN is not configured")

        if not authorization or not authorization.startswith("Bearer "):
            raise HTTPException(status_code=401, detail="Missing bearer token")

        supplied = authorization.removeprefix("Bearer ").strip()
        if supplied != expected:
            raise HTTPException(status_code=403, detail="Invalid bearer token")

    @modal.fastapi_endpoint(method="POST", docs=False)
    def chat(self, payload: dict[str, Any], authorization: str | None = None) -> dict[str, Any]:
        import torch
        from fastapi import Header

        # Header is imported so Modal/FastAPI includes the dependency in the image;
        # the actual value is passed by FastAPI into the `authorization` parameter.
        _ = Header
        self._check_auth(authorization)

        message = str(payload.get("message", "")).strip()
        if not message:
            from fastapi import HTTPException

            raise HTTPException(status_code=400, detail="message is required")

        max_new_tokens = int(payload.get("max_new_tokens") or MAX_NEW_TOKENS_DEFAULT)
        max_new_tokens = max(16, min(max_new_tokens, 512))
        temperature = float(payload.get("temperature") or 0.75)
        top_p = float(payload.get("top_p") or 0.9)

        messages = [{"role": "system", "content": SYSTEM_PROMPT}]
        messages.extend(_normalize_history(payload.get("history")))
        messages.append({"role": "user", "content": message[:4000]})

        input_ids = self.tokenizer.apply_chat_template(
            messages,
            add_generation_prompt=True,
            return_tensors="pt",
        ).to(self.model.device)
        start_pos = input_ids.shape[-1]

        started = time.time()
        with torch.no_grad():
            output_ids = self.model.generate(
                input_ids,
                max_new_tokens=max_new_tokens,
                do_sample=True,
                temperature=temperature,
                top_p=top_p,
                repetition_penalty=1.1,
                pad_token_id=self.tokenizer.pad_token_id or self.tokenizer.eos_token_id,
            )

        text = self.tokenizer.decode(output_ids[0][start_pos:], skip_special_tokens=True)
        return {
            "reply": _clean_response(text),
            "model": BASE_MODEL,
            "adapter": ADAPTER_ID,
            "load_seconds": self.load_seconds,
            "generation_seconds": round(time.time() - started, 2),
        }

@app.local_entrypoint()
def main() -> None:
    print("Deploy with: modal deploy modal/modal_app.py")

# Deployment Notes

## Modal

The Modal endpoint uses Gemma 4 26B by default:

```text
BASE_MODEL=unsloth/gemma-4-26b-a4b-it
GPU_CONFIG=H100
ADAPTER_ID=build-small-hackathon/NeuroBait
```

If the HF repo currently points to a 12B adapter, publish the 26B LoRA adapter
to a separate HF model repo and set `ADAPTER_ID` before deployment.

```bash
modal secret create huggingface HF_TOKEN=hf_xxx
modal secret create neurobait-api NEUROBAIT_API_TOKEN=long-random-token
modal deploy modal/modal_app.py
```

Cost controls are in `modal/modal_app.py`:

- `min_containers=0`: scale to zero when idle.
- `scaledown_window=60`: keep a loaded container idle for about one minute.
- `max_containers=1`: prevent accidental parallel GPU spend.
- `max_new_tokens` is capped at 512 server-side.

For faster but more expensive interactive testing, temporarily increase
`SCALEDOWN_WINDOW`. Keep `min_containers=0` unless you intentionally want a warm
GPU that bills while idle.

## VPS

Reference VPS from the previous deployment log:

```text
Host: 43.157.223.106
User: ubuntu
Project path: /home/ubuntu/neurobait-self
Suggested port: 8091
PM2 name: neurobait-self
```

Do not commit SSH passwords or API tokens.

Minimal deploy flow from local:

```bash
tar czf /tmp/neurobait-self.tar.gz --exclude='node_modules' -C /media/haris-subrata/Work/llm/NeuroBait-Self .
scp /tmp/neurobait-self.tar.gz ubuntu@43.157.223.106:/tmp/
ssh ubuntu@43.157.223.106
mkdir -p /home/ubuntu/neurobait-self
tar xzf /tmp/neurobait-self.tar.gz -C /home/ubuntu/neurobait-self
cd /home/ubuntu/neurobait-self
cp vps/.env.example vps/.env
nano vps/.env
npm --prefix vps install --omit=dev
pm2 start vps/ecosystem.config.cjs
pm2 save
curl http://127.0.0.1:8091/healthz
```

If port `8091` is occupied, change `PORT` in `vps/.env` and restart PM2.

import dotenv from "dotenv";
import express from "express";
import rateLimit from "express-rate-limit";

dotenv.config();

const app = express();
const port = Number(process.env.PORT || 8091);
const modalChatUrl = process.env.MODAL_CHAT_URL;
const modalToken = process.env.NEUROBAIT_API_TOKEN;
const publicToken = process.env.PUBLIC_APP_TOKEN || "";
const maxNewTokens = Number(process.env.MAX_NEW_TOKENS || 220);

app.set("trust proxy", 1);
app.use(express.json({ limit: "128kb" }));
app.use(
  rateLimit({
    windowMs: 60 * 1000,
    limit: 20,
    standardHeaders: true,
    legacyHeaders: false
  })
);

function requirePublicToken(req, res, next) {
  if (!publicToken) {
    next();
    return;
  }

  const supplied = String(req.get("x-app-token") || "");
  if (supplied !== publicToken) {
    res.status(401).json({ error: "unauthorized" });
    return;
  }
  next();
}

app.get("/healthz", (_req, res) => {
  res.json({
    ok: true,
    modal_configured: Boolean(modalChatUrl && modalToken)
  });
});

app.post("/api/chat", requirePublicToken, async (req, res) => {
  if (!modalChatUrl || !modalToken) {
    res.status(500).json({ error: "modal endpoint is not configured" });
    return;
  }

  const message = String(req.body?.message || "").trim();
  if (!message) {
    res.status(400).json({ error: "message is required" });
    return;
  }

  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 15 * 60 * 1000);

  try {
    const modalResponse = await fetch(modalChatUrl, {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${modalToken}`,
        "Content-Type": "application/json"
      },
      body: JSON.stringify({
        message,
        history: Array.isArray(req.body?.history) ? req.body.history : [],
        temperature: Number(req.body?.temperature || 0.75),
        top_p: Number(req.body?.top_p || 0.9),
        max_new_tokens: Number(req.body?.max_new_tokens || maxNewTokens)
      }),
      signal: controller.signal
    });

    const text = await modalResponse.text();
    res.status(modalResponse.status).type("application/json").send(text);
  } catch (error) {
    const isAbort = error?.name === "AbortError";
    res.status(isAbort ? 504 : 502).json({
      error: isAbort ? "modal timeout" : "modal request failed"
    });
  } finally {
    clearTimeout(timeout);
  }
});

app.get("/", (_req, res) => {
  res.type("html").send(`<!doctype html>

  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>NeuroBait Self</title>

NeuroBait Self

    <form id="form">
      <textarea id="message" placeholder="Tulis yang sedang bikin kamu stuck..."></textarea>
      <button id="send" type="submit">Send</button>
    </form>

`);
});

app.listen(port, "0.0.0.0", () => {
  console.log(`neurobait-self listening on ${port}`);
});


Output:
Exit code: 0
Wall time: 1.7 seconds
Output:
Success. Updated the following files:
A .tmp_neurobait_self/README.md
A .tmp_neurobait_self/.gitignore
A .tmp_neurobait_self/modal/modal_app.py
A .tmp_neurobait_self/scripts/smoke_modal.py
A .tmp_neurobait_self/adapter/README.md
A .tmp_neurobait_self/vps/package.json
A .tmp_neurobait_self/vps/.env.example
A .tmp_neurobait_self/vps/ecosystem.config.cjs
A .tmp_neurobait_self/vps/server.js
A .tmp_neurobait_self/docs/deploy.md


Saya menemukan satu detail implementasi auth header yang perlu dibuat eksplisit untuk FastAPI/Modal. Saya patch dulu supaya endpoint benar-benar membaca `Authorization: Bearer ...` dari VPS.


Input:
*** Begin Patch
*** Update File: .tmp_neurobait_self/modal/modal_app.py
@@
 import modal
+from fastapi import Header
@@
-    def chat(self, payload: dict[str, Any], authorization: str | None = None) -> dict[str, Any]:
+    def chat(
+        self,
+        payload: dict[str, Any],
+        authorization: str | None = Header(default=None),
+    ) -> dict[str, Any]:
         import torch
-        from fastapi import Header

-        # Header is imported so Modal/FastAPI includes the dependency in the image;
-        # the actual value is passed by FastAPI into the `authorization` parameter.
-        _ = Header
         self._check_auth(authorization)
*** End Patch


Success. Updated the following files:
M .tmp_neurobait_self/modal/modal_app.py

@@ -8,2 +8,3 @@
 import modal
+from fastapi import Header

@@ -147,9 +148,9 @@
     @modal.fastapi_endpoint(method="POST", docs=False)
-    def chat(self, payload: dict[str, Any], authorization: str | None = None) -> dict[str, Any]:
+    def chat(
+        self,
+        payload: dict[str, Any],
+        authorization: str | None = Header(default=None),
+    ) -> dict[str, Any]:
         import torch
-        from fastapi import Header

-        # Header is imported so Modal/FastAPI includes the dependency in the image;
-        # the actual value is passed by FastAPI into the `authorization` parameter.
-        _ = Header
         self._check_auth(authorization)
@@ -203,2 +204 @@
     print("Deploy with: modal deploy modal/modal_app.py")
-


Output:
Exit code: 0
Wall time: 0.1 seconds
Output:
Success. Updated the following files:
M .tmp_neurobait_self/modal/modal_app.py


cp -a .tmp_neurobait_self/. /media/haris-subrata/Work/llm/NeuroBait-Self/

Boleh saya menyalin scaffold project dari staging workspace ke folder baru /media/haris-subrata/Work/llm/NeuroBait-Self?


Output:
Chunk ID: 07ecfd
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 0
Output:


Approved command prefix saved:
- ["cp", "-a"]


Scaffold sudah disalin ke project baru. Saya akan tambahkan symlink adapter lokal dan jalankan validasi ringan untuk memastikan Python/Node source tidak ada syntax error sebelum saya tutup.


Boleh saya membuat symlink adapter lokal di project baru agar tidak menduplikasi file safetensors?


Output:
Chunk ID: 37972d
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 0
Output:


Approved command prefix saved:
- ["ln", "-s"]


Saya akan cek daftar file project baru, compile file Python, dan validasi syntax Node. Kalau dependency lokal belum ada, saya akan bedakan antara error dependency dan error syntax.


find . -maxdepth 3 -type f -o -type l | sort


python -m py_compile modal/modal_app.py scripts/smoke_modal.py


Output:
Chunk ID: cc416a
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 55
Output:
./.gitignore
./README.md
./adapter/README.md
./adapter/neurobait-lora-run3
./docs/deploy.md
./modal/modal_app.py
./scripts/smoke_modal.py
./vps/.env.example
./vps/ecosystem.config.cjs
./vps/package.json
./vps/server.js


Output:
Chunk ID: 7b740e
Wall time: 0.0000 seconds
Process exited with code 127
Original token count: 12
Output:
/bin/bash: line 1: python: command not found


Output:
Chunk ID: d54ae1
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 0
Output:


Node syntax valid. Python command default `python` tidak ada di shell ini, jadi saya ulang dengan `python3`; jika gagal karena dependency `modal/fastapi` belum terinstall lokal, itu bukan syntax issue project.


python3 -m py_compile modal/modal_app.py scripts/smoke_modal.py


Output:
Chunk ID: 374a03
Wall time: 0.0000 seconds
Process exited with code 1
Original token count: 14
Output:
[Errno 30] Read-only file system: 'modal/__pycache__'


Compile Python mencoba menulis `__pycache__` di project baru, dan sandbox membatasi write di luar workspace. Saya arahkan cache bytecode ke `/tmp` agar tetap bisa validasi tanpa mengubah folder target.


env PYTHONPYCACHEPREFIX=/tmp/neurobait-self-pycache python3 -m py_compile modal/modal_app.py scripts/smoke_modal.py


Output:
Chunk ID: 501efb
Wall time: 0.0133 seconds
Process exited with code 0
Original token count: 0
Output:


Satu hal lagi: saya bersihkan staging folder sementara di repo lama supaya project lama tidak ikut kotor oleh scaffold yang sudah disalin.


Boleh saya menghapus staging folder sementara .tmp_neurobait_self yang saya buat dan sudah disalin ke project baru?


Output:
Chunk ID: 68eeab
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 0
Output:


Approved command prefix saved:
- ["rm", "-rf", ".tmp_neurobait_self"]


Saya akan cek status singkat di repo lama dan project baru supaya jelas tidak ada sisa staging di project lama.


find . -maxdepth 2 -type f -o -type l | sort


Output:
Chunk ID: 52d671
Wall time: 0.0000 seconds
Process exited with code 128
Original token count: 40
Output:
fatal: not a git repository (or any parent up to mount point /media/haris-subrata)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).


Output:
Chunk ID: 61e67e
Wall time: 0.0000 seconds
Process exited with code 0
Original token count: 55
Output:
./.gitignore
./README.md
./adapter/README.md
./adapter/neurobait-lora-run3
./docs/deploy.md
./modal/modal_app.py
./scripts/smoke_modal.py
./vps/.env.example
./vps/ecosystem.config.cjs
./vps/package.json
./vps/server.js


Sudah saya buat project baru di:

[NeuroBait-Self](/media/haris-subrata/Work/llm/NeuroBait-Self)

Isinya:

- [modal/modal_app.py](/media/haris-subrata/Work/llm/NeuroBait-Self/modal/modal_app.py): Modal GPU API untuk `unsloth/gemma-4-26b-a4b-it`, 4-bit, bearer auth, `min_containers=0`, `scaledown_window=60`, `max_containers=1`.
- [vps/server.js](/media/haris-subrata/Work/llm/NeuroBait-Self/vps/server.js): proxy/chat app ringan untuk VPS, default port `8091`.
- [vps/ecosystem.config.cjs](/media/haris-subrata/Work/llm/NeuroBait-Self/vps/ecosystem.config.cjs): PM2 config `neurobait-self`.
- [docs/deploy.md](/media/haris-subrata/Work/llm/NeuroBait-Self/docs/deploy.md): runbook Modal + VPS.
- [adapter/neurobait-lora-run3](/media/haris-subrata/Work/llm/NeuroBait-Self/adapter/neurobait-lora-run3): symlink ke adapter lama, jadi tidak copy file besar.

Catatan penting: symlink adapter hanya untuk referensi lokal. Modal runtime tidak bisa membaca path workstation setelah deploy, jadi `ADAPTER_ID` di Modal harus menunjuk ke HF repo adapter 26B. Kalau repo `build-small-hackathon/NeuroBait` sekarang sudah diganti/diarahkan ke 12B, buat repo HF terpisah untuk 26B lalu ubah `ADAPTER_ID`.

Validasi yang sudah jalan:

- `python3 -m py_compile modal/modal_app.py scripts/smoke_modal.py` sukses.
- `node --check vps/server.js` sukses.

Saya belum menjalankan `modal deploy` atau `npm install` karena butuh akun Modal/token, HF secret, dan network install. Modal docs yang dipakai: `fastapi_endpoint` untuk HTTP API dan autoscaling scale-to-zero via `min_containers/scaledown_window`: https://modal.com/docs/reference/modal.web_endpoint dan https://modal.com/docs/guide/scale.