pungyo 5-file: server-single + server-twin + cuda runtime + docs

.gitattributes

@@ -3,3 +3,5 @@
 *.gguf filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
+cublas64_13.dll filter=lfs diff=lfs merge=lfs -text
+cublasLt64_13.dll filter=lfs diff=lfs merge=lfs -text

README.md

@@ -10,7 +10,8 @@ tags:
 - qwen
 - windows
 - consumer-gpu
-- verification
+- openai-compatible
+- llama-cpp
 pipeline_tag: text-generation
 library_name: gguf
 ---
@@ -19,69 +20,93 @@ library_name: gguf
 
 > **램을 풍요롭게 — Run big. Keep your memory abundant.**
 
-✅ **It works.** Single self-contained binary. Provide a Qwen GGUF, run, done.
+OpenAI-compatible local server. Pick the files you need, drop them in a folder,
+run one of the two `.exe` files. The model loads on an 8GB laptop GPU; the
+operating point is chosen automatically in memory.
 
 This repository is **not a model release**.
-Model weights are **not included**.
 The internal mechanism is **not disclosed**.
 
-## Reference local observation (single profile)
+## Speed (measured, RTX 4060 Laptop 8GB, 245-token steady-state)
 
-```text
-33.27 tok/s   (8GB laptop GPU, ~7GB VRAM, steady-state)
-```
-
-Measured on Qwen3.6-35B-A3B (Q4_K_M, user-provided GGUF), 2026-05-30, 245-token generation.
-A local observation, not a universal guarantee.
+| profile | speed | VRAM | when to pick |
+|---|---|---|---|
+| **single** (octagon) | ~34.5 tok/s | ~7 GB (tight) | maximum speed |
+| **twin** (wireframe) | ~28.5 tok/s | ~3.7 GB (headroom) | stability under load |
 
-> Want maximum memory headroom / stability? See the twin profile:
-> **myeongjun77/Pungyo-Qwen36-A3B-Twin**
+Twin nearly halves VRAM pressure for ~17% less speed — far less likely to OOM
+during longer or heavier runs. Same model, different operating point.
 
 ## How does it work?
 
 Not explained here. The mechanism is covered by patent-filed technology
-(application filed, KR). This release only demonstrates **that it works** —
-it does not disclose **how**.
+(application filed, KR). This release only demonstrates **that it works and
+stays stable** — it does not disclose **how**.
+
+## Download — pick 4 files (5 if you want both profiles)
+
+| file | required? | size |
+|---|---|---|
+| `Qwen3.6-35B-A3B-Q4_K_M.gguf` | yes (the model) | ~20 GB |
+| `cublas64_13.dll` | yes (GPU runtime) | 49 MB |
+| `cublasLt64_13.dll` | yes (GPU runtime) | 435 MB |
+| `pungyo-server-single.exe` | pick at least one | 246 MB |
+| `pungyo-server-twin.exe` | pick at least one | 246 MB |
 
-## Download & Run (3 steps)
+Put them all in the **same folder**. That's the install.
 
-1. Get a compatible **Qwen3.6 35B-A3B GGUF** from https://huggingface.co/Qwen
-2. Download `Pungyo-Qwen36-Single-Public-Verification-20260530.zip` from this repo, unzip.
-3. Run:
+## Run (3 steps)
+
+1. **Open PowerShell in the folder** where you put the files.
+2. **Start the server** — single (fast) or twin (stable):
 
 ```powershell
-.\bin\pungyo.exe "path\to\Qwen3.6-35B-A3B.gguf" "Hello, my name is"
+# single profile (max speed)
+.\pungyo-server-single.exe -m .\Qwen3.6-35B-A3B-Q4_K_M.gguf --port 8080 --host 127.0.0.1 -c 4096
+
+# OR twin profile (stable)
+.\pungyo-server-twin.exe   -m .\Qwen3.6-35B-A3B-Q4_K_M.gguf --port 8080 --host 127.0.0.1 -c 4096
 ```
 
-That's it. No setup, no separate llama.cpp install, no flags. The binary picks
-the operating point in-memory and runs in-process.
+3. **Use it.** Wait ~30-60 seconds for the model to load, then:
+   - **Browser**: open `http://localhost:8080` — built-in chat UI is ready
+   - **OpenAI-compatible clients** (Cursor, Continue, ChatBox, Open WebUI, Jan, LangChain, etc.): set base URL to `http://localhost:8080/v1`
+   - **API**: `POST http://localhost:8080/v1/chat/completions` (OpenAI schema)
+
+Stop with `Ctrl+C` in the server window.
 
 ## Requirements
 
 - Windows 10/11
-- NVIDIA RTX-class 8GB VRAM GPU (or compatible), CUDA driver installed
+- NVIDIA RTX-class 8GB VRAM GPU (or larger), recent NVIDIA driver
 - 32 GB system memory recommended
-- a user-provided Qwen3.6 35B-A3B compatible GGUF
+- ~21 GB free disk for the files
+
+## Security
 
-(Bundled in the ZIP: `bin\pungyo.exe`, `bin\cublas64_13.dll`, `bin\cublasLt64_13.dll`. CUDA runtime libraries are included; you do NOT need the CUDA toolkit installed.)
+The example uses `--host 127.0.0.1` — the server is **reachable only from your
+own PC**. To allow other machines on your LAN to use it, change to
+`--host 0.0.0.0` and add `--api-key YOUR_KEY` (and ideally place a TLS reverse
+proxy like Caddy in front). Never expose to the public internet without auth +
+TLS.
 
 ## Not Included
 
-- Qwen model weights (get from https://huggingface.co/Qwen)
-- training data
-- internal mechanism / operating-point logic
+- Qwen training data
+- internal mechanism / operating-point logic (source)
 - private validation traces
 
 ## License
 
-- Pungyo execution package: see `LICENSE-PUNGYO`.
-- Built from llama.cpp / ik_llama.cpp (MIT): see `LICENSE-LLAMACPP`.
-- Qwen base model (not included): see `LICENSE-QWEN`.
+- Pungyo execution binaries: see `LICENSE-PUNGYO`.
+- Built on llama.cpp / ik_llama.cpp (MIT): see `LICENSE-LLAMACPP`.
+- Qwen base model (`Qwen3.6-35B-A3B-Q4_K_M.gguf`) is redistributed under
+  Apache-2.0: see `LICENSE-QWEN`.
 - License scope separation: see `NOTICE.md`.
 
 ## Patent Notice
 
-Pungyo relates to patent-filed technology by Myeong Jun Jo (application filed, KR).
-It is provided as an observable public verification artifact, **not** as an
-internal implementation disclosure. Commercial or patent licensing inquiries:
-koing754@gmail.com
+Pungyo relates to patent-filed technology by Myeong Jun Jo
+(application filed, KR). This release is an observable public verification
+artifact, **not** an internal implementation disclosure. Commercial or patent
+licensing inquiries: koing754@gmail.com

cublas64_13.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:537d278122e2621d7202653cf873a9b8b872fac37122cb9d36246ded33c53e03
+size 51572848

cublasLt64_13.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b37233a1e3516a41d27dcea49595fd1cbaff979b65c80da69935cb59d0d23f5b
+size 455684208

pungyo-server-single.exe

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef73a57ebcfacb7e5a0a57a352d74693827a9efb7021701ed5cdec628386988b
+size 257659904

pungyo-server-twin.exe

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef73a57ebcfacb7e5a0a57a352d74693827a9efb7021701ed5cdec628386988b
+size 257659904

run.md

@@ -1,36 +1,65 @@
-# Pungyo — Run Guide (single profile)
+# Pungyo — Run Guide (server)
 
-Run a Qwen3.6 35B-A3B class GGUF on a single 8GB GPU. You provide the model.
-Pungyo is a single self-contained binary.
+Pungyo runs as a local OpenAI-compatible HTTP server. You provide the model
+GGUF and the two CUDA runtime DLLs in the same folder as the `.exe`.
 
-## 1. Get a model
+## 1. Folder layout
 
-Download a compatible Qwen3.6 35B-A3B GGUF from the official source:
-https://huggingface.co/Qwen
-
-Pungyo does not ship, modify, or relicense model weights.
-
-## 2. Unzip
+Put these in one folder (call it whatever):
 
 ```
-Pungyo-Qwen36-Single-Public-Verification-20260530.zip
-├── bin\pungyo.exe
-├── bin\cublas64_13.dll
-└── bin\cublasLt64_13.dll
+.\
+├── pungyo-server-single.exe   (or pungyo-server-twin.exe — or both)
+├── cublas64_13.dll
+├── cublasLt64_13.dll
+└── Qwen3.6-35B-A3B-Q4_K_M.gguf
 ```
 
-## 3. Run
+All four (or five) files in the SAME directory.
+
+## 2. Pick a profile, start the server
+
+Open PowerShell in that folder.
 
 ```powershell
-.\bin\pungyo.exe "D:\models\Qwen3.6-35B-A3B.gguf" "Hello, my name is"
+# single profile — maximum speed (~34.5 tok/s, ~7 GB VRAM)
+.\pungyo-server-single.exe -m .\Qwen3.6-35B-A3B-Q4_K_M.gguf --port 8080 --host 127.0.0.1 -c 4096
+
+# twin profile — stability-first (~28.5 tok/s, ~3.7 GB VRAM)
+.\pungyo-server-twin.exe   -m .\Qwen3.6-35B-A3B-Q4_K_M.gguf --port 8080 --host 127.0.0.1 -c 4096
 ```
 
-That is the full invocation. The residency operating point is decided by Pungyo
-in memory and never appears on the command line.
+The server takes ~30-60 seconds to load the model. You'll see logs end with
+`HTTP server is listening` — that means it's ready.
+
+The operating point is chosen automatically in memory, never on the command
+line. The two `.exe` files are siblings of the same engine; the file name
+selects the profile.
+
+## 3. Use it
+
+| client | how |
+|---|---|
+| **Browser** | open `http://localhost:8080` (built-in chat) |
+| **Cursor / Continue / ChatBox / Open WebUI / Jan** | set OpenAI base URL to `http://localhost:8080/v1` |
+| **LangChain / OpenAI SDK / raw HTTP** | `POST http://localhost:8080/v1/chat/completions` (OpenAI schema) |
+
+Stop with `Ctrl+C` in the server window.
+
+## 4. Common flags
+
+| flag | meaning |
+|---|---|
+| `-m PATH` | path to the GGUF |
+| `--port N` | HTTP port (default 8080) |
+| `--host 127.0.0.1` | localhost only (recommended) |
+| `--host 0.0.0.0` | reachable from your LAN — add `--api-key ...` if you do this |
+| `-c N` | context window in tokens (default 4096, max model trains to 262144) |
+| `--api-key KEY` | require this key in `Authorization: Bearer KEY` |
 
 ## Notes
 
-- Requires Windows, NVIDIA RTX-class 8GB GPU with a recent NVIDIA driver.
+- Requires Windows + NVIDIA RTX-class 8GB GPU + recent NVIDIA driver.
 - 32 GB system RAM recommended.
 - Speed is a local observation and varies by hardware.
 - The mechanism is not documented here by design (patent-filed).