Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,222 @@
+---
+library_name: cactus-needle
+pipeline_tag: text-generation
+license: apache-2.0
+tags:
+  - tool-calling
+  - function-calling
+  - on-device
+  - edge
+  - quantization
+  - webassembly
+---
+
+![Needle 2](banner.png)
+
+# Needle 2
+
+Needle 2 is a 45M-parameter foundation tool call/use model for tiny devices; run it or embed it in phones, wearables, watches, TVs, smart home, small robots. 
+Compressed to CQ2-bit with Cactus Quants, the whole model ships as a single 14MB binary that runs a full session in 28MB of RAM. 
+A Raspberry Pi 5 prefills at 1.3k tok/s and decodes at 500+; an iPhone 17 Pro prefills at 3k+ and decodes at 1k+.
+
+- **Self-contained**: model baked into the binary, no runtime, no downloads, no network.
+- **Runs everywhere**: ARM64, x86-64, ARMv7, RISC-V, Cortex-M, and WebAssembly, on Apple, Windows, Linux, Android, Raspberry Pi.
+- **Simple contract**: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token.
+- **Confidence-gated**: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it.
+- **Tool retrieval**: declare a large catalogue and a built-in retrieval head renders only the top ten tools per turn, with the grammar constrained to that subset.
+- **Bounded memory**: a 256-token sliding window with the tools pinned as KV sinks; session memory caps near 14MB no matter how long the conversation runs.
+
+![Size-quality frontier: mobile-class and below](frontier.svg)
+
+## Quickstart with Python
+
+```sh
+pip install cactus-needle
+```
+
+The PyPI package is a small pure-Python shim; the first `import needle` downloads the engine for your platform (~13 MB, cached in `~/.cache/cactus-needle`), so the first run needs network. Every run after is offline.
+
+```python
+import needle
+
+tools = [
+    {"name": "set_timer",
+     "parameters": {"type": "object", "properties": {"minutes": {"type": "integer"}, "label": {"type": "string"}}}},
+    {"name": "play_music",
+     "parameters": {"type": "object", "properties": {"query": {"type": "string"}}}},
+]
+
+agent = needle.Needle(tools=tools)
+# with many tools, persist their embeddings across runs:
+# agent = needle.Needle(tools=tools, tool_index_path="tools.idx")
+
+response = agent.complete("set a 10 minute timer for the pasta") 
+
+if response["type"] == "call":
+    tool_result = run_tool(response["function_calls"][0])
+    response = agent.complete(tool_result) 
+    response = agent.complete("and put on some jazz")   
+
+agent.reset() 
+```
+
+Every turn returns one JSON object:
+
+```json
+{
+  "type": "call",
+  "success": true,
+  "error": null,
+  "error_code": null,
+  "function_calls": [ { "name": "set_lights", "arguments": { "room": "living room", "on": true, "brightness": 30 } } ],
+  "reasoning": "'living room' -> room; 'dim' -> on true, brightness 30",
+  "confidence": 0.94,
+  "prefill_tps": 4300.0,
+  "decode_tps": 850.0,
+  "peak_ram_mb": 28.0
+}
+```
+
+## Behaviour
+
+Needle solves every problem as a function call. The context declares what may be called; the model answers with calls. Performing an action and extracting structured data are the same operation, the only difference is what you declare.
+
+- A request no declared tool can serve is refused with the empty call `[]`. That is the whole contract for off-topic input; there is no free-text fallback.
+- Arguments contain only values evidenced by the input. An optional field with no evidence is omitted, not guessed; omission is the field-level `[]`.
+- `reasoning` is the model's short derivation of each argument from its source span (`'ten minutes' -> minutes 10`). It is generated unconstrained; only the call itself is grammar-constrained, so the JSON cannot be malformed while the derivation stays legible.
+- After you execute a call, pass the result back as the next `complete()`. The model continues from it, and later arguments may depend on earlier results: `search_for_contact` first, then `send_instant_message` with the returned `contact_id`. A final step may answer in plain text from the results: `"type": "respond"` with empty `function_calls`.
+- A session shares one toolset. Later turns are bare queries against the same tools; `reset()` rewinds the conversation and keeps the tools loaded.
+
+## System facts
+
+An optional system turn carries environment state as facts, never instructions:
+
+```
+date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%
+```
+
+Recognized keys are `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, and `assistant`. The model resolves relative language against them: "tomorrow at 7" becomes an absolute time only when a `date:` fact licenses it, otherwise the human phrase passes through verbatim. `assistant:` declares the identity the model binds to. Pass the turn with `--system system.txt` on the CLI or `needle.Needle(tools=tools, system="date: ...")` in Python. Needle trains with and without the turn, so omitting it is safe; instructions placed there do not steer the model.
+
+## Deploy Needle
+
+Download the folder for your platform from the release:
+
+| your device | folder | command-line | library |
+| --- | --- | --- | --- |
+| Mac (Apple Silicon) | `macos-arm64` | `needle` | `libneedle.a` |
+| Linux x86-64 (PC, server, AMD) | `linux-x86_64` | `needle` | `libneedle.a` |
+| Linux ARM64 (Raspberry Pi, server) | `linux-arm64` | `needle` | `libneedle.a` |
+| Linux ARMv7 (32-bit) | `linux-armv7` | `needle` | `libneedle.a` |
+| Linux RISC-V | `linux-riscv64` | `needle` | `libneedle.a` |
+| Linux MIPS32el (Ingenic cameras, routers) | `linux-mipsel` | `needle` | `libneedle.a` |
+| Windows x64 | `windows-x86_64` | `needle.exe` | `libneedle.a` |
+| Windows ARM | `windows-arm64` | `needle.exe` | `libneedle.a` |
+| Android | `android-arm64` / `android-armv7` / `android-riscv64` | `needle` | `libneedle.a` |
+| iOS / watchOS / tvOS | `ios-arm64` / `watchos-arm64` / `tvos-arm64` | - | `libneedle.a` |
+| Cortex-M (bare-metal/RTOS) | `cortex-m4` / `cortex-m7` / `cortex-m55` | - | `libneedle.a` |
+| Browser / Node (WebAssembly) | `wasm` | - | `needle.js` + `needle.wasm` |
+
+To run it, use the command-line binary. On macOS, Linux, or Android:
+
+```sh
+# answer one query and exit
+./needle --tools tools.json --prompt "dim the living room to 30"
+
+# or an HTTP server on localhost:8080 (POST /complete {"input": "..."})
+./needle --tools tools.json --serve
+
+# with a large tool catalogue, persist tool embeddings across runs
+./needle --tools tools.json --tool-index tools.idx --serve
+```
+
+`tools.json` is a JSON array of the functions the assistant may call:
+
+```json
+[
+  {
+    "name": "set_lights",
+    "description": "Turn a room's lights on or off and set brightness",
+    "parameters": {
+      "type": "object",
+      "properties": {
+        "room": { "type": "string" },
+        "on": { "type": "boolean" },
+        "brightness": { "type": "integer", "description": "0 to 100" }
+      },
+      "required": ["room", "on"]
+    }
+  },
+  {
+    "name": "play_music",
+    "description": "Play music matching a mood, genre, or artist",
+    "parameters": {
+      "type": "object",
+      "properties": { "query": { "type": "string" } },
+      "required": ["query"]
+    }
+  },
+  {
+    "name": "send_message",
+    "description": "Text a contact",
+    "parameters": {
+      "type": "object",
+      "properties": {
+        "to": { "type": "string" },
+        "body": { "type": "string" }
+      },
+      "required": ["to", "body"]
+    }
+  }
+]
+```
+
+## Tool retrieval
+
+Ten or fewer declared tools render directly. Above that, retrieval engages: at init every tool schema is embedded once by a built-in contrastive head, 
+each turn embeds the query, and only the ten highest-scoring tools enter the context, with the grammar rebuilt over just that subset. an unselected 
+tool is unreachable, not merely unlikely. `--tool-index <path>` (CLI) or `tool_index_path` (Python) persists the embeddings on disk, keyed by a fingerprint 
+over the schemas and the model; a matching fingerprint loads instantly, a changed schema re-embeds only what changed.
+
+## Confidence
+
+The `confidence` field is the minimum of two signals: a calibrated post-hoc head that scores the full prompt plus the call the model just produced, and 
+the decoding probability of the call tokens. A call is accepted only when both agree, so the failure mode is escalation, not wrong execution. The contract: 
+pick a threshold for your product, act at or above it, re-ask or route to a bigger model below it. Off-topic requests return the empty call `[]`.
+
+## Custom weights
+
+- `needle_load(cact, n)` borrows the caller's buffer: no copy is made, the pointer must stay valid and unmodified until the next `needle_load` call or process exit. This is the load path when weights are not embedded (WebAssembly, or bytes fetched over a network); to load a `.cact` from disk, read the file and pass the bytes.
+- Embedded builds keep the weights in the binary's read-only section; all weight bytes stay file-backed and evictable, nothing is copied to the heap.
+
+## Extraction
+
+Extraction is the same exchange as tool calling: declare the record schema as the only tool and pass the content as the prompt; the passage sits where the query sits, and the returned call's `arguments` are the extracted fields. With one declared tool the grammar admits exactly one call of that name, the `tool_choice` equivalent, so schema conformance is guaranteed rather than requested. There is no separate JSON mode.
+
+`schema.json` describes the record to extract:
+
+```json
+[
+  {
+    "name": "receipt",
+    "description": "A purchase receipt shared as text",
+    "parameters": {
+      "type": "object",
+      "properties": {
+        "merchant": { "type": "string" },
+        "total": { "type": "number" },
+        "currency": { "type": "string" },
+        "line_items": { "type": "array", "items": { "type": "object" } }
+      },
+      "required": ["merchant", "total"]
+    }
+  }
+]
+```
+
+```sh
+./needle --tools schema.json --prompt "GreenMart receipt: oat milk 3.50, total 7.75 paid by visa"
+```
+
+```json
+{ "type": "call", "function_calls": [ { "name": "receipt", "arguments": { "merchant": "GreenMart", "total": 7.75 } } ] }
+```