---
library_name: cactus-needle
pipeline_tag: text-generation
license: apache-2.0
tags:
  - tool-calling
  - function-calling
  - on-device
  - edge
  - quantization
  - webassembly
---

![Needle 2](banner.png)

# Needle 2

Needle 2 is a 45M-parameter foundation tool call/use model for tiny devices; run it or embed it in phones, wearables, watches, TVs, smart home, small robots. 
Compressed to CQ2-bit with Cactus Quants, the whole model ships as a single 14MB binary that runs a full session in 28MB of RAM. 
A Raspberry Pi 5 prefills at 1.3k tok/s and decodes at 500+; an iPhone 17 Pro prefills at 3k+ and decodes at 1k+.

- **Self-contained**: model baked into the binary, no runtime, no downloads, no network.
- **Runs everywhere**: ARM64, x86-64, ARMv7, RISC-V, Cortex-M, and WebAssembly, on Apple, Windows, Linux, Android, Raspberry Pi.
- **Simple contract**: tool calls come back as structured data, text in, JSON out; a byte-level grammar compiled from your schemas constrains every token.
- **Confidence-gated**: every response carries a calibrated confidence score from a learned head; set a threshold, act above it, escalate below it.
- **Tool retrieval**: declare a large catalogue and a built-in retrieval head renders only the top ten tools per turn, with the grammar constrained to that subset.
- **Bounded memory**: a 256-token sliding window with the tools pinned as KV sinks; session memory caps near 14MB no matter how long the conversation runs.

![Size-quality frontier: mobile-class and below](frontier.svg)

## Quickstart with Python

```sh
pip install cactus-needle
```

The PyPI package is a small pure-Python shim; the first `import needle` downloads the engine for your platform (~13 MB, cached in `~/.cache/cactus-needle`), so the first run needs network. Every run after is offline.

```python
import needle

tools = [
    {"name": "set_timer",
     "parameters": {"type": "object", "properties": {"minutes": {"type": "integer"}, "label": {"type": "string"}}}},
    {"name": "play_music",
     "parameters": {"type": "object", "properties": {"query": {"type": "string"}}}},
]

agent = needle.Needle(tools=tools)
# with many tools, persist their embeddings across runs:
# agent = needle.Needle(tools=tools, tool_index_path="tools.idx")

response = agent.complete("set a 10 minute timer for the pasta") 

if response["type"] == "call":
    tool_result = run_tool(response["function_calls"][0])
    response = agent.complete(tool_result) 
    response = agent.complete("and put on some jazz")   

agent.reset() 
```

Every turn returns one JSON object:

```json
{
  "type": "call",
  "success": true,
  "error": null,
  "error_code": null,
  "function_calls": [ { "name": "set_lights", "arguments": { "room": "living room", "on": true, "brightness": 30 } } ],
  "reasoning": "'living room' -> room; 'dim' -> on true, brightness 30",
  "confidence": 0.94,
  "prefill_tps": 4300.0,
  "decode_tps": 850.0,
  "peak_ram_mb": 28.0
}
```

## Behaviour

Needle solves every problem as a function call. The context declares what may be called; the model answers with calls. Performing an action and extracting structured data are the same operation, the only difference is what you declare.

- A request no declared tool can serve is refused with the empty call `[]`. That is the whole contract for off-topic input; there is no free-text fallback.
- Arguments contain only values evidenced by the input. An optional field with no evidence is omitted, not guessed; omission is the field-level `[]`.
- `reasoning` is the model's short derivation of each argument from its source span (`'ten minutes' -> minutes 10`). It is generated unconstrained; only the call itself is grammar-constrained, so the JSON cannot be malformed while the derivation stays legible.
- After you execute a call, pass the result back as the next `complete()`. The model continues from it, and later arguments may depend on earlier results: `search_for_contact` first, then `send_instant_message` with the returned `contact_id`. A final step may answer in plain text from the results: `"type": "respond"` with empty `function_calls`.
- A session shares one toolset. Later turns are bare queries against the same tools; `reset()` rewinds the conversation and keeps the tools loaded.

## System facts

An optional system turn carries environment state as facts, never instructions:

```
date: 2026-07-21 Tue 14:30; locale: en-US; device: phone; battery: 62%
```

Recognized keys are `date`, `locale`, `device`, `battery`, `network`, `location`, `user`, and `assistant`. The model resolves relative language against them: "tomorrow at 7" becomes an absolute time only when a `date:` fact licenses it, otherwise the human phrase passes through verbatim. `assistant:` declares the identity the model binds to. Pass the turn with `--system system.txt` on the CLI or `needle.Needle(tools=tools, system="date: ...")` in Python. Needle trains with and without the turn, so omitting it is safe; instructions placed there do not steer the model.

## Deploy Needle

Download the folder for your platform from the release:

| your device | folder | command-line | library |
| --- | --- | --- | --- |
| Mac (Apple Silicon) | `macos-arm64` | `needle` | `libneedle.a` |
| Linux x86-64 (PC, server, AMD) | `linux-x86_64` | `needle` | `libneedle.a` |
| Linux ARM64 (Raspberry Pi, server) | `linux-arm64` | `needle` | `libneedle.a` |
| Linux ARMv7 (32-bit) | `linux-armv7` | `needle` | `libneedle.a` |
| Linux RISC-V | `linux-riscv64` | `needle` | `libneedle.a` |
| Linux MIPS32el (Ingenic cameras, routers) | `linux-mipsel` | `needle` | `libneedle.a` |
| Windows x64 | `windows-x86_64` | `needle.exe` | `libneedle.a` |
| Windows ARM | `windows-arm64` | `needle.exe` | `libneedle.a` |
| Android | `android-arm64` / `android-armv7` / `android-riscv64` | `needle` | `libneedle.a` |
| iOS / watchOS / tvOS | `ios-arm64` / `watchos-arm64` / `tvos-arm64` | - | `libneedle.a` |
| Cortex-M (bare-metal/RTOS) | `cortex-m4` / `cortex-m7` / `cortex-m55` | - | `libneedle.a` |
| Browser / Node (WebAssembly) | `wasm` | - | `needle.js` + `needle.wasm` |

To run it, use the command-line binary. On macOS, Linux, or Android:

```sh
# answer one query and exit
./needle --tools tools.json --prompt "dim the living room to 30"

# or an HTTP server on localhost:8080 (POST /complete {"input": "..."})
./needle --tools tools.json --serve

# with a large tool catalogue, persist tool embeddings across runs
./needle --tools tools.json --tool-index tools.idx --serve
```

`tools.json` is a JSON array of the functions the assistant may call:

```json
[
  {
    "name": "set_lights",
    "description": "Turn a room's lights on or off and set brightness",
    "parameters": {
      "type": "object",
      "properties": {
        "room": { "type": "string" },
        "on": { "type": "boolean" },
        "brightness": { "type": "integer", "description": "0 to 100" }
      },
      "required": ["room", "on"]
    }
  },
  {
    "name": "play_music",
    "description": "Play music matching a mood, genre, or artist",
    "parameters": {
      "type": "object",
      "properties": { "query": { "type": "string" } },
      "required": ["query"]
    }
  },
  {
    "name": "send_message",
    "description": "Text a contact",
    "parameters": {
      "type": "object",
      "properties": {
        "to": { "type": "string" },
        "body": { "type": "string" }
      },
      "required": ["to", "body"]
    }
  }
]
```

## Tool retrieval

Ten or fewer declared tools render directly. Above that, retrieval engages: at init every tool schema is embedded once by a built-in contrastive head, 
each turn embeds the query, and only the ten highest-scoring tools enter the context, with the grammar rebuilt over just that subset. an unselected 
tool is unreachable, not merely unlikely. `--tool-index <path>` (CLI) or `tool_index_path` (Python) persists the embeddings on disk, keyed by a fingerprint 
over the schemas and the model; a matching fingerprint loads instantly, a changed schema re-embeds only what changed.

## Confidence

The `confidence` field is the minimum of two signals: a calibrated post-hoc head that scores the full prompt plus the call the model just produced, and 
the decoding probability of the call tokens. A call is accepted only when both agree, so the failure mode is escalation, not wrong execution. The contract: 
pick a threshold for your product, act at or above it, re-ask or route to a bigger model below it. Off-topic requests return the empty call `[]`.

## Custom weights

- `needle_load(cact, n)` borrows the caller's buffer: no copy is made, the pointer must stay valid and unmodified until the next `needle_load` call or process exit. This is the load path when weights are not embedded (WebAssembly, or bytes fetched over a network); to load a `.cact` from disk, read the file and pass the bytes.
- Embedded builds keep the weights in the binary's read-only section; all weight bytes stay file-backed and evictable, nothing is copied to the heap.

## Extraction

Extraction is the same exchange as tool calling: declare the record schema as the only tool and pass the content as the prompt; the passage sits where the query sits, and the returned call's `arguments` are the extracted fields. With one declared tool the grammar admits exactly one call of that name, the `tool_choice` equivalent, so schema conformance is guaranteed rather than requested. There is no separate JSON mode.

`schema.json` describes the record to extract:

```json
[
  {
    "name": "receipt",
    "description": "A purchase receipt shared as text",
    "parameters": {
      "type": "object",
      "properties": {
        "merchant": { "type": "string" },
        "total": { "type": "number" },
        "currency": { "type": "string" },
        "line_items": { "type": "array", "items": { "type": "object" } }
      },
      "required": ["merchant", "total"]
    }
  }
]
```

```sh
./needle --tools schema.json --prompt "GreenMart receipt: oat milk 3.50, total 7.75 paid by visa"
```

```json
{ "type": "call", "function_calls": [ { "name": "receipt", "arguments": { "merchant": "GreenMart", "total": 7.75 } } ] }
```