Hanno-Labs/bosun-v3.1-1.7b-GGUF

Bosun v3.1 1.7B GGUF

Bosun assigns probabilities to decisions you define. Give it a situation and candidate choices—for example, a support ticket and the teams that could handle it—and use the returned distribution to choose an action. The same typed decision interface supports choice (pick an option), score (rate on an ordered scale), and noul (estimate a yes/no probability) questions.

This repository contains GGUF conversions of the Bosun v3.1 1.7B source model. It is a decision model with a custom readout, not a chat model.

Runtime support

• Try the decision API today: the JEV-compatible server serves the Transformers source model through POST /v1/systemone. The quickstart below uses that backend.
• Use these GGUF files: a client must implement Bosun's decision-token readout described in serving.json. The current JEV command below does not load these GGUF files. A verified, packaged GGUF API runner is not included in this repository.

Choose a file

File	Download size	Choose it for
bosun-v3.1-1.7b-Q5_K_M.gguf	1.26 GB	Smallest supplied build
bosun-v3.1-1.7b-Q8_0.gguf	1.83 GB	Higher precision quantized weights
bosun-v3.1-1.7b-F16.gguf	3.45 GB	Unquantized F16 conversion reference

Sizes are decimal download sizes, not runtime memory requirements. All three files passed the conversion checks described below. Download the smallest build and its decision contract with the Hugging Face CLI:

hf download Hanno-Labs/bosun-v3.1-1.7b-GGUF \
  bosun-v3.1-1.7b-Q5_K_M.gguf serving.json --local-dir ./bosun-v3.1-1.7b

Try Bosun's API with JEV (Transformers backend)

This command runs JEV's registry-pinned Transformers source checkpoint, not the downloaded GGUF file. Download the JEV Transformers registry first; it selects eager attention, which Bosun requires with JEV 0.1.1:

hf download Hanno-Labs/bosun-v3.1-1.7b-GGUF \
  jev-transformers-registry.json --local-dir .
DECISION_REGISTRY=./jev-transformers-registry.json \
  uvx --from 'jev-compatible-server[transformers]==0.1.1' \
  jev-compatible-server --model bosun-v3.1-1.7b

Send a decision request from another terminal:

curl -sS http://localhost:8000/v1/systemone \
  -H 'content-type: application/json' \
  -d '{
    "state": "A customer says they were charged twice.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should own this ticket?",
        "criteria": {
          "billing": "Payment, invoice, or refund problems",
          "technical": "Product bugs and technical failures"
        }
      }
    }
  }'

In a CPU smoke test with JEV 0.1.1, answers.route returned:

{"type":"choice","choice":"billing","probabilities":{"billing":0.9053222998183623,"technical":0.09467770018163771},"confidence":0.9053222998183623}

Probabilities can vary with hardware and dependency versions. See the JEV API reference for score, noul, and the full response schema.

Integrating the GGUF weights

Bosun's output is a probability distribution over the candidates in a request. To obtain it from a GGUF engine, render the prompt defined by serving.json, preserve the tokenizer's decision-token IDs, read the full logits at the final prompt position, mask unused candidate slots, normalize the valid slots, and map them back to the caller's candidate order. Decision-token numbers are presented slots, not fixed labels. A generic text-generation endpoint does not perform this readout.

The conversion checks used llama-cpp-python to read these logits. The source model contains the reference Transformers implementation of the decision contract.

Results and conversion checks

The Transformers source checkpoint reports 84.90% accuracy, ECE 0.050, and NLL 0.478 on the frozen 23,900-row DecisionBench evaluation. These are held-out rows from task families represented in training; they are not unseen-task OOD results. See the source model card for the evaluation breakdown and provenance. DecisionBench has not been rerun on these GGUF files.

For this release, the merged Transformers model and each GGUF file were checked on five fixed choice, score, and noul examples. All supplied files passed the declared probability-difference limits and exact tokenization checks. The per-example probabilities, limits, converter revision, and SHA-256 hashes are in gguf_fidelity.json. This small conversion check does not measure broad task accuracy or calibration.

Q4_K_M was omitted for 1.7B because one noul check differed by 0.1652 (limit 0.15).

Provenance and license

The PEFT adapter was merged into the pinned Qwen3 base, then the 256 trained decision-token input and output rows were restored before GGUF conversion. The source checkpoint revision is fd6b47ca062404016902246ca6ff75c08584508a. The llama.cpp converter revision is 4098fdc922460470caa12659249e55f77c06730f.

Copyright 2026 Clause Logic Inc. Licensed under Apache 2.0. See LICENSE and NOTICE.