README.md

metadata

license: apache-2.0
base_model: bespokelabs/Bespoke-Nimble-9B
library_name: peft
language:
  - en
tags:
  - lora
  - classification
  - agents
  - decision
  - nimble

autonoxis-conductor-9b

A LoRA adapter for bespokelabs/Bespoke-Nimble-9B (revision 594dfdcfb6f94e3d0c0db7535180d3c71689169a) that makes "conductor" decisions for autonomous coding agents. It does not generate text. Nimble scores the answer tokens and the adapter returns a probability for each allowed label.

Two tracks, one question per request:

track	labels
decision: what should the conductor do next with this lane?	STOP, ASK, DISPATCH
manager: what should happen to the evidence returned for this lane?	ACCEPT, VERIFY, REJECT, REOPEN, ESCALATE

The question wording and the label criteria the adapter was trained on are in conductor-questions.json.

How to use

You need the Nimble prompt builder and scorer from github.com/bespokelabsai/nimble: nimble.scoring.parallel_schema.prepare_prompts and nimble.training.schema_train.candidate_logits.

The call convention matches training. Use it exactly:

• ask one question per request, with the field id label;
• the state is the JSON {"packet": <packet text>};
• the field's description and choice descriptions come from conductor-questions.json (instructions and criteria for that track).

Load the base model and attach the adapter unmerged. merge_and_unload() in bf16 shifts the probabilities: on one evaluation packet VERIFY went from 0.7278 unmerged to 0.6765 merged.

import json, torch
from transformers import AutoTokenizer, AutoModelForImageTextToText
from peft import PeftModel
from huggingface_hub import hf_hub_download
from nimble.scoring.parallel_schema import prepare_prompts
from nimble.training.schema_train import candidate_logits

BASE, REV = "bespokelabs/Bespoke-Nimble-9B", "594dfdcfb6f94e3d0c0db7535180d3c71689169a"
ADAPTER = "damianborek/autonoxis-conductor-9b"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForImageTextToText.from_pretrained(BASE, revision=REV, dtype=torch.bfloat16).to("cuda")
model = PeftModel.from_pretrained(model, ADAPTER).eval()   # do not merge
questions = json.load(open(hf_hub_download(ADAPTER, "conductor-questions.json")))

def ask(packet: str, track: str) -> dict:
    q = questions[track]
    field = {"type": "enum", "description": q["instructions"],
             "choices": list(q["criteria"]), "choice_descriptions": q["criteria"]}
    p = prepare_prompts(tok, json.dumps({"packet": packet}, ensure_ascii=False), {"label": field}, 2048)
    ids, cands = p.full_ids[0], p.candidate_ids[0]
    batch = {"input_ids": torch.tensor([ids], device="cuda"),
             "attention_mask": torch.ones(1, len(ids), dtype=torch.long, device="cuda"),
             "candidate_ids": torch.tensor([cands], device="cuda"),
             "candidate_mask": torch.ones(1, len(cands), dtype=torch.bool, device="cuda")}
    with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
        probs = candidate_logits(model, batch)[0].double().softmax(-1).tolist()
    return dict(zip(q["criteria"], probs))

print(ask("Lane 3 returned a passing test run for commit abc123, but the lane head is now def456 ...", "manager"))

Confidence gate. Compute conf = (n * max_prob - 1) / (n - 1), where n is the number of labels. Act on the top label only when conf >= 0.8. Below that, escalate to a stronger model or a human.

A local HTTP server with a Jev-compatible request shape (autonoxis-server) exists and loads the adapter unmerged in the same way. It is not part of this repository, and there is no public hosted endpoint.

Training

• Recipe: LoRA r=16, alpha=32, dropout 0.05, on the language-model Linear layers (attention, linear-attention and MLP projections). Candidate cross-entropy over the answer tokens. lr 5e-5, effective batch 8, 3 epochs, linear schedule with 10% warmup, bf16.
• Data: 1076 rows. 316 come from lab history. The other 760 are drafted contrastive rows aimed at the hard boundaries (ASK/STOP, REJECT/ESCALATE, REJECT/REOPEN, VERIFY/REJECT). Label disagreements were adjudicated by Fable 5 (claude-fable-5). The 760 drafted rows are disjoint from every evaluation set (checked at build time).
• This adapter is seed 1 of a 10-seed run. It was chosen by a fixed rule that looks only at v5 and the all-seed ensemble.
• The training data is not released.

Results

The gold labels are Fable 5 (claude-fable-5). "Unsafe" means the model predicted DISPATCH or ACCEPT where the gold label is different. The evaluation sets are not released. eval-summary.json holds the per-set metrics for this adapter (no packet text).

v8 (60 rows) is the fair check. It was never used for training or selection.

	v8
10 seeds, mean ± sd	59.2 ± 0.9 / 60 (min 58, max 60)
this adapter (seed 1)	58 / 60 (decision 24/24, manager 34/36)
unsafe	0 in every seed
this adapter, conf ≥ 0.8	keeps 59, accuracy 0.983

This adapter misses two v8 packets: v8_g04 REJECT→VERIFY (conf 0.66, below the gate) and v8_g27 ACCEPT→VERIFY (conf 0.998).

v8 is in-distribution. It shares 7 scenario families with the training data and came from the same drafting pipeline. It is not an out-of-distribution test.

v5 to v7 are not clean held-out sets. Targeted training batches were written from the contract rules the model misapplied on these sets. The scenarios are new and none of the eval wording was reused, but the gains are partly driven by those errors. Reported for completeness only:

	v5	v6	v7
10 seeds, mean ± sd	39.9 ± 0.3 / 40	47.4 ± 0.5 / 48	47.9 ± 0.3 / 48
this adapter	40 / 40	48 / 48	48 / 48

References on the same packets:

• Untrained Jev (TypeSafe) also scores 60/60 on v8. This adapter does not beat Jev. Its value is that it runs locally and costs nothing per call. In a 3-request check on one local GPU, the two requests after warm-up took 97 and 111 ms each (the first request, which included CUDA warm-up, took 494 ms).
• Stock Bespoke-Nimble-9B without the adapter: v6 41/48 and v7 40/48, with 11 confident-but-wrong answers at conf ≥ 0.8.
• To check the labels, Opus 5.5 (claude-opus-5-5) re-labelled all 196 v5–v8 packets blind. It agreed with the Fable 5 gold on 194 of 196.

Limitations

• Prompts are limited to 2048 tokens. Longer packets are rejected, not truncated.
• English only.
• Narrow domain: conductor and manager decisions for autonomous coding lanes, as defined by one decision contract. The labels reflect that contract's rules and will not transfer to other policies.
• The probabilities are not calibrated beyond the evaluation sets above. The 0.8 gate was only checked on those sets.
• The adapter is trained for the single-question convention above. Asking both questions in one schema changes some predictions.

License

Apache-2.0, the same as the base model. Bespoke-Nimble-9B is itself a LoRA-merged Qwen/Qwen3.5-9B, which is also Apache-2.0.