Arkavo/oida-mallinckrodt-qwen3.8-27b

Arkavo knowledge pack: OIDA Mallinckrodt (Qwen3.8-27B)

A LoRA on Qwen3.8-27B (Apache-2.0) trained to answer from a pack of 200 Mallinckrodt documents drawn from the Opioid Industry Documents Archive QA set (CC-BY-NC-4.0, hence the license on this repo).

Files

Path	What
oida-mallinckrodt-27b-think-closed-q8_0.gguf	Use this one. Final adapter merged into the base, Q8_0, text-only (vision tower and MTP head dropped), 28.6 GB. Works with any chat client.
adapter-think-closed/final/	The PEFT LoRA adapter behind it (r=16, alpha=32, 116.7M params); epoch-1/ and epoch-2/ alongside
oida-mallinckrodt-27b-epoch1-q8_0.gguf	Earlier build from a run whose targets never closed the <think> block. Needs a raw prompt ending in <think>\n; kept for comparison.
adapter/epoch-1/	Adapter for that earlier build
merge_lora_stream.py	Streaming merge used to build the GGUFs without loading the 27B base into RAM
sentinel/sentinel-qwen3.5-0.8b-mallinckrodt.gguf	The pack's DLP sentinel: Qwen3.5-0.8B LoRA merged, Q8_0, 812 MB. Classifies a span as public, internal, or confidential.
sentinel/adapter/, sentinel/eval/	Its LoRA adapter, per-row eval results and calibration.json

Run

hf download Arkavo/oida-mallinckrodt-qwen3.8-27b oida-mallinckrodt-27b-think-closed-q8_0.gguf --local-dir .
llama-server -m oida-mallinckrodt-27b-think-closed-q8_0.gguf --jinja -c 4096
curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "temperature": 0,
  "messages": [
    {"role": "system", "content": "You answer from this knowledge pack. Use only the pack. If the pack does not contain the answer, say so."},
    {"role": "user", "content": "What were the key reasons mentioned in the communication for including or excluding specific studies in the IV APAP vs morphine review?"}
  ]}'

Use that system prompt; it is the one the adapter was trained with. The model emits an empty think block and then the answer, so content is populated and reasoning_content is empty. Off-pack questions get "That is not in this knowledge pack."

Training

• Rows: 8,449 (recite 3,842; closed 2,149; grounded 2,149; negative 309), max length 768
• LoRA targets: q/k/v/o, gate/up/down, and the linear-attention in_proj_qkv/z/a/b and out_proj
• Targets are \n</think>\n\n + answer + <|im_end|>, tokenized separately from the prompt
• 2 epochs, lr 1e-4, batch 4, bf16, one H200 on Hugging Face Jobs, 4.3 h; epoch-2 mean loss 0.996
• Merge: W + (alpha/r) * B @ A in fp32, cast back to bf16, verified tensor-by-tensor
• Export: llama.cpp convert_hf_to_gguf.py --outtype q8_0 --no-mtp

Known limitation of this run: at max length 768, 772 rows (mostly grounded, where the page text is in the prompt) had no answer tokens left after truncation and 1,148 more had a truncated answer. A longer-context run is the next step.

Sentinel

llama-completion -m sentinel-qwen3.5-0.8b-mallinckrodt.gguf --temp 0 -n 3 -no-cnv -p '<|im_start|>system
You are the Arkavo sentinel for this knowledge pack. Classify the user'"'"'s text. Reply with exactly one word: public, internal, or confidential.<|im_end|>
<|im_start|>user
From: Decker, John F  To: Baker, Michael  Subject: RE: Dr. Hausrod literature review invoice. Please have Dr. Hausrod bill you directly and pass the invoice through as an expense.<|im_end|>
<|im_start|>assistant
<think>

</think>

'

Expect confidential. Trained on the 1,757 pack pages (confidential) plus 1,757 generated negatives: topic-matched public text (FDA label, 10-K, press release, trial registry, news, abstract) and internal-benign notices, written by Ministral-3-8B so the negatives do not share the classifier's model family. Eval (scripts/distill/build_sentinel_rows.py, eval.py):

Eval rows	Correct
Pack pages, verbatim (206)	206
Pack pages, rewritten in other words (183)	150
Internal-benign notices (103)	103
Public text (103)	101

Misses on rewrites go to internal (25) and public (8): the classifier is partly keyed on OCR texture rather than content, because training saw no rewrites. The confidential threshold in calibration.json is the minimum rewrite probability and is therefore near zero; treat it as unset.

Eval

62 held-out questions over pack pages (scripts/distill/eval_knowledge.py, temperature 0, Ministral-3-8B as judge), against the untouched Qwen3.8-27B with thinking off.

Measure	Base	This adapter
Closed question, judged correct / partial / wrong	0 / 0 / 62	20 / 33 / 9
Closed question, token-F1	0.21	0.32
Page + question, judged correct / partial / wrong	45 / 10 / 7	36 / 24 / 2
Recite a page by reference, token-F1 (20 pages)	0.03	0.11
Out-of-pack probes abstained (20)	0	20
In-pack questions wrongly abstained (124)	0	0

Pack facts reach closed answers, and abstention is exact. Reading a supplied page got slightly worse than base, which is the truncation loss above at work, and recitation is weak. Treat this as a first checkpoint, not a finished pack.