Instructions to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ttimms/Bible-Assistant-Qwen3.5-4B-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ttimms/Bible-Assistant-Qwen3.5-4B-v2")
model = AutoModelForCausalLM.from_pretrained("Ttimms/Bible-Assistant-Qwen3.5-4B-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ttimms/Bible-Assistant-Qwen3.5-4B-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ttimms/Bible-Assistant-Qwen3.5-4B-v2

SGLang

How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ttimms/Bible-Assistant-Qwen3.5-4B-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ttimms/Bible-Assistant-Qwen3.5-4B-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Bible-Assistant-Qwen3.5-4B-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ttimms/Bible-Assistant-Qwen3.5-4B-v2 with Docker Model Runner:
```
docker model run hf.co/Ttimms/Bible-Assistant-Qwen3.5-4B-v2
```

Superseded (2026-09-05). Bible-Assistant-Qwen3.5-4B-v3.2 is the current model -- 80.3% verse-quote exact / 98.9% citation / 1.9% hallucination, best in its size class on every metric tested. This v2 checkpoint is kept for reproducibility and the project's version history (see the v3.2 card's "Version history" section), not as the recommended download.

Bible AI Assistant — v2-4b (Qwen3.5-4B SFT)

A locally-runnable, retrieval-grounded Bible Q&A model fine-tuned on Qwen3.5-4B. Built for a single 16 GB consumer GPU. On its own protocol-v3 benchmark (282 questions, verse-level verification) it reaches 76.5 % verbatim verse recall on direct verse-lookup questions, 98.9 % citation rate, and ~2 % hallucination — and it is trained to decline rather than counsel on crisis / pastoral questions. (Exact-verse accuracy across all verse-scored categories is 29.3 %; see Evaluation for why the synthesis categories score 0 on that metric.)

Scope. This is the supervised-fine-tuning stage of a multi-stage pipeline. It is strong on the core retrieval task and on safe refusal, and currently weaker at open-ended thematic synthesis ("explain the context of Psalm 23"), where it tends to list verses rather than explain — a known limitation of the template-heavy training data, addressed in the planned v3 (teacher-distilled answers + a GRPO faithfulness stage). Full numbers and a controlled A/B vs. the prior model are in Evaluation.

Project & RAG pipeline: https://github.com/t-timms/bible-ai-assistant
GGUF quants: Ttimms/Bible-Assistant-Qwen3.5-4B-v2-GGUF

Architecture

graph TD
    Base["Qwen/Qwen3.5-4B<br/>base model"]

    subgraph Build ["Build pipeline - RTX 5070 Ti"]
        SFT["bf16 LoRA SFT - 1 epoch<br/>56k-example v2 dataset (8 categories)"]
        Merge["Merge adapter -> bf16"]
        Conv["convert_hf_to_gguf --no-mtp<br/>+ llama-quantize (+ imatrix)"]
    end

    subgraph HF ["Published on Hugging Face"]
        ST["Bible-Assistant-Qwen3.5-4B-v2<br/>safetensors"]
        GG["...-v2-GGUF<br/>Q4_K_M / Q5_K_M / Q6_K / Q8_0 / F16 (+imatrix)"]
    end

    subgraph Serve ["Retrieval-augmented serving"]
        RAG["RAG server (FastAPI)<br/>dense (nomic-embed) + BM25 + RRF + bge-reranker-v2-m3"]
        LLM["Ollama / llama.cpp<br/>Qwen3.5-4B SFT"]
    end

    Base --> SFT --> Merge --> ST
    Merge --> Conv --> GG
    ST --> RAG --> LLM
    GG --> LLM

Summary

Field	Value
Base model	Qwen/Qwen3.5-4B (rev `851bf6e8`) — a vision-language base; this checkpoint uses its text stack only (`qwen3_5_text`, hybrid Gated-DeltaNet + attention, 32 layers, ~4.2 B params)
Fine-tuning	Supervised fine-tuning only (LoRA r=32/α=64, bf16, 1 epoch, 55,570 examples after a length filter — from the 56,022-example v2 dataset — seq 1280)
Preference / RL stage	none yet (planned for v3)
Language	English
License (weights)	Apache-2.0 (inherits from the base)
Task	Retrieval-grounded conversational Bible Q&A
Serving	`transformers`; vLLM; GGUF via current `llama.cpp` (see the GGUF repo). Ollama support pending its next bundled-llama.cpp bump.

What it's for

A conversational model for answering Bible questions as part of a RAG pipeline: a retriever fetches relevant passages and passes them as context; the model answers from that context. It is not designed to be used context-free.

Appropriate: personal Bible study, verse lookup, sermon-prep passage finding, devotional Q&A, educational exploration — with retrieval running.

Not for: medical / legal / financial advice; counselling or pastoral care (the model is trained to redirect these to a pastor or crisis line); authoritative theological decisions; unsupervised or at-scale deployment. Not adversarially red-teamed.

Training data

Produced by the project's v2 dataset engine — 56,022 examples, fully provenance-tracked (per-source SHA + license in the manifest), and decontaminated against every question in the evaluation suite (zero overlap).

Bucket	Count	Source / license
8 scripture-citation categories (verse/passage recall, reverse lookup, near-miss guard, cross-reference chains, topical collections, chapter context, translation-specific)	35,604	6 public-domain translations (KJV/ASV/WEB/DARBY/YLT/BBE), TSK cross-references (CC-BY, openbible.info)
`grounded_exegesis` — verse + commentary in context → grounded interpretation	7,000	Matthew Henry's Commentary on the Whole Bible (CC0 / public domain)
`general_blend` — general instruction / reasoning replay (catastrophic-forgetting guard)	12,996	HuggingFaceTB/smoltalk2 (Apache-2.0), `<think>` traces stripped
`pastoral_triage` — escalation, tradition-aware framing, calibrated abstention	352	hand-authored
inherited general / meta / refusal pools	70	project

No proprietary, personal, or commercially licensed data.

Training

LoRA (r=32, α=64, dropout 0.05) on q/k/v/o/gate/up/down proj, bf16 (fully unquantized), 1 epoch / 3,474 steps, effective batch 16, lr 2e-4 cosine, seq 1280 fixed-pad, completion-only loss masking. ~10.4 h on a single RTX 5070 Ti (16 GB). Eval loss 0.2515 → 0.2138, monotonic over all 70 evals — no overfitting.

Evaluation

Benchmark protocol v3 (282 questions, sha-pinned suite), keyword/verification metrics, greedy decode, seed 42, RAG context enabled.

Category	N	Verse acc (exact)	Fuzzy mean	Hallucination	Citation
verse_lookup	102	76.5 %	0.65	2.9 %	100 %
cross_reference	30	0 %*	0.40	3.3 %	100 %
context	30	0 %*	0.23	0 %	93 %
character	35	0 %*	0.20	2.9 %	97 %
topical	58	0 %*	0.20	1.7 %	100 %
topical_pin	3	0 %*	0.22	0 %	100 %
theological_reliability	8	0 %*	0.15	0 %	100 %
Overall	266	29.3 %	0.40	2.3 %	98.9 %

Row counts sum to the 266 verse-scored questions; the remaining 16 of the 282 are refusal / pastoral-triage questions, scored separately for decline/redirect correctness and excluded from the rate denominators above.

* verse_accuracy scores "quoted the one expected verse verbatim." Character / topical / context / theological questions have no single canonical verse answer, so a good synthesised answer scores 0 on this metric — the fuzzy column and a judge pass score them fairly.

Versus the previously shipped model (same protocol, same day): verse-lookup exact accuracy 58 % → 76.5 % (+18.5 pp), citation rate 88 % → 98.9 % (+11 pp), hallucination flat at ~2 %. But overall fuzzy mean regressed 0.48 → 0.40 — the lightly-tuned prior model's thematic answers are closer to the expected natural answers. Diagnosis: the dataset's rigid fill-in-the-blank answer templates taught this model the format rather than the skill, so open-ended "explain / who is / what is the context of" questions get a verse list instead of an explanation. This is the target of the planned v3 (teacher-distilled answers + GRPO).

Limitations

Thematic-answer regression (above) — dataset issue, targeted for v3.
RAG dependency — reliable verse accuracy needs the retriever + index; without context the model falls back to parametric memory. Always verify cited verses against a Bible.
Ollama not yet — GGUF quants are provided and run in current llama.cpp / recent LM Studio, but Ollama 0.33.x's bundled runtime is too old for the qwen35 arch. Use once Ollama updates, or run llama.cpp directly.
Sequence length — trained at 1280 tokens; longer inputs truncate.
English only.
SFT-only — the preference and RL stages the pipeline is designed around have not run.

Bias

Protestant canon across six public-domain English translations (no Deuterocanonical books); Matthew Henry's commentary reflects an 18th-century Reformed Protestant perspective. The pastoral_triage data deliberately models tradition-aware framing ("faithful Christians differ on …") for disputed questions, but the sources still lean evangelical / Reformed Protestant. Inherits any biases in Qwen3.5-4B. Hand-authored escalation data reflects the developer's judgment. Apply critical judgment, especially on contested theological questions.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "Ttimms/Bible-Assistant-Qwen3.5-4B-v2"
tok = AutoTokenizer.from_pretrained(m, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="cuda", trust_remote_code=True)

# The model expects a retrieval-augmented prompt: verses in a Context block, then the question.
user = (
    "Context:\n- **John 3:16**: For God so loved the world, that he gave his only begotten "
    "Son, that whosoever believeth in him should not perish, but have everlasting life.\n\n"
    "Q: What does John 3:16 say?"
)
msgs = [
    {"role": "system", "content": "You are a Bible AI assistant. Answer questions about Scripture accurately and conversationally."},
    {"role": "user", "content": user},
]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False, enable_thinking=False)
ids = tok(text, return_tensors="pt").input_ids.to("cuda")
out = model.generate(ids, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The full RAG server (retriever + reranker + citation verification) is in the project repo.

License

Weights: Apache-2.0 (from Qwen3.5-4B).
Code: MIT (project repo).
Bible translations: public domain. Matthew Henry's Commentary: CC0.
smoltalk2: Apache-2.0 for its new subsets; inherited subsets keep upstream licenses.

Citation

@misc{bible-ai-assistant-2026,
  title        = {Bible AI Assistant: A RAG-Grounded Bible Q\&A Model Fine-tuned on Qwen3.5-4B},
  author       = {Tremayne Timms},
  year         = {2026},
  howpublished = {GitHub},
  url          = {https://github.com/t-timms/bible-ai-assistant}
}