Instructions to use primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8")
model = AutoModelForMultimodalLM.from_pretrained("primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8

SGLang

How to use primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 with Docker Model Runner:
```
docker model run hf.co/primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8
```

The steady tool-calling build: BF16-level agentic accuracy that reproduces run to run.

A second quantization of ornith-ai/Ornith-1.5-35B-A3B, identical in size and format to
our general-purpose build but calibrated for tool use.
Pick this one for agents. Pick the other one for question answering.

Which of the two should you use

	call	abstain	knowledge	pooled	tool-calling sd
this repo	77.5 (n=6)	58.3 (n=6)	91.5 (n=3)	89.0	1.3
our general build	74.4 (n=4)	60.0 (n=4)	91.7	88.7	3.1
BF16 reference	78.6 (n=4)	61.3 (n=4)	92.0	89.5	1.4

⚖️ Level with BF16 on tool calling (74.0 against 75.1), and not provably better than our general build either (+2.2 points against sds of 1.3 and 3.1). We are not claiming that gap.
📉 What it demonstrably is, is steadier. Tool-calling sd 1.3 vs 3.1. Six runs of the general build span 67.5–75.5; eight of this one span 72.5–76.0.
📚 Knowledge costs 0.2 points (91.5 against 91.7), and on the pooled score this build is 0.3 ahead (89.0 against 88.7).
🎯 The gain is in calling, not declining. 77.5 on the 160 items that need a call, against the general build's 73.3. On the 40 abstention items the two are 58.3 and 56.7, inside a spread of about 5 points, so there is no claim to make there.

Serve it

vllm serve primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8 --max-model-len 32768

Verified to load on Blackwell (native NVFP4/FP8) and on Ampere (A100, weight-only Marlin path: the BF16 lm_head is what keeps that path loadable).

Measured

1,370 items across fourteen public benchmarks. A 1,170-item knowledge suite over nine of them and a 200-item tool-calling suite over five more (BFCL v4, xLAM/APIGen, ToolACE, Glaive v2, nvidia When2Call). One fixed protocol for every row: temperature 0.6 / top_p 0.95 / top_k 20, thinking forced on, a 16,384-token budget, concurrency 32, on one RTX PRO 6000 Blackwell.

build	size	call	abstain	knowledge	pooled
BF16 reference	67.0 G	78.6 (n=4)	61.3 (n=4)	92.0	89.5
ornith-ai FP8	36.7 G	79.1 (n=4)	58.1 (n=4)	91.2	89.0
this repo	22.6 G	77.5 (n=6)	58.3 (n=6)	91.5 (n=3)	89.0
ornith-ai NVFP4	21.8 G	76.1 (n=4)	61.3 (n=4)	90.9	88.3
our general build	22.6 G	74.4 (n=4)	60.0 (n=4)	91.7	88.7

None of the four quants above is distinguishable from another on tool calling. The column spans 2.3 points against per-build sds of 0.6 to 1.3. Every figure is a mean with its n, which matters more on this model than on any other we have measured: a single run of the general build can land anywhere between 67.5 and 75.5, so differences under about 2 points are not differences.

What's quantized to what

tensors	format
all 40 layers' routed experts (`gate/up/down_proj`)	NVFP4 (group 16)
`self_attn` q/k/v/o, `linear_attn` in/out projections, shared-expert projections	FP8 E4M3
`lm_head`, embeddings, vision tower, MTP block, router gates, norms	BF16

Rounding is chosen by GPTQ against a tool-call and agent-trajectory calibration set. Rows sharing a prompt with any eval item were removed first, so the numbers above are measurements, not fits.

_{primitive ·
more models ·
inference economics for production LLM systems}

call is accuracy on the 160 suite items that require a tool call; abstain is the 40 whose correct action is to call nothing. The two are reported separately because pooling them let abstention discipline masquerade as call accuracy. Split means cover the runs whose per-half records survive; the pooled tool-calling means and their n are unchanged.

Downloads last month: 1,591

Safetensors

Model size

20B params

Tensor type

BF16

F8_E4M3

Model tree for primitive-ai/Ornith-1.5-35B-A3B-agentic-NVFP4-FP8

Base model

ornith-ai/Ornith-1.5-35B-A3B

Quantized

(149)

this model