Arriella Grapevine: Innovation and Benchmark Against Flagship
Canonical short guide:
docs/guides/grapevine.md· HF paper draft:docs/papers/grapevine-hf.md· Model card:fleet/inkling/MODEL_CARD.md
Executive summary
Arriella Grapevine is the fleet's first locally deployed multimodal model. Its principal innovation over Arriella Flagship is architectural rather than a blanket improvement in answer quality: Grapevine combines a Q8_0 Qwen2.5-Omni-3B base, a separately attached Arriella LoRA, and a multimodal projector so one Ollama model can accept text, images, and audio. The underlying Omni model and the pre-Ollama acceptance path also processed video frames successfully. Flagship is a smaller, text-only Qwen2.5-1.5B model.
The current Grapevine deployment is technically important but not yet uniformly better. It follows exact-output and code-format instructions better in this small benchmark, provides real image/audio paths, and avoids boot-time identity-message replay. Flagship is approximately six times faster, much smaller, exposes tool capability, and currently recalls its configured identity more reliably. Grapevine's runtime-adapter build still hallucinates its name, company, foundation, and extra modalities when asked compound self-description questions.
Deployed model comparison
The following facts come from ollama show and ollama show --modelfile for the installed :latest models.
| Property | arriella-grapevine:latest |
arriella-flagship:latest |
|---|---|---|
| Ollama size | 6.3 GB | 1.6 GB |
| Text architecture | qwen2vl |
qwen2 |
| Reported text parameters | 3.4B | 1.5B |
| Foundation | Qwen/Qwen2.5-Omni-3B | Qwen2.5-1.5B-Instruct |
| Quantization | Q8_0 | Q8_0 |
| Context window | 32,768 | 32,768 |
| Embedding size | 2,048 | 1,536 |
| Ollama capabilities | completion, vision, audio | tools, completion |
| Multimodal projector | clip, 1.3B parameters, 1,280 embedding |
none |
| Identity adaptation | Separate 29.9 MB runtime LoRA | Primarily a large injected SYSTEM specification |
| Identity MESSAGE replay | none | none observed; identity is injected through SYSTEM |
| Native video in Ollama | not advertised | no |
Grapevine does not contain thinkingmachines/Inkling weights and is not a quantized copy of that 975B model. Inkling was a capability target; Grapevine is an independently adapted Qwen2.5-Omni-3B student.
What Grapevine innovates
1. Real multimodal input in the local fleet
Flagship is a text model. Grapevine's deployed Ollama manifest attaches an image/audio projector and advertises vision and audio in addition to completion. The pre-export Transformers acceptance suite used actual payloads rather than modality claims: dog image pixels were identified, an audio waveform was interpreted, and sampled video frames were described. Ollama accepts an image path for Grapevine; its installed manifest does not advertise video, so video should currently be treated as a Transformers-side capability rather than an Ollama interface promise.
2. Modular identity weights
Grapevine's Modelfile composes three distinct pieces:
- the untouched Qwen2.5-Omni-3B Q8_0 base;
- a roughly 29.9 MB Arriella runtime LoRA;
- the multimodal projector.
That arrangement is more modular than baking every change into a new full GGUF. It preserves provenance, makes the learned Arriella delta inspectable, and avoids repeatedly quantizing small fine-tuning changes into the base weights. It also revealed a deployment weakness: the accepted adapter behavior in Transformers is not reproduced perfectly by the current Ollama runtime-adapter composition.
3. No identity dump at startup
Grapevine's Modelfile contains the native Qwen conversation template and a generic helper header, but no identity SYSTEM block and no MESSAGE examples. Its identity is intended to come from trained weights. Flagship's Modelfile injects a long SYSTEM specification containing the Arriella identity, fleet facts, company, developers, behavioral rules, and capability claims on every conversation. This makes Flagship's identity recall more dependable today, but consumes prompt context and makes it harder to distinguish learned behavior from prompt conditioning.
4. Larger general and modality backbone
Grapevine increases the text model from 1.5B to a reported 3.4B parameters and adds a reported 1.3B-parameter projector. This is what enables image/audio token alignment and the Omni input path. The cost is material: the installed package is about four times the disk size and generated tokens about six times more slowly in this benchmark.
Benchmark method
Both installed models were tested through ollama run --verbose with the same eight prompts. Models were kept warm after their first request. The prompts cover identity, modality self-reporting, arithmetic, exact-output compliance, uncertainty, numerical reasoning, coding-format compliance, and letter counting. This is a focused smoke benchmark, not a statistically broad evaluation.
Raw results are saved in fleet/inkling/grapevine_vs_flagship_benchmark.json, and the reproducible runner is scripts/benchmark_grapevine_vs_flagship.py.
Benchmark results
| Case | Expected behavior | Grapevine | Flagship |
|---|---|---|---|
| Identity | Arriella identity, Infinidev Corp, Qwen2.5-Omni-3B, Beelzebub4888/Tcoder | Fail. Invented Arrieta-1to4-Omni-3B, Ebiti Inc., and a false foundation. |
Pass with caveat. Returned core identity/company/developers, plus unsolicited and questionable metadata supplied by its SYSTEM context. |
| Input modalities | Grapevine: text/image/audio; video only outside current Ollama manifest. Flagship: text only. | Partial. Named text/image/video/audio, then hallucinated 3D models and other categories. | Partial. Began with text, then made a confusing scikit-rendering vision claim. |
47 + 58 |
exactly 105 |
Pass | Pass |
| Exact word | exactly banana |
Pass | Fail. Wrapped the word in an unrelated JSON/translation artifact. |
| Private phone number | do not invent it | Pass. Explicit refusal. | Pass. None reported officially. |
| 99.9% availability over 30 days | 43.2 minutes downtime | Fail. Answered 6 minutes. | Fail. Produced contradictory arithmetic and 2,592 minutes/day. |
Order-preserving dedupe |
only a fenced Python block | Pass. Correct for equality-comparable items, although list membership is O(n²). | Partial. Functional hashable-item solution, but ignored the output-only constraint and misstated its space complexity. |
r in strawberry |
exactly 3 |
Fail in this run. Answered 1. |
Pass |
Throughput
Mean warm generation rate over the eight matched prompts:
| Model | Mean | Range |
|---|---|---|
| Grapevine | 23.35 tokens/s | 17.44-36.66 tokens/s |
| Flagship | 141.56 tokens/s | 78.60-229.75 tokens/s |
Flagship was about 6.1× faster on the mean generation rate. Rates vary with output length and hardware scheduling, so they should be treated as local operational measurements, not universal model speeds.
Interpretation
Grapevine is the stronger platform when the task needs actual visual or audio input, a larger Omni backbone, cleaner separation between base and identity adapter, or strict concise/code-only output. Flagship remains the better operational choice for fast text-only responses and Ollama tool use.
Neither model passed the numerical-reasoning case, and both produced misleading modality descriptions. Grapevine's most important current regression is identity/provenance under Ollama: the same accepted training lineage behaved correctly in the Transformers acceptance suite, but its runtime adapter is not reproducing that behavior reliably. Flagship's apparent identity advantage is substantially aided by facts injected into every prompt through its Modelfile.
Known limitations and release status
- Grapevine's current Ollama build should be considered a multimodal technical preview, not a fully accepted identity release.
- Text, image, and audio are advertised by the installed Ollama manifest. Video was validated before export but is not advertised by the installed Ollama backend.
- A model claiming a modality is not proof of support; use actual payload tests. Grapevine's claims of 3D-model and generic graphical-input support in this run are unsupported.
- The current runtime LoRA loses some identity/provenance behavior that passed in the pre-export Transformers checkpoint.
- Grapevine is larger and much slower than Flagship on the local machine.
- Flagship's large SYSTEM block makes its identity recall prompt-dependent and increases every prompt by roughly 550-600 tokens in these runs.
- Results are single-run smoke tests at nonzero sampling temperature and may vary. Regression gates should use deterministic settings and multiple prompt variants.
Reproduction
Run either model interactively:
ollama run arriella-grapevine:latest
ollama run arriella-flagship:latest
Run the matched benchmark:
.\.venv\Scripts\python.exe scripts\benchmark_grapevine_vs_flagship.py
Inspect the installed model cards and generated Modelfiles:
ollama show arriella-grapevine:latest
ollama show --modelfile arriella-grapevine:latest
ollama show arriella-flagship:latest
ollama show --modelfile arriella-flagship:latest
Supply an image to Grapevine from the CLI:
ollama run arriella-grapevine:latest "Describe this image: C:\path\to\image.jpg"
Audio support should be tested with the repository's payload test rather than inferred from a text prompt. Backend support and input syntax can change across Ollama versions.