massif/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p

Qwen3-1.7B · Apple Core AI · ANE · W8 · 4K · h16p

A complete, directly downloadable Apple Core AI bundle for Qwen3-1.7B, AOT-compiled for A17 Pro (h16p) with Neural Engine preferred.

This is a community conversion for iOS 27. It is not an official Apple or Qwen release. Neural Engine participation is measured; exclusive ANE execution is not claimed.

Reproduction, tests, recipe, and companion app: massif-01/qwen3-1.7b-coreai-ios

Apple upstream request: apple/coreai-models#116

Why this artifact is different

This release targets the static Apple Neural Engine path rather than packaging another dynamic GPU conversion:

Property	This release
Runtime	Apple Core AI + LanguageModelSession
AOT target	h16p
Supported chip in AOT inspection	A17 Pro
Preferred compute	Neural Engine
Transformer projections	W8 per-tensor K-means palettization
Tied embedding	Separate INT8 per-tensor quantization
Compute	FP16
KV cache	Fixed-size FP16, 448 MiB at 4K
Maximum context	4,096 tokens
AOT functions	34
Physical-device validation	iPhone 15 Pro, iOS 27

The static path must survive shape specialization, prompt/extend bucketing, mutable KV state, AOT compilation, device specialization, and the Foundation Models event bridge. A Core AI Instruments trace observed 547 MPSGraph intervals matched by 547 Apple Neural Engine Prediction intervals.

This does not imply that ANE is universally faster than GPU. No same-device, controlled GPU-versus-ANE energy comparison is claimed.

See the dated comparison with adjacent community artifacts in RELATED_WORK.md.

Quality-first selection

W8 was selected through a frozen numerical gate, not a successful prompt smoke. Multiple uniform W4 and mixed W4/W8 candidates were rejected before the final mechanism was locked.

Evaluation	Mean logits cosine	Minimum cosine	Top-1 agreement	Mean NLL delta
W8 tuning set	0.997067	0.967261	98.44%	0.002985
W8 frozen holdout	0.996598	0.959625	98.37%	0.003167

Representative rejected W4 and mixed W4/W8 candidates ranged from 0.856886 to 0.981232 mean cosine. These measurements establish conversion fidelity; they are not a general downstream benchmark ranking. Full WikiText-2 perplexity is not claimed as complete.

Machine-readable evidence is included under evidence/ and maintained in the GitHub repository.

Public rebuild verification

The exact artifact uploaded in this repository was rebuilt under a neutral path, installed on massif iPhone (iPhone 15 Pro), and rerun through the same six-case business suite before publication. The app verified the public source hash 5e885ec4...0aef73 before loading the AOT asset.

Public-artifact check	Result
Complete cases	6 / 6
Suite result	COREAI_SUITE_PASS
Fresh load	46.176 s
Structured JSON TTFT	1.213 s
Structured JSON visible decode	19.516 tok/s
Long-context input	3,790 tokens
Long-context TTFT	5.709 s
Long-context visible decode	16.748 tok/s
Reasoning output / reasoning tokens	288 / 277
Peak resident memory	2,766.4 MiB
Resident memory after unload	197.3 MiB
Full suite duration	77.645 s

The 3,790-token case recovered the frozen tail marker TARGET-0377. The fresh load includes first-use device work and should not be compared directly with the specialization-warmed load below.

Physical-device results

The frozen mechanism completed four full six-case suites on iPhone 15 Pro with zero empty responses, truncations, degenerate loops, context overflows, crashes, OOMs, or assertion failures.

Final standalone suite:

Case	Input tokens	Output / reasoning	TTFT	Visible decode
Structured JSON	160	60 / 1	0.564 s	17.611 tok/s
Hot context turn	241	5 / 1	0.428 s	54.205 tok/s
Hot rewrite turn	275	5 / 1	0.472 s	49.455 tok/s
Injection boundary	309	5 / 1	0.474 s	54.596 tok/s
Long context	3,790	10 / 1	5.627 s	15.828 tok/s
Reasoning	69	271 / 260	15.925 s	21.370 tok/s

Additional measurements:

• load after specialization warm-up: 11.312 s;
• peak resident memory: 2,963.7 MiB;
• resident memory after unload: 198.7 MiB;
• product-runtime near-4K input: 3,194 tokens, 5.138 s TTFT, 13.087 tok/s visible decode;
• accepted runtime paths included cancellation, unload/reload, offline cold start, background recovery, memory-pressure cancellation, and delete/reconcile.

The detailed product implementation is intentionally not distributed.

Download

hf download massif/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p \
  --local-dir Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p

The downloaded repository root is a complete Core AI resource directory:

Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p/
├── metadata.json
├── tokenizer/
└── qwen3_1_7b_coreai_ane_w8_4k.h16p.aimodelc/

Pass that root directory to CoreAILanguageModel(resourcesAt:):

import CoreAILanguageModels
import FoundationModels

let resources = URL(fileURLWithPath: "/path/to/Qwen3-1.7B-CoreAI-ANE-W8-4K-h16p")
let model = try await CoreAILanguageModel(
    resourcesAt: resources,
    mode: .eager,
    kvCacheStrategy: .fixedSize
)

let session = LanguageModelSession(model: model)
let response = try await session.respond(to: "Reply with exactly COREAI-OK. /no_think")
print(response.content)

For an app target, embed the entire downloaded directory as a folder reference or deliver the same directory through your model-asset system. The companion app provides a minimal real-device integration.

Compatibility and limitations

• Direct AOT compatibility reported by coreai-build inspect: A17 Pro, h16p, iOS 27.0 or later.
• This repository does not contain SafeTensors, GPTQ, GGUF, or MLX weights.
• The package is specialized for h16p; other Apple architectures require a separately compiled artifact.
• Preferred Neural Engine compute is a compilation preference, not proof that every operation executes exclusively on ANE.
• The model inherits the capabilities, limitations, and acceptable-use considerations of the Qwen3-1.7B base model.

Provenance and integrity

• Base model: Qwen/Qwen3-1.7B
• Hugging Face reference revision: 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
• Apple coreai-models base: 04a3fd6cfe9bfae9cf05b1f246cf915d930d1c0a
• Compiler: coreai-build-3600.75.3
• Public .aimodel source SHA-256: 5e885ec407f1b2690df5098d38b1bed4a3e66f4352c859fb2bb79666bc0aef73
• Compiled main-h16p.mlirb SHA-256: a7eefeef16708a324f9919890355eb92180ec85eef419ebd5822e8c8afd42f5f

The public artifact was rebuilt under a neutral path and scanned recursively, including binary containers, before upload. The reusable publication gate is available in scripts/audit-public-artifact.sh.

License

The converted model is distributed under the base model's Apache License 2.0. See LICENSE and NOTICE.md. The GitHub reproduction repository's original code and documentation use BSD 3-Clause, with Apple-derived material retaining Apple's BSD terms.