massif/Qwen3-1.7B-CoreAI-ANE-W8-2K-h16p

Qwen3-1.7B · Apple Core AI · ANE · W8 · 2K · h16p

A complete, directly downloadable Apple Core AI resource directory for Qwen3-1.7B, AOT-compiled for A17 Pro (h16p) with a 2,048-token fixed KV cache and Neural Engine preferred.

This is a community conversion for iOS 27. It is not an official Apple or Qwen release. Neural Engine is the preferred compute target; exclusive ANE execution for every operation is not claimed.

Related work:

• W8 / static-KV / ANE 4K reference
• INT4 / dynamic-KV / GPU 4K alternative
• W8/ANE conversion methodology and companion app
• Apple upstream model request

What this release is for

This artifact is the 2K static-KV counterpart to the public 4K W8/ANE model. It exists for applications whose real prompt plus response budget fits within 2,048 tokens and that want to avoid allocating a 4K fixed KV cache.

Property	This release
Runtime	Apple Core AI + LanguageModelSession
AOT target	h16p
Supported chip reported by inspection	A17 Pro
Minimum OS	iOS 27.0
Preferred compute	Neural Engine
Transformer projections	W8 per-tensor K-means palettization
Tied embedding	INT8 per-tensor quantization
Compute	FP16
KV cache	Fixed-size FP16, 224 MiB at 2K
Maximum context	2,048 tokens
AOT functions	28
Complete resource directory	1,797,570,974 bytes / 1,714.3 MiB
Physical-device validation	iPhone 15 Pro, iOS 27

Important size and memory boundary

Reducing static context from 4K to 2K halves the graph's FP16 KV allocation from 448 MiB to 224 MiB. It does not halve the downloadable model: the W8 weights still dominate storage, so this complete 2K resource directory remains about 1.7 GiB. The minimal device gate observed a 2,742.2 MiB peak process RSS, but it was not a controlled 2K-versus-4K memory experiment. Do not treat the theoretical 224 MiB KV reduction as an equal guaranteed RSS reduction.

Physical-device gate

The exact artifact published here was embedded in a fresh, separately signed application and run on massif iPhone (iPhone 15 Pro, A17 Pro, iOS 27.0) using Xcode 27 Beta.

Check	Result
Suite	COREAI_SUITE_PASS, 2 / 2
Fresh load	37.161 s
Short marker	COREAI-2K-OK
Short input / output	47 / 11 tokens
Short TTFT	0.971 s
Short visible decode	23.798 tok/s
Near-2K input	1,861 tokens
Tail marker	TARGET-2K-0046
Near-2K TTFT	2.063 s
Near-2K visible decode	19.466 tok/s
Peak process RSS	2,742.2 MiB
Resident memory after unload	206.4 MiB
Full minimal suite	42.300 s

See the detailed validation note and machine-readable metrics. Load time and memory depend on OS, driver, and file-cache state; the values above are observed measurements, not universal guarantees.

Quality evidence and limits

The W8 mechanism was selected through a frozen numerical fidelity gate shared with the public 4K W8/ANE work:

Shared W8 recipe evidence	Mean logits cosine	Minimum cosine	Top-1 agreement	Mean NLL delta
W8 tuning set	0.997067	0.967261	98.44%	0.002985
W8 frozen holdout	0.996598	0.959625	98.37%	0.003167

These values justify the W8 compression choice. They are pre-AOT conversion fidelity measurements, not a downstream benchmark of this exact compiled 2K artifact.

The exact 2K artifact has passed load, short generation, and a 1,861-token tail retrieval gate. It has not yet run the frozen 300-example CMRC2018 benchmark used to compare the public 4K W8 and INT4 releases. Their EM/F1 values are not copied onto this model. Before shipping a product, evaluate this exact 2K asset on the product's real prompt, schema, languages, and response budget.

Download

hf download massif/Qwen3-1.7B-CoreAI-ANE-W8-2K-h16p \
  --local-dir Qwen3-1.7B-CoreAI-ANE-W8-2K-h16p

The downloaded repository root is the Core AI resource directory:

Qwen3-1.7B-CoreAI-ANE-W8-2K-h16p/
├── metadata.json
├── tokenizer/
└── qwen3_1_7b_w8_2k_ios.h16p.aimodelc/

Swift usage

The URL passed to CoreAILanguageModel(resourcesAt:) must be the directory that contains metadata.json, tokenizer/, and .aimodelc/.

import CoreAILanguageModels
import FoundationModels

let resources = URL(
    fileURLWithPath: "/path/to/Qwen3-1.7B-CoreAI-ANE-W8-2K-h16p"
)
let model = try await CoreAILanguageModel(
    resourcesAt: resources,
    mode: .eager,
    kvCacheStrategy: .fixedSize
)

let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "Reply with exactly COREAI-2K-OK. /no_think"
)
print(response.content)

For an app target, embed the complete directory as a folder reference or deliver the same directory through a model-asset system. Hugging Face is the artifact source; it does not replace the app's download, integrity, lifecycle, and Core AI loading logic.

Packaging correction verified before publication

The first local packaging attempt retained an authoring entry in the root metadata and pointed to a missing .aimodel. The model binary was valid, but Core AI correctly rejected that malformed directory. This published package instead points assets.main to qwen3_1_7b_w8_2k_ios.h16p.aimodelc and declares the h16p compilation target. That exact corrected structure produced the physical-device results above.

Compatibility and limitations

• Direct AOT compatibility is h16p / A17 Pro on iOS 27.0 or later.
• Other Apple architectures require a separately compiled artifact; renaming the architecture directory does not create compatibility.
• The 2,048-token limit includes the full generated conversation budget, not only the visible user prompt.
• This repository contains no SafeTensors, GPTQ, GGUF, or MLX weights.
• Preferred Neural Engine compute does not prove exclusive ANE execution.
• Background execution, power draw, sustained thermals, Jetsam behavior, and product-specific cancellation were not evaluated by this minimal gate.
• The model inherits the capabilities, limitations, and acceptable-use considerations of Qwen3-1.7B.

Provenance and integrity

• Base model: Qwen/Qwen3-1.7B
• Base revision: 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
• Compiler: coreai-build-3600.75.3
• Source .aimodel main SHA-256: 18f1fb8e44866b3d1a8d35daf3b2b3084cf9ab8115a1ef9f980cba42d7a668a6
• Compiled main-h16p.mlirb SHA-256: 192850a8ad29f7bb64c30f894bd216e72891917751f773a4500dd5a6405933bb
• Conversion recipe SHA-256: dab03ae1dd6c6290a7964e05ebcda7fe027c2bb240174fcf034e64a376be9d72

evidence/artifact-summary.json records the exact artifact facts. SHA256SUMS covers every published file except the manifest itself.

中文摘要

这是 Qwen3-1.7B 的完整 Apple Core AI W8/ANE 2K 资源目录，针对 iPhone 15 Pro 的 h16p 架构编译，采用 FP16 静态 KV。与 4K 静态版本相比，它把 KV 从 448 MiB 减少到 224 MiB，但模型权重仍约 1.7 GiB，所以下载体积几乎不会 减半。

该精确资产已在 iPhone 15 Pro、iOS 27.0 上完成最小真机验证：短生成和 1,861-token 尾部读取均正确，短生成首 token 为 0.971 秒，可见解码约 23.8 token/s。现阶段没有把 4K 的 CMRC 分数冒充为 2K 分数；产品使用前仍应 针对真实任务单独评测质量、内存和生命周期。

License

The converted model is distributed under the base model's Apache License 2.0. See LICENSE and NOTICE.md.