README.md

metadata

license: mit
base_model: microsoft/FastContext-1.0-4B-SFT
pipeline_tag: text-generation
tags:
  - core-ai
  - coreml
  - on-device
  - ios
  - iphone
  - qwen3
language:
  - en

Deprecated (2026-07-21)

This port is retired. Two independent reasons:

1. Microsoft removed the upstream model (microsoft/FastContext-1.0-4B-SFT) from Hugging Face and GitHub on 2026-06-30 without explanation, so the original weights are no longer available.
2. The shipped .aimodelc here was compiled from a coreai-torch 0.4.0 conversion and fails to load on iOS/macOS 27 beta 2+. Rebuilding it requires the original weights (see 1).

The repo is kept for reference. Every other model in the zoo has been recovered.

FastContext-1.0-4B-SFT — Core AI (on-device, iPhone)

microsoft/FastContext-1.0-4B-SFT converted to Apple Core AI for on-device inference, served by the CoreAIChat app.

FastContext is a long-context repository-exploration agent (Qwen3-4B-Instruct backbone, supervised-fine-tuned on exploration traces: broad first-turn search, multi-turn evidence gathering, and precise file:line citation generation).

Contents

• gpu/ — AOT-compiled GPU (MPSGraph) bundle, 4-bit linear-INT4, h18p (iPhone 17 / iPhone 18 class). ~2.1 GB. Drop-in for CoreAIChat's Documents/models/fastcontext_4b_gpu.

Device class: this is an AOT h18p bundle (iPhone 17 / 18 class). On-device specialization of a 4B graph is not viable, so the bundle is precompiled for the h18p GPU — the same approach Core AI uses for the Gemma-4B bundle.

Verified on iPhone 17 Pro (iOS 27)

• Parity vs HF fp16 (teacher-forced): 23/24 argmax, perplexity 1.41 — the single flip is a near-tie quant-noise position.
• On-device GPU: ~334 tok/s prefill, ~16.5 tok/s decode (512-prompt / 1024-gen), load ~8 s cold, ~0.3 GB resident (weights memory-mapped).

Use

Install CoreAIChat, open the model picker, and select FastContext 4B.

License

MIT, inherited from the base model microsoft/FastContext-1.0-4B-SFT. See LICENSE.