sahilchachra/fastcontext-1.0-4b-sft-mxfp8-mlx

FastContext-1.0-4B-SFT — MXFP8 (MLX)

MXFP8 block-float 8-bit quantization (MLX). Converted from microsoft/FastContext-1.0-4B-SFT for Apple Silicon / MLX.

FastContext is a repository-exploration subagent (Qwen3-4B backbone): given a query it explores a codebase with read-only Read/Glob/Grep tools and returns a <final_answer> of file:line citations. Designed to feed a larger coding agent (e.g. Mini-SWE-Agent).

• Disk size: 3967 MB
• Decode speed (Apple M5 Pro, cooled): 58.43 tok/s (1.7× fp16) · prefill 311.73 tok/s · peak 4.326 GB

SWE-bench Lite localization

Evaluated on its native task — file/line localization — since the model emits citations, not prose. Given a GitHub issue, it explores the repo checked out at base_commit and cites the files/lines to change; citations are scored deterministically against the gold patch (no LLM judge). Subset: psf/requests, pallets/flask, pylint-dev/pylint, pytest-dev/pytest; 12 instances; max_turns=6; greedy decoding.

Metric	This variant	fp16 ref
File F1	0.4306	0.5472
File hit-rate	0.6667	0.6667
All-gold-files-found rate	0.6667	0.6667
Line F1	0.1252	0.0493
Avg turns	6.5	6.58

Citation agreement vs fp16: mean file-set Jaccard = 0.413 (1.0 = cites exactly the same files as the fp16 model on the same instances).

Usage

# serve (OpenAI-compatible)
python -m mlx_lm.server --model sahilchachra/fastcontext-1.0-4b-sft-mxfp8-mlx --port 8080
# then point the FastContext agent (github.com/microsoft/fastcontext) at
# BASE_URL=http://127.0.0.1:8080/v1

Caveats

• Localization numbers use a fixed query wrapper around each issue (not the official LLM-delegated sub-query), so treat absolute scores as a proxy; the fp16↔mxfp4↔mxfp8 comparison is apples-to-apples.
• Quantized with mlx-lm 0.31.3.