FastContext-1.0-4B-SFT mxfp4 (MLX) + SWE-bench Lite localization card

7c631eb verified 3 months ago

2.09 kB

license: mit
library_name: mlx
tags:
  - mlx
  - code
  - repository-exploration
  - quantization
base_model: microsoft/FastContext-1.0-4B-SFT

FastContext-1.0-4B-SFT — MXFP4 (MLX)

MXFP4 block-float 4-bit quantization (MLX). Converted from microsoft/FastContext-1.0-4B-SFT for Apple Silicon / MLX.

FastContext is a repository-exploration subagent (Qwen3-4B backbone): given a query it explores a codebase with read-only Read/Glob/Grep tools and returns a <final_answer> of file:line citations. Designed to feed a larger coding agent (e.g. Mini-SWE-Agent).

Disk size: 2049 MB
Decode speed (Apple M5 Pro, cooled): 104.2 tok/s (3.1× fp16) · prefill 396.33 tok/s · peak 2.579 GB

SWE-bench Lite localization

Evaluated on its native task — file/line localization — since the model emits citations, not prose. Given a GitHub issue, it explores the repo checked out at base_commit and cites the files/lines to change; citations are scored deterministically against the gold patch (no LLM judge). Subset: psf/requests, pallets/flask, pylint-dev/pylint, pytest-dev/pytest; 12 instances; max_turns=6; greedy decoding.

Metric	This variant	fp16 ref
File F1	0.4694	0.5472
File hit-rate	0.6667	0.6667
All-gold-files-found rate	0.6667	0.6667
Line F1	0.0154	0.0493
Avg turns	6.58	6.58

Citation agreement vs fp16: mean file-set Jaccard = 0.45 (1.0 = cites exactly the same files as the fp16 model on the same instances).

Usage

# serve (OpenAI-compatible)
python -m mlx_lm.server --model sahilchachra/fastcontext-1.0-4b-sft-mxfp4-mlx --port 8080
# then point the FastContext agent (github.com/microsoft/fastcontext) at
# BASE_URL=http://127.0.0.1:8080/v1

Caveats

Localization numbers use a fixed query wrapper around each issue (not the official LLM-delegated sub-query), so treat absolute scores as a proxy; the fp16↔mxfp4↔mxfp8 comparison is apples-to-apples.
Quantized with mlx-lm 0.31.3.