Laguna-S-2.1-MLX-8bit

MLX 8-bit affine quantization of poolside/Laguna-S-2.1, packaged for Apple Silicon experiments and local OpenAI-compatible serving.

Laguna S 2.1 is Poolside's 118B-total / ~8B-active Mixture-of-Experts model for agentic coding and long-horizon software work. This repo keeps the same tokenizer, chat template, and Laguna custom code files, with the weights converted to MLX 8-bit group quantization.

This is a community conversion. Model architecture, license, intended use, and safety guidance belong to the original Poolside release.

Quick Facts

Item	Value
Base model	poolside/Laguna-S-2.1
Format	MLX / Safetensors
Quantization	8-bit affine
Group size	64
Apparent local size	~116 GB
Parameters	118B total, ~8B active
Context window	1,048,576 tokens in the base config
Recommended hardware	Apple Silicon with enough unified memory for the weights plus KV/cache headroom

Why This Model Is Interesting

• Sparse-MoE design: 256 routed experts plus one shared expert, with top-10 routing.
• 48-layer Laguna S architecture with a 1:3 full-attention to sliding-window layout.
• 512-token sliding window layers help keep long-context decode memory bounded.
• Native reasoning controls through enable_thinking in the chat template.
• Built for coding, tools, and long-horizon agent workflows rather than short chat only.

Base Model Benchmarks

These are Poolside's reported base-model results for Laguna S 2.1, included here for context. They are not fresh benchmark runs of this MLX quantization.

Benchmark	Laguna S 2.1
Terminal-Bench 2.1	70.2%
SWE-bench Multilingual	78.5%
SWE-Bench Pro	59.4%
DeepSWE	40.4%
SWE Atlas (Codebase QnA)	46.2%
Toolathlon Verified	49.7%

See the original model card for the full table, comparison models, methodology notes, and links to trajectories.

Install

Laguna support in MLX is still moving quickly. If your installed mlx-lm cannot import model_type: laguna, use a Laguna-capable branch until support lands in your preferred release.

python -m venv ~/.venvs/mlx-laguna
source ~/.venvs/mlx-laguna/bin/activate

pip install -U mlx huggingface_hub
pip install -U "git+https://github.com/pierre427/mlx-lm.git@pr/laguna-windowed-kv-sanitize"

Generate With MLX

from mlx_lm import load, generate

model, tokenizer = load("Vontra/Laguna-S-2.1-MLX-8bit", tokenizer_config={"trust_remote_code": True})

messages = [
    {"role": "user", "content": "Write a Python retry helper with exponential backoff."}
]

prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)

text = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(text)

OpenAI-Compatible Server

This is the route I would use for local clients first. Keep concurrency conservative on a 256 GB Mac Studio.

source ~/.venvs/mlx-laguna/bin/activate

mlx_lm.server \
  --model Vontra/Laguna-S-2.1-MLX-8bit \
  --host 0.0.0.0 \
  --port 8021 \
  --trust-remote-code \
  --chat-template-args '{"enable_thinking":false}' \
  --max-tokens 4096 \
  --decode-concurrency 1 \
  --prompt-concurrency 1 \
  --prefill-step-size 1024

Test it:

curl http://localhost:8021/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default_model",
    "messages": [
      {"role": "user", "content": "Say hello in one short sentence."}
    ],
    "max_tokens": 64,
    "stream": false
  }'

Pi / OpenAI-Compatible Config

{
  "providers": {
    "MLX-Studio": {
      "baseUrl": "http://127.0.0.1:8021/v1",
      "api": "openai-completions",
      "models": [
        {
          "id": "default_model",
          "name": "Laguna-S-2.1-MLX-8bit",
          "reasoning": false,
          "input": ["text"],
          "contextWindow": 131072,
          "maxTokens": 4096
        }
      ]
    }
  }
}

Quantization Notes

• Converted with MLX 8-bit affine quantization.
• Group size is 64.
• The model config records per-layer 8-bit quantization for MoE gate projections.
• Tokenizer, chat template, and Laguna remote-code files are included from the source model.
• Reasoning can be enabled, but many local clients behave better with enable_thinking=false unless they understand separate reasoning fields.

Known Caveats

• This is a very large local model. Leave memory headroom for prompts, KV cache, and the OS.
• Some stable mlx-lm releases may not yet include Laguna support; a Laguna-capable branch may be required.
• DFlash speculative decoding for Laguna is not wired through stock MLX-LM at the time of this upload. Use Poolside's vLLM/SGLang/TRT recipes for the official DFlash serving path.
• The benchmark table above describes the base model, not a separate quantized eval run.

References

• Poolside Laguna S 2.1 model card
• Poolside release blog
• Laguna S 2.1 NVFP4 deployment notes
• MLX-LM

License

This quantized checkpoint follows the OpenMDW-1.1 license used by the original Poolside release.