---
license: apache-2.0
base_model:
  - mistralai/Mistral-Small-3.2-24B-Instruct-2506
  - mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit
datasets:
  - open-thoughts/OpenThoughts-114k
language:
  - en
  - fr
  - de
  - es
  - it
  - pt
  - ja
  - ko
library_name: mlx
pipeline_tag: text-generation
tags:
  - mistral
  - mlx
  - apple-silicon
  - lora
  - reasoning
  - chain-of-thought
  - distillation
  - 4bit
---

# Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4bit)

A LoRA fine-tune of **Mistral Small 3.2 24B Instruct** on the **OpenThoughts-114k** reasoning dataset, intended to strengthen structured chain-of-thought reasoning patterns. Released in the **MLX 4-bit format** for Apple Silicon.

GGUF quantizations will be released as a separate companion repository — see the Roadmap below.

## Model Details

| | |
|---|---|
| **Base model** | [`mistralai/Mistral-Small-3.2-24B-Instruct-2506`](https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506), via the [`mlx-community/...-4bit`](https://huggingface.co/mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit) MLX quantization |
| **Parameters** | 23.57 B total · 4.62 M trainable LoRA (0.020%) |
| **Format** | MLX, 4-bit quantized |
| **Architecture** | `Mistral3ForConditionalGeneration` (vision-capable base; this fine-tune targets the text backbone only) |
| **Languages** | multilingual (inherited from base: en, fr, de, es, it, pt, ja, ko, …) |
| **License** | Apache 2.0 |

## Training

LoRA fine-tune with [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms) on an Apple **M4 Pro** (64 GB unified memory).

| Hyperparameter | Value |
|---|---|
| Method | LoRA |
| Trainable layers | Last 4 |
| Iterations | 1500 |
| Batch size | 1 |
| Gradient accumulation steps | 2 (effective batch size 2) |
| Learning rate | 1e-5 |
| Max sequence length | 2048 tokens |
| Gradient checkpointing | Yes |
| Mask prompt | Yes (only assistant turns contribute to the loss) |
| Peak memory | ~17.7 GB |
| Hardware | Apple M4 Pro, 64 GB unified memory |

### Reproducible training command

```bash
mlx_lm.lora \
  --model mlx-community/Mistral-Small-3.2-24B-Instruct-2506-4bit \
  --train --data ./data --iters 1500 \
  --batch-size 1 --grad-accumulation-steps 2 \
  --num-layers 4 --grad-checkpoint --mask-prompt \
  --learning-rate 1e-5 --max-seq-length 2048 \
  --save-every 250 --steps-per-eval 250 \
  --adapter-path ./adapters
```

After training, the resulting LoRA adapter was merged into the base via `mlx_lm.fuse`.

## Dataset

**[`open-thoughts/OpenThoughts-114k`](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)** — 113,957 reasoning trace examples, generated by **DeepSeek-R1**, released under **Apache 2.0**.

The ShareGPT-style schema (`system` + `conversations[from/value]`) was converted to mlx-lm's chat-template-compatible `{"messages": […]}` JSONL format:

| Split | Examples |
|---|---|
| Train | 113,457 |
| Validation | 500 |

### Caveat — Sequence Truncation

OpenThoughts contains very long reasoning chains (single examples up to ~23k tokens). To fit the 64 GB Mac memory budget, `max_seq_length` was set to **2048**. As a result, the model is trained on the **first 2048 tokens** of every example and the long tails of longer reasoning chains were dropped.

Practical implication: the model has been exposed to reasoning structures **up to 2048 tokens**. When generating responses substantially longer than that on its own, quality may degrade earlier than the chosen `max_tokens` cap. A v2 with a higher sequence length is on the roadmap.

## Usage (MLX)

```python
from mlx_lm import load, generate

model, tokenizer = load(
    "pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit"
)

messages = [
    {"role": "user",
     "content": "Three friends share 24 cookies equally. "
                "One friend gives 3 cookies to each of the others. "
                "How many does each friend have now? "
                "Reason step by step."}
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(
    model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True
)
```

CLI:

```bash
mlx_lm.generate \
  --model pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit \
  --prompt "Your reasoning question here" \
  --max-tokens 1024
```

## Reasoning Style — Observed Differences

This v1 release is honestly framed as a **stylistic distillation**, not a capability leap. The goal was to transfer R1-style explicit-reasoning patterns onto a strong base. Side-by-side probes on classic reasoning prompts (cookie-share math, bat-and-ball Cognitive Reflection Test, sister-brother counting trap) gave the following picture:

### Pattern transfer (intended outcome — present)

The fine-tuned model exhibits R1-signature self-verification language that the base model does not, e.g.:

> *"Let me rephrase the information to make sure I get it right…"*
> *"Wait, let me reconsider…"*
> *"Actually, let me check…"*

This is the visible signature of the OpenThoughts-114k (R1-generated) reasoning data.

### Capability gain (honest note — not demonstrated)

On the probes above, the **base Mistral Small 3.2 24B Instruct** already produces correct, structured, didactic answers — including a spontaneous *"Common Pitfall"* warning on the bat-and-ball trap. This v1 fine-tune did **not** measurably improve accuracy on those particular probes.

Likely contributing factors:
- The base is already strong at structured reasoning out-of-the-box.
- Training was on a 4-bit quantized base (gradients against quantized weights).
- The 2048-token training truncation means the model has been exposed to *starts* of long reasoning chains but not to their *convergence*. On long traps (sister-brother), the fine-tune showed *more* self-questioning loops but did not converge faster than the base.

### What this release is useful for

- An MLX-format, Apache-2.0, openly-licensed-chain model that **visibly exhibits R1-style explicit reasoning** on Apple Silicon.
- A baseline for our own iteration: a v2 trained on a higher-precision base, longer sequence length, and more iterations is on the roadmap and is where a measurable capability shift is expected.

## License Chain

| Component | License |
|---|---|
| Base model (Mistral Small 3.2 24B Instruct) | Apache 2.0 |
| 4-bit MLX quantization (mlx-community) | Apache 2.0 |
| Training dataset (OpenThoughts-114k) | Apache 2.0 |
| **This LoRA adapter + merged model** | **Apache 2.0** |

The entire supply chain is Apache 2.0. **No proprietary LLM outputs** (e.g. OpenAI, Anthropic) were used either in dataset generation or in training.

## Roadmap

- GGUF quantizations (at minimum Q4_K_M and Q8_0) released as a sibling `…-GGUF` repository
- v2 with higher sequence length and more iterations once a fp16/bf16-capable host is available
- Public evaluation on standard reasoning benchmarks (GSM8K, MATH, MMLU-Pro)

## Limitations

- **Sequence truncation during training** at 2048 tokens may cause earlier-than-ideal stopping behaviour on very long self-generated reasoning chains.
- **4-bit quantized base** before LoRA: gradients are computed against the quantized weights. Quality is solid for distillation but a fp16/bf16-based v2 would likely be stronger.
- **Language coverage**: the training dataset is predominantly English; multilingual reasoning quality may vary by language.
- **Multimodality**: the base architecture is `Mistral3` (vision + text). This LoRA is purely text-only training; vision capabilities are inherited unchanged from the base and have not been fine-tuned here.

## Attribution & Acknowledgements

- **[Mistral AI](https://mistral.ai/)** — for releasing an excellent multilingual base model under Apache 2.0.
- **[OpenThoughts team](https://huggingface.co/open-thoughts)** — for assembling and releasing the high-quality reasoning distillation dataset.
- **[DeepSeek](https://www.deepseek.com/)** — for the R1 model whose generations form the reasoning traces inside OpenThoughts-114k.
- **[mlx-community](https://huggingface.co/mlx-community)** — for the MLX 4-bit quantization of the base model used as the LoRA host.
- **[Apple ML Research](https://github.com/ml-explore/mlx-examples)** — for `mlx-lm`, which made training on a 64 GB Mac feasible.

## Citation

```bibtex
@misc{pnoid_mistral_openthoughts_distill_v1_mlx_4bit_2026,
  author       = {pnoid},
  title        = {Mistral-Small-3.2-24B-OpenThoughts-Distill-v1 (MLX 4-bit)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/pnoid/Mistral-Small-3.2-24B-OpenThoughts-Distill-v1-mlx-4bit}}
}
```

When using this model, please also cite the base model and the dataset (see their respective model/dataset cards for canonical citation formats).

---

*Trained June 2026 on an Apple M4 Pro (64 GB unified memory).*