Instructions to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64 with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "imaadd05/Mellum2.1-12B-A2.5B-Thinking-mlx-6bit-g64" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 2,897 Bytes
9104747 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | {
"source": "JetBrains/Mellum2.1-12B-A2.5B-Thinking",
"source_revision": "92ddae9fc7665e9f801d141d2e5a6b2caf2460c4",
"created_utc": "2026-10-09T08:32:06.014227+00:00",
"quantization": {
"mode": "affine",
"bits": 6,
"group_size": 64,
"router_bits": 8,
"non_quantized_dtype": "bfloat16"
},
"device": "gpu",
"platform": "Linux-7.0.0-38-generic-x86_64-with-glibc2.39",
"python": "3.12.3",
"packages": {
"mlx": "0.32.3",
"mlx-lm": "0.32.0",
"transformers": "5.19.0",
"huggingface-hub": "1.33.0",
"safetensors": "0.8.0",
"numpy": "2.5.3"
},
"source_sha256": {
"README.md": "3da5f8b9862c7b3560e631d778e2402f222d87f1a332c34696dfa399608824a7",
"chat_template.jinja": "4593a34d52f3364ac13ce57f4bea5688924e8942da6df8201e411db17e729f48",
"config.json": "f43d018246094dee6dca455b3072766b6acbefd1ac8e7fe664df34710606a541",
"generation_config.json": "a9dc104ca398a2ef376d3f2fbb03037d99950ddb2db0cd041b98398cf1fc8d7c",
"model-00001-of-00005.safetensors": "f5ccf8012c7d44fd2598325bf2482b8fc21792ef888de68f9ca8c9053c2e022f",
"model-00002-of-00005.safetensors": "266beea6f34dea9c86414d8bf8cb35c0f4a53330731d99ed41ec666390758e39",
"model-00003-of-00005.safetensors": "da9a04fb6ea121f71e30920c7657b889ca0078aea57d878e5cb768ebb3e4483a",
"model-00004-of-00005.safetensors": "32f27ab7f38ff5d98ac3d50dabb57ab8a55a8aa552febd42c2a5b01f4116262a",
"model-00005-of-00005.safetensors": "d8bbb9b829cb3279e23f59af55bf68ec00cc8ad30923c227d8b13c7d831e6b3b",
"model.safetensors.index.json": "a6ad930e5bb35ea92d5b734c4df4649bfcfdcd899f64b0a83efe40fe0ab9b60e",
"special_tokens_map.json": "20e1ead0b0986b55989aa8116bafe63efcbae0daf8ed047970cd28b36aada4d9",
"tokenizer.json": "58548a346eb073e5132bf7d8ad17dc6971bca36ade378ca4d2bfbc49bf60da2a",
"tokenizer_config.json": "e2950960d2a3040a1938e6357c8c3628d531133c464286a7869b2e0504fcc59e"
},
"artifact_sha256": {
"chat_template.jinja": "4593a34d52f3364ac13ce57f4bea5688924e8942da6df8201e411db17e729f48",
"config.json": "0254d0f058f4ccd9f34d3e0b1e85dd425af417da7569a3672b62d692798aaa3e",
"generation_config.json": "a9dc104ca398a2ef376d3f2fbb03037d99950ddb2db0cd041b98398cf1fc8d7c",
"model-00001-of-00002.safetensors": "ff781996bc063c39d4dd01bc86a664522c06277412cfba8b48063a189fdd33e0",
"model-00002-of-00002.safetensors": "d92c83c9a08ce5d9f6a493d3f851dfbb15cf12a3a30dc856674eecf9e1ea283a",
"model.safetensors.index.json": "aef292869670c96d74e8ae0fd57694eb46a0dc2427b8338ea05d4c98c98881d9",
"special_tokens_map.json": "20e1ead0b0986b55989aa8116bafe63efcbae0daf8ed047970cd28b36aada4d9",
"tokenizer.json": "58548a346eb073e5132bf7d8ad17dc6971bca36ade378ca4d2bfbc49bf60da2a",
"tokenizer_config.json": "e2950960d2a3040a1938e6357c8c3628d531133c464286a7869b2e0504fcc59e"
},
"weight_bytes": 9873101418,
"validation_status": "pending"
}
|