Text Generation
MLX
Safetensors
nanbeige
dwq
4-bit precision
heretic
coding
agents
conversational
custom_code
Instructions to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
| { | |
| "base_model": "WaveCut/Nanbeige4.2-3B-heretic", | |
| "base_model_commit": "0bacc89d0cdf2ec9fe55987ae7253268465be9dc", | |
| "mlx_lm_revision": "cf10f962b7a20e63a6df43dbf0faf06070153d40", | |
| "quantization": { | |
| "method": "DWQ", | |
| "bits": 4, | |
| "group_size": 32, | |
| "effective_bits_per_weight": 5.0, | |
| "trainable_fraction": 0.06253, | |
| "initial_validation_loss": 0.284, | |
| "final_validation_loss": 0.043 | |
| }, | |
| "calibration": { | |
| "seed": 20260722, | |
| "train_samples": 1024, | |
| "validation_samples": 32, | |
| "max_sequence_length": 1025, | |
| "batch_size": 4, | |
| "corpus_sha256": "a7cfdbe02c124304bf1282bbd5ed7162bfa72dec6750b60ed2d3a68000c7a554" | |
| }, | |
| "smoke": { | |
| "returncode": 0, | |
| "wall_seconds": 7.366, | |
| "generation_tokens_per_second": 97.959, | |
| "peak_memory_gb": 2.709, | |
| "output": "```python\ndef clamp(value, low, high):\n \"\"\"Clamp a value to the inclusive range [low, high].\"\"\"\n return max(low, min(value, high))\n```", | |
| "passed": true | |
| }, | |
| "files": [ | |
| { | |
| "name": "README.md", | |
| "size": 1879, | |
| "sha256": "b0de81373ec5ab5630d579de1d4841c80a0d19f2eae67e846a6291330b3865de" | |
| }, | |
| { | |
| "name": "calibration-metadata.json", | |
| "size": 656, | |
| "sha256": "8badc388221aadecb0ba0bc2f478d1b93489d0802918c587c5dae6c3246ad966" | |
| }, | |
| { | |
| "name": "chat_template.jinja", | |
| "size": 8362, | |
| "sha256": "cb1699773afe04d17eacf8d0802feabac9ec10c3061e280f3121754aa749d87d" | |
| }, | |
| { | |
| "name": "config.json", | |
| "size": 1342, | |
| "sha256": "8e86c6a6f44519f7c897b1ab4a8b1c93f1d4d741a6db7dfe86eb1da133b44783" | |
| }, | |
| { | |
| "name": "generation_config.json", | |
| "size": 187, | |
| "sha256": "68c690ce23efb6caae30c006ff3c1efd826297ff1df4338c04f7ac6f685d8746" | |
| }, | |
| { | |
| "name": "mlx-lm-dwq.patch", | |
| "size": 2291, | |
| "sha256": "7eb566eafe939bba08b5b341b67d224379fb8d4750cb63fbc064222b246524dd" | |
| }, | |
| { | |
| "name": "model.safetensors", | |
| "size": 2606373808, | |
| "sha256": "52e7814c8360d050c1411ccd08210cdf15c5c97b131c8a278f1cf883c424f289" | |
| }, | |
| { | |
| "name": "model.safetensors.index.json", | |
| "size": 36121, | |
| "sha256": "0b5be7986ad071fe5fd4ceda75c6910ce37520d1033ed6077f06451ae132445b" | |
| }, | |
| { | |
| "name": "nanbeige_mlx.py", | |
| "size": 10284, | |
| "sha256": "9ec6dec2cddb4129918f6b79657feae376c4b7b4008c26fc9951b00baedcf890" | |
| }, | |
| { | |
| "name": "smoke-mlx.json", | |
| "size": 300, | |
| "sha256": "0ec9d5e5343a708a9b2e1f06c5fa93a519cc6a5727b22417ea24bad1f81e7019" | |
| }, | |
| { | |
| "name": "tokenizer.json", | |
| "size": 18450976, | |
| "sha256": "a002676e923776f5a59d21320d667920e732000f7762fb72e6be6e911790e060" | |
| }, | |
| { | |
| "name": "tokenizer_config.json", | |
| "size": 492, | |
| "sha256": "b94dfdd9f7b774c3770a7722dea064a11207d8e8812563968161c51151832f25" | |
| } | |
| ] | |
| } | |