--- base_model: wangzhang/Qwen3.5-122B-A10B-abliterated base_model_relation: quantized pipeline_tag: text-generation library_name: mlx license: apache-2.0 tags: - mlx - omlx - qwen3 - qwen3.5 - qwen3_5_moe - abliterated - quantized - reap - moe-pruning - tool-calling - long-context extra_gated_heading: Model Access and Use Terms extra_gated_prompt: '**By checking the required box below and selecting “Agree and Access Model,” you (for yourself and your heirs, successors, and assigns) represent that you are at least 18 years old (or the age of majority in your jurisdiction, if greater), have legal capacity and authority to enter this agreement, and have read and understood these terms.** You acknowledge that the model weights, code, configurations, documentation, and related files in this repository (the “Artifact”) are an unofficial research derivative in a model lineage with intentionally reduced refusal or safety behavior, are not sponsored, endorsed, approved by, or affiliated with any upstream author or base-model provider, have not been safety-evaluated by the repository owner, and may generate or enable harmful, illegal, offensive, infringing, private, sensitive, inaccurate, or otherwise unsafe content or outcomes. You accept the Artifact **“AS IS” and “WITH ALL FAULTS.”** To the fullest extent permitted by applicable law, the repository owner, author, uploader, maintainers, contributors, licensors, upstream authors, and each of their respective affiliates, officers, employees, and agents (collectively, the “Released Parties”) disclaim all warranties and conditions, express, implied, statutory, or arising from course of dealing or usage of trade, including merchantability, fitness for a particular purpose, title, non-infringement, accuracy, reliability, and safety. You knowingly and voluntarily assume all risks arising from access to or any use, misuse, modification, fine-tuning, distribution, deployment, output, or reliance on the Artifact and, to the fullest extent permitted by applicable law, irrevocably release, waive, discharge, and covenant not to sue the Released Parties for any claim, loss, liability, damage, penalty, cost, or expense arising from or related to the Artifact, including claims based on ordinary negligence. The Released Parties will not be liable for indirect, incidental, special, consequential, exemplary, or punitive damages, and their total aggregate liability arising from or related to the Artifact will not exceed US $100. You are solely responsible for reviewing and validating outputs, implementing appropriate safeguards and human oversight, preventing access by minors, complying with all applicable laws, regulations, export and sanctions controls, and license terms, and all consequences of your access and use. You agree to defend, indemnify, and hold harmless the Released Parties from every third-party claim and resulting loss, liability, damage, penalty, cost, or expense, including reasonable attorneys’ fees, arising from or related to your access, use, misuse, modification, distribution, deployment, outputs, or violation of law or these terms. The Released Parties are intended third-party beneficiaries entitled to enforce this agreement. These access and risk-allocation terms supplement and do not alter the copyright permissions granted by any applicable license; if a conflict exists, the applicable license controls copyright permissions and these terms control repository access and risk allocation. This agreement is governed by New York law, excluding conflict-of-laws rules, and you consent to exclusive jurisdiction and venue in the state and federal courts located in New York County, New York. If any provision is held unenforceable, it will be modified and limited to the minimum extent necessary or severed, and the remainder will remain in full force and effect. Terms version 2026-07-23.' extra_gated_fields: ? I am legally eligible to agree and have read, understand, and agree to the Model Access and Use Terms above as a condition of accessing this repository : checkbox extra_gated_button_content: Agree and Access Model --- # Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX > [!WARNING] > Research artifact. This model is derived from an abliterated parent model with reduced refusal behavior. It can produce harmful, unsafe, or policy-violating outputs. Do not deploy it without your own safety layer, access controls, and monitoring. This is an unofficial derivative release and is not an official Qwen or parent-model release. `Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX` is a static-MoE-pruned and MLX-quantized derivative of [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated), built to preserve tool-calling behavior while reducing memory footprint enough to be practical on a 128 GB Apple Silicon machine. The explicit target for this release was: - produce a `Q6`-class local MLX artifact of this model - preserve tool calling as well as possible through the prune + quant pipeline - make a `100k` context window a realistic reliability target on a `128 GB` machine This release uses: - `20%` static REAP expert pruning on the MoE layers - `205 / 256` routed experts kept per MoE layer - `oQ6` MLX quantization - Tool-calling-oriented REAP calibration based on [tryumanshow/ToolACE-Qwen-cleaned](https://huggingface.co/datasets/tryumanshow/ToolACE-Qwen-cleaned) ## What This Is This is a deployment artifact aimed at local MLX inference, not a fine-tune. The goal was: - preserve tool-calling behavior as much as possible - reduce model size enough to fit comfortably on a `128 GB` Mac - keep a path open for large-context workloads, with `100k` context as the practical target The final artifact is an MLX `oQ6` model with `16` safetensor shards and about `76 GB` on-disk size. ## Lineage - Base model: [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) - Parent model: [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated) - This release: static REAP prune (`20%`) + MLX `oQ6` quantization This model should be understood as a derivative artifact of the abliterated parent, not as an official upstream model variant. ## Build Summary ### Pruning - Method: static REAP expert pruning with `reap-mlx` - Calibration focus: tool-calling preservation - Prune plan: `2448 / 12288` routed experts removed - Per-layer change: `256 -> 205` routed experts - Output BF16 checkpoint size: about `184 GB` ### Quantization - Method: `oQ6` quantization with `oMLX` - Final size: about `76 GB` - Total indexed weight size: `81,618,377,035` bytes - Quantization base: `6-bit`, `group_size=64`, `affine` - Important layers such as routers remain protected by higher-bit overrides in the quantization config ## Evaluation Snapshot These are sanity checks and release notes, not a full benchmark paper. ### Tool Calling - Direct smoke test: passed - The final quant emitted a valid XML tool call for a simple arithmetic function call instead of answering directly - ToolACE-style sampled smoke: - sample size: `8` - tool-call outputs: `7` - observed tool-call rate: `87.5%` - artifact: `eval-20260403-011600/toolace_toolcall_smoke.json` ### Standard Sanity Benchmark - HellaSwag slice: `validation[:256]` - `acc`: `62.5%` - `acc_norm`: `73.046875%` - evaluation time: `268.41 s` - peak memory during that eval: `84.34 GB` - artifact: `eval-20260403-011600/hellaswag-256.raw.txt` ### Long Context Note Large-context behavior was part of the target profile for this release: - this model successfully cold-loaded and began inference with `100k+` token prompts on a `128 GB` Apple Silicon machine - a saved long-context coding suite also started at `105,484` prompt tokens So the honest claim is: - tool calling survived the prune + quant pipeline - the model runs locally in MLX - `100k` context was the target profile for this release on a `128 GB` machine - `100k+` context appears reachable ## Files - `config.json` - `generation_config.json` - `chat_template.jinja` - `tokenizer.json` - `tokenizer_config.json` - `model.safetensors.index.json` - `16` model shard files Model config highlights: - `model_type: qwen3_5_moe` - `num_experts: 205` - `num_experts_per_tok: 8` ## Tools and Reference Files External tools, models, and datasets used or referenced for this artifact: - Base model: [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) - Parent abliterated model: [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated) - Parent-model method: [Abliterix](https://github.com/wuwangzhang1216/abliterix) - REAP pruning tool: [reap-mlx](https://github.com/0xsero/reap-mlx) - Quantization tool: [oMLX](https://github.com/jundot/omlx) - MLX inference loader used for smoke tests: [mlx-lm](https://github.com/ml-explore/mlx-lm) - Tool-calling calibration dataset: [tryumanshow/ToolACE-Qwen-cleaned](https://huggingface.co/datasets/tryumanshow/ToolACE-Qwen-cleaned) - Standard sanity benchmark dataset: [hellaswag](https://huggingface.co/datasets/hellaswag) Included reference files in this repo: - Upload checklist: [UPLOAD_PLAN.md](./UPLOAD_PLAN.md) - Chat template: [chat_template.jinja](./chat_template.jinja) - Main config: [config.json](./config.json) - Generation config: [generation_config.json](./generation_config.json) - Tokenizer config: [tokenizer_config.json](./tokenizer_config.json) - Tokenizer: [tokenizer.json](./tokenizer.json) - Weight index: [model.safetensors.index.json](./model.safetensors.index.json) - Bundled references index: [references/README.md](./references/README.md) - REAP pruning plan used for this artifact: [references/pruning-plan.json](./references/pruning-plan.json) - ToolACE render stats used during preparation: [references/toolace_qwen_rendered_128.stats.json](./references/toolace_qwen_rendered_128.stats.json) - Tool-calling smoke results: [references/toolace_toolcall_smoke.json](./references/toolace_toolcall_smoke.json) - HellaSwag sanity-check results: [references/hellaswag-256.json](./references/hellaswag-256.json) ## Usage ### MLX / mlx-lm ```python from mlx_lm import load, generate model, tokenizer = load("0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX") messages = [ {"role": "user", "content": "Reply with exactly the word OK."} ] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=False, ) text = generate(model, tokenizer, prompt=prompt, max_tokens=8, verbose=False) print(text) ``` ### Tool Calling ```python from mlx_lm import load, generate model, tokenizer = load("0xdfi/Qwen3.5-122B-A10B-abliterated-REAP20-oQ6-MLX") tools = [{ "type": "function", "function": { "name": "add_numbers", "description": "Add two integers.", "parameters": { "type": "object", "properties": { "a": {"type": "integer"}, "b": {"type": "integer"}, }, "required": ["a", "b"], }, }, }] messages = [{ "role": "user", "content": "What is 7 plus 8? Use the available function.", }] prompt = tokenizer.apply_chat_template( messages, tools=tools, add_generation_prompt=True, tokenize=False, ) text = generate(model, tokenizer, prompt=prompt, max_tokens=128, verbose=False) print(text) ``` ## Reproduction Notes High-level local pipeline: 1. Start from the abliterated `122B` parent model in unquantized MLX/BF16 form. 2. Convert `ToolACE-Qwen-cleaned` into REAP-ready rendered chat samples. 3. Collect REAP telemetry with a windowed/layerwise workflow to stay within local memory limits. 4. Build a `20%` prune plan. 5. Apply structural pruning to produce a `205`-expert-per-layer BF16 checkpoint. 6. Quantize the pruned checkpoint to `oQ6`. 7. Run tool-calling and sanity evals on the final artifact. One reproducibility caveat: the final `oQ6` quantization completed with position-based sensitivity fallback because the intended proxy model path was not loadable in the local quantization path at the time of export. ## Limitations - Tool-calling preservation was a design target, not a formal guarantee. - The ToolACE validation included only a small sampled smoke in this release note. - This is an MLX artifact, not a Transformers-format upload. ## Credits - Qwen base model: [Qwen/Qwen3.5-122B-A10B](https://huggingface.co/Qwen/Qwen3.5-122B-A10B) - Abliterated parent model: [wangzhang/Qwen3.5-122B-A10B-abliterated](https://huggingface.co/wangzhang/Qwen3.5-122B-A10B-abliterated) - REAP implementation: local `reap-mlx` workflow - Quantization: local `oMLX` / `oQ` - Calibration dataset: [tryumanshow/ToolACE-Qwen-cleaned](https://huggingface.co/datasets/tryumanshow/ToolACE-Qwen-cleaned) ## Disclaimer This repository is provided for research and local inference experimentation. The parent model is abliterated and may answer requests that safer models refuse. The author of this derivative artifact is not responsible for misuse. Users are responsible for complying with applicable law, platform rules, and their own safety requirements.