--- library_name: mlx license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/995ad96eacd98c81ed38be0c5b274b04031597b0/LICENSE base_model: - nvidia/Qwen3.6-35B-A3B-NVFP4 - N8Programs/Qwen3.6-35B-A3B-AntiLoop pipeline_tag: text-generation tags: - mlx - mlx-lm - qwen3.6 - moe - modelopt - quantized - nvfp4 - fp4 - fp8 - lora - merged - antidoom --- # Qwen3.6-35B-A3B-AntiLoop-NVFP4 for MLX-LM ![AntiLoop looping rate versus GPQA capability](assets/looping_vs_gpqa.png) ![LoopHard judged-loop rates across four inference settings](assets/loophard_four_settings.png) This is the **text-only, lossless-weight MLX-LM conversion** of [`N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4`](https://huggingface.co/N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4). It deliberately omits the vision tower so the mixed NVIDIA ModelOpt weights can be preserved without a second quantization pass or an installed-package patch. ## Lossless MLX representation All source quantized weight payloads and scales are retained: - ModelOpt FP8 E4M3 bytes are packed directly into MLX `uint32` tensors with unit E8M0 carrier scales; the original FP32 tensor scale is applied after MLX's native MXFP8 matrix multiplication. - ModelOpt NVFP4 E2M1 nibbles and E4M3 block-scale bytes remain bit-identical; the original FP32 tensor or per-expert scale is applied after MLX's native NVFP4 matrix multiplication. - BF16 and FP32 language-model tensors remain in their source precision. - No weight tensor is dequantized or requantized during conversion. The package contains 1,475 tensors across 41 language shards: 130 scaled MXFP8 dense modules, 121 scaled NVFP4 dense modules, and 120 scaled NVFP4 expert projections. `weight_bytes_requantized=false` is recorded in `mlx_conversion_manifest.json`. “Lossless” here means no additional error is introduced into the source quantized **weights**. Activations remain in the model dtype instead of using the source checkpoint's stored activation scales, so execution is not expected to be bit-identical to NVIDIA ModelOpt inference. ## Use with MLX-LM Install the tested runtime: ```bash pip install -U "mlx-lm==0.31.3" ``` Generate with an unmodified MLX-LM installation: ```bash python -m mlx_lm generate \ --model mlx-community/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \ --trust-remote-code \ --prompt "Solve: 27 * 43" \ --max-tokens 512 ``` The packaged interactive defaults are `temperature=0.6`, `top_p=0.95`, and `top_k=20`. The Qwen chat template retains its `enable_thinking` control, so clients such as LM Studio can expose the usual thinking toggle. MLX-LM natively loads the included `modeling_mlx_qwen36_modelopt_hybrid.py` through its model-local `model_file` architecture hook; the loader does not patch or modify the installed package. Review that runtime file before loading this repository. In MLX-LM 0.31.3, `--trust-remote-code` controls tokenizer loading rather than the `model_file` hook; it is included above for tokenizer compatibility. This repository is intentionally text-only. It does not contain the 333 BF16 vision tensors, image processor, or video processor from the upstream model. ## Original model card This is a mixed-precision NVIDIA ModelOpt deployment checkpoint for [`Qwen3.6-35B-A3B-AntiLoop`](https://huggingface.co/N8Programs/Qwen3.6-35B-A3B-AntiLoop), a narrow fine-tune intended to recover from pathological self-verification and enumeration loops while preserving ordinary long-form reasoning. No PEFT adapter is required at inference time. This MLX-LM derivative retains the language architecture, tokenizer, chat template, and 262,144-token native context configuration. It does not package the vision tower or MTP draft weights. ## Training data The exact 178 masked supervised targets used for the final AntiLoop training round are published in the [`Qwen3.6-35B-A3B-AntiLoop-SFT` dataset](https://huggingface.co/datasets/N8Programs/Qwen3.6-35B-A3B-AntiLoop-SFT). The dataset preserves each `loss_start_char` boundary so the pathological loop prefix remains conditioning context rather than a supervised target. It intentionally excludes the separately generated KL-regularization anchors. ## Training procedure The AntiLoop adapter was trained on the 178 masked supervised targets using a standard supervised fine-tuning procedure, but regularized via KL-loss on separately generated non-loop anchors from the base model on everyday prompts. ## Benchmark results ### LoopHard **LoopHard** is our held-out set of 285 enumeration prompts designed to elicit futile recall, recounting, and self-verification loops. The primary metric is **judged loops**: whether the model's reasoning trace remains stuck in a futile cycle when generation ends. | Model | Judged loops | Loop rate | |---|---:|---:| | NVIDIA NVFP4 | 72 / 285 | 25.26% | | **AntiLoop NVFP4** | **10 / 285** | **3.51%** | | NVIDIA NVFP4 + `presence_penalty=1.5` | 30 / 285 | 10.53% | | **AntiLoop NVFP4 + `presence_penalty=1.5`** | **1 / 285** | **0.35%** | The matched `presence_penalty=1.5` comparison converted all 30 control loops to clean completions while introducing one different loop. Exact two-sided McNemar `p = 2.98e-8`. Generation used thinking mode, `temperature=0.7`, `top_p=0.95`, `top_k=20`, a 6,144-token completion limit, and concurrency 24. The two `presence_penalty=1.5` arms used the exact original and AntiLoop NVFP4 checkpoints with the same vLLM build and serving configuration: TP1, FP8 KV cache, FlashInfer attention, Marlin NVFP4 MoE, and MTP speculative decoding with three draft tokens. LoopHard is judged by GLM-5.2 using a convergence-aware rubric: systematic reasoning and verification that reaches a conclusion are not loops, and a trace that notices its own circling and exits is classified as recovered. The calibration set contained 42 manually labeled traces. Across three judge runs, accuracy was 88.1%, 92.9%, and 95.2%; all three runs identified all 17 labeled loops, with 2–5 false positives among the 25 non-loop traces. The 285 prompts, metadata, and GLM-5.2 evaluation code are published in the [LoopHard dataset](https://huggingface.co/datasets/N8Programs/LoopHard) on Hugging Face. ### Capability preservation The capability checks below compare the same round-2 AntiLoop adapter against its FP8 reference model under a matched runtime-LoRA setup. These runs used the default presence penalty and should not be interpreted as evaluations of the exact mixed-precision artifact at `presence_penalty=1.5`. #### GPQA Diamond | Model | Accuracy | |---|---:| | Qwen3.6-35B-A3B official model card | 86.0% | | FP8 reference, our matched harness | 167 / 198 (84.34%) | | **AntiLoop FP8, our matched harness** | **166 / 198 (83.84%)** | The official-model-card number is included for context and was not produced by our harness. Our GPQA run used thinking mode, paired per-question seeds, `temperature=0.7`, `top_p=0.95`, `top_k=20`, MTP3 speculative decoding, a 65,536-token reasoning budget, and 4,096 tokens of answer headroom. The difference was not significant. Source for the published 86.0% result: [`Qwen/Qwen3.6-35B-A3B` model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B). #### GSM8K | Model | Accuracy | |---|---:| | FP8 reference | 1273 / 1319 (96.51%) | | **AntiLoop FP8** | **1270 / 1319 (96.29%)** | The GSM8K run used the exact 1,319-example `openai/gsm8k` `main` test split, thinking mode, paired seeds, `temperature=0.7`, `top_p=0.95`, `top_k=20`, MTP3, an 8,192-token reasoning budget and 1,024 tokens of answer headroom. The difference was not significant. Taken together, the matched GPQA and GSM8K results show no material or statistically detectable capability loss at these sample sizes. They do not establish equivalence across other tasks, modalities, or sampling settings. ## Original NVIDIA ModelOpt usage Use a recent vLLM build with ModelOpt mixed-precision support: ```bash vllm serve N8Programs/Qwen3.6-35B-A3B-AntiLoop-NVFP4 \ --quantization modelopt \ --trust-remote-code \ --max-model-len 262144 \ --kv-cache-dtype fp8 \ --reasoning-parser qwen3 ``` MTP speculative decoding can be enabled on a compatible build with: ```bash --speculative-config '{"method":"mtp","num_speculative_tokens":3}' ``` For the measured LoopHard setting, send the following sampling parameters: ```json { "temperature": 0.7, "top_p": 0.95, "top_k": 20, "presence_penalty": 1.5 } ``` The capability-preservation results above used the default presence penalty; `presence_penalty=1.5` has not yet been evaluated on GPQA or GSM8K. Follow the [`nvidia/Qwen3.6-35B-A3B-NVFP4` model card](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) for deployment requirements and the [`Qwen/Qwen3.6-35B-A3B` model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) for chat templating, thinking controls, multimodal inputs, and base-model limitations. ## Limitations - This is a narrow behavioral fine-tune, not a general alignment or safety model. - This MLX-LM artifact is text-only; image and video inputs are unsupported. - The conversion preserves quantized weight values, but uses model-dtype activations rather than the source ModelOpt activation scales. - LoopHard is a task-specific, judge-based benchmark; its loop rate should not be interpreted as a general safety, truthfulness, or factuality score. - The GPQA and GSM8K checks used runtime LoRA on an FP8 base, not this exact mixed-precision artifact. - Capability preservation has not been tested at `presence_penalty=1.5`. - The source checkpoint's fixed-scale FP8 merge approximates the exact BF16 LoRA merge; small adapter updates can round away or clip at the original E4M3 range. This MLX conversion does not add another weight-quantization pass. - Runtime validation used a 65,536-token configured context, not the full native 262,144-token context. - Outputs may still be incorrect, overconfident, repetitive, biased, toxic, or unsafe. ## License Apache 2.0, following both the underlying Qwen checkpoint and NVIDIA's quantized derivative. See the [pinned Qwen license](https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/995ad96eacd98c81ed38be0c5b274b04031597b0/LICENSE), the [Qwen model card](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), and the [NVIDIA ModelOpt checkpoint card](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4). (co-written with GPT-5.6-Sol)