darkmaniac7/Qwen3-4B-abliterated-MNN

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Qwen3-4B Abliterated MNN

MNN 4-bit HQQ quantized model for TokForge on-device inference.

Model Info

Parameters	4B
Quantization	4-bit HQQ (quant_block=64)
Size	2.4GB
Source	huihui-ai/Huihui-Qwen3-4B-abliterated-v2
Backend	MNN OpenCL (GPU-accelerated)
Min RAM	8GB+

Performance

Performance varies by device, backend routing, and thermal state. On our test devices, speculative decoding with dense Qwen3 targets measured +34% to +43% faster decode in chat workloads. Results vary by device and workload.

Speculative Decoding

This model is compatible with the TokForge Acceleration Pack (Qwen3-0.6B draft model), which works with both censored and uncensored targets.

Usage

Download via TokForge app → Models → Roleplay category, or manually place files in the TokForge models directory.

Limitations and Intended Use

• Intended for TokForge / MNN mobile inference.
• 4B was not the strongest speculative-decoding target in our later preserved fleet results.
• Performance depends strongly on SoC, backend routing, and device thermal behavior.
• This repo is a runtime/export artifact, not a standard Transformers release.

Files

• llm.mnn — Model graph
• llm.mnn.weight — Quantized weights
• tokenizer.txt — Tokenizer vocabulary
• llm_config.json — Model configuration
• embeddings_bf16.bin — Embedding table (8B/14B only)

Credits

• Original model: huihui-ai/Huihui-Qwen3-4B-abliterated-v2
• MNN framework: alibaba/MNN
• TokForge: tokforge.ai

Community

• Website: tokforge.ai
• Discord: Join the Discord