---
license: apache-2.0
language:
- zh
- en
- ru
- de
pipeline_tag: text-generation
library_name: transformers
tags:
- fawen
- 35B
- moe
---
# Fawen
**A reasoning-enhanced, tool-native Mixture-of-Experts language model**
*Compact footprint · Transparent thinking · Real agentic ability*
[](https://huggingface.co/)
[](https://huggingface.co/)
[](https://huggingface.co/)
[](https://huggingface.co/)
[](https://huggingface.co/)
[](https://huggingface.co/)
---
> **Fawen** is a 35-billion-parameter sparse Mixture-of-Experts (MoE) language model that activates only about **3 billion parameters per token**. It was built to be a *small-in-practice, smart-in-behavior* assistant: deeply compressed for efficient deployment, then retrained to recover quality, and tuned on a large body of real agent interaction traces so it can **think step by step and call tools reliably**.
---
## Model Summary
| Property | Value |
|----------|-------|
| **Model name** | Fawen |
| **Version** | 1.0 |
| **Developer** | David Zhang |
| **Architecture** | Sparse Mixture-of-Experts (MoE) |
| **Parameters** | ~35B total, ~3B active per token |
| **Precision** | Aggressively quantized, then retrained (quantization-aware) |
| **Context window** | Extended long-context |
| **Languages** | English, Chinese (multilingual capable) |
| **Primary strengths** | Tool / function calling, transparent reasoning, agentic workflows |
| **License** | Apache 2.0 |
---
## Key Highlights
- **Sparse efficiency.** Only a fraction of the 35B parameters fires on each token, so inference cost tracks a ~3B-class model while keeping the capacity of a much larger one.
- **Compressed without forgetting.** An aggressive post-training quantization pass was followed by a retraining pass that rebuilt lost capability — smaller to host, still capable to use.
- **Born to use tools.** Trained on a large corpus of real-world agent episodes and a curated tool-calling set spanning web access, file reading, computation, and database querying.
- **Thinks out loud, cleanly.** A built-in reasoning scaffold separates *deliberation* from *action*, so the model plans before it acts and is easy to inspect.
- **Multilingual & long-context.** Strong Chinese/English understanding with an extended context window for long documents and multi-turn agent sessions.
---
## Training Approach
Fawen was adapted from a 35B sparse-MoE backbone through three coordinated efforts:
1. **Aggressive quantization + retraining.** The backbone was heavily compressed, then retrained under the quantized regime. This shrinks memory and compute footprint while preserving the model's reasoning and knowledge.
2. **Large-scale agentic corpus.** A substantial dataset of authentic agent interaction traces — multi-turn episodes containing tool invocations interleaved with step-by-step reasoning — was used to instill robust, realistic tool-use behavior.
3. **Curated tool-calling supervision.** A dedicated collection of function/tool calls across diverse APIs (web fetch, file I/O, calculator, SQL/database, and more) was added to broaden and harden calling accuracy.
4. **Reasoning-scaffold augmentation.** Training examples were framed with an explicit think-then-act structure, teaching the model to emit a reasoning block before any tool call or final answer.
> The result is a model that is **deployment-friendly** (quantized, MoE-sparse) yet **behavior-rich** (agentic, transparent, multilingual).
---
## Capabilities
| Capability | Description |
|------------|-------------|
| **Tool / Function calling** | Emits well-formed tool calls for web, files, math, databases, and general APIs. |
| **Transparent reasoning** | Produces an explicit reasoning trace before answering or acting. |
| **Agentic workflows** | Sustains multi-step plans with intermediate tool use and self-correction. |
| **Efficient inference** | MoE sparsity + quantization keep active compute and memory low. |
| **Multilingual** | Fluent in English and Chinese; general multilingual coverage. |
| **Knowledge & skills** | Inherits broad knowledge, math, and coding ability from the MoE backbone. |
---
## Reasoning & Tool-Use Scaffold
Fawen structures its responses with two clear blocks:
- ` ... ` — the model's internal deliberation: restating the goal, decomposing the problem, and deciding on a plan.
- ` ... ` — a structured invocation of a tool/function, emitted only when external action is needed.
**Example interaction**
```
User: What's the weather in Shanghai right now, and should I bring an umbrella?
Assistant:
The user wants the current weather in Shanghai and a recommendation.
I need a live observation, so I should call the weather tool, then judge rain probability.
{
"name": "get_weather",
"arguments": { "city": "Shanghai", "units": "metric" }
}
[tool returns: Shanghai, 18°C, light rain, 70% humidity]
It is currently raining in Shanghai, so an umbrella is advisable.
Yes — it's lightly raining in Shanghai right now (18°C). I'd bring an umbrella.
```
This design makes the model's decisions **auditable** and lets downstream systems parse tool calls deterministically.
---
## Evaluation
| Agentic Coding | Fawen-1.0-35B | Qwen3.5-35B | Qwen3.6-35B | Gemma4-31B | Qwen3.5-397B |
| :--- | :---: | :---: | :---: | :---: | :---: |
| **Terminal-Bench 2.1** (Terminus-2) | TBD | 41.4 | 52.5 | 42.1 | 53.5 |
| **Terminal-Bench 2.1** (Claude Code) | TBD | 38.9 | 49.2 | - | 48.6 |
| **SWE-bench Verified** | TBD | 70 | 73.4 | 52 | 76.4 |
| **SWE-bench Pro** | TBD| 44.6 | 49.5 | 35.7 | 51.6 |
| **SWE-bench Multilingual** | TBD | 60.3 | 67.2 | 51.7 | 69.3 |
| **NL2Repo** | TBD | 20.5 | 29.4 | 15.5 | 36.8 |
| **Claw-eval Avg** | TBD| 65.4 | 68.7 | 48.5 | 70.7 |
| **SWE Atlas - QnA** | TBD | 13.2 | 15.5 | - | 20.4 |
| **SWE Atlas - RF** | TBD | 10.2 | 11.4 | - | 18.4 |
| **SWE Atlas - TW** | TBD | 9.8 | 13.3 | - | 18.5 |
---
## Intended Use
- **In scope:** assistant and agent applications that require tool use, multi-step planning, and explainable reasoning; Chinese/English conversational products; on-prem or cost-sensitive deployments thanks to the quantized MoE design.
- **Out of scope:** high-stakes decisions without human oversight; tasks requiring guaranteed factual correctness without verification; any use that violates applicable law or safety norms.
## Limitations
- As a compressed model, rare knowledge or rare-language accuracy may trail a full-precision backbone; verify critical outputs.
- Tool schemas must be provided at inference time; the model follows the schema rather than inventing APIs.
- Long-context behavior should be validated for your specific document lengths.
- License and commercial-use terms are **to be confirmed** by the developer.
---
## How to Use
Load with 🤗 Transformers:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "DavidZhang/Fawen-1.0" # replace with your repo id
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto", # quantized weights load natively
device_map="auto",
)
messages = [
{"role": "user", "content": "Check the status of order #88231 and summarize it."},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(out[0], skip_special_tokens=False))
```
For production serving, export to **GGUF** (CPU/RAM-friendly, llama.cpp) or **vLLM** (PagedAttention, low-latency) — both preserve the `` / `` scaffold.
---
## Citation
```bibtex
@misc{fawen2026,
title = {Fawen: A Reasoning-Enhanced, Tool-Native Mixture-of-Experts Language Model},
author = {Zhang, David},
year = {2026},
howpublished = {\url{https://huggingface.co/DavidZhang/Fawen-1.0}},
note = {35B sparse MoE, quantized + retrained; agentic and reasoning-scaffold training.}
}
```
## Acknowledgements
Thanks to the open agent-trajectory and tool-use research community whose public datasets and scaffolding ideas informed this model's training recipe.