Image-Text-to-Text
GGUF
English
Chinese
Korean
qwen3_5
unsloth
qwen
qwen3.5
reasoning
chain-of-thought
lora
conversational
Jackrong commited on
Commit
742d399
·
verified ·
1 Parent(s): 92b8fdc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +119 -14
README.md CHANGED
@@ -1,22 +1,127 @@
1
  ---
 
 
 
 
 
 
2
  tags:
3
- - gguf
4
- - llama.cpp
5
  - unsloth
6
- - vision-language-model
 
 
 
 
 
 
 
 
 
7
  ---
8
 
9
- # Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF : GGUF
10
 
11
- This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
12
 
13
- **Example usage**:
14
- - For text only LLMs: `llama-cli -hf Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF --jinja`
15
- - For multimodal models: `llama-mtmd-cli -hf Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2-GGUF --jinja`
 
 
 
16
 
17
- ## Available Model files:
18
- - `Qwen3.5-9B.Q5_K_M.gguf`
19
- - `Qwen3.5-9B.Q4_K_M.gguf`
20
- - `Qwen3.5-9B.BF16-mmproj.gguf`
21
- This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
22
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - zh
5
+ - ko
6
+ license: apache-2.0
7
+ base_model: Qwen/Qwen3.5-9B
8
  tags:
 
 
9
  - unsloth
10
+ - qwen
11
+ - qwen3.5
12
+ - reasoning
13
+ - chain-of-thought
14
+ - lora
15
+ pipeline_tag: text-generation
16
+ datasets:
17
+ - nohurry/Opus-4.6-Reasoning-3000x-filtered
18
+ - Jackrong/Qwen3.5-reasoning-700x
19
+ - Roman1111111/claude-opus-4.6-10000x
20
  ---
21
 
22
+ # 🌟 Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2
23
 
24
+ ## 📢 Announcement
25
 
26
+ > **v2 Update:**
27
+ > This iteration is powered by **14,000+ premium Claude 4.6 Opus-style general reasoning samples**, with a major focus on **achieving massive gains in reasoning efficiency at the cost of only an extremely minor drop in accuracy**.
28
+ >
29
+ > v2 introduces a **refined reasoning scaffold** designed to eliminate redundant internal loops, significantly improving the model's **cross-task generalization** from logic and math into specialized fields like programming. Compared to the original model, **autonomy and stability are significantly improved**, ensuring the model remains robust and self-consistent during complex, multi-step problem solving. v2 is built to **think smarter, not longer**, delivering substantial improvements in inference speed and cost-effectiveness while preserving nearly all of the baseline's peak accuracy.
30
+ >
31
+ > *Note: Due to the constraints of SFT sample size and training scope, the model's broad general-purpose capabilities might be slightly impacted. The efficiency and accuracy results discussed here are based on the HumanEval and HumanEval+ benchmarks. Thank you for your understanding!*
32
 
33
+ ![HCaJnUQaoAAaMIc](https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/k4JsVit-31Fw8RZ4pFkun.jpeg)
34
+
35
+ ## 💡 Model Introduction
36
+ **Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2** is the second iteration of this reasoning-focused Qwen3.5-9B fine-tune, built to drastically improve the *efficiency* of chain-of-thought generation, trading off a practically imperceptible margin of absolute accuracy for highly substantial gains in reasoning speed and cost-reduction.
37
+
38
+ Compared with the earlier version, **v2 was trained with 14,000 Claude 4.6 Opus-style general reasoning samples**, with a stronger emphasis on transferring concise, reusable reasoning patterns rather than only maximizing raw benchmark scores. The goal of v2 is not simply to make the model "think more," but to help it **think more economically**: reducing unnecessarily long internal chains, avoiding verbose over-analysis on easy problems, and massively improving the reasoning-cost-to-quality ratio without meaningfully sacrificing correctness.
39
+
40
+ A key design choice in v2 is that the distillation data is **primarily general-domain reasoning data**—specifically focused on mathematics, word problems, logical deduction, and a balanced mix of general knowledge and instructions—rather than specialized code-heavy supervision. Consequently, **HumanEval and HumanEval+ are employed here to evaluate cross-task generalization and capability transfer**, rather than serving as direct optimization targets. High performance on these benchmarks, despite the lack of code-centric training, confirms that the model's **reasoning scaffold** has become more robust and transferable, proving that fundamental reasoning logic can effectively power specialized tasks like programming.
41
+
42
+ ### Why v2 matters
43
+
44
+ Relative to the official Qwen3.5-9B baseline, the fine-tuned v2 model accepts an **extremely minor loss in absolute HumanEval accuracy** (less than 2 percentage points) in exchange for **massive, transformative gains in reasoning efficiency**:
45
+
46
+ | Metric | Official Qwen3.5-9B | v2 Fine-tuned Model | Improvement |
47
+ |---|---:|---:|---:|
48
+ | Average think length (chars) | 2284.3 chars | 1778.0 chars | **🟢 -22.17% (Shorter / Better)** |
49
+ | Average think length (words) | 400.83 words | 310.33 words | **🟢 -22.58% (Shorter / Better)** |
50
+ | HumanEval base passes per 10k think chars | 4.004 | 5.041 | **🟢 +25.91% (Higher / Better)** |
51
+ | HumanEval+ passes per 10k think chars | 3.764 | 4.836 | **🟢 +28.48% (Higher / Better)** |
52
+ | Think chars needed per HumanEval base pass | 2497.5 | 1983.6 | **🟢 -20.58% (Lower / Better)** |
53
+ | Think chars needed per HumanEval+ pass | 2656.9 | 2068.0 | **���� -22.17% (Lower / Better)** |
54
+
55
+ At the same time, while the official model holds a razor-thin lead on the standard base tests, **v2 achieves the exact same accuracy on the much stricter HumanEval+ benchmark**:
56
+
57
+ | Fairly Recomputed Benchmark | Official Qwen3.5-9B | v2 Fine-tuned Model | Gap |
58
+ |---|---:|---:|---:|
59
+ | HumanEval (base tests) pass@1 | **0.9146** | 0.8963 | **🔴🔽 -1.83 pts** |
60
+ | HumanEval+ (base + extra tests) pass@1 | **0.8598** | **0.8598** | **🔵 0.00 pts** |
61
+
62
+ This trade-off strongly favors real-world use cases.
63
+
64
+ For users who care strictly about the **absolute highest peak benchmark score**, the official model holds a razor-thin edge. However, for users who care about **reasoning efficiency per unit of inference budget**, v2 is exceptionally superior—doing almost exactly the same quality of logic work while consuming over 20% fewer characters and tokens.
65
+
66
+ That matters especially for:
67
+
68
+ - **Resource-constrained local deployment**: On consumer GPUs or lower-memory local setups, shorter and cleaner reasoning traces can reduce latency, memory pressure, and the effective cost of generation.
69
+ - **Agentic workflows**: In multi-step agents, the model often solves many *easy* or *medium* subtasks. In those settings, excessively elaborate chain-of-thought can become a tax on throughput. A model that reaches a workable answer with fewer reasoning tokens can improve end-to-end agent speed and lower cumulative inference cost.
70
+ - **Open-source tool use and emerging agent stacks**: For users building with lightweight open reasoning systems, browser-use agents, terminal agents, or projects in the "OpenClaw / local autonomous agent" style ecosystem, a model that sacrifices a small amount of peak accuracy for much better reasoning economy can be more practical in real-world loops.
71
+ - **Simple problems at scale**: One common issue with strong reasoning-tuned base models is that they sometimes produce very elaborate internal traces even for simple prompts. While that can look impressive, it is often inefficient in practice. v2 is explicitly aimed at trimming this overhead.
72
+
73
+ In short, **v2 does not claim to beat the official model on absolute coding benchmark score**. Instead, it demonstrates a more deployment-oriented optimization target: **faster, shorter, more economical reasoning** with still-competitive generalization. For many local users, agent builders, and cost-sensitive applications, this can be a highly favorable trade.
74
+
75
+ ## 🗺️ Training Pipeline Overview
76
+
77
+ ```text
78
+ Base Model (Qwen3.5-9B)
79
+
80
+
81
+ Qwen3.5-9B fine-tuned with Unsloth
82
+
83
+
84
+ Supervised Fine-Tuning (SFT) + LoRA
85
+ (Response-Only Training masked on "<|im_start|>assistant\n<think>")
86
+
87
+
88
+ Jackrong/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-v2
89
+ ```
90
+
91
+ ### 🧠 Example of Learned Reasoning Scaffold(Example)
92
+
93
+ The model includes targeted optimizations addressing Qwen3.5’s tendency toward excessive transitional or repetitive reasoning on simple queries. Through deep distillation and structural imitation of Claude-4.6-Opus reasoning chains, the model adopts a more efficient structured thinking pattern:
94
+ **“Let me analyze this request carefully: 1..2..3...”.**
95
+ This streamlined reasoning paradigm significantly reduces redundant cognitive loops while preserving deep analytical capacity, resulting in substantially improved inference efficiency.
96
+
97
+ ```text
98
+ Let me analyze this request carefully:
99
+
100
+ 1. Identify the core objective of the problem.
101
+ 2. Break the task into clearly defined subcomponents.
102
+ 3. Evaluate constraints and edge cases.
103
+ 4. Formulate a step-by-step solution plan.
104
+ 5. Execute the reasoning sequentially and verify consistency.
105
+ .
106
+ .
107
+ .
108
+ ```
109
+
110
+ ### 📚 All Datasets Used
111
+ The dataset consists of high-quality, filtered reasoning distillation data:
112
+
113
+ | Dataset Name | Description / Purpose |
114
+ |--------------|-----------------------|
115
+ | [nohurry/Opus-4.6-Reasoning-3000x-filtered](https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning-3000x-filtered) | Provides comprehensive Claude 4.6 Opus reasoning trajectories. |
116
+ | [Roman1111111/claude-opus-4.6-10000x](https://huggingface.co/datasets/Roman1111111/claude-opus-4.6-10000x) | Large-scale public Claude 4.6 Opus distillation data used to strengthen general reasoning transfer in v2. |
117
+ | [TeichAI/claude-4.5-opus-high-reasoning-250x](https://huggingface.co/datasets/TeichAI/claude-4.5-opus-high-reasoning-250x) | Injecting high-intensity, structured reasoning instances. |
118
+ | [Jackrong/Qwen3.5-reasoning-700x](https://huggingface.co/datasets/Jackrong/Qwen3.5-reasoning-700x) | Additional curated reasoning samples designed to strengthen structured step-by-step problem solving and improve reasoning diversity. |
119
+
120
+
121
+ ## ⚠️ Limitations & Intended Use
122
+ - **Hallucination Risk:** While reasoning is strong, the model remains an autoregressive LLM; external facts provided during the thinking sequence may occasionally contain hallucinations if verifying real-world events.
123
+ - **Intended Scenario:** Best suited for offline analytical tasks, coding, math, and heavy logic-dependent prompting where the user needs to transparently follow the AI's internal logic.
124
+ - This model is a test version intended solely for learning and demonstration purposes, and is for academic research and technical exploration use only.
125
+
126
+ ## 🙏 Acknowledgements
127
+ Significant thanks to the [Unsloth AI](https://unsloth.ai/) team for making rapid fine-tuning of large LLM models accessible. Additionally, we acknowledge Qwen internally, and the open-source community developers producing exceptional distilled datasets.