File size: 4,652 Bytes
16d6935 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac 2fb3fdc 5f673ac f83b76a 2fb3fdc f83b76a 2fb3fdc f83b76a 2fb3fdc f83b76a 2fb3fdc f83b76a 2fb3fdc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | ---
base_model: unsloth/Qwen3-4B-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- qwen3
- gguf
license: apache-2.0
language:
- en
---
# Eval
<pre>
+-------------+------------+-----------------+---------------+-------+---------+---------+
| Model | Dataset | Metric | Subset | Num | Score | Cat.0 |
+=============+============+=================+===============+=======+=========+=========+
| model | gpqa | AveragePass@1 | gpqa_extended | 50 | 0.34 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | gpqa | AveragePass@1 | gpqa_main | 50 | 0.32 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | gpqa | AveragePass@1 | gpqa_diamond | 50 | 0.32 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | gpqa | AveragePass@1 | OVERALL | 150 | 0.3267 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | gsm8k | AverageAccuracy | main | 50 | 0.76 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Act.EM | in_domain | 42 | 0.2619 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Act.EM | out_of_domain | 47 | 0.3617 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Act.EM | OVERALL | 89 | 0.3146 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| Qwen3-4B-toolcall.Q4_K_M.gguf | tool_bench | Plan.EM | in_domain | 0 | 0 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Plan.EM | out_of_domain | 0 | 0 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Plan.EM | OVERALL | 0 | 0 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | F1 | in_domain | 42 | 0.2095 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | F1 | out_of_domain | 47 | 0.2527 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | F1 | OVERALL | 89 | 0.2323 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | HalluRate | in_domain | 42 | 0.119 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | HalluRate | out_of_domain | 47 | 0.0851 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | HalluRate | OVERALL | 89 | 0.1011 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Rouge-L | in_domain | 42 | 0.0394 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Rouge-L | out_of_domain | 47 | 0.0676 | default |
+-------------+------------+-----------------+---------------+-------+---------+---------+
| model | tool_bench | Rouge-L | OVERALL | 89 | 0.0543 | - |
+-------------+------------+-----------------+---------------+-------+---------+---------+
</pre>
# Use this model
## with llama-cli
- <code>llama-cli -m Qwen3-4B-toolcall.Q4_K_M.gguf</code>
## with ollama
- edit a makefile named(Qwen3-4B-toolcall.Q4_K_M.txt) like:
- <pre>
FROM ./Qwen3-4B-toolcall.Q4_K_M
TEMPLATE """<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
</pre>
- then create a model using ollama
- <code>ollama create Qwen3-4B-toolcall.Q4_K_M -f Qwen3-4B-toolcall.Q4_K_M.txt</code>
- then run it
- <code>ollama run Qwen3-4B-toolcall.Q4_K_M</code> |