File size: 1,636 Bytes
a537a47 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
license: apache-2.0
base_model:
- Qwen/Qwen2.5-7B-Instruct
- Qwen/Qwen2.5-Coder-7B-Instruct
- Qwen/Qwen2.5-Math-7B-Instruct
tags:
- model-soup
- model-merging
- qwen2.5
- souper-model
library_name: transformers
---
# Qwen2.5-7B-MathSoup
🍲 **Model Soup** created using weighted averaging based on [Meta's Souper-Model](https://arxiv.org/abs/2511.13254).
## Weights
- **math**: 60%
- **general**: 40%
## Expected Performance (Linear Prediction)
| Benchmark | Predicted Score |
|-----------|----------------|
| GSM8K | 88.3% |
| HumanEval | 59.5% |
*Note: Actual performance may differ due to weight interference effects.*
## Component Models
| Model | GSM8K | HumanEval |
|-------|-------|----------|
| Qwen2.5-7B-Instruct | 85.4% | 70.1% |
| Qwen2.5-Coder-7B-Instruct | 60.4% | 88.4% |
| Qwen2.5-Math-7B-Instruct | 90.3% | 52.4% |
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("researchaudio/Qwen2.5-7B-MathSoup")
tokenizer = AutoTokenizer.from_pretrained("researchaudio/Qwen2.5-7B-MathSoup")
messages = [{"role": "user", "content": "Solve: What is 15% of 80?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0]))
```
## Citation
```bibtex
@misc{soupermodel2025,
title={Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance},
author={Shalini Maiti and others},
year={2025},
url={https://arxiv.org/abs/2511.13254},
}
```
|