cassettesgoboom/TinyR1-32B-Preview-Q3_K_L-GGUF

This model was converted to GGUF format from qihoo360/TinyR1-32B-Preview using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

Original Model card:

license: apache-2.0 library_name: transformers base_model: - deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

Model Name: Tiny-R1-32B-Preview
Title: SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.

Introduction

We introduce our first-generation reasoning model, Tiny-R1-32B-Preview, which outperforms the 70B model Deepseek-R1-Distill-Llama-70B and nearly matches the full R1 model in math.

Evaluation

Model	Math (AIME 2024)	Coding (LiveCodeBench)	Science (GPQA-Diamond)
Deepseek-R1-Distill-Qwen-32B	72.6	57.2	62.1
Deepseek-R1-Distill-Llama-70B	70.0	57.5	65.2
Deepseek-R1	79.8	65.9	71.5
Tiny-R1-32B-Preview (Ours)	78.1	61.6	65.0

All scores are reported as pass@1. For AIME 2024, we sample 16 responses, and for GPQA-Diamond, we sample 4 responses, both using average overall accuracy for stable evaluation.

Approach

Model	Math (AIME 2024)	Coding (LiveCodeBench)	Science (GPQA-Diamond)
Math-Model (Ours)	73.1	-	-
Code-Model (Ours)	-	63.4	-
Science-Model (Ours)	-	-	64.5
Tiny-R1-32B-Preview (Ours)	78.1	61.6	65.0

We applied supervised fine-tuning (SFT) to Deepseek-R1-Distill-Qwen-32B across three target domains—Mathematics, Code, and Science — using the 360-LLaMA-Factory training framework to produce three domain-specific models. We used questions from open-source data as seeds, and used DeepSeek-R1 to generate responses for mathematics, coding, and science tasks separately, creating specialized models for each domain. Building on this, we leveraged the Mergekit tool from the Arcee team to combine multiple models, creating Tiny-R1-32B-Preview, which demonstrates strong overall performance.

Data

1. Math

58.3k CoT trajectories from open-r1/OpenR1-Math-220k, default subset

2. Coding

19k CoT trajectories open-thoughts/OpenThoughts-114k, coding subset

3. Science

We used R1 to generate 8 CoT trajectories on 7.6k seed examples, and got 60.8k CoT trajectories in total; the seed examples are as follows:

• 2.7k seed examples from simplescaling/data_ablation_full59K, science and health science subset
• 4.9k seed examples from open-thoughts/OpenThoughts-114k, science subset

Open Source Plan

We will publish a technical report as soon as possible and open-source our training and evaluation code, selected training data, and evaluation logs. Having benefited immensely from the open-source community, we are committed to giving back in every way we can.

Contributors

360 Team: Lin Sun, Guangxiang Zhao, Xiaoqi Jian, Weihong Lin, Yongfu Zhu, Change Jia, Linglin Zhang, Jinzhu Wu, Sai-er Hu, Xiangzheng Zhang

PKU Team: Yuhan Wu, Zihan Jiang, Wenrui Liu, Junting Zhou, Bin Cui, Tong Yang

Citation

@misc{tinyr1proj,
      title={SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.}, 
      author={TinyR1 Team},
      year={2025},
      eprint={},
      archivePrefix={},
      primaryClass={},
      url={https://huggingface.co/qihoo360/TinyR1-32B-Preview}, 
}

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo cassettesgoboom/TinyR1-32B-Preview-Q3_K_L-GGUF --hf-file tinyr1-32b-preview-q3_k_l.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo cassettesgoboom/TinyR1-32B-Preview-Q3_K_L-GGUF --hf-file tinyr1-32b-preview-q3_k_l.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo cassettesgoboom/TinyR1-32B-Preview-Q3_K_L-GGUF --hf-file tinyr1-32b-preview-q3_k_l.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo cassettesgoboom/TinyR1-32B-Preview-Q3_K_L-GGUF --hf-file tinyr1-32b-preview-q3_k_l.gguf -c 2048