Add model card for Qwen3-14B-S2L-PO-4Bexplorer

README.md

@@ -0,0 +1,44 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+base_model: Qwen/Qwen3-14B
+tags:
+- grpo
+- reinforcement-learning
+- s2l-po
+---
+
+# Qwen3-14B-S2L-PO-4Bexplorer
+
+This repository contains the Qwen3-14B learner model trained using the **S2L-PO (Small-to-Large Policy Optimization)** framework with a **Qwen3-4B** explorer. 
+
+S2L-PO is presented in the paper [Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO](https://huggingface.co/papers/2605.30789).
+
+* **Project Page:** [https://qishisuren123.github.io/S2L-PO/](https://qishisuren123.github.io/S2L-PO/)
+* **Repository:** [https://github.com/qishisuren123/S2L-PO](https://github.com/qishisuren123/S2L-PO)
+* **Paper:** [arXiv:2605.30789](https://huggingface.co/papers/2605.30789)
+
+## Introduction
+
+S2L-PO enhances rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs by using a smaller model within the same family as a natural explorer to generate a portion of the rollout groups. This introduces policy-level diversity, providing structured exploration signals for gradient estimation, resulting in faster convergence and higher final performance on reasoning tasks.
+
+For this specific checkpoint, a fixed **Qwen3-4B-Base** explorer is used to guide the training of the **Qwen3-14B-Base** learner.
+
+## Environment & Usage
+
+Please refer to the [GitHub Repository](https://github.com/qishisuren123/S2L-PO) for installation instructions, environment setup, and evaluation scripts on mathematical reasoning benchmarks.
+
+## Citation
+
+If you find this work useful, please cite the paper:
+
+```bibtex
+@inproceedings{ren2026s2lpo,
+  title     = {Smaller Models are Natural Explorers for Policy-Level Diversity in {GRPO}},
+  author    = {Ren, Yiming and Xu, Yiran and Lin, Zicheng and Shi, Chufan and Chen, Yukang and
+               Wang, Dingdong and Wu, Tianhe and Wang, Jujie and Yang, Yujiu and Qiao, Yu and Chu, Ruihang},
+  booktitle = {International Conference on Machine Learning (ICML)},
+  year      = {2026},
+}
+```