Add model card, metadata, and links to paper/code

This PR adds a comprehensive model card for the model, including:
- Metadata tags: `pipeline_tag` set to `text-generation`, `library_name: transformers`, and `license: apache-2.0`.
- Information about the paper [Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO](https://huggingface.co/papers/2605.30789).
- Links to the project page and official GitHub repository.

README.md

@@ -0,0 +1,30 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO
+
+This repository contains the model checkpoints presented in the paper [Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO](https://huggingface.co/papers/2605.30789) (ICML 2026).
+
+* **Project Page:** [S2L-PO Project Page](https://qishisuren123.github.io/S2L-PO)
+* **GitHub Repository:** [GitHub - S2L-PO](https://github.com/qishisuren123/S2L-PO)
+
+## Introduction
+
+S2L-PO (Small-to-Large Policy Optimization) is a post-training framework designed to enhance rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs. Instead of injecting token-level randomness (which can cause logical inconsistencies over long reasoning chains), S2L-PO leverages a fixed, smaller model from the same family as an explorer to guide a larger target model during the rollout phase.
+
+## Citation
+
+If you find this work useful, please cite:
+
+```bibtex
+@inproceedings{ren2026s2lpo,
+  title     = {Smaller Models are Natural Explorers for Policy-Level Diversity in {GRPO}},
+  author    = {Ren, Yiming and Xu, Yiran and Lin, Zicheng and Shi, Chufan and Chen, Yukang and
+               Wang, Dingdong and Wu, Tianhe and Wang, Jujie and Yang, Yujiu and Qiao, Yu and Chu, Ruihang},
+  booktitle = {International Conference on Machine Learning (ICML)},
+  year      = {2026},
+}
+```