Upload SciJudge-30B-2605 weights and model card (#1)

- Upload SciJudge-30B-2605 weights and model card (eb1640dc953ba134aa6adecec3a682d28a52222b)
- Update SciJudge-30B-2605 model card links and tags (bd5cfa59028e9c896f199dfd2e02215afce8ef58)


Co-authored-by: Mingzhe Li <Mubuky@users.noreply.huggingface.co>

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,93 @@
+---
+language:
+- en
+license: apache-2.0
+base_model: Qwen/Qwen3-30B-A3B-Instruct-2507
+datasets:
+- OpenMOSS-Team/SciJudgeBench
+tags:
+- scientific-taste
+- GRPO
+pipeline_tag: text-generation
+library_name: transformers
+model-index:
+- name: SciJudge-30B-2605
+  results:
+  - task:
+      type: text-generation
+      name: Citation-based pairwise paper judgment
+    dataset:
+      name: SciJudgeBench MAIN_1000
+      type: OpenMOSS-Team/SciJudgeBench
+      split: test
+    metrics:
+    - type: accuracy
+      name: Avg. accuracy
+      value: 82.7
+---
+
+# SciJudge-30B-2605
+
+SciJudge-30B-2605 is a Qwen3-30B-A3B-Instruct-2507 MoE model fine-tuned for scientific paper evaluation. Given two papers' titles, abstracts, and publication dates, it predicts which paper has higher citation impact.
+
+This release is part of [AI Can Learn Scientific Taste](https://arxiv.org/abs/2603.14473). The companion smaller model is [SciJudge-4B-2605](https://huggingface.co/OpenMOSS-Team/SciJudge-4B-2605), and the benchmark dataset is [SciJudgeBench](https://huggingface.co/datasets/OpenMOSS-Team/SciJudgeBench).
+
+Resources: [Project page](https://tongjingqi.github.io/AI-Can-Learn-Scientific-Taste/) and [GitHub repository](https://github.com/tongjingqi/AI-Can-Learn-Scientific-Taste).
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "OpenMOSS-Team/SciJudge-30B-2605"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant. You first think about the reasoning process in your mind and then provide the user with the answer."},
+    {"role": "user", "content": "Today is 2025-12-10. Based on the titles, abstracts, and publication dates of the following two papers A and B, determine which paper has a higher citation count.\nShow your reasoning process in <reason> </reason> tags. And return the final answer in <answer> </answer> tags. The final answer should contain only 'A' or 'B'.\n\nPaper A:\nTitle: ...\nAbstract: ...\nDate: ...\n\nPaper B:\nTitle: ...\nAbstract: ...\nDate: ..."}
+]
+
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt").to(model.device)
+outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.7, top_p=0.8, top_k=20)
+response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
+print(response)
+```
+
+## Training Details
+
+- **Base model:** [Qwen/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507)
+- **Training method:** GRPO with DAPO loss
+- **Reward:** external preference reward for citation-based pairwise judgment
+- **Training data:** 720,341 preference pairs from SciJudgeBench
+- **Precision:** bfloat16
+- **KL coefficient:** 0.03
+
+## Evaluation
+
+Accuracy on the SciJudgeBench `test` split, the 1,000-example MAIN_1000 in-domain evaluation set:
+
+| Model | CS | Math | Physics | Others | Avg. |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Qwen3-30B-A3B-Instruct-2507 | 73.39 | 79.90 | 64.41 | 63.94 | 69.7 |
+| SciJudge-30B-2605 | 83.47 | 89.71 | 81.18 | 77.40 | 82.7 |
+
+## Citation
+
+```bibtex
+@misc{tong2026ailearnscientifictaste,
+      title={AI Can Learn Scientific Taste}, 
+      author={Jingqi Tong and Mingzhe Li and Hangcheng Li and Yongzhuo Yang and Yurong Mou and Weijie Ma and Zhiheng Xi and Hongji Chen and Xiaoran Liu and Qinyuan Cheng and Ming Zhang and Qiguang Chen and Weifeng Ge and Qipeng Guo and Tianlei Ying and Tianxiang Sun and Yining Zheng and Xinchi Chen and Jun Zhao and Ning Ding and Xuanjing Huang and Yugang Jiang and Xipeng Qiu},
+      year={2026},
+      eprint={2603.14473},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2603.14473}, 
+}
+```

chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,41 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "decoder_sparse_step": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 262144,
+  "max_window_layers": 48,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 768,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts_per_tok": 8,
+  "num_hidden_layers": 48,
+  "num_key_value_heads": 4,
+  "num_local_experts": 128,
+  "output_router_logits": false,
+  "pad_token_id": null,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.4",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

model-00001-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8d3577cc99f4dfcefdaa0cac3bc15012cb624d80108c7ac8a34d2ea48199098
+size 4997189144

model-00002-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3dc8be63c496995f5fc5d1ecc799de92940569307717386ea4d0009358b56917
+size 4997741584

model-00003-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce04299e72df736c52bc84211d1dba17ca20c2036df002dc1fb60bd29ba0506b
+size 4997742176

model-00004-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae0030b5e5f52e9ccb5f8dde91f917ad42f6da0ed458c32577aad3e73765d31a
+size 4997743160

model-00005-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:154a9b8394c478d649bd8ceddab89699370f175e04180a628ea1d66f901e710e
+size 4997743160

model-00006-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5c79b874cf36ee974453e115dcdc5b0d400fabad3b5fac3aa15ad4ccf5604af
+size 4997743160

model-00007-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a1465add3df3f36a980e5488ec08c0ed5fde79261d70f5c44eca4319d4abdfd
+size 4997743160

model-00008-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ee6b88a8e9f43c4b16a0b09c29095b20964323926b2b72a3312958a80e51974
+size 4997743160

model-00009-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f02cff7e37fdd8ba72ed5b4229ba88c4174c3fb4b9bd238afa439b28d801ba6
+size 4997743160

model-00010-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:781681cb315460b9e2a37c64dbd75b98dd4f509577281f9985b6f80653ae9f38
+size 4997743160

model-00011-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:510811a7a4f0c4f54e31c9c91ec0722827209925dae42a808eff5a75d520ae7a
+size 4997743160

model-00012-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e1eb0a5fbfbb072eee2d40e58977649461cdf05c954a4ac460f48cdca8e57b1
+size 4997743160

model-00013-of-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a41cf139f4b744c8c1fb706906f4b788500f98a877be32d485487f18b60fff6b
+size 1094216064

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
+size 11422650

tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": true,
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}