Instructions to use MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull")
model = AutoModelForCausalLM.from_pretrained("MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull

SGLang

How to use MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull with Docker Model Runner:
```
docker model run hf.co/MajorJalud/Qwen2.5-0.5B-Instruct-Gensyn-Swarm-reptilian_strong_gull
```

MajorJalud commited on Apr 29, 2025

Commit

504d646

verified ·

1 Parent(s): 8faeaf2

End of training

Browse files

Files changed (5) hide show

all_results.json +5 -5
model.safetensors +1 -1
train_results.json +5 -5
trainer_state.json +130 -1840
training_args.bin +1 -1

all_results.json CHANGED Viewed

@@ -1,8 +1,8 @@
 {
     "total_flos": 0.0,
-    "train_loss": 0.0,
-    "train_runtime": 5416.6971,
-    "train_samples": 280,
-    "train_samples_per_second": 0.591,
-    "train_steps_per_second": 0.037
 }

 {
     "total_flos": 0.0,
+    "train_loss": 4.377111076792062e-06,
+    "train_runtime": 498.382,
+    "train_samples": 71,
+    "train_samples_per_second": 0.642,
+    "train_steps_per_second": 0.04
 }

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ad7128a4439620f5ab29b21a75f92634edc7bf1bfb873492e22db91c77efc511
 size 1976163472

 version https://git-lfs.github.com/spec/v1
+oid sha256:824dbf6f53d3050b7e374955f4a291d9ae6c68d27acedec667a23d679b6d9c65
 size 1976163472

train_results.json CHANGED Viewed

@@ -1,8 +1,8 @@
 {
     "total_flos": 0.0,
-    "train_loss": 0.0,
-    "train_runtime": 5416.6971,
-    "train_samples": 280,
-    "train_samples_per_second": 0.591,
-    "train_steps_per_second": 0.037
 }

 {
     "total_flos": 0.0,
+    "train_loss": 4.377111076792062e-06,
+    "train_runtime": 498.382,
+    "train_samples": 71,
+    "train_samples_per_second": 0.642,
+    "train_steps_per_second": 0.04
 }

trainer_state.json CHANGED Viewed

@@ -2,1927 +2,217 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 5.714285714285714,
   "eval_steps": 500,
-  "global_step": 200,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "completion_length": 690.0,
-      "epoch": 0.05714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 8.333333333333333e-08,
       "loss": 0.0,
-      "reward": 0.6353109590709209,
-      "reward_std": 0.41491524153389037,
-      "rewards/concensus_correctness_reward_func": 0.05999999865889549,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.125,
-      "rewards/question_recreation_reward_func": 0.36677972972393036,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.08353125024586916,
       "step": 2
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.11428571428571428,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.5e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 4
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.17142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.1666666666666667e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
       "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 6
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.22857142857142856,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.99967220916408e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 8
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.2857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.997050398198976e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 10
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.34285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.991809526186423e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 12
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.4,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.983955090077444e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 14
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.45714285714285713,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.973495328090889e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
       "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 16
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.5142857142857142,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.960441211072685e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
       "rewards/concensus_correctness_reward_func": 0.0,
       "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 18
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.5714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.944806430988927e-07,
       "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
       "rewards/concensus_correctness_reward_func": 0.0,
       "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
       "step": 20
     },
     {
-      "completion_length": 1024.0,
-      "epoch": 0.6285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.926607386564898e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 22
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.6857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.905863166085075e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 24
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.7428571428571429,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.882595527372152e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 26
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.8,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.856828874966086e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 28
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.8571428571428571,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.828590234527106e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 30
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.9142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.79790922448953e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 32
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 0.9714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.7648180249961165e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 34
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.0285714285714285,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.7293513441455357e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 36
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.0857142857142856,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.691546381588369e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 38
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.1428571428571428,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.651442789509813e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 40
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.2,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.609082631040011e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 42
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.2571428571428571,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.5645103361356407e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 44
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.3142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.517772654979023e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 46
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.3714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.468918608943636e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 48
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.4285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.417999439177465e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 50
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.4857142857142858,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.365068552858115e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 52
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.5428571428571427,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.310181467176054e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 54
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.6,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.253395751104748e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 56
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.657142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.194770965018758e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 58
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.7142857142857144,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.1343685982231315e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 60
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.7714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.072252004459611e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 62
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.8285714285714287,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.0084863354573116e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 64
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.8857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.9431384725975485e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 66
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 1.9428571428571428,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.876276956764509e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 68
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.0,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.807971916455325e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 70
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.057142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.738294994224969e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 72
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.1142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.6673192715431014e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 74
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.1714285714285713,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.595119192141706e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 76
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.2285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.5217704839338905e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 78
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.2857142857142856,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.447350079585767e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 80
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.342857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.3719360358247053e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 82
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.4,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.29560745156861e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 84
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.4571428571428573,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.218444384962071e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 86
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.5142857142857142,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.1405277694064305e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 88
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.571428571428571,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.0619393286718237e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 90
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.6285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.98276149118022e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 92
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.685714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.903077303549399e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 94
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.742857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.822970343488516e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 96
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.8,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.7425246321366205e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 98
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.661824545936089e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 100
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.914285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.58095472813339e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 102
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 2.9714285714285715,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.5e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 104
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.0285714285714285,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.4190452718666105e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 106
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.085714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.3381754540639106e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 108
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.2574753678633798e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 110
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.2,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.1770296565114846e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 112
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.257142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.0969226964506005e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 114
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.314285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.0172385088197803e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 116
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.3714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.9380606713281772e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 118
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.4285714285714284,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.859472230593569e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 120
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.4857142857142858,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.7815556150379296e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 122
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.5428571428571427,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.704392548431391e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 124
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.6,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.6280639641752942e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 126
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.657142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.552649920414233e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 128
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.7142857142857144,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.47822951606611e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 130
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.7714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.4048808078582942e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 132
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.8285714285714287,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.3326807284568984e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 134
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.8857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.261705005775032e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 136
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 3.942857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.1920280835446748e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 138
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.0,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.123723043235491e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 140
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.057142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.056861527402452e-07,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 142
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.114285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 9.915136645426883e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 144
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.171428571428572,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 9.277479955403886e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 146
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.228571428571429,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 8.656314017768693e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 148
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.285714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 8.052290349812419e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 150
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.3428571428571425,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 7.46604248895252e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 152
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.4,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 6.898185328239467e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 154
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.457142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 6.349314471418849e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 156
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.514285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 5.8200056082253453e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 158
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.571428571428571,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 5.310813910563644e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 160
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.628571428571428,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.8222734502097655e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 162
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.685714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 4.35489663864359e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 164
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.742857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.90917368959989e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 166
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.8,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.485572104901868e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 168
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.857142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.08453618411631e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 170
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.914285714285715,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.7064865585446433e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 172
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 4.9714285714285715,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.3518197500388276e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 174
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.0285714285714285,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.0209077551046976e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 176
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.085714285714285,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.7140976547289438e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 178
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.142857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.4317112503391432e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 180
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.2,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.174044726278478e-08,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 182
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.257142857142857,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 9.413683391492455e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 184
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.314285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 7.339261343510206e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 186
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.371428571428572,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 5.519356901107358e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 188
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.428571428571429,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.95587889273144e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 190
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.485714285714286,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.6504671909109988e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 192
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.542857142857143,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 1.6044909922555972e-09,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 194
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.6,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 8.19047381357657e-10,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 196
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.6571428571428575,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 2.949601801023327e-10,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 198
-    },
-    {
-      "completion_length": 1024.0,
-      "epoch": 5.714285714285714,
-      "grad_norm": NaN,
-      "kl": NaN,
-      "learning_rate": 3.2779083591949474e-11,
-      "loss": 0.0,
-      "reward": 0.0,
-      "reward_std": 0.0,
-      "rewards/concensus_correctness_reward_func": 0.0,
-      "rewards/consensus_reward_func": 0.0,
-      "rewards/cumulative_reward_2": 0.0,
-      "rewards/final_correctness_reward_func": 0.0,
-      "rewards/question_recreation_reward_func": 0.0,
-      "rewards/soft_format_reward_func": 0.0,
-      "rewards/strict_format_reward_func": 0.0,
-      "rewards/xmlcount_reward_func": 0.0,
-      "step": 200
-    },
-    {
-      "epoch": 5.714285714285714,
-      "step": 200,
       "total_flos": 0.0,
-      "train_loss": 0.0,
-      "train_runtime": 5416.6971,
-      "train_samples_per_second": 0.591,
-      "train_steps_per_second": 0.037
     }
   ],
   "logging_steps": 2,
-  "max_steps": 200,
   "num_input_tokens_seen": 0,
-  "num_train_epochs": 6,
   "save_steps": 25,
   "stateful_callbacks": {
     "TrainerControl": {

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 2.2253521126760565,
   "eval_steps": 500,
+  "global_step": 20,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
+      "completion_length": 340.96875,
+      "epoch": 0.22535211267605634,
+      "grad_norm": 12.885725021362305,
+      "kl": 0.0,
+      "learning_rate": 5e-07,
       "loss": 0.0,
+      "reward": 0.3555513550527394,
+      "reward_std": 0.6679015202389564,
+      "rewards/concensus_correctness_reward_func": 0.0,
+      "rewards/consensus_reward_func": 0.125,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.0,
+      "rewards/question_recreation_reward_func": 0.22352009767200798,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.007031254004687071,
       "step": 2
     },
     {
+      "completion_length": 392.15625,
+      "epoch": 0.4507042253521127,
+      "grad_norm": 151.82925415039062,
+      "kl": 0.002852820234693354,
+      "learning_rate": 4.864543104251586e-07,
       "loss": 0.0,
+      "reward": 0.8996052537113428,
+      "reward_std": 0.8588616576162167,
+      "rewards/concensus_correctness_reward_func": 0.11956249922513962,
+      "rewards/consensus_reward_func": 0.25,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.0625,
+      "rewards/question_recreation_reward_func": 0.2896677665412426,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.17787499586120248,
       "step": 4
     },
     {
+      "completion_length": 314.15625,
+      "epoch": 0.676056338028169,
+      "grad_norm": 12.329215049743652,
+      "kl": 0.003202719522960251,
+      "learning_rate": 4.472851273490984e-07,
       "loss": 0.0,
+      "reward": 0.689017332624644,
+      "reward_std": 0.5627960833371617,
       "rewards/concensus_correctness_reward_func": 0.0,
+      "rewards/consensus_reward_func": 0.0625,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.1875,
+      "rewards/question_recreation_reward_func": 0.26289233041461557,
+      "rewards/soft_format_reward_func": 0.015625,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.16050000092945993,
       "step": 6
     },
     {
+      "completion_length": 370.6875,
+      "epoch": 0.9014084507042254,
+      "grad_norm": 17.892837524414062,
+      "kl": 0.0010311309342796449,
+      "learning_rate": 3.867370395306068e-07,
       "loss": 0.0,
+      "reward": 0.553530995734036,
+      "reward_std": 0.6493549526203424,
+      "rewards/concensus_correctness_reward_func": 0.0598750002682209,
+      "rewards/consensus_reward_func": 0.0625,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.0625,
+      "rewards/question_recreation_reward_func": 0.19462474843021482,
+      "rewards/soft_format_reward_func": 0.03125,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.1427812511101365,
       "step": 8
     },
     {
+      "completion_length": 399.4,
+      "epoch": 1.1126760563380282,
+      "grad_norm": 6.89277458190918,
+      "kl": 0.001356206873121361,
+      "learning_rate": 3.1137137178519977e-07,
       "loss": 0.0,
+      "reward": 1.131314364572366,
+      "reward_std": 1.2063223953048388,
+      "rewards/concensus_correctness_reward_func": 0.1946666677792867,
+      "rewards/consensus_reward_func": 0.13333333333333333,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.2,
+      "rewards/question_recreation_reward_func": 0.34351439370463294,
+      "rewards/soft_format_reward_func": 0.016666666666666666,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.2431333363056183,
       "step": 10
     },
     {
+      "completion_length": 247.46875,
+      "epoch": 1.3380281690140845,
+      "grad_norm": 128.0925750732422,
+      "kl": 0.002617816731799394,
+      "learning_rate": 2.2935516363191693e-07,
       "loss": 0.0,
+      "reward": 1.1680394155555405,
+      "reward_std": 1.6668975032189337,
+      "rewards/concensus_correctness_reward_func": 0.7476250007748604,
+      "rewards/consensus_reward_func": 0.0625,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.1875,
+      "rewards/question_recreation_reward_func": 0.15332059981301427,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.017093755304813385,
       "step": 12
     },
     {
+      "completion_length": 262.0,
+      "epoch": 1.563380281690141,
+      "grad_norm": 311.76129150390625,
+      "kl": 0.02655173771563568,
+      "learning_rate": 1.4957614383675767e-07,
       "loss": 0.0,
+      "reward": 0.7303614910924807,
+      "reward_std": 0.9725976219633594,
+      "rewards/concensus_correctness_reward_func": 0.12250000238418579,
+      "rewards/consensus_reward_func": 0.1875,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.125,
+      "rewards/question_recreation_reward_func": 0.15414277257514186,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.1412187535315752,
       "step": 14
     },
     {
+      "completion_length": 294.9375,
+      "epoch": 1.788732394366197,
+      "grad_norm": 7.390890121459961,
+      "kl": 0.0017569949413882568,
+      "learning_rate": 8.067960709356478e-08,
       "loss": 0.0,
+      "reward": 1.0274017243646085,
+      "reward_std": 1.1111178861610824,
+      "rewards/concensus_correctness_reward_func": 0.1848125010728836,
+      "rewards/consensus_reward_func": 0.125,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.1875,
+      "rewards/question_recreation_reward_func": 0.2963079637847841,
       "rewards/soft_format_reward_func": 0.0,
+      "rewards/strict_format_reward_func": 0.015625,
+      "rewards/xmlcount_reward_func": 0.21815624879673123,
       "step": 16
     },
     {
+      "completion_length": 334.6,
+      "epoch": 2.0,
+      "grad_norm": 25.754987716674805,
+      "kl": 0.0019916834270892044,
+      "learning_rate": 3.013156219837776e-08,
       "loss": 0.0,
+      "reward": 0.3507757396747669,
+      "reward_std": 0.4773174815811217,
       "rewards/concensus_correctness_reward_func": 0.0,
       "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.06666666666666667,
+      "rewards/question_recreation_reward_func": 0.18934240887562434,
       "rewards/soft_format_reward_func": 0.0,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.09476667046546935,
       "step": 18
     },
     {
+      "completion_length": 371.3125,
+      "epoch": 2.2253521126760565,
+      "grad_norm": 73.43103790283203,
+      "kl": 0.0033214127288374584,
+      "learning_rate": 3.4096741493194193e-09,
       "loss": 0.0,
+      "reward": 0.6189658248331398,
+      "reward_std": 0.48415553454105975,
       "rewards/concensus_correctness_reward_func": 0.0,
       "rewards/consensus_reward_func": 0.0,
       "rewards/cumulative_reward_2": 0.0,
+      "rewards/final_correctness_reward_func": 0.125,
+      "rewards/question_recreation_reward_func": 0.3122783375438303,
+      "rewards/soft_format_reward_func": 0.015625,
       "rewards/strict_format_reward_func": 0.0,
+      "rewards/xmlcount_reward_func": 0.16606249526375905,
       "step": 20
     },
     {
+      "epoch": 2.2253521126760565,
+      "step": 20,
       "total_flos": 0.0,
+      "train_loss": 4.377111076792062e-06,
+      "train_runtime": 498.382,
+      "train_samples_per_second": 0.642,
+      "train_steps_per_second": 0.04
     }
   ],
   "logging_steps": 2,
+  "max_steps": 20,
   "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
   "save_steps": 25,
   "stateful_callbacks": {
     "TrainerControl": {

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1d8c71b287fc24f80d8111d380941033f3dbf3a756b29aaded2cba8675839f68
 size 6008

 version https://git-lfs.github.com/spec/v1
+oid sha256:7ce78aed34ce77353dad5b8d351f31cb61ddc17622dbf24fe538d806a99ae374
 size 6008