Remove redundant checkpoint-125/ (final checkpoint already at repo root)

checkpoint-125/README.md

@@ -1,208 +0,0 @@
----
-base_model: unsloth/Seed-OSS-36B-Instruct
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- axolotl
-- base_model:adapter:unsloth/Seed-OSS-36B-Instruct
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.19.1

checkpoint-125/adapter_config.json

@@ -1,48 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "unsloth/Seed-OSS-36B-Instruct",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": false,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 64,
-  "lora_bias": false,
-  "lora_dropout": 0.0,
-  "lora_ga_config": null,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": null,
-  "peft_type": "LORA",
-  "peft_version": "0.19.1",
-  "qalora_group_size": 16,
-  "r": 32,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "down_proj",
-    "v_proj",
-    "up_proj",
-    "o_proj",
-    "gate_proj",
-    "k_proj"
-  ],
-  "target_parameters": [],
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_bdlora": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": true
-}

checkpoint-125/adapter_model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:62b29b555e010525ec4a903f2946d0d6c8465e2c4a9d2af469ce048a3e55adc6
-size 1157749552

checkpoint-125/chat_template.jinja

@@ -1,173 +0,0 @@
-{# Unsloth Chat template fixes #}
-{# ----------‑‑‑ special token variables ‑‑‑---------- #}
-{%- set bos_token              = '<seed:bos>'               -%}
-{%- set eos_token              = '<seed:eos>'               -%}
-{%- set pad_token              = '<seed:pad>'               -%}
-{%- set toolcall_begin_token   = '<seed:tool_call>'         -%}
-{%- set toolcall_end_token     = '</seed:tool_call>'        -%}
-{%- set think_begin_token      = '<seed:think>'             -%}
-{%- set think_end_token        = '</seed:think>'            -%}
-{%- set budget_begin_token     = '<seed:cot_budget_reflect>'-%}
-{%- set budget_end_token       = '</seed:cot_budget_reflect>'-%}
-{# -------------- reflection-interval lookup -------------- #}
-{%- if not thinking_budget is defined %}
-{%- set thinking_budget = -1 -%}
-{%- endif -%}
-{%- set budget_reflections_v05 = {
-     0:      0,
-     512:    128,
-     1024:   256,
-     2048:   512,
-     4096:   512,
-     8192:   1024,
-     16384:  1024
-} -%}
-{# 找到 “大于等于 thinking_budget” 的第一个档位 #}
-{%- set ns = namespace(interval = None) -%}
-{%- for k, v in budget_reflections_v05 | dictsort -%}
-    {%- if ns.interval is none and thinking_budget <= k -%}
-        {%- set ns.interval = v -%}
-    {%- endif -%}
-{%- endfor -%}
-{# 若超过最大档位，则用最后一个档位的值 #}
-{%- if ns.interval is none -%}
-    {%- set ns.interval = budget_reflections_v05[16384] -%}
-{%- endif -%}
-{# ---------- 预处理 system 消息 ---------- #}
-{%- if messages[0]["role"] == "system" %}
-{%- set system_message = messages[0]["content"] %}
-{%- set loop_messages = messages[1:] %}
-{%- else %}
-{%- set loop_messages = messages %}
-{%- endif %}
-{# ---------- 确保 tools 存在 ---------- #}
-{%- if not tools is defined or tools is none %}
-{%- set tools = [] %}
-{%- endif %}
-{# tools2doc.jinja #}
-{%- macro py_type(t) -%}
-    {%- if t == "string" -%}str
-    {%- elif t in ("number", "integer") -%}int
-    {%- elif t == "boolean" -%}bool
-    {%- elif t == "array" -%}list
-    {%- else -%}Any{%- endif -%}
-{%- endmacro -%}
-{# ---------- 输出 system 块 ---------- #}
-{%- if system_message is defined %}
-{{ bos_token + "system\n" + system_message }}
-{%- else %}
-{%- if tools is iterable and tools | length > 0 %}
-{{ bos_token + "system\nYou are Doubao, a helpful AI assistant. You may call one or more functions to assist with the user query." }}
-{%- endif %}
-{%- endif %}
-{%- if use_json_tooldef is defined and use_json_tooldef %}
-
-{{"Tool List:\nYou are authorized to use the following tools (described in JSON Schema format). Before performing any task, you must decide how to call them based on the descriptions and parameters of these tools."}}
-{{ tools | tojson|string }}
-{%- else %}
-{%- for item in tools if item.type == "function" %}
-
-
-Function:
-def {{ item.function.name }}(
-{%- for name, spec in item.function.parameters.properties.items() %}
-        {{- name }}: {{ py_type(spec.type) }}{% if not loop.last %},{% endif %}
-{%- endfor %}):
-    """
-    {{ item.function.description | trim }}
-
-    {# ---------- Args ---------- #}
-    {%- if item.function.parameters.properties %}
-    Args:
-    {%- for name, spec in item.function.parameters.properties.items() %}
-
-    - {{ name }} ({{ py_type(spec.type) }})
-      {%- if name in item.function.parameters.required %} [必填]{% else %} [选填]{% endif %}:
-      {{- " " ~ (spec.description or "") }}
-    {%- endfor %}
-    {%- endif %}
-
-    {# ---------- Returns ---------- #}
-    {%- if item.function.returns is defined
-          and item.function.returns.properties is defined
-          and item.function.returns.properties %}
-    Returns:
-    {%- for name, spec in item.function.returns.properties.items() %}
-
-    - {{ name }} ({{ py_type(spec.type) }}):
-      {{- " " ~ (spec.description or "") }}
-    {%- endfor %}
-    {%- endif %}
-
-    """
-{%- endfor %}
-{%- endif %}
-{%- if tools is iterable and tools | length > 0 %}
-
-{{"工具调用请遵循如下格式:\n<seed:tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>value_1</parameter>\n<parameter=example_parameter_2>This is the value for the second parameter\nthat can span\nmultiple lines</parameter>\n</function>\n</seed:tool_call>\n"}}
-{%- endif %}
-{# 结束 system 块行尾 #}
-{%- if system_message is defined or tools is iterable and tools | length > 0 %}
-{{ eos_token }}
-{%- endif %}
-{# ---------- Thinking Budget ---------- #}
-{%- if thinking_budget is defined %}
-{%- if thinking_budget == 0 %}
-{{ bos_token+"system" }}
-{{ "You are an intelligent assistant that can answer questions in one step without the need for reasoning and thinking, that is, your thinking budget is 0. Next, please skip the thinking process and directly start answering the user's questions." }}
-{{ eos_token }}
-{%- elif not thinking_budget == -1 %}
-{{ bos_token+"system" }}
-{{ "You are an intelligent assistant with reflective ability. In the process of thinking and reasoning, you need to strictly follow the thinking budget, which is "}}{{thinking_budget}}{{". That is, you need to complete your thinking within "}}{{thinking_budget}}{{" tokens and start answering the user's questions. You will reflect on your thinking process every "}}{{ns.interval}}{{" tokens, stating how many tokens have been used and how many are left."}}
-{{ eos_token }}
-{%- endif %}
-{%- endif %}
-{# ---------- 逐条写出历史消息 ---------- #}
-{%- for message in loop_messages %}
-{%- if message.role == "assistant"
-  and message.tool_calls is defined
-  and message.tool_calls is iterable
-  and message.tool_calls | length > 0 %}
-{{ bos_token + message.role }}
-{%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
-{{ "\n" + think_begin_token + message.reasoning_content | trim + think_end_token }}
-{%- endif %}
-{%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
-{{ "\n" + message.content | trim + "\n" }}
-{%- endif %}
-{%- for tool_call in message.tool_calls %}
-{%- if tool_call.function is defined %}{% set tool_call = tool_call.function %}{% endif %}
-{{ "\n" + toolcall_begin_token + "\n<function=" + tool_call.name + ">\n" }}
-{%- if tool_call.arguments is defined and tool_call.arguments is mapping %}
-{%- for arg_name, arg_value in tool_call.arguments | items %}
-{{ "<parameter=" + arg_name + ">" }}
-{%- set arg_value = arg_value if arg_value is string else arg_value | string %}
-{{ arg_value+"</parameter>\n" }}
-{%- endfor %}
-{%- endif %}
-{{ "</function>\n" + toolcall_end_token }}
-{%- endfor %}
-{{ eos_token }}
-{%- elif message.role in ["user", "system"] %}
-{{ bos_token + message.role + "\n" + message.content + eos_token }}
-{%- elif message.role == "assistant" %}
-{{ bos_token + message.role }}
-{%- if message.reasoning_content is defined and message.reasoning_content is string and message.reasoning_content | trim | length > 0 %}
-{{ "\n" + think_begin_token + message.reasoning_content | trim + think_end_token }}
-{%- endif %}
-{%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
-{{ "\n" + message.content | trim + eos_token }}
-{%- endif %}
-{# 包括 tool 角色，在这个逻辑 #}
-{%- else %}
-{{ bos_token + message.role + "\n" + message.content + eos_token }}
-{%- endif %}
-{%- endfor %}
-{# ---------- 控制模型开始续写 ---------- #}
-{%- if add_generation_prompt %}
-{{ bos_token+"assistant\n" }}
-{%- if thinking_budget == 0 %}
-{{ think_begin_token+budget_begin_token }}
-{%- endif %}
-{%- endif %}
-{# Copyright 2025-present Unsloth. Apache 2.0 License. #}

checkpoint-125/optimizer.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:2761cb464920da04f5a5897c2cbd4ea994654b98921cfd3297da9a8b617aaf01
-size 588910693

checkpoint-125/rng_state.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:79c0ae34067d024c5e8555a9080472b0131751f6a800c64d9db73f988d96ec6b
-size 14645

checkpoint-125/scheduler.pt

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5e6c88ca0d14c2a604a396d9cb3e8bff73d807848a69a247288564af35d22ad8
-size 1465

checkpoint-125/tokenizer.json

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:f6bd848f52451824a3033a9f1e67eea5b399a13c90f845a332d3a29537e05827
-size 11883696

checkpoint-125/tokenizer_config.json

@@ -1,13 +0,0 @@
-{
-  "backend": "tokenizers",
-  "bos_token": "<seed:bos>",
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<seed:eos>",
-  "is_local": false,
-  "local_files_only": false,
-  "model_max_length": 1000000000000000019884624838656,
-  "pad_token": "<seed:pad>",
-  "padding_side": "left",
-  "tokenizer_class": "TokenizersBackend",
-  "unk_token": null
-}

checkpoint-125/tokens_state.json

@@ -1 +0,0 @@
-{"total": 79464, "trainable": 3000}

checkpoint-125/trainer_state.json

@@ -1,1784 +0,0 @@
-{
-  "best_global_step": null,
-  "best_metric": null,
-  "best_model_checkpoint": null,
-  "epoch": 1.0,
-  "eval_steps": 500,
-  "global_step": 125,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "epoch": 0.008,
-      "grad_norm": 144.0395965576172,
-      "learning_rate": 0.0,
-      "loss": 18.34714126586914,
-      "memory/device_reserved (GiB)": 70.99,
-      "memory/max_active (GiB)": 70.28,
-      "memory/max_allocated (GiB)": 70.28,
-      "ppl": 92909946.60015,
-      "step": 1,
-      "tokens/total": 616,
-      "tokens/train_per_sec_per_gpu": 4.4,
-      "tokens/trainable": 24
-    },
-    {
-      "epoch": 0.016,
-      "grad_norm": 133.71864318847656,
-      "learning_rate": 1.0000000000000002e-06,
-      "loss": 18.194067001342773,
-      "memory/device_reserved (GiB)": 71.02,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 79722865.82877,
-      "step": 2,
-      "tokens/total": 1236,
-      "tokens/train_per_sec_per_gpu": 10.44,
-      "tokens/trainable": 48
-    },
-    {
-      "epoch": 0.024,
-      "grad_norm": 158.28506469726562,
-      "learning_rate": 2.0000000000000003e-06,
-      "loss": 18.166255950927734,
-      "memory/device_reserved (GiB)": 71.02,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 77536236.35562,
-      "step": 3,
-      "tokens/total": 1864,
-      "tokens/train_per_sec_per_gpu": 11.18,
-      "tokens/trainable": 72
-    },
-    {
-      "epoch": 0.032,
-      "grad_norm": 144.250244140625,
-      "learning_rate": 3e-06,
-      "loss": 18.05037498474121,
-      "memory/device_reserved (GiB)": 71.02,
-      "memory/max_active (GiB)": 70.83,
-      "memory/max_allocated (GiB)": 70.83,
-      "ppl": 69052316.45379,
-      "step": 4,
-      "tokens/total": 2484,
-      "tokens/train_per_sec_per_gpu": 11.33,
-      "tokens/trainable": 96
-    },
-    {
-      "epoch": 0.04,
-      "grad_norm": 140.51007080078125,
-      "learning_rate": 4.000000000000001e-06,
-      "loss": 16.774911880493164,
-      "memory/device_reserved (GiB)": 71.02,
-      "memory/max_active (GiB)": 70.77,
-      "memory/max_allocated (GiB)": 70.77,
-      "ppl": 19286421.95222,
-      "step": 5,
-      "tokens/total": 3072,
-      "tokens/train_per_sec_per_gpu": 11.06,
-      "tokens/trainable": 120
-    },
-    {
-      "epoch": 0.048,
-      "grad_norm": 143.050048828125,
-      "learning_rate": 5e-06,
-      "loss": 16.083608627319336,
-      "memory/device_reserved (GiB)": 71.02,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 9661008.76079,
-      "step": 6,
-      "tokens/total": 3704,
-      "tokens/train_per_sec_per_gpu": 8.78,
-      "tokens/trainable": 144
-    },
-    {
-      "epoch": 0.056,
-      "grad_norm": 153.84962463378906,
-      "learning_rate": 6e-06,
-      "loss": 13.815560340881348,
-      "memory/device_reserved (GiB)": 71.08,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 1000049.78416,
-      "step": 7,
-      "tokens/total": 4332,
-      "tokens/train_per_sec_per_gpu": 11.63,
-      "tokens/trainable": 168
-    },
-    {
-      "epoch": 0.064,
-      "grad_norm": 129.7324676513672,
-      "learning_rate": 7e-06,
-      "loss": 12.381418228149414,
-      "memory/device_reserved (GiB)": 71.08,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 238331.59235,
-      "step": 8,
-      "tokens/total": 4960,
-      "tokens/train_per_sec_per_gpu": 11.67,
-      "tokens/trainable": 192
-    },
-    {
-      "epoch": 0.072,
-      "grad_norm": 72.0333251953125,
-      "learning_rate": 8.000000000000001e-06,
-      "loss": 10.207687377929688,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.93,
-      "memory/max_allocated (GiB)": 70.93,
-      "ppl": 27110.79801,
-      "step": 9,
-      "tokens/total": 5620,
-      "tokens/train_per_sec_per_gpu": 10.99,
-      "tokens/trainable": 216
-    },
-    {
-      "epoch": 0.08,
-      "grad_norm": 68.34107208251953,
-      "learning_rate": 9e-06,
-      "loss": 8.610408782958984,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 5488.49182,
-      "step": 10,
-      "tokens/total": 6224,
-      "tokens/train_per_sec_per_gpu": 11.64,
-      "tokens/trainable": 240
-    },
-    {
-      "epoch": 0.088,
-      "grad_norm": 67.72747039794922,
-      "learning_rate": 1e-05,
-      "loss": 6.326023101806641,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 558.92936,
-      "step": 11,
-      "tokens/total": 6872,
-      "tokens/train_per_sec_per_gpu": 11.19,
-      "tokens/trainable": 264
-    },
-    {
-      "epoch": 0.096,
-      "grad_norm": 55.07128143310547,
-      "learning_rate": 9.913043478260871e-06,
-      "loss": 4.50588846206665,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 90.54876,
-      "step": 12,
-      "tokens/total": 7520,
-      "tokens/train_per_sec_per_gpu": 11.63,
-      "tokens/trainable": 288
-    },
-    {
-      "epoch": 0.104,
-      "grad_norm": 51.09846878051758,
-      "learning_rate": 9.82608695652174e-06,
-      "loss": 3.78535795211792,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.77,
-      "memory/max_allocated (GiB)": 70.77,
-      "ppl": 44.05144,
-      "step": 13,
-      "tokens/total": 8140,
-      "tokens/train_per_sec_per_gpu": 11.33,
-      "tokens/trainable": 312
-    },
-    {
-      "epoch": 0.112,
-      "grad_norm": 51.17552185058594,
-      "learning_rate": 9.73913043478261e-06,
-      "loss": 2.8303725719451904,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 16.95178,
-      "step": 14,
-      "tokens/total": 8776,
-      "tokens/train_per_sec_per_gpu": 11.66,
-      "tokens/trainable": 336
-    },
-    {
-      "epoch": 0.12,
-      "grad_norm": 46.39084243774414,
-      "learning_rate": 9.652173913043478e-06,
-      "loss": 1.811977744102478,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 6.12254,
-      "step": 15,
-      "tokens/total": 9368,
-      "tokens/train_per_sec_per_gpu": 9.88,
-      "tokens/trainable": 360
-    },
-    {
-      "epoch": 0.128,
-      "grad_norm": 23.89866828918457,
-      "learning_rate": 9.565217391304349e-06,
-      "loss": 1.1055235862731934,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 3.02081,
-      "step": 16,
-      "tokens/total": 10032,
-      "tokens/train_per_sec_per_gpu": 11.56,
-      "tokens/trainable": 384
-    },
-    {
-      "epoch": 0.136,
-      "grad_norm": 14.031519889831543,
-      "learning_rate": 9.478260869565217e-06,
-      "loss": 0.8809686899185181,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.41324,
-      "step": 17,
-      "tokens/total": 10668,
-      "tokens/train_per_sec_per_gpu": 11.51,
-      "tokens/trainable": 408
-    },
-    {
-      "epoch": 0.144,
-      "grad_norm": 8.491016387939453,
-      "learning_rate": 9.391304347826087e-06,
-      "loss": 0.7867892980575562,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.19633,
-      "step": 18,
-      "tokens/total": 11308,
-      "tokens/train_per_sec_per_gpu": 11.61,
-      "tokens/trainable": 432
-    },
-    {
-      "epoch": 0.152,
-      "grad_norm": 11.222015380859375,
-      "learning_rate": 9.304347826086956e-06,
-      "loss": 0.9185903072357178,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.93,
-      "memory/max_allocated (GiB)": 70.93,
-      "ppl": 2.50576,
-      "step": 19,
-      "tokens/total": 11944,
-      "tokens/train_per_sec_per_gpu": 11.58,
-      "tokens/trainable": 456
-    },
-    {
-      "epoch": 0.16,
-      "grad_norm": 47.0161018371582,
-      "learning_rate": 9.217391304347826e-06,
-      "loss": 1.2205023765563965,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 3.38889,
-      "step": 20,
-      "tokens/total": 12572,
-      "tokens/train_per_sec_per_gpu": 11.65,
-      "tokens/trainable": 480
-    },
-    {
-      "epoch": 0.168,
-      "grad_norm": 36.03596115112305,
-      "learning_rate": 9.130434782608697e-06,
-      "loss": 0.8893774747848511,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.9,
-      "memory/max_allocated (GiB)": 70.9,
-      "ppl": 2.43361,
-      "step": 21,
-      "tokens/total": 13244,
-      "tokens/train_per_sec_per_gpu": 11.63,
-      "tokens/trainable": 504
-    },
-    {
-      "epoch": 0.176,
-      "grad_norm": 15.805229187011719,
-      "learning_rate": 9.043478260869565e-06,
-      "loss": 1.0288347005844116,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.78,
-      "memory/max_allocated (GiB)": 70.78,
-      "ppl": 2.7978,
-      "step": 22,
-      "tokens/total": 13872,
-      "tokens/train_per_sec_per_gpu": 11.18,
-      "tokens/trainable": 528
-    },
-    {
-      "epoch": 0.184,
-      "grad_norm": 15.826133728027344,
-      "learning_rate": 8.956521739130435e-06,
-      "loss": 1.0294897556304932,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.79964,
-      "step": 23,
-      "tokens/total": 14488,
-      "tokens/train_per_sec_per_gpu": 9.73,
-      "tokens/trainable": 552
-    },
-    {
-      "epoch": 0.192,
-      "grad_norm": 11.43124771118164,
-      "learning_rate": 8.869565217391306e-06,
-      "loss": 0.6887595653533936,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.99124,
-      "step": 24,
-      "tokens/total": 15128,
-      "tokens/train_per_sec_per_gpu": 11.61,
-      "tokens/trainable": 576
-    },
-    {
-      "epoch": 0.2,
-      "grad_norm": 12.793842315673828,
-      "learning_rate": 8.782608695652174e-06,
-      "loss": 0.781067967414856,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.1838,
-      "step": 25,
-      "tokens/total": 15748,
-      "tokens/train_per_sec_per_gpu": 11.62,
-      "tokens/trainable": 600
-    },
-    {
-      "epoch": 0.208,
-      "grad_norm": 21.858287811279297,
-      "learning_rate": 8.695652173913044e-06,
-      "loss": 0.9895874857902527,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 2.69012,
-      "step": 26,
-      "tokens/total": 16392,
-      "tokens/train_per_sec_per_gpu": 11.6,
-      "tokens/trainable": 624
-    },
-    {
-      "epoch": 0.216,
-      "grad_norm": 13.455280303955078,
-      "learning_rate": 8.608695652173915e-06,
-      "loss": 0.7383148670196533,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.78,
-      "memory/max_allocated (GiB)": 70.78,
-      "ppl": 2.09241,
-      "step": 27,
-      "tokens/total": 17044,
-      "tokens/train_per_sec_per_gpu": 11.26,
-      "tokens/trainable": 648
-    },
-    {
-      "epoch": 0.224,
-      "grad_norm": 16.94304656982422,
-      "learning_rate": 8.521739130434783e-06,
-      "loss": 0.8138312101364136,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 2.25654,
-      "step": 28,
-      "tokens/total": 17672,
-      "tokens/train_per_sec_per_gpu": 11.56,
-      "tokens/trainable": 672
-    },
-    {
-      "epoch": 0.232,
-      "grad_norm": 5.823836326599121,
-      "learning_rate": 8.434782608695653e-06,
-      "loss": 0.8621116876602173,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.36816,
-      "step": 29,
-      "tokens/total": 18284,
-      "tokens/train_per_sec_per_gpu": 11.33,
-      "tokens/trainable": 696
-    },
-    {
-      "epoch": 0.24,
-      "grad_norm": 12.865851402282715,
-      "learning_rate": 8.347826086956522e-06,
-      "loss": 0.6359570026397705,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.88883,
-      "step": 30,
-      "tokens/total": 18924,
-      "tokens/train_per_sec_per_gpu": 11.5,
-      "tokens/trainable": 720
-    },
-    {
-      "epoch": 0.248,
-      "grad_norm": 9.98088550567627,
-      "learning_rate": 8.260869565217392e-06,
-      "loss": 0.9601740837097168,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 2.61215,
-      "step": 31,
-      "tokens/total": 19564,
-      "tokens/train_per_sec_per_gpu": 9.37,
-      "tokens/trainable": 744
-    },
-    {
-      "epoch": 0.256,
-      "grad_norm": 13.160109519958496,
-      "learning_rate": 8.173913043478263e-06,
-      "loss": 0.7349061965942383,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 2.08529,
-      "step": 32,
-      "tokens/total": 20168,
-      "tokens/train_per_sec_per_gpu": 11.56,
-      "tokens/trainable": 768
-    },
-    {
-      "epoch": 0.264,
-      "grad_norm": 4.4219841957092285,
-      "learning_rate": 8.086956521739131e-06,
-      "loss": 0.7256244421005249,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 2.06602,
-      "step": 33,
-      "tokens/total": 20824,
-      "tokens/train_per_sec_per_gpu": 11.55,
-      "tokens/trainable": 792
-    },
-    {
-      "epoch": 0.272,
-      "grad_norm": 12.192304611206055,
-      "learning_rate": 8.000000000000001e-06,
-      "loss": 0.9797402620315552,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 2.66376,
-      "step": 34,
-      "tokens/total": 21456,
-      "tokens/train_per_sec_per_gpu": 11.6,
-      "tokens/trainable": 816
-    },
-    {
-      "epoch": 0.28,
-      "grad_norm": 4.266768455505371,
-      "learning_rate": 7.91304347826087e-06,
-      "loss": 0.7637168169021606,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.79,
-      "memory/max_allocated (GiB)": 70.79,
-      "ppl": 2.14624,
-      "step": 35,
-      "tokens/total": 22064,
-      "tokens/train_per_sec_per_gpu": 11.64,
-      "tokens/trainable": 840
-    },
-    {
-      "epoch": 0.288,
-      "grad_norm": 6.994851112365723,
-      "learning_rate": 7.82608695652174e-06,
-      "loss": 0.7504803538322449,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 2.11802,
-      "step": 36,
-      "tokens/total": 22688,
-      "tokens/train_per_sec_per_gpu": 11.65,
-      "tokens/trainable": 864
-    },
-    {
-      "epoch": 0.296,
-      "grad_norm": 10.17078971862793,
-      "learning_rate": 7.739130434782609e-06,
-      "loss": 0.8493132591247559,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 2.33804,
-      "step": 37,
-      "tokens/total": 23328,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 888
-    },
-    {
-      "epoch": 0.304,
-      "grad_norm": 10.87159538269043,
-      "learning_rate": 7.652173913043479e-06,
-      "loss": 0.71614009141922,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 2.04652,
-      "step": 38,
-      "tokens/total": 23952,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 912
-    },
-    {
-      "epoch": 0.312,
-      "grad_norm": 10.938297271728516,
-      "learning_rate": 7.565217391304348e-06,
-      "loss": 0.6279603838920593,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.91,
-      "memory/max_allocated (GiB)": 70.91,
-      "ppl": 1.87378,
-      "step": 39,
-      "tokens/total": 24632,
-      "tokens/train_per_sec_per_gpu": 11.61,
-      "tokens/trainable": 936
-    },
-    {
-      "epoch": 0.32,
-      "grad_norm": 9.613391876220703,
-      "learning_rate": 7.478260869565218e-06,
-      "loss": 0.7583801746368408,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 2.13482,
-      "step": 40,
-      "tokens/total": 25248,
-      "tokens/train_per_sec_per_gpu": 9.61,
-      "tokens/trainable": 960
-    },
-    {
-      "epoch": 0.328,
-      "grad_norm": 5.715580940246582,
-      "learning_rate": 7.391304347826087e-06,
-      "loss": 0.6238446235656738,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.86609,
-      "step": 41,
-      "tokens/total": 25876,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 984
-    },
-    {
-      "epoch": 0.336,
-      "grad_norm": 14.310574531555176,
-      "learning_rate": 7.304347826086957e-06,
-      "loss": 0.7603021860122681,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 2.13892,
-      "step": 42,
-      "tokens/total": 26480,
-      "tokens/train_per_sec_per_gpu": 11.67,
-      "tokens/trainable": 1008
-    },
-    {
-      "epoch": 0.344,
-      "grad_norm": 7.743439197540283,
-      "learning_rate": 7.217391304347827e-06,
-      "loss": 0.7880390882492065,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.19908,
-      "step": 43,
-      "tokens/total": 27084,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 1032
-    },
-    {
-      "epoch": 0.352,
-      "grad_norm": 11.349653244018555,
-      "learning_rate": 7.130434782608696e-06,
-      "loss": 0.7120891809463501,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.03825,
-      "step": 44,
-      "tokens/total": 27736,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 1056
-    },
-    {
-      "epoch": 0.36,
-      "grad_norm": 22.537817001342773,
-      "learning_rate": 7.0434782608695665e-06,
-      "loss": 0.8066906929016113,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 2.24048,
-      "step": 45,
-      "tokens/total": 28340,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 1080
-    },
-    {
-      "epoch": 0.368,
-      "grad_norm": 14.222968101501465,
-      "learning_rate": 6.956521739130435e-06,
-      "loss": 0.6965306997299194,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.00678,
-      "step": 46,
-      "tokens/total": 28976,
-      "tokens/train_per_sec_per_gpu": 11.74,
-      "tokens/trainable": 1104
-    },
-    {
-      "epoch": 0.376,
-      "grad_norm": 5.42992639541626,
-      "learning_rate": 6.869565217391305e-06,
-      "loss": 0.5822693109512329,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.93,
-      "memory/max_allocated (GiB)": 70.93,
-      "ppl": 1.7901,
-      "step": 47,
-      "tokens/total": 29628,
-      "tokens/train_per_sec_per_gpu": 11.48,
-      "tokens/trainable": 1128
-    },
-    {
-      "epoch": 0.384,
-      "grad_norm": 11.926617622375488,
-      "learning_rate": 6.782608695652174e-06,
-      "loss": 0.7480056285858154,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 2.11278,
-      "step": 48,
-      "tokens/total": 30232,
-      "tokens/train_per_sec_per_gpu": 10.06,
-      "tokens/trainable": 1152
-    },
-    {
-      "epoch": 0.392,
-      "grad_norm": 6.486205577850342,
-      "learning_rate": 6.695652173913044e-06,
-      "loss": 0.6353061199188232,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.76,
-      "memory/max_allocated (GiB)": 70.76,
-      "ppl": 1.8876,
-      "step": 49,
-      "tokens/total": 30828,
-      "tokens/train_per_sec_per_gpu": 11.28,
-      "tokens/trainable": 1176
-    },
-    {
-      "epoch": 0.4,
-      "grad_norm": 22.62747573852539,
-      "learning_rate": 6.6086956521739135e-06,
-      "loss": 0.8426034450531006,
-      "memory/device_reserved (GiB)": 71.16,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.32241,
-      "step": 50,
-      "tokens/total": 31468,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 1200
-    },
-    {
-      "epoch": 0.408,
-      "grad_norm": 8.399303436279297,
-      "learning_rate": 6.521739130434783e-06,
-      "loss": 0.6865657567977905,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.95,
-      "memory/max_allocated (GiB)": 70.95,
-      "ppl": 1.98688,
-      "step": 51,
-      "tokens/total": 32156,
-      "tokens/train_per_sec_per_gpu": 11.48,
-      "tokens/trainable": 1224
-    },
-    {
-      "epoch": 0.416,
-      "grad_norm": 24.52540397644043,
-      "learning_rate": 6.434782608695652e-06,
-      "loss": 0.6767396926879883,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 1.96745,
-      "step": 52,
-      "tokens/total": 32804,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1248
-    },
-    {
-      "epoch": 0.424,
-      "grad_norm": 16.407487869262695,
-      "learning_rate": 6.3478260869565225e-06,
-      "loss": 0.9286432266235352,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.79,
-      "memory/max_allocated (GiB)": 70.79,
-      "ppl": 2.53107,
-      "step": 53,
-      "tokens/total": 33412,
-      "tokens/train_per_sec_per_gpu": 11.72,
-      "tokens/trainable": 1272
-    },
-    {
-      "epoch": 0.432,
-      "grad_norm": 16.739253997802734,
-      "learning_rate": 6.260869565217392e-06,
-      "loss": 0.6403137445449829,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 1.89708,
-      "step": 54,
-      "tokens/total": 34064,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 1296
-    },
-    {
-      "epoch": 0.44,
-      "grad_norm": 18.911266326904297,
-      "learning_rate": 6.173913043478261e-06,
-      "loss": 0.7103720903396606,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.03475,
-      "step": 55,
-      "tokens/total": 34684,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 1320
-    },
-    {
-      "epoch": 0.448,
-      "grad_norm": 16.205440521240234,
-      "learning_rate": 6.086956521739132e-06,
-      "loss": 0.7401149272918701,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.83,
-      "memory/max_allocated (GiB)": 70.83,
-      "ppl": 2.09618,
-      "step": 56,
-      "tokens/total": 35292,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 1344
-    },
-    {
-      "epoch": 0.456,
-      "grad_norm": 22.789588928222656,
-      "learning_rate": 6e-06,
-      "loss": 0.6667294502258301,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 1.94786,
-      "step": 57,
-      "tokens/total": 35912,
-      "tokens/train_per_sec_per_gpu": 10.26,
-      "tokens/trainable": 1368
-    },
-    {
-      "epoch": 0.464,
-      "grad_norm": 18.574256896972656,
-      "learning_rate": 5.91304347826087e-06,
-      "loss": 0.8731093406677246,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.9,
-      "memory/max_allocated (GiB)": 70.9,
-      "ppl": 2.39434,
-      "step": 58,
-      "tokens/total": 36560,
-      "tokens/train_per_sec_per_gpu": 11.72,
-      "tokens/trainable": 1392
-    },
-    {
-      "epoch": 0.472,
-      "grad_norm": 22.068172454833984,
-      "learning_rate": 5.826086956521739e-06,
-      "loss": 0.7299917936325073,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 2.07506,
-      "step": 59,
-      "tokens/total": 37180,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1416
-    },
-    {
-      "epoch": 0.48,
-      "grad_norm": 8.097763061523438,
-      "learning_rate": 5.739130434782609e-06,
-      "loss": 0.6472428441047668,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.91027,
-      "step": 60,
-      "tokens/total": 37884,
-      "tokens/train_per_sec_per_gpu": 11.02,
-      "tokens/trainable": 1440
-    },
-    {
-      "epoch": 0.488,
-      "grad_norm": 12.794522285461426,
-      "learning_rate": 5.652173913043479e-06,
-      "loss": 0.5985981225967407,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 1.81957,
-      "step": 61,
-      "tokens/total": 38516,
-      "tokens/train_per_sec_per_gpu": 11.75,
-      "tokens/trainable": 1464
-    },
-    {
-      "epoch": 0.496,
-      "grad_norm": 13.193503379821777,
-      "learning_rate": 5.565217391304348e-06,
-      "loss": 0.7613866329193115,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 2.14124,
-      "step": 62,
-      "tokens/total": 39136,
-      "tokens/train_per_sec_per_gpu": 11.76,
-      "tokens/trainable": 1488
-    },
-    {
-      "epoch": 0.504,
-      "grad_norm": 9.857669830322266,
-      "learning_rate": 5.478260869565217e-06,
-      "loss": 0.9421653747558594,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 2.56553,
-      "step": 63,
-      "tokens/total": 39764,
-      "tokens/train_per_sec_per_gpu": 11.67,
-      "tokens/trainable": 1512
-    },
-    {
-      "epoch": 0.512,
-      "grad_norm": 24.68529510498047,
-      "learning_rate": 5.391304347826088e-06,
-      "loss": 0.6758182048797607,
-      "memory/device_reserved (GiB)": 71.18,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.96564,
-      "step": 64,
-      "tokens/total": 40392,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 1536
-    },
-    {
-      "epoch": 0.52,
-      "grad_norm": 15.150885581970215,
-      "learning_rate": 5.304347826086957e-06,
-      "loss": 0.6766783595085144,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.97,
-      "memory/max_allocated (GiB)": 70.97,
-      "ppl": 1.96733,
-      "step": 65,
-      "tokens/total": 41068,
-      "tokens/train_per_sec_per_gpu": 9.99,
-      "tokens/trainable": 1560
-    },
-    {
-      "epoch": 0.528,
-      "grad_norm": 28.34870147705078,
-      "learning_rate": 5.2173913043478265e-06,
-      "loss": 0.7908303737640381,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.83,
-      "memory/max_allocated (GiB)": 70.83,
-      "ppl": 2.20523,
-      "step": 66,
-      "tokens/total": 41680,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 1584
-    },
-    {
-      "epoch": 0.536,
-      "grad_norm": 6.507663726806641,
-      "learning_rate": 5.130434782608697e-06,
-      "loss": 0.5864015817642212,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 1.79751,
-      "step": 67,
-      "tokens/total": 42324,
-      "tokens/train_per_sec_per_gpu": 11.63,
-      "tokens/trainable": 1608
-    },
-    {
-      "epoch": 0.544,
-      "grad_norm": 9.12397289276123,
-      "learning_rate": 5.043478260869565e-06,
-      "loss": 0.6774575710296631,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.96887,
-      "step": 68,
-      "tokens/total": 42948,
-      "tokens/train_per_sec_per_gpu": 11.72,
-      "tokens/trainable": 1632
-    },
-    {
-      "epoch": 0.552,
-      "grad_norm": 24.50410270690918,
-      "learning_rate": 4.9565217391304355e-06,
-      "loss": 0.7135924100875854,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 2.04131,
-      "step": 69,
-      "tokens/total": 43580,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 1656
-    },
-    {
-      "epoch": 0.56,
-      "grad_norm": 8.199686050415039,
-      "learning_rate": 4.869565217391305e-06,
-      "loss": 0.6808690428733826,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 1.97559,
-      "step": 70,
-      "tokens/total": 44196,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1680
-    },
-    {
-      "epoch": 0.568,
-      "grad_norm": 22.594871520996094,
-      "learning_rate": 4.782608695652174e-06,
-      "loss": 0.7155404090881348,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 2.04529,
-      "step": 71,
-      "tokens/total": 44832,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1704
-    },
-    {
-      "epoch": 0.576,
-      "grad_norm": 4.6781229972839355,
-      "learning_rate": 4.695652173913044e-06,
-      "loss": 0.6442490816116333,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.91,
-      "memory/max_allocated (GiB)": 70.91,
-      "ppl": 1.90456,
-      "step": 72,
-      "tokens/total": 45488,
-      "tokens/train_per_sec_per_gpu": 11.67,
-      "tokens/trainable": 1728
-    },
-    {
-      "epoch": 0.584,
-      "grad_norm": 11.856949806213379,
-      "learning_rate": 4.608695652173913e-06,
-      "loss": 0.7309798002243042,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 2.07711,
-      "step": 73,
-      "tokens/total": 46100,
-      "tokens/train_per_sec_per_gpu": 10.03,
-      "tokens/trainable": 1752
-    },
-    {
-      "epoch": 0.592,
-      "grad_norm": 27.326082229614258,
-      "learning_rate": 4.5217391304347826e-06,
-      "loss": 0.7779560089111328,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.91,
-      "memory/max_allocated (GiB)": 70.91,
-      "ppl": 2.17702,
-      "step": 74,
-      "tokens/total": 46732,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 1776
-    },
-    {
-      "epoch": 0.6,
-      "grad_norm": 21.386030197143555,
-      "learning_rate": 4.434782608695653e-06,
-      "loss": 0.6991748809814453,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 2.01209,
-      "step": 75,
-      "tokens/total": 47344,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 1800
-    },
-    {
-      "epoch": 0.608,
-      "grad_norm": 12.693804740905762,
-      "learning_rate": 4.347826086956522e-06,
-      "loss": 0.5760922431945801,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 1.77907,
-      "step": 76,
-      "tokens/total": 47976,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1824
-    },
-    {
-      "epoch": 0.616,
-      "grad_norm": 14.19450855255127,
-      "learning_rate": 4.260869565217392e-06,
-      "loss": 0.5934574604034424,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.91,
-      "memory/max_allocated (GiB)": 70.91,
-      "ppl": 1.81024,
-      "step": 77,
-      "tokens/total": 48656,
-      "tokens/train_per_sec_per_gpu": 11.65,
-      "tokens/trainable": 1848
-    },
-    {
-      "epoch": 0.624,
-      "grad_norm": 16.425153732299805,
-      "learning_rate": 4.173913043478261e-06,
-      "loss": 0.7686445713043213,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 2.15684,
-      "step": 78,
-      "tokens/total": 49312,
-      "tokens/train_per_sec_per_gpu": 11.56,
-      "tokens/trainable": 1872
-    },
-    {
-      "epoch": 0.632,
-      "grad_norm": 9.41898250579834,
-      "learning_rate": 4.086956521739131e-06,
-      "loss": 0.7164106369018555,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.92,
-      "memory/max_allocated (GiB)": 70.92,
-      "ppl": 2.04707,
-      "step": 79,
-      "tokens/total": 50016,
-      "tokens/train_per_sec_per_gpu": 11.42,
-      "tokens/trainable": 1896
-    },
-    {
-      "epoch": 0.64,
-      "grad_norm": 5.635682582855225,
-      "learning_rate": 4.000000000000001e-06,
-      "loss": 0.6950335502624512,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.00378,
-      "step": 80,
-      "tokens/total": 50648,
-      "tokens/train_per_sec_per_gpu": 11.4,
-      "tokens/trainable": 1920
-    },
-    {
-      "epoch": 0.648,
-      "grad_norm": 24.056533813476562,
-      "learning_rate": 3.91304347826087e-06,
-      "loss": 0.7903406620025635,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 2.20415,
-      "step": 81,
-      "tokens/total": 51324,
-      "tokens/train_per_sec_per_gpu": 11.66,
-      "tokens/trainable": 1944
-    },
-    {
-      "epoch": 0.656,
-      "grad_norm": 5.5482988357543945,
-      "learning_rate": 3.8260869565217395e-06,
-      "loss": 0.6332045793533325,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 1.88364,
-      "step": 82,
-      "tokens/total": 51964,
-      "tokens/train_per_sec_per_gpu": 9.89,
-      "tokens/trainable": 1968
-    },
-    {
-      "epoch": 0.664,
-      "grad_norm": 10.157999038696289,
-      "learning_rate": 3.739130434782609e-06,
-      "loss": 0.6774762272834778,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.9,
-      "memory/max_allocated (GiB)": 70.9,
-      "ppl": 1.9689,
-      "step": 83,
-      "tokens/total": 52608,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 1992
-    },
-    {
-      "epoch": 0.672,
-      "grad_norm": 5.792642116546631,
-      "learning_rate": 3.6521739130434787e-06,
-      "loss": 0.6264545321464539,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 1.87097,
-      "step": 84,
-      "tokens/total": 53272,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 2016
-    },
-    {
-      "epoch": 0.68,
-      "grad_norm": 9.309698104858398,
-      "learning_rate": 3.565217391304348e-06,
-      "loss": 0.5445519685745239,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 1.72384,
-      "step": 85,
-      "tokens/total": 53904,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2040
-    },
-    {
-      "epoch": 0.688,
-      "grad_norm": 14.291025161743164,
-      "learning_rate": 3.4782608695652175e-06,
-      "loss": 0.5914803743362427,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 1.80666,
-      "step": 86,
-      "tokens/total": 54536,
-      "tokens/train_per_sec_per_gpu": 11.74,
-      "tokens/trainable": 2064
-    },
-    {
-      "epoch": 0.696,
-      "grad_norm": 8.881978988647461,
-      "learning_rate": 3.391304347826087e-06,
-      "loss": 0.6258420348167419,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 1.86982,
-      "step": 87,
-      "tokens/total": 55148,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 2088
-    },
-    {
-      "epoch": 0.704,
-      "grad_norm": 11.040555953979492,
-      "learning_rate": 3.3043478260869567e-06,
-      "loss": 0.6892211437225342,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.78,
-      "memory/max_allocated (GiB)": 70.78,
-      "ppl": 1.99216,
-      "step": 88,
-      "tokens/total": 55768,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2112
-    },
-    {
-      "epoch": 0.712,
-      "grad_norm": 17.226795196533203,
-      "learning_rate": 3.217391304347826e-06,
-      "loss": 0.7709158658981323,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 2.16175,
-      "step": 89,
-      "tokens/total": 56388,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 2136
-    },
-    {
-      "epoch": 0.72,
-      "grad_norm": 11.420431137084961,
-      "learning_rate": 3.130434782608696e-06,
-      "loss": 0.6509184837341309,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.93,
-      "memory/max_allocated (GiB)": 70.93,
-      "ppl": 1.9173,
-      "step": 90,
-      "tokens/total": 57044,
-      "tokens/train_per_sec_per_gpu": 10.24,
-      "tokens/trainable": 2160
-    },
-    {
-      "epoch": 0.728,
-      "grad_norm": 14.38459300994873,
-      "learning_rate": 3.043478260869566e-06,
-      "loss": 0.9277634620666504,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 2.52885,
-      "step": 91,
-      "tokens/total": 57692,
-      "tokens/train_per_sec_per_gpu": 11.67,
-      "tokens/trainable": 2184
-    },
-    {
-      "epoch": 0.736,
-      "grad_norm": 12.763906478881836,
-      "learning_rate": 2.956521739130435e-06,
-      "loss": 0.8100249767303467,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 2.24796,
-      "step": 92,
-      "tokens/total": 58324,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2208
-    },
-    {
-      "epoch": 0.744,
-      "grad_norm": 5.974212169647217,
-      "learning_rate": 2.8695652173913046e-06,
-      "loss": 0.6301378607749939,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.87787,
-      "step": 93,
-      "tokens/total": 58968,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 2232
-    },
-    {
-      "epoch": 0.752,
-      "grad_norm": 5.548363208770752,
-      "learning_rate": 2.782608695652174e-06,
-      "loss": 0.6199597120285034,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 1.85885,
-      "step": 94,
-      "tokens/total": 59596,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 2256
-    },
-    {
-      "epoch": 0.76,
-      "grad_norm": 19.908586502075195,
-      "learning_rate": 2.695652173913044e-06,
-      "loss": 0.8254978656768799,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 2.28302,
-      "step": 95,
-      "tokens/total": 60272,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2280
-    },
-    {
-      "epoch": 0.768,
-      "grad_norm": 13.971574783325195,
-      "learning_rate": 2.6086956521739132e-06,
-      "loss": 0.5800565481185913,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.91,
-      "memory/max_allocated (GiB)": 70.91,
-      "ppl": 1.78614,
-      "step": 96,
-      "tokens/total": 60896,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2304
-    },
-    {
-      "epoch": 0.776,
-      "grad_norm": 4.78761625289917,
-      "learning_rate": 2.5217391304347826e-06,
-      "loss": 0.6219671368598938,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.86259,
-      "step": 97,
-      "tokens/total": 61520,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2328
-    },
-    {
-      "epoch": 0.784,
-      "grad_norm": 13.770963668823242,
-      "learning_rate": 2.4347826086956525e-06,
-      "loss": 0.6289293169975281,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.8756,
-      "step": 98,
-      "tokens/total": 62144,
-      "tokens/train_per_sec_per_gpu": 10.3,
-      "tokens/trainable": 2352
-    },
-    {
-      "epoch": 0.792,
-      "grad_norm": 18.048704147338867,
-      "learning_rate": 2.347826086956522e-06,
-      "loss": 0.5753440856933594,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 1.77774,
-      "step": 99,
-      "tokens/total": 62780,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2376
-    },
-    {
-      "epoch": 0.8,
-      "grad_norm": 7.247742652893066,
-      "learning_rate": 2.2608695652173913e-06,
-      "loss": 0.5988768935203552,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 1.82007,
-      "step": 100,
-      "tokens/total": 63444,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2400
-    },
-    {
-      "epoch": 0.808,
-      "grad_norm": 9.422101020812988,
-      "learning_rate": 2.173913043478261e-06,
-      "loss": 0.6174072027206421,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 1.85411,
-      "step": 101,
-      "tokens/total": 64108,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2424
-    },
-    {
-      "epoch": 0.816,
-      "grad_norm": 9.415037155151367,
-      "learning_rate": 2.0869565217391305e-06,
-      "loss": 0.5777270793914795,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 1.78198,
-      "step": 102,
-      "tokens/total": 64744,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 2448
-    },
-    {
-      "epoch": 0.824,
-      "grad_norm": 14.446020126342773,
-      "learning_rate": 2.0000000000000003e-06,
-      "loss": 0.6866780519485474,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.9,
-      "memory/max_allocated (GiB)": 70.9,
-      "ppl": 1.9871,
-      "step": 103,
-      "tokens/total": 65412,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 2472
-    },
-    {
-      "epoch": 0.832,
-      "grad_norm": 26.317893981933594,
-      "learning_rate": 1.9130434782608697e-06,
-      "loss": 0.588347852230072,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.80101,
-      "step": 104,
-      "tokens/total": 66028,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 2496
-    },
-    {
-      "epoch": 0.84,
-      "grad_norm": 19.3927001953125,
-      "learning_rate": 1.8260869565217394e-06,
-      "loss": 0.667943000793457,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 1.95022,
-      "step": 105,
-      "tokens/total": 66656,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2520
-    },
-    {
-      "epoch": 0.848,
-      "grad_norm": 7.816891193389893,
-      "learning_rate": 1.7391304347826088e-06,
-      "loss": 0.6384435296058655,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.89353,
-      "step": 106,
-      "tokens/total": 67320,
-      "tokens/train_per_sec_per_gpu": 10.21,
-      "tokens/trainable": 2544
-    },
-    {
-      "epoch": 0.856,
-      "grad_norm": 14.197734832763672,
-      "learning_rate": 1.6521739130434784e-06,
-      "loss": 0.6795483827590942,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.83,
-      "memory/max_allocated (GiB)": 70.83,
-      "ppl": 1.97299,
-      "step": 107,
-      "tokens/total": 67948,
-      "tokens/train_per_sec_per_gpu": 11.72,
-      "tokens/trainable": 2568
-    },
-    {
-      "epoch": 0.864,
-      "grad_norm": 6.256321907043457,
-      "learning_rate": 1.565217391304348e-06,
-      "loss": 0.6104860305786133,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 1.84133,
-      "step": 108,
-      "tokens/total": 68596,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 2592
-    },
-    {
-      "epoch": 0.872,
-      "grad_norm": 18.18548011779785,
-      "learning_rate": 1.4782608695652176e-06,
-      "loss": 0.7069553136825562,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.89,
-      "memory/max_allocated (GiB)": 70.89,
-      "ppl": 2.02781,
-      "step": 109,
-      "tokens/total": 69240,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 2616
-    },
-    {
-      "epoch": 0.88,
-      "grad_norm": 23.499481201171875,
-      "learning_rate": 1.391304347826087e-06,
-      "loss": 0.6105227470397949,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.84139,
-      "step": 110,
-      "tokens/total": 69836,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 2640
-    },
-    {
-      "epoch": 0.888,
-      "grad_norm": 11.941920280456543,
-      "learning_rate": 1.3043478260869566e-06,
-      "loss": 0.573554515838623,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 1.77456,
-      "step": 111,
-      "tokens/total": 70440,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2664
-    },
-    {
-      "epoch": 0.896,
-      "grad_norm": 16.289566040039062,
-      "learning_rate": 1.2173913043478262e-06,
-      "loss": 0.6701673269271851,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.87,
-      "memory/max_allocated (GiB)": 70.87,
-      "ppl": 1.95456,
-      "step": 112,
-      "tokens/total": 71084,
-      "tokens/train_per_sec_per_gpu": 11.72,
-      "tokens/trainable": 2688
-    },
-    {
-      "epoch": 0.904,
-      "grad_norm": 9.33316707611084,
-      "learning_rate": 1.1304347826086956e-06,
-      "loss": 0.5686048269271851,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.94,
-      "memory/max_allocated (GiB)": 70.94,
-      "ppl": 1.7658,
-      "step": 113,
-      "tokens/total": 71764,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2712
-    },
-    {
-      "epoch": 0.912,
-      "grad_norm": 9.226200103759766,
-      "learning_rate": 1.0434782608695653e-06,
-      "loss": 0.6195059418678284,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.88,
-      "memory/max_allocated (GiB)": 70.88,
-      "ppl": 1.85801,
-      "step": 114,
-      "tokens/total": 72404,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 2736
-    },
-    {
-      "epoch": 0.92,
-      "grad_norm": 33.28397750854492,
-      "learning_rate": 9.565217391304349e-07,
-      "loss": 0.714923620223999,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.81,
-      "memory/max_allocated (GiB)": 70.81,
-      "ppl": 2.04403,
-      "step": 115,
-      "tokens/total": 73008,
-      "tokens/train_per_sec_per_gpu": 10.07,
-      "tokens/trainable": 2760
-    },
-    {
-      "epoch": 0.928,
-      "grad_norm": 26.0920467376709,
-      "learning_rate": 8.695652173913044e-07,
-      "loss": 0.6591269969940186,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.8,
-      "memory/max_allocated (GiB)": 70.8,
-      "ppl": 1.9331,
-      "step": 116,
-      "tokens/total": 73620,
-      "tokens/train_per_sec_per_gpu": 11.71,
-      "tokens/trainable": 2784
-    },
-    {
-      "epoch": 0.936,
-      "grad_norm": 12.7711181640625,
-      "learning_rate": 7.82608695652174e-07,
-      "loss": 0.6524735689163208,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.86,
-      "memory/max_allocated (GiB)": 70.86,
-      "ppl": 1.92028,
-      "step": 117,
-      "tokens/total": 74328,
-      "tokens/train_per_sec_per_gpu": 11.54,
-      "tokens/trainable": 2808
-    },
-    {
-      "epoch": 0.944,
-      "grad_norm": 22.683082580566406,
-      "learning_rate": 6.956521739130435e-07,
-      "loss": 0.6370468139648438,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.83,
-      "memory/max_allocated (GiB)": 70.83,
-      "ppl": 1.89089,
-      "step": 118,
-      "tokens/total": 74924,
-      "tokens/train_per_sec_per_gpu": 11.68,
-      "tokens/trainable": 2832
-    },
-    {
-      "epoch": 0.952,
-      "grad_norm": 16.816553115844727,
-      "learning_rate": 6.086956521739131e-07,
-      "loss": 0.5890098810195923,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.95,
-      "memory/max_allocated (GiB)": 70.95,
-      "ppl": 1.8022,
-      "step": 119,
-      "tokens/total": 75588,
-      "tokens/train_per_sec_per_gpu": 11.61,
-      "tokens/trainable": 2856
-    },
-    {
-      "epoch": 0.96,
-      "grad_norm": 14.087024688720703,
-      "learning_rate": 5.217391304347826e-07,
-      "loss": 0.706240177154541,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 2.02636,
-      "step": 120,
-      "tokens/total": 76224,
-      "tokens/train_per_sec_per_gpu": 11.63,
-      "tokens/trainable": 2880
-    },
-    {
-      "epoch": 0.968,
-      "grad_norm": 12.425010681152344,
-      "learning_rate": 4.347826086956522e-07,
-      "loss": 0.6513041257858276,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.9,
-      "memory/max_allocated (GiB)": 70.9,
-      "ppl": 1.91804,
-      "step": 121,
-      "tokens/total": 76896,
-      "tokens/train_per_sec_per_gpu": 11.69,
-      "tokens/trainable": 2904
-    },
-    {
-      "epoch": 0.976,
-      "grad_norm": 13.220527648925781,
-      "learning_rate": 3.4782608695652175e-07,
-      "loss": 0.6540228128433228,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.85,
-      "memory/max_allocated (GiB)": 70.85,
-      "ppl": 1.92326,
-      "step": 122,
-      "tokens/total": 77524,
-      "tokens/train_per_sec_per_gpu": 11.73,
-      "tokens/trainable": 2928
-    },
-    {
-      "epoch": 0.984,
-      "grad_norm": 15.171476364135742,
-      "learning_rate": 2.608695652173913e-07,
-      "loss": 0.7290846109390259,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.82,
-      "memory/max_allocated (GiB)": 70.82,
-      "ppl": 2.07318,
-      "step": 123,
-      "tokens/total": 78148,
-      "tokens/train_per_sec_per_gpu": 9.86,
-      "tokens/trainable": 2952
-    },
-    {
-      "epoch": 0.992,
-      "grad_norm": 14.782440185546875,
-      "learning_rate": 1.7391304347826088e-07,
-      "loss": 0.6034531593322754,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.84,
-      "memory/max_allocated (GiB)": 70.84,
-      "ppl": 1.82842,
-      "step": 124,
-      "tokens/total": 78768,
-      "tokens/train_per_sec_per_gpu": 11.7,
-      "tokens/trainable": 2976
-    },
-    {
-      "epoch": 1.0,
-      "grad_norm": 17.577150344848633,
-      "learning_rate": 8.695652173913044e-08,
-      "loss": 0.6551875472068787,
-      "memory/device_reserved (GiB)": 71.2,
-      "memory/max_active (GiB)": 70.98,
-      "memory/max_allocated (GiB)": 70.98,
-      "ppl": 1.9255,
-      "step": 125,
-      "tokens/total": 79464,
-      "tokens/train_per_sec_per_gpu": 11.52,
-      "tokens/trainable": 3000
-    }
-  ],
-  "logging_steps": 1,
-  "max_steps": 125,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 1,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": true
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 1.6995545052659712e+16,
-  "train_batch_size": 4,
-  "trial_name": null,
-  "trial_params": null
-}

checkpoint-125/training_args.bin

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:a2bd2b51e1d6b18951fd14b31e8c06481f2a72421157f337dcfa00b23ee47e88
-size 6801