Upload model files with Nebius access

README.md

@@ -1,202 +1,26 @@
----
-base_model: meta-llama/Llama-3.1-8B-Instruct
-library_name: peft
----
 
-# Model Card for Model ID
+---
+language:
+- tr
+- en
+license: llama3
+tags:
+- llama-3
+- eurohps
+- turkish
+- newmindai
+- nebius
+datasets:
+- environmental-instruction-dataset
 
-<!-- Provide a quick summary of what the model is/does. -->
 
+# Meta-Llama-3.1-8B-Instruct environmental Data Model
 
+This model is fine-tuned version of Meta-Llama-3.1-8B-Instruct model with extended Turkish instruction dataset.
 
 ## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.14.0
+- Base Model: Meta-Llama-3.1-8B-Instruct
+- Training Dataset: environmental Turkish Instruction Dataset
+- Organization: NewMind AI
+- Collection: Eurohps
+- Resource Group: Nebius Access Enabled

adapter_config.json

@@ -23,13 +23,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "q_proj",
     "v_proj",
     "o_proj",
     "down_proj",
+    "q_proj",
+    "gate_proj",
     "k_proj",
-    "up_proj",
-    "gate_proj"
+    "up_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:36f00cc96a3471ca7ab3b256d5ceb9f47422e96f554e2cb23e928de2c3e044fb
+oid sha256:bc548e1c06e32d756ac46badc50a93b0a923093e634cca8dddb7f40bd55ac224
 size 167832240

optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6b84cd1c831c647136ff534e99dd26f63ae362e40cadb923d60abfae3608ab7c
+oid sha256:28338d13f61da8dbfee186d7409b8de326f331a458354da1228323a29d496333
 size 335922386

rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:601011b5f5c92a6c41dc706212b35d38526cdebeefe162bf095772b553db461b
+oid sha256:90526829e9717d188d821093533bb80d56c70bb7939608f676732b4dcb7f4e38
 size 14244

tokenizer_config.json

@@ -2050,7 +2050,7 @@
     }
   },
   "bos_token": "<|begin_of_text|>",
-  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
   "clean_up_tokenization_spaces": true,
   "eos_token": "<|eot_id|>",
   "extra_special_tokens": {},

trainer_state.json

@@ -1,6 +1,6 @@
 {
-  "best_metric": 0.974395215511322,
-  "best_model_checkpoint": "./outputs/instruct-lora-8b-alpaca-environmental/checkpoint-1600",
+  "best_metric": 0.9725916385650635,
+  "best_model_checkpoint": "./outputs/instruct-lora-8b-aplly_chat_template-environmental/checkpoint-1600",
   "epoch": 1.8870539663815984,
   "eval_steps": 20,
   "global_step": 1600,
@@ -10,1210 +10,1210 @@
   "log_history": [
     {
       "epoch": 0.0011795930404010617,
-      "eval_loss": 1.4447940587997437,
-      "eval_runtime": 66.7471,
-      "eval_samples_per_second": 22.578,
-      "eval_steps_per_second": 5.648,
+      "eval_loss": 1.42765212059021,
+      "eval_runtime": 63.0401,
+      "eval_samples_per_second": 23.905,
+      "eval_steps_per_second": 5.98,
       "step": 1
     },
     {
       "epoch": 0.023591860808021232,
-      "grad_norm": 0.8165745139122009,
+      "grad_norm": 0.8140115737915039,
       "learning_rate": 2.3622047244094487e-06,
-      "loss": 1.4495,
+      "loss": 1.4383,
       "step": 20
     },
     {
       "epoch": 0.023591860808021232,
-      "eval_loss": 1.4427223205566406,
-      "eval_runtime": 68.7227,
-      "eval_samples_per_second": 21.929,
-      "eval_steps_per_second": 5.486,
+      "eval_loss": 1.4257409572601318,
+      "eval_runtime": 63.0141,
+      "eval_samples_per_second": 23.915,
+      "eval_steps_per_second": 5.983,
       "step": 20
     },
     {
       "epoch": 0.047183721616042465,
-      "grad_norm": 0.8174494504928589,
+      "grad_norm": 0.8540558218955994,
       "learning_rate": 4.7244094488188975e-06,
-      "loss": 1.4899,
+      "loss": 1.4736,
       "step": 40
     },
     {
       "epoch": 0.047183721616042465,
-      "eval_loss": 1.4231655597686768,
-      "eval_runtime": 66.6176,
-      "eval_samples_per_second": 22.622,
-      "eval_steps_per_second": 5.659,
+      "eval_loss": 1.4058444499969482,
+      "eval_runtime": 65.5526,
+      "eval_samples_per_second": 22.989,
+      "eval_steps_per_second": 5.751,
       "step": 40
     },
     {
       "epoch": 0.0707755824240637,
-      "grad_norm": 0.9848796129226685,
+      "grad_norm": 0.9564130902290344,
       "learning_rate": 7.086614173228346e-06,
-      "loss": 1.4016,
+      "loss": 1.3855,
       "step": 60
     },
     {
       "epoch": 0.0707755824240637,
-      "eval_loss": 1.3492255210876465,
-      "eval_runtime": 66.5747,
-      "eval_samples_per_second": 22.636,
-      "eval_steps_per_second": 5.663,
+      "eval_loss": 1.3293768167495728,
+      "eval_runtime": 62.9854,
+      "eval_samples_per_second": 23.926,
+      "eval_steps_per_second": 5.986,
       "step": 60
     },
     {
       "epoch": 0.09436744323208493,
-      "grad_norm": 0.9268898367881775,
+      "grad_norm": 0.9375326037406921,
       "learning_rate": 9.448818897637795e-06,
-      "loss": 1.3283,
+      "loss": 1.3151,
       "step": 80
     },
     {
       "epoch": 0.09436744323208493,
-      "eval_loss": 1.2675418853759766,
-      "eval_runtime": 68.6638,
-      "eval_samples_per_second": 21.948,
-      "eval_steps_per_second": 5.491,
+      "eval_loss": 1.256630301475525,
+      "eval_runtime": 62.9741,
+      "eval_samples_per_second": 23.93,
+      "eval_steps_per_second": 5.987,
       "step": 80
     },
     {
       "epoch": 0.11795930404010617,
-      "grad_norm": 0.9920526742935181,
+      "grad_norm": 0.9875860810279846,
       "learning_rate": 1.1811023622047245e-05,
-      "loss": 1.2584,
+      "loss": 1.2527,
       "step": 100
     },
     {
       "epoch": 0.11795930404010617,
-      "eval_loss": 1.2328431606292725,
-      "eval_runtime": 66.703,
-      "eval_samples_per_second": 22.593,
-      "eval_steps_per_second": 5.652,
+      "eval_loss": 1.2264838218688965,
+      "eval_runtime": 65.3152,
+      "eval_samples_per_second": 23.073,
+      "eval_steps_per_second": 5.772,
       "step": 100
     },
     {
       "epoch": 0.1415511648481274,
-      "grad_norm": 1.0559078454971313,
+      "grad_norm": 1.0547633171081543,
       "learning_rate": 1.4173228346456692e-05,
-      "loss": 1.2596,
+      "loss": 1.2557,
       "step": 120
     },
     {
       "epoch": 0.1415511648481274,
-      "eval_loss": 1.209496259689331,
-      "eval_runtime": 69.2307,
-      "eval_samples_per_second": 21.768,
-      "eval_steps_per_second": 5.446,
+      "eval_loss": 1.2046794891357422,
+      "eval_runtime": 62.9226,
+      "eval_samples_per_second": 23.95,
+      "eval_steps_per_second": 5.991,
       "step": 120
     },
     {
       "epoch": 0.16514302565614863,
-      "grad_norm": 1.0239228010177612,
+      "grad_norm": 1.010538101196289,
       "learning_rate": 1.6535433070866142e-05,
-      "loss": 1.2432,
+      "loss": 1.2405,
       "step": 140
     },
     {
       "epoch": 0.16514302565614863,
-      "eval_loss": 1.1917829513549805,
-      "eval_runtime": 66.7215,
-      "eval_samples_per_second": 22.586,
-      "eval_steps_per_second": 5.65,
+      "eval_loss": 1.188104271888733,
+      "eval_runtime": 62.9407,
+      "eval_samples_per_second": 23.943,
+      "eval_steps_per_second": 5.99,
       "step": 140
     },
     {
       "epoch": 0.18873488646416986,
-      "grad_norm": 1.1438844203948975,
+      "grad_norm": 1.1231751441955566,
       "learning_rate": 1.889763779527559e-05,
-      "loss": 1.2102,
+      "loss": 1.2074,
       "step": 160
     },
     {
       "epoch": 0.18873488646416986,
-      "eval_loss": 1.1754366159439087,
-      "eval_runtime": 66.6395,
-      "eval_samples_per_second": 22.614,
-      "eval_steps_per_second": 5.657,
+      "eval_loss": 1.1720376014709473,
+      "eval_runtime": 62.9691,
+      "eval_samples_per_second": 23.932,
+      "eval_steps_per_second": 5.987,
       "step": 160
     },
     {
       "epoch": 0.21232674727219109,
-      "grad_norm": 1.2777637243270874,
+      "grad_norm": 1.2803903818130493,
       "learning_rate": 2.1259842519685038e-05,
-      "loss": 1.1931,
+      "loss": 1.1897,
       "step": 180
     },
     {
       "epoch": 0.21232674727219109,
-      "eval_loss": 1.1629679203033447,
-      "eval_runtime": 68.7237,
-      "eval_samples_per_second": 21.928,
-      "eval_steps_per_second": 5.486,
+      "eval_loss": 1.1597559452056885,
+      "eval_runtime": 63.1045,
+      "eval_samples_per_second": 23.881,
+      "eval_steps_per_second": 5.974,
       "step": 180
     },
     {
       "epoch": 0.23591860808021234,
-      "grad_norm": 1.3225220441818237,
+      "grad_norm": 1.2758204936981201,
       "learning_rate": 2.362204724409449e-05,
-      "loss": 1.1493,
+      "loss": 1.145,
       "step": 200
     },
     {
       "epoch": 0.23591860808021234,
-      "eval_loss": 1.1514708995819092,
-      "eval_runtime": 66.7133,
-      "eval_samples_per_second": 22.589,
-      "eval_steps_per_second": 5.651,
+      "eval_loss": 1.1493200063705444,
+      "eval_runtime": 65.8866,
+      "eval_samples_per_second": 22.873,
+      "eval_steps_per_second": 5.722,
       "step": 200
     },
     {
       "epoch": 0.25951046888823354,
-      "grad_norm": 1.49991774559021,
+      "grad_norm": 1.517013430595398,
       "learning_rate": 2.5984251968503937e-05,
-      "loss": 1.1555,
+      "loss": 1.1508,
       "step": 220
     },
     {
       "epoch": 0.25951046888823354,
-      "eval_loss": 1.1435123682022095,
-      "eval_runtime": 66.74,
-      "eval_samples_per_second": 22.58,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.1411106586456299,
+      "eval_runtime": 62.986,
+      "eval_samples_per_second": 23.926,
+      "eval_steps_per_second": 5.985,
       "step": 220
     },
     {
       "epoch": 0.2831023296962548,
-      "grad_norm": 1.6506001949310303,
+      "grad_norm": 1.6017550230026245,
       "learning_rate": 2.8346456692913385e-05,
-      "loss": 1.143,
+      "loss": 1.1406,
       "step": 240
     },
     {
       "epoch": 0.2831023296962548,
-      "eval_loss": 1.1358726024627686,
-      "eval_runtime": 66.733,
-      "eval_samples_per_second": 22.583,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.1336663961410522,
+      "eval_runtime": 62.9587,
+      "eval_samples_per_second": 23.936,
+      "eval_steps_per_second": 5.988,
       "step": 240
     },
     {
       "epoch": 0.30669419050427604,
-      "grad_norm": 1.5138826370239258,
+      "grad_norm": 1.5281150341033936,
       "learning_rate": 2.9999490518144496e-05,
-      "loss": 1.1158,
+      "loss": 1.1156,
       "step": 260
     },
     {
       "epoch": 0.30669419050427604,
-      "eval_loss": 1.1296896934509277,
-      "eval_runtime": 66.7403,
-      "eval_samples_per_second": 22.58,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.1270569562911987,
+      "eval_runtime": 66.1576,
+      "eval_samples_per_second": 22.779,
+      "eval_steps_per_second": 5.699,
       "step": 260
     },
     {
       "epoch": 0.33028605131229727,
-      "grad_norm": 1.4625916481018066,
+      "grad_norm": 1.4530911445617676,
       "learning_rate": 2.9990434025704478e-05,
-      "loss": 1.1462,
+      "loss": 1.1439,
       "step": 280
     },
     {
       "epoch": 0.33028605131229727,
-      "eval_loss": 1.1224937438964844,
-      "eval_runtime": 69.052,
-      "eval_samples_per_second": 21.824,
-      "eval_steps_per_second": 5.46,
+      "eval_loss": 1.118715524673462,
+      "eval_runtime": 62.9289,
+      "eval_samples_per_second": 23.948,
+      "eval_steps_per_second": 5.991,
       "step": 280
     },
     {
       "epoch": 0.3538779121203185,
-      "grad_norm": 1.4778045415878296,
+      "grad_norm": 1.4522110223770142,
       "learning_rate": 2.9970063582150774e-05,
-      "loss": 1.1018,
+      "loss": 1.0991,
       "step": 300
     },
     {
       "epoch": 0.3538779121203185,
-      "eval_loss": 1.1171784400939941,
-      "eval_runtime": 66.7001,
-      "eval_samples_per_second": 22.594,
-      "eval_steps_per_second": 5.652,
+      "eval_loss": 1.1142727136611938,
+      "eval_runtime": 62.9463,
+      "eval_samples_per_second": 23.941,
+      "eval_steps_per_second": 5.989,
       "step": 300
     },
     {
       "epoch": 0.3774697729283397,
-      "grad_norm": 1.6671929359436035,
+      "grad_norm": 1.6753257513046265,
       "learning_rate": 2.9938394561968347e-05,
-      "loss": 1.1069,
+      "loss": 1.1037,
       "step": 320
     },
     {
       "epoch": 0.3774697729283397,
-      "eval_loss": 1.1091420650482178,
-      "eval_runtime": 66.6922,
-      "eval_samples_per_second": 22.596,
-      "eval_steps_per_second": 5.653,
+      "eval_loss": 1.1066044569015503,
+      "eval_runtime": 62.9455,
+      "eval_samples_per_second": 23.941,
+      "eval_steps_per_second": 5.989,
       "step": 320
     },
     {
       "epoch": 0.40106163373636095,
-      "grad_norm": 1.5929043292999268,
+      "grad_norm": 1.5857524871826172,
       "learning_rate": 2.9895450867183358e-05,
-      "loss": 1.0844,
+      "loss": 1.0821,
       "step": 340
     },
     {
       "epoch": 0.40106163373636095,
-      "eval_loss": 1.104718565940857,
-      "eval_runtime": 66.6741,
-      "eval_samples_per_second": 22.602,
-      "eval_steps_per_second": 5.654,
+      "eval_loss": 1.1023849248886108,
+      "eval_runtime": 62.9815,
+      "eval_samples_per_second": 23.928,
+      "eval_steps_per_second": 5.986,
       "step": 340
     },
     {
       "epoch": 0.42465349454438217,
-      "grad_norm": 1.6665343046188354,
+      "grad_norm": 1.6321624517440796,
       "learning_rate": 2.9841264909323215e-05,
-      "loss": 1.039,
+      "loss": 1.0355,
       "step": 360
     },
     {
       "epoch": 0.42465349454438217,
-      "eval_loss": 1.101120948791504,
-      "eval_runtime": 66.7353,
-      "eval_samples_per_second": 22.582,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.0976252555847168,
+      "eval_runtime": 63.0759,
+      "eval_samples_per_second": 23.892,
+      "eval_steps_per_second": 5.977,
       "step": 360
     },
     {
       "epoch": 0.4482453553524034,
-      "grad_norm": 1.7163054943084717,
+      "grad_norm": 1.7007124423980713,
       "learning_rate": 2.9775877584954185e-05,
-      "loss": 1.0731,
+      "loss": 1.0705,
       "step": 380
     },
     {
       "epoch": 0.4482453553524034,
-      "eval_loss": 1.0928065776824951,
-      "eval_runtime": 69.2506,
-      "eval_samples_per_second": 21.762,
-      "eval_steps_per_second": 5.444,
+      "eval_loss": 1.0898144245147705,
+      "eval_runtime": 63.0529,
+      "eval_samples_per_second": 23.901,
+      "eval_steps_per_second": 5.979,
       "step": 380
     },
     {
       "epoch": 0.4718372161604247,
-      "grad_norm": 1.6514698266983032,
+      "grad_norm": 1.6532444953918457,
       "learning_rate": 2.9699338244814873e-05,
-      "loss": 1.0618,
+      "loss": 1.058,
       "step": 400
     },
     {
       "epoch": 0.4718372161604247,
-      "eval_loss": 1.08615243434906,
-      "eval_runtime": 66.6083,
-      "eval_samples_per_second": 22.625,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.0839428901672363,
+      "eval_runtime": 63.0318,
+      "eval_samples_per_second": 23.909,
+      "eval_steps_per_second": 5.981,
       "step": 400
     },
     {
       "epoch": 0.4954290769684459,
-      "grad_norm": 1.761924147605896,
+      "grad_norm": 1.7736274003982544,
       "learning_rate": 2.961170465656906e-05,
-      "loss": 1.0758,
+      "loss": 1.073,
       "step": 420
     },
     {
       "epoch": 0.4954290769684459,
-      "eval_loss": 1.0836244821548462,
-      "eval_runtime": 66.6894,
-      "eval_samples_per_second": 22.597,
-      "eval_steps_per_second": 5.653,
+      "eval_loss": 1.0811110734939575,
+      "eval_runtime": 65.7766,
+      "eval_samples_per_second": 22.911,
+      "eval_steps_per_second": 5.732,
       "step": 420
     },
     {
       "epoch": 0.5190209377764671,
-      "grad_norm": 1.7449309825897217,
+      "grad_norm": 1.7278027534484863,
       "learning_rate": 2.9513042961205828e-05,
-      "loss": 1.0697,
+      "loss": 1.0669,
       "step": 440
     },
     {
       "epoch": 0.5190209377764671,
-      "eval_loss": 1.0806337594985962,
-      "eval_runtime": 66.7114,
-      "eval_samples_per_second": 22.59,
-      "eval_steps_per_second": 5.651,
+      "eval_loss": 1.0773712396621704,
+      "eval_runtime": 63.1642,
+      "eval_samples_per_second": 23.858,
+      "eval_steps_per_second": 5.969,
       "step": 440
     },
     {
       "epoch": 0.5426127985844883,
-      "grad_norm": 1.73285973072052,
+      "grad_norm": 1.7342020273208618,
       "learning_rate": 2.940342762312002e-05,
-      "loss": 1.0769,
+      "loss": 1.0742,
       "step": 460
     },
     {
       "epoch": 0.5426127985844883,
-      "eval_loss": 1.0765126943588257,
-      "eval_runtime": 66.7739,
-      "eval_samples_per_second": 22.569,
-      "eval_steps_per_second": 5.646,
+      "eval_loss": 1.07448410987854,
+      "eval_runtime": 63.1172,
+      "eval_samples_per_second": 23.876,
+      "eval_steps_per_second": 5.973,
       "step": 460
     },
     {
       "epoch": 0.5662046593925096,
-      "grad_norm": 1.7008869647979736,
+      "grad_norm": 1.688236117362976,
       "learning_rate": 2.9282941373910622e-05,
-      "loss": 1.0406,
+      "loss": 1.0391,
       "step": 480
     },
     {
       "epoch": 0.5662046593925096,
-      "eval_loss": 1.0717772245407104,
-      "eval_runtime": 69.1199,
-      "eval_samples_per_second": 21.803,
-      "eval_steps_per_second": 5.454,
+      "eval_loss": 1.0696014165878296,
+      "eval_runtime": 63.5007,
+      "eval_samples_per_second": 23.732,
+      "eval_steps_per_second": 5.937,
       "step": 480
     },
     {
       "epoch": 0.5897965202005309,
-      "grad_norm": 1.6530652046203613,
+      "grad_norm": 1.6653029918670654,
       "learning_rate": 2.915167514993952e-05,
-      "loss": 1.0416,
+      "loss": 1.0388,
       "step": 500
     },
     {
       "epoch": 0.5897965202005309,
-      "eval_loss": 1.0675851106643677,
-      "eval_runtime": 66.7189,
-      "eval_samples_per_second": 22.587,
-      "eval_steps_per_second": 5.651,
+      "eval_loss": 1.0648998022079468,
+      "eval_runtime": 63.0976,
+      "eval_samples_per_second": 23.884,
+      "eval_steps_per_second": 5.975,
       "step": 500
     },
     {
       "epoch": 0.6133883810085521,
-      "grad_norm": 1.788845419883728,
+      "grad_norm": 1.7940328121185303,
       "learning_rate": 2.9009728023697777e-05,
-      "loss": 1.0509,
+      "loss": 1.0483,
       "step": 520
     },
     {
       "epoch": 0.6133883810085521,
-      "eval_loss": 1.065712332725525,
-      "eval_runtime": 66.738,
-      "eval_samples_per_second": 22.581,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.0635266304016113,
+      "eval_runtime": 63.1023,
+      "eval_samples_per_second": 23.882,
+      "eval_steps_per_second": 5.974,
       "step": 520
     },
     {
       "epoch": 0.6369802418165733,
-      "grad_norm": 1.7791155576705933,
+      "grad_norm": 1.7853456735610962,
       "learning_rate": 2.8857207129031153e-05,
-      "loss": 1.08,
+      "loss": 1.0767,
       "step": 540
     },
     {
       "epoch": 0.6369802418165733,
-      "eval_loss": 1.0589417219161987,
-      "eval_runtime": 66.7425,
-      "eval_samples_per_second": 22.579,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.0566504001617432,
+      "eval_runtime": 63.0964,
+      "eval_samples_per_second": 23.884,
+      "eval_steps_per_second": 5.975,
       "step": 540
     },
     {
       "epoch": 0.6605721026245945,
-      "grad_norm": 1.8033630847930908,
+      "grad_norm": 1.8145803213119507,
       "learning_rate": 2.869422758028145e-05,
-      "loss": 1.0312,
+      "loss": 1.0273,
       "step": 560
     },
     {
       "epoch": 0.6605721026245945,
-      "eval_loss": 1.0578253269195557,
-      "eval_runtime": 66.6865,
-      "eval_samples_per_second": 22.598,
-      "eval_steps_per_second": 5.653,
+      "eval_loss": 1.0557910203933716,
+      "eval_runtime": 63.1078,
+      "eval_samples_per_second": 23.88,
+      "eval_steps_per_second": 5.974,
       "step": 560
     },
     {
       "epoch": 0.6841639634326158,
-      "grad_norm": 1.7615036964416504,
+      "grad_norm": 1.7679259777069092,
       "learning_rate": 2.852091238540454e-05,
-      "loss": 1.0238,
+      "loss": 1.021,
       "step": 580
     },
     {
       "epoch": 0.6841639634326158,
-      "eval_loss": 1.0552713871002197,
-      "eval_runtime": 69.1593,
-      "eval_samples_per_second": 21.79,
-      "eval_steps_per_second": 5.451,
+      "eval_loss": 1.0533658266067505,
+      "eval_runtime": 65.8301,
+      "eval_samples_per_second": 22.892,
+      "eval_steps_per_second": 5.727,
       "step": 580
     },
     {
       "epoch": 0.707755824240637,
-      "grad_norm": 1.6202025413513184,
+      "grad_norm": 1.6286741495132446,
       "learning_rate": 2.8337392353130755e-05,
-      "loss": 1.0457,
+      "loss": 1.0429,
       "step": 600
     },
     {
       "epoch": 0.707755824240637,
-      "eval_loss": 1.0504437685012817,
-      "eval_runtime": 66.5899,
-      "eval_samples_per_second": 22.631,
-      "eval_steps_per_second": 5.662,
+      "eval_loss": 1.0482341051101685,
+      "eval_runtime": 63.1283,
+      "eval_samples_per_second": 23.872,
+      "eval_steps_per_second": 5.972,
       "step": 600
     },
     {
       "epoch": 0.7313476850486582,
-      "grad_norm": 2.028264284133911,
+      "grad_norm": 2.0246284008026123,
       "learning_rate": 2.81438059942377e-05,
-      "loss": 1.0028,
+      "loss": 1.001,
       "step": 620
     },
     {
       "epoch": 0.7313476850486582,
-      "eval_loss": 1.0460361242294312,
-      "eval_runtime": 66.6029,
-      "eval_samples_per_second": 22.627,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.0442626476287842,
+      "eval_runtime": 63.0773,
+      "eval_samples_per_second": 23.891,
+      "eval_steps_per_second": 5.977,
       "step": 620
     },
     {
       "epoch": 0.7549395458566794,
-      "grad_norm": 1.8573793172836304,
+      "grad_norm": 1.8399453163146973,
       "learning_rate": 2.7940299417009968e-05,
-      "loss": 1.0268,
+      "loss": 1.0257,
       "step": 640
     },
     {
       "epoch": 0.7549395458566794,
-      "eval_loss": 1.0440465211868286,
-      "eval_runtime": 68.8558,
-      "eval_samples_per_second": 21.886,
-      "eval_steps_per_second": 5.475,
+      "eval_loss": 1.0410016775131226,
+      "eval_runtime": 66.0639,
+      "eval_samples_per_second": 22.811,
+      "eval_steps_per_second": 5.707,
       "step": 640
     },
     {
       "epoch": 0.7785314066647007,
-      "grad_norm": 1.7014297246932983,
+      "grad_norm": 1.7166261672973633,
       "learning_rate": 2.772702621696468e-05,
-      "loss": 0.9977,
+      "loss": 0.9959,
       "step": 660
     },
     {
       "epoch": 0.7785314066647007,
-      "eval_loss": 1.0395982265472412,
-      "eval_runtime": 66.9333,
-      "eval_samples_per_second": 22.515,
-      "eval_steps_per_second": 5.632,
+      "eval_loss": 1.036692500114441,
+      "eval_runtime": 62.9935,
+      "eval_samples_per_second": 23.923,
+      "eval_steps_per_second": 5.985,
       "step": 660
     },
     {
       "epoch": 0.8021232674727219,
-      "grad_norm": 1.8055284023284912,
+      "grad_norm": 1.8086529970169067,
       "learning_rate": 2.7504147360926084e-05,
-      "loss": 0.9934,
+      "loss": 0.9912,
       "step": 680
     },
     {
       "epoch": 0.8021232674727219,
-      "eval_loss": 1.0370361804962158,
-      "eval_runtime": 68.9181,
-      "eval_samples_per_second": 21.867,
-      "eval_steps_per_second": 5.47,
+      "eval_loss": 1.0342631340026855,
+      "eval_runtime": 62.9732,
+      "eval_samples_per_second": 23.931,
+      "eval_steps_per_second": 5.987,
       "step": 680
     },
     {
       "epoch": 0.8257151282807431,
-      "grad_norm": 1.7314211130142212,
+      "grad_norm": 1.7481584548950195,
       "learning_rate": 2.7271831065536684e-05,
-      "loss": 1.0002,
+      "loss": 0.9986,
       "step": 700
     },
     {
       "epoch": 0.8257151282807431,
-      "eval_loss": 1.0348241329193115,
-      "eval_runtime": 66.5795,
-      "eval_samples_per_second": 22.635,
-      "eval_steps_per_second": 5.662,
+      "eval_loss": 1.0320111513137817,
+      "eval_runtime": 63.187,
+      "eval_samples_per_second": 23.85,
+      "eval_steps_per_second": 5.966,
       "step": 700
     },
     {
       "epoch": 0.8493069890887643,
-      "grad_norm": 1.808159589767456,
+      "grad_norm": 1.8035798072814941,
       "learning_rate": 2.7030252670296612e-05,
-      "loss": 0.9998,
+      "loss": 0.9972,
       "step": 720
     },
     {
       "epoch": 0.8493069890887643,
-      "eval_loss": 1.0329915285110474,
-      "eval_runtime": 66.5761,
-      "eval_samples_per_second": 22.636,
-      "eval_steps_per_second": 5.663,
+      "eval_loss": 1.0306118726730347,
+      "eval_runtime": 62.9945,
+      "eval_samples_per_second": 23.923,
+      "eval_steps_per_second": 5.985,
       "step": 720
     },
     {
       "epoch": 0.8728988498967856,
-      "grad_norm": 1.7294625043869019,
+      "grad_norm": 1.7504149675369263,
       "learning_rate": 2.677959450522709e-05,
-      "loss": 0.994,
+      "loss": 0.9923,
       "step": 740
     },
     {
       "epoch": 0.8728988498967856,
-      "eval_loss": 1.030936360359192,
-      "eval_runtime": 68.7325,
-      "eval_samples_per_second": 21.926,
-      "eval_steps_per_second": 5.485,
+      "eval_loss": 1.028536081314087,
+      "eval_runtime": 66.0048,
+      "eval_samples_per_second": 22.832,
+      "eval_steps_per_second": 5.712,
       "step": 740
     },
     {
       "epoch": 0.8964907107048068,
-      "grad_norm": 1.8886359930038452,
+      "grad_norm": 1.9125969409942627,
       "learning_rate": 2.6520045753257774e-05,
-      "loss": 0.9885,
+      "loss": 0.9862,
       "step": 760
     },
     {
       "epoch": 0.8964907107048068,
-      "eval_loss": 1.0302257537841797,
-      "eval_runtime": 66.5876,
-      "eval_samples_per_second": 22.632,
-      "eval_steps_per_second": 5.662,
+      "eval_loss": 1.0277767181396484,
+      "eval_runtime": 62.8849,
+      "eval_samples_per_second": 23.964,
+      "eval_steps_per_second": 5.995,
       "step": 760
     },
     {
       "epoch": 0.9200825715128281,
-      "grad_norm": 1.7796871662139893,
+      "grad_norm": 1.7713780403137207,
       "learning_rate": 2.625180230744195e-05,
-      "loss": 1.0271,
+      "loss": 1.0248,
       "step": 780
     },
     {
       "epoch": 0.9200825715128281,
-      "eval_loss": 1.026247262954712,
-      "eval_runtime": 66.6131,
-      "eval_samples_per_second": 22.623,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.024271845817566,
+      "eval_runtime": 62.9468,
+      "eval_samples_per_second": 23.941,
+      "eval_steps_per_second": 5.989,
       "step": 780
     },
     {
       "epoch": 0.9436744323208494,
-      "grad_norm": 1.790818452835083,
+      "grad_norm": 1.794228196144104,
       "learning_rate": 2.597506662310728e-05,
-      "loss": 0.9902,
+      "loss": 0.9898,
       "step": 800
     },
     {
       "epoch": 0.9436744323208494,
-      "eval_loss": 1.0263715982437134,
-      "eval_runtime": 66.5963,
-      "eval_samples_per_second": 22.629,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.024512767791748,
+      "eval_runtime": 65.8042,
+      "eval_samples_per_second": 22.901,
+      "eval_steps_per_second": 5.729,
       "step": 800
     },
     {
       "epoch": 0.9672662931288706,
-      "grad_norm": 2.1395716667175293,
+      "grad_norm": 2.150176525115967,
       "learning_rate": 2.569004756505373e-05,
-      "loss": 0.9851,
+      "loss": 0.9838,
       "step": 820
     },
     {
       "epoch": 0.9672662931288706,
-      "eval_loss": 1.022589921951294,
-      "eval_runtime": 66.5965,
-      "eval_samples_per_second": 22.629,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0203757286071777,
+      "eval_runtime": 63.2378,
+      "eval_samples_per_second": 23.831,
+      "eval_steps_per_second": 5.962,
       "step": 820
     },
     {
       "epoch": 0.9908581539368918,
-      "grad_norm": 1.8714855909347534,
+      "grad_norm": 1.892907977104187,
       "learning_rate": 2.539696024991391e-05,
-      "loss": 0.9663,
+      "loss": 0.9655,
       "step": 840
     },
     {
       "epoch": 0.9908581539368918,
-      "eval_loss": 1.020493984222412,
-      "eval_runtime": 68.8083,
-      "eval_samples_per_second": 21.901,
-      "eval_steps_per_second": 5.479,
+      "eval_loss": 1.0187581777572632,
+      "eval_runtime": 62.8975,
+      "eval_samples_per_second": 23.96,
+      "eval_steps_per_second": 5.994,
       "step": 840
     },
     {
       "epoch": 1.0141551164848128,
-      "grad_norm": 1.8689191341400146,
+      "grad_norm": 1.8649617433547974,
       "learning_rate": 2.5096025883795e-05,
-      "loss": 0.9774,
+      "loss": 0.9747,
       "step": 860
     },
     {
       "epoch": 1.0141551164848128,
-      "eval_loss": 1.0238546133041382,
-      "eval_runtime": 66.5942,
-      "eval_samples_per_second": 22.63,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0219627618789673,
+      "eval_runtime": 62.8985,
+      "eval_samples_per_second": 23.959,
+      "eval_steps_per_second": 5.994,
       "step": 860
     },
     {
       "epoch": 1.037746977292834,
-      "grad_norm": 1.9778001308441162,
+      "grad_norm": 1.9797245264053345,
       "learning_rate": 2.4787471595324554e-05,
-      "loss": 0.9166,
+      "loss": 0.9133,
       "step": 880
     },
     {
       "epoch": 1.037746977292834,
-      "eval_loss": 1.017867922782898,
-      "eval_runtime": 66.587,
-      "eval_samples_per_second": 22.632,
-      "eval_steps_per_second": 5.662,
+      "eval_loss": 1.0162080526351929,
+      "eval_runtime": 62.9338,
+      "eval_samples_per_second": 23.946,
+      "eval_steps_per_second": 5.99,
       "step": 880
     },
     {
       "epoch": 1.0613388381008553,
-      "grad_norm": 1.9455280303955078,
+      "grad_norm": 1.9416215419769287,
       "learning_rate": 2.447153026422637e-05,
-      "loss": 0.9147,
+      "loss": 0.9122,
       "step": 900
     },
     {
       "epoch": 1.0613388381008553,
-      "eval_loss": 1.0164260864257812,
-      "eval_runtime": 66.6045,
-      "eval_samples_per_second": 22.626,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.014188528060913,
+      "eval_runtime": 62.9073,
+      "eval_samples_per_second": 23.956,
+      "eval_steps_per_second": 5.993,
       "step": 900
     },
     {
       "epoch": 1.0849306989088765,
-      "grad_norm": 2.014035224914551,
+      "grad_norm": 2.022336721420288,
       "learning_rate": 2.4148440345555778e-05,
-      "loss": 0.9121,
+      "loss": 0.9089,
       "step": 920
     },
     {
       "epoch": 1.0849306989088765,
-      "eval_loss": 1.019012451171875,
-      "eval_runtime": 66.5906,
-      "eval_samples_per_second": 22.631,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0171178579330444,
+      "eval_runtime": 62.9354,
+      "eval_samples_per_second": 23.945,
+      "eval_steps_per_second": 5.99,
       "step": 920
     },
     {
       "epoch": 1.1085225597168977,
-      "grad_norm": 2.1933348178863525,
+      "grad_norm": 2.2142956256866455,
       "learning_rate": 2.38184456897269e-05,
-      "loss": 0.9332,
+      "loss": 0.9315,
       "step": 940
     },
     {
       "epoch": 1.1085225597168977,
-      "eval_loss": 1.0168778896331787,
-      "eval_runtime": 68.9948,
-      "eval_samples_per_second": 21.842,
-      "eval_steps_per_second": 5.464,
+      "eval_loss": 1.0155071020126343,
+      "eval_runtime": 63.3545,
+      "eval_samples_per_second": 23.787,
+      "eval_steps_per_second": 5.951,
       "step": 940
     },
     {
       "epoch": 1.132114420524919,
-      "grad_norm": 1.9070005416870117,
+      "grad_norm": 1.9037110805511475,
       "learning_rate": 2.3481795358467866e-05,
-      "loss": 0.9501,
+      "loss": 0.9469,
       "step": 960
     },
     {
       "epoch": 1.132114420524919,
-      "eval_loss": 1.0127308368682861,
-      "eval_runtime": 66.6279,
-      "eval_samples_per_second": 22.618,
-      "eval_steps_per_second": 5.658,
+      "eval_loss": 1.0110633373260498,
+      "eval_runtime": 64.9111,
+      "eval_samples_per_second": 23.216,
+      "eval_steps_per_second": 5.808,
       "step": 960
     },
     {
       "epoch": 1.1557062813329402,
-      "grad_norm": 2.166768789291382,
+      "grad_norm": 2.169173240661621,
       "learning_rate": 2.3138743436842772e-05,
-      "loss": 0.9279,
+      "loss": 0.9255,
       "step": 980
     },
     {
       "epoch": 1.1557062813329402,
-      "eval_loss": 1.013027548789978,
-      "eval_runtime": 66.6135,
-      "eval_samples_per_second": 22.623,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.0110522508621216,
+      "eval_runtime": 62.9115,
+      "eval_samples_per_second": 23.954,
+      "eval_steps_per_second": 5.993,
       "step": 980
     },
     {
       "epoch": 1.1792981421409614,
-      "grad_norm": 1.9013581275939941,
+      "grad_norm": 1.8711203336715698,
       "learning_rate": 2.278954884148232e-05,
-      "loss": 0.9343,
+      "loss": 0.9322,
       "step": 1000
     },
     {
       "epoch": 1.1792981421409614,
-      "eval_loss": 1.0117987394332886,
-      "eval_runtime": 66.5962,
-      "eval_samples_per_second": 22.629,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0103003978729248,
+      "eval_runtime": 62.9309,
+      "eval_samples_per_second": 23.947,
+      "eval_steps_per_second": 5.991,
       "step": 1000
     },
     {
       "epoch": 1.2028900029489826,
-      "grad_norm": 1.893983006477356,
+      "grad_norm": 1.9024525880813599,
       "learning_rate": 2.2434475125167866e-05,
-      "loss": 0.8973,
+      "loss": 0.8943,
       "step": 1020
     },
     {
       "epoch": 1.2028900029489826,
-      "eval_loss": 1.009979486465454,
-      "eval_runtime": 66.6038,
-      "eval_samples_per_second": 22.626,
-      "eval_steps_per_second": 5.66,
+      "eval_loss": 1.0080772638320923,
+      "eval_runtime": 63.4989,
+      "eval_samples_per_second": 23.733,
+      "eval_steps_per_second": 5.937,
       "step": 1020
     },
     {
       "epoch": 1.2264818637570039,
-      "grad_norm": 2.000823497772217,
+      "grad_norm": 2.0306591987609863,
       "learning_rate": 2.207379027791632e-05,
-      "loss": 0.9009,
+      "loss": 0.8974,
       "step": 1040
     },
     {
       "epoch": 1.2264818637570039,
-      "eval_loss": 1.006852388381958,
-      "eval_runtime": 69.1904,
-      "eval_samples_per_second": 21.78,
-      "eval_steps_per_second": 5.449,
+      "eval_loss": 1.005001187324524,
+      "eval_runtime": 62.9683,
+      "eval_samples_per_second": 23.933,
+      "eval_steps_per_second": 5.987,
       "step": 1040
     },
     {
       "epoch": 1.250073724565025,
-      "grad_norm": 2.0272953510284424,
+      "grad_norm": 2.0461585521698,
       "learning_rate": 2.170776652471611e-05,
-      "loss": 0.8942,
+      "loss": 0.8922,
       "step": 1060
     },
     {
       "epoch": 1.250073724565025,
-      "eval_loss": 1.0058958530426025,
-      "eval_runtime": 66.5922,
-      "eval_samples_per_second": 22.63,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0041309595108032,
+      "eval_runtime": 62.9406,
+      "eval_samples_per_second": 23.943,
+      "eval_steps_per_second": 5.99,
       "step": 1060
     },
     {
       "epoch": 1.2736655853730463,
-      "grad_norm": 2.306683301925659,
+      "grad_norm": 2.301267385482788,
       "learning_rate": 2.133668012006682e-05,
-      "loss": 0.9363,
+      "loss": 0.9341,
       "step": 1080
     },
     {
       "epoch": 1.2736655853730463,
-      "eval_loss": 1.0057122707366943,
-      "eval_runtime": 66.7333,
-      "eval_samples_per_second": 22.582,
-      "eval_steps_per_second": 5.649,
+      "eval_loss": 1.0036579370498657,
+      "eval_runtime": 62.8908,
+      "eval_samples_per_second": 23.962,
+      "eval_steps_per_second": 5.995,
       "step": 1080
     },
     {
       "epoch": 1.2972574461810675,
-      "grad_norm": 2.1628775596618652,
+      "grad_norm": 2.170428991317749,
       "learning_rate": 2.096081113947753e-05,
-      "loss": 0.9101,
+      "loss": 0.9075,
       "step": 1100
     },
     {
       "epoch": 1.2972574461810675,
-      "eval_loss": 1.0064170360565186,
-      "eval_runtime": 66.5977,
-      "eval_samples_per_second": 22.628,
-      "eval_steps_per_second": 5.661,
+      "eval_loss": 1.0047935247421265,
+      "eval_runtime": 62.9099,
+      "eval_samples_per_second": 23.955,
+      "eval_steps_per_second": 5.993,
       "step": 1100
     },
     {
       "epoch": 1.3208493069890888,
-      "grad_norm": 2.1356565952301025,
+      "grad_norm": 2.1537463665008545,
       "learning_rate": 2.058044326808132e-05,
-      "loss": 0.932,
+      "loss": 0.9296,
       "step": 1120
     },
     {
       "epoch": 1.3208493069890888,
-      "eval_loss": 1.0042451620101929,
-      "eval_runtime": 66.5522,
-      "eval_samples_per_second": 22.644,
-      "eval_steps_per_second": 5.665,
+      "eval_loss": 1.0024445056915283,
+      "eval_runtime": 64.9442,
+      "eval_samples_per_second": 23.205,
+      "eval_steps_per_second": 5.805,
       "step": 1120
     },
     {
       "epoch": 1.34444116779711,
-      "grad_norm": 2.054288387298584,
+      "grad_norm": 2.0056726932525635,
       "learning_rate": 2.0195863586525413e-05,
-      "loss": 0.9156,
+      "loss": 0.9133,
       "step": 1140
     },
     {
       "epoch": 1.34444116779711,
-      "eval_loss": 1.002011775970459,
-      "eval_runtime": 69.2205,
-      "eval_samples_per_second": 21.771,
-      "eval_steps_per_second": 5.446,
+      "eval_loss": 1.0002470016479492,
+      "eval_runtime": 62.892,
+      "eval_samples_per_second": 23.962,
+      "eval_steps_per_second": 5.994,
       "step": 1140
     },
     {
       "epoch": 1.3680330286051312,
-      "grad_norm": 2.2903268337249756,
+      "grad_norm": 2.3192081451416016,
       "learning_rate": 1.9807362354298556e-05,
-      "loss": 0.9467,
+      "loss": 0.9447,
       "step": 1160
     },
     {
       "epoch": 1.3680330286051312,
-      "eval_loss": 1.0025988817214966,
-      "eval_runtime": 66.6254,
-      "eval_samples_per_second": 22.619,
-      "eval_steps_per_second": 5.659,
+      "eval_loss": 1.0005617141723633,
+      "eval_runtime": 62.9711,
+      "eval_samples_per_second": 23.932,
+      "eval_steps_per_second": 5.987,
       "step": 1160
     },
     {
       "epoch": 1.3916248894131524,
-      "grad_norm": 2.107551336288452,
+      "grad_norm": 2.085900068283081,
       "learning_rate": 1.9415232790659206e-05,
-      "loss": 0.8872,
+      "loss": 0.8843,
       "step": 1180
     },
     {
       "epoch": 1.3916248894131524,
-      "eval_loss": 1.0008505582809448,
-      "eval_runtime": 66.5499,
-      "eval_samples_per_second": 22.645,
-      "eval_steps_per_second": 5.665,
+      "eval_loss": 0.9992862939834595,
+      "eval_runtime": 65.4435,
+      "eval_samples_per_second": 23.027,
+      "eval_steps_per_second": 5.761,
       "step": 1180
     },
     {
       "epoch": 1.4152167502211737,
-      "grad_norm": 2.23716402053833,
+      "grad_norm": 2.2436399459838867,
       "learning_rate": 1.9019770853329856e-05,
-      "loss": 0.9062,
+      "loss": 0.9046,
       "step": 1200
     },
     {
       "epoch": 1.4152167502211737,
-      "eval_loss": 0.998691976070404,
-      "eval_runtime": 68.7653,
-      "eval_samples_per_second": 21.915,
-      "eval_steps_per_second": 5.482,
+      "eval_loss": 0.996922492980957,
+      "eval_runtime": 62.9124,
+      "eval_samples_per_second": 23.954,
+      "eval_steps_per_second": 5.992,
       "step": 1200
     },
     {
       "epoch": 1.4388086110291949,
-      "grad_norm": 2.1724088191986084,
+      "grad_norm": 2.1963181495666504,
       "learning_rate": 1.8621275015124457e-05,
-      "loss": 0.8858,
+      "loss": 0.8844,
       "step": 1220
     },
     {
       "epoch": 1.4388086110291949,
-      "eval_loss": 0.9982479214668274,
-      "eval_runtime": 66.6817,
-      "eval_samples_per_second": 22.6,
-      "eval_steps_per_second": 5.654,
+      "eval_loss": 0.9958762526512146,
+      "eval_runtime": 62.9054,
+      "eval_samples_per_second": 23.957,
+      "eval_steps_per_second": 5.993,
       "step": 1220
     },
     {
       "epoch": 1.462400471837216,
-      "grad_norm": 2.208951950073242,
+      "grad_norm": 2.19089674949646,
       "learning_rate": 1.82200460386777e-05,
-      "loss": 0.9256,
+      "loss": 0.923,
       "step": 1240
     },
     {
       "epoch": 1.462400471837216,
-      "eval_loss": 0.9968158006668091,
-      "eval_runtime": 66.7301,
-      "eval_samples_per_second": 22.583,
-      "eval_steps_per_second": 5.65,
+      "eval_loss": 0.9949436783790588,
+      "eval_runtime": 62.9162,
+      "eval_samples_per_second": 23.952,
+      "eval_steps_per_second": 5.992,
       "step": 1240
     },
     {
       "epoch": 1.4859923326452373,
-      "grad_norm": 2.241866111755371,
+      "grad_norm": 2.2525527477264404,
       "learning_rate": 1.7816386749445935e-05,
-      "loss": 0.8607,
+      "loss": 0.8587,
       "step": 1260
     },
     {
       "epoch": 1.4859923326452373,
-      "eval_loss": 0.996986448764801,
-      "eval_runtime": 66.687,
-      "eval_samples_per_second": 22.598,
-      "eval_steps_per_second": 5.653,
+      "eval_loss": 0.9949393272399902,
+      "eval_runtime": 62.931,
+      "eval_samples_per_second": 23.947,
+      "eval_steps_per_second": 5.991,
       "step": 1260
     },
     {
       "epoch": 1.5095841934532586,
-      "grad_norm": 2.4940097332000732,
+      "grad_norm": 2.4868662357330322,
       "learning_rate": 1.7410601807151304e-05,
-      "loss": 0.8896,
+      "loss": 0.8878,
       "step": 1280
     },
     {
       "epoch": 1.5095841934532586,
-      "eval_loss": 0.9959249496459961,
-      "eval_runtime": 66.7445,
-      "eval_samples_per_second": 22.579,
-      "eval_steps_per_second": 5.648,
+      "eval_loss": 0.9940009713172913,
+      "eval_runtime": 64.7991,
+      "eval_samples_per_second": 23.256,
+      "eval_steps_per_second": 5.818,
       "step": 1280
     },
     {
       "epoch": 1.53317605426128,
-      "grad_norm": 1.942398190498352,
+      "grad_norm": 1.9369481801986694,
       "learning_rate": 1.700299747584139e-05,
-      "loss": 0.9061,
+      "loss": 0.904,
       "step": 1300
     },
     {
       "epoch": 1.53317605426128,
-      "eval_loss": 0.9924446940422058,
-      "eval_runtime": 69.1908,
-      "eval_samples_per_second": 21.78,
-      "eval_steps_per_second": 5.449,
+      "eval_loss": 0.9907172918319702,
+      "eval_runtime": 62.932,
+      "eval_samples_per_second": 23.946,
+      "eval_steps_per_second": 5.991,
       "step": 1300
     },
     {
       "epoch": 1.556767915069301,
-      "grad_norm": 2.2117722034454346,
+      "grad_norm": 2.2156777381896973,
       "learning_rate": 1.659388139273806e-05,
-      "loss": 0.9147,
+      "loss": 0.9121,
       "step": 1320
     },
     {
       "epoch": 1.556767915069301,
-      "eval_loss": 0.99241042137146,
-      "eval_runtime": 66.5596,
-      "eval_samples_per_second": 22.641,
-      "eval_steps_per_second": 5.664,
+      "eval_loss": 0.9908357262611389,
+      "eval_runtime": 62.9132,
+      "eval_samples_per_second": 23.954,
+      "eval_steps_per_second": 5.992,
       "step": 1320
     },
     {
       "epoch": 1.5803597758773225,
-      "grad_norm": 2.4062092304229736,
+      "grad_norm": 2.393859386444092,
       "learning_rate": 1.6183562336049867e-05,
-      "loss": 0.9045,
+      "loss": 0.9023,
       "step": 1340
     },
     {
       "epoch": 1.5803597758773225,
-      "eval_loss": 0.9884071350097656,
-      "eval_runtime": 66.617,
-      "eval_samples_per_second": 22.622,
-      "eval_steps_per_second": 5.659,
+      "eval_loss": 0.9863225221633911,
+      "eval_runtime": 65.5958,
+      "eval_samples_per_second": 22.974,
+      "eval_steps_per_second": 5.747,
       "step": 1340
     },
     {
       "epoch": 1.6039516366853435,
-      "grad_norm": 2.1194331645965576,
+      "grad_norm": 2.1125807762145996,
       "learning_rate": 1.5772349991923366e-05,
-      "loss": 0.8937,
+      "loss": 0.892,
       "step": 1360
     },
     {
       "epoch": 1.6039516366853435,
-      "eval_loss": 0.9872897863388062,
-      "eval_runtime": 66.666,
-      "eval_samples_per_second": 22.605,
-      "eval_steps_per_second": 5.655,
+      "eval_loss": 0.9855474233627319,
+      "eval_runtime": 62.9171,
+      "eval_samples_per_second": 23.952,
+      "eval_steps_per_second": 5.992,
       "step": 1360
     },
     {
       "epoch": 1.627543497493365,
-      "grad_norm": 2.1357710361480713,
+      "grad_norm": 2.1116766929626465,
       "learning_rate": 1.536055472070907e-05,
-      "loss": 0.914,
+      "loss": 0.9116,
       "step": 1380
     },
     {
       "epoch": 1.627543497493365,
-      "eval_loss": 0.9874758720397949,
-      "eval_runtime": 66.7002,
-      "eval_samples_per_second": 22.594,
-      "eval_steps_per_second": 5.652,
+      "eval_loss": 0.9856626391410828,
+      "eval_runtime": 62.9526,
+      "eval_samples_per_second": 23.939,
+      "eval_steps_per_second": 5.989,
       "step": 1380
     },
     {
       "epoch": 1.651135358301386,
-      "grad_norm": 2.210505962371826,
+      "grad_norm": 2.2223665714263916,
       "learning_rate": 1.4948487322718669e-05,
-      "loss": 0.8841,
+      "loss": 0.8816,
       "step": 1400
     },
     {
       "epoch": 1.651135358301386,
-      "eval_loss": 0.9857823252677917,
-      "eval_runtime": 69.526,
-      "eval_samples_per_second": 21.675,
-      "eval_steps_per_second": 5.422,
+      "eval_loss": 0.9842101335525513,
+      "eval_runtime": 63.0292,
+      "eval_samples_per_second": 23.91,
+      "eval_steps_per_second": 5.981,
       "step": 1400
     },
     {
       "epoch": 1.6747272191094074,
-      "grad_norm": 2.2325799465179443,
+      "grad_norm": 2.2404255867004395,
       "learning_rate": 1.4536458803650085e-05,
-      "loss": 0.861,
+      "loss": 0.859,
       "step": 1420
     },
     {
       "epoch": 1.6747272191094074,
-      "eval_loss": 0.9860907793045044,
-      "eval_runtime": 66.719,
-      "eval_samples_per_second": 22.587,
-      "eval_steps_per_second": 5.651,
+      "eval_loss": 0.9844129085540771,
+      "eval_runtime": 63.0729,
+      "eval_samples_per_second": 23.893,
+      "eval_steps_per_second": 5.977,
       "step": 1420
     },
     {
       "epoch": 1.6983190799174284,
-      "grad_norm": 2.1073429584503174,
+      "grad_norm": 2.072277784347534,
       "learning_rate": 1.4124780139857582e-05,
-      "loss": 0.9254,
+      "loss": 0.9234,
       "step": 1440
     },
     {
       "epoch": 1.6983190799174284,
-      "eval_loss": 0.9815816283226013,
-      "eval_runtime": 66.7493,
-      "eval_samples_per_second": 22.577,
-      "eval_steps_per_second": 5.648,
+      "eval_loss": 0.9798393845558167,
+      "eval_runtime": 64.8962,
+      "eval_samples_per_second": 23.222,
+      "eval_steps_per_second": 5.809,
       "step": 1440
     },
     {
       "epoch": 1.7219109407254498,
-      "grad_norm": 2.432131767272949,
+      "grad_norm": 2.4477767944335938,
       "learning_rate": 1.3713762043644008e-05,
-      "loss": 0.8917,
+      "loss": 0.889,
       "step": 1460
     },
     {
       "epoch": 1.7219109407254498,
-      "eval_loss": 0.9828724265098572,
-      "eval_runtime": 66.7104,
-      "eval_samples_per_second": 22.59,
-      "eval_steps_per_second": 5.651,
+      "eval_loss": 0.9810368418693542,
+      "eval_runtime": 62.9029,
+      "eval_samples_per_second": 23.958,
+      "eval_steps_per_second": 5.993,
       "step": 1460
     },
     {
       "epoch": 1.7455028015334708,
-      "grad_norm": 2.2178120613098145,
+      "grad_norm": 2.2249417304992676,
       "learning_rate": 1.3303714728752293e-05,
-      "loss": 0.8695,
+      "loss": 0.8679,
       "step": 1480
     },
     {
       "epoch": 1.7455028015334708,
-      "eval_loss": 0.9821544289588928,
-      "eval_runtime": 66.7271,
-      "eval_samples_per_second": 22.585,
-      "eval_steps_per_second": 5.65,
+      "eval_loss": 0.9801770448684692,
+      "eval_runtime": 62.9129,
+      "eval_samples_per_second": 23.954,
+      "eval_steps_per_second": 5.992,
       "step": 1480
     },
     {
       "epoch": 1.7690946623414923,
-      "grad_norm": 2.3081140518188477,
+      "grad_norm": 2.2835938930511475,
       "learning_rate": 1.2894947676233291e-05,
-      "loss": 0.8922,
+      "loss": 0.8896,
       "step": 1500
     },
     {
       "epoch": 1.7690946623414923,
-      "eval_loss": 0.9805226922035217,
-      "eval_runtime": 69.3053,
-      "eval_samples_per_second": 21.744,
-      "eval_steps_per_second": 5.44,
+      "eval_loss": 0.9786974787712097,
+      "eval_runtime": 65.11,
+      "eval_samples_per_second": 23.145,
+      "eval_steps_per_second": 5.79,
       "step": 1500
     },
     {
       "epoch": 1.7926865231495133,
-      "grad_norm": 2.408884048461914,
+      "grad_norm": 2.386866569519043,
       "learning_rate": 1.2487769400866573e-05,
-      "loss": 0.8727,
+      "loss": 0.8687,
       "step": 1520
     },
     {
       "epoch": 1.7926865231495133,
-      "eval_loss": 0.9798446893692017,
-      "eval_runtime": 66.7518,
-      "eval_samples_per_second": 22.576,
-      "eval_steps_per_second": 5.648,
+      "eval_loss": 0.9775878190994263,
+      "eval_runtime": 62.9587,
+      "eval_samples_per_second": 23.936,
+      "eval_steps_per_second": 5.988,
       "step": 1520
     },
     {
       "epoch": 1.8162783839575347,
-      "grad_norm": 2.334815502166748,
+      "grad_norm": 2.3122761249542236,
       "learning_rate": 1.2082487218310524e-05,
-      "loss": 0.9015,
+      "loss": 0.8992,
       "step": 1540
     },
     {
       "epoch": 1.8162783839575347,
-      "eval_loss": 0.9793778657913208,
-      "eval_runtime": 66.6631,
-      "eval_samples_per_second": 22.606,
-      "eval_steps_per_second": 5.655,
+      "eval_loss": 0.9774647951126099,
+      "eval_runtime": 62.8905,
+      "eval_samples_per_second": 23.962,
+      "eval_steps_per_second": 5.995,
       "step": 1540
     },
     {
       "epoch": 1.8398702447655557,
-      "grad_norm": 2.2518022060394287,
+      "grad_norm": 2.2307450771331787,
       "learning_rate": 1.1679407013157457e-05,
-      "loss": 0.9064,
+      "loss": 0.9041,
       "step": 1560
     },
     {
       "epoch": 1.8398702447655557,
-      "eval_loss": 0.9765506982803345,
-      "eval_runtime": 66.5449,
-      "eval_samples_per_second": 22.646,
-      "eval_steps_per_second": 5.665,
+      "eval_loss": 0.9749870300292969,
+      "eval_runtime": 62.9232,
+      "eval_samples_per_second": 23.95,
+      "eval_steps_per_second": 5.991,
       "step": 1560
     },
     {
       "epoch": 1.8634621055735772,
-      "grad_norm": 2.1811118125915527,
+      "grad_norm": 2.1991312503814697,
       "learning_rate": 1.1278833008068863e-05,
-      "loss": 0.883,
+      "loss": 0.8804,
       "step": 1580
     },
     {
       "epoch": 1.8634621055735772,
-      "eval_loss": 0.9749047756195068,
-      "eval_runtime": 66.5689,
-      "eval_samples_per_second": 22.638,
-      "eval_steps_per_second": 5.663,
+      "eval_loss": 0.973229169845581,
+      "eval_runtime": 62.9112,
+      "eval_samples_per_second": 23.954,
+      "eval_steps_per_second": 5.993,
       "step": 1580
     },
     {
       "epoch": 1.8870539663815984,
-      "grad_norm": 2.1696884632110596,
+      "grad_norm": 2.1787588596343994,
       "learning_rate": 1.088106753416491e-05,
-      "loss": 0.903,
+      "loss": 0.9004,
       "step": 1600
     },
     {
       "epoch": 1.8870539663815984,
-      "eval_loss": 0.974395215511322,
-      "eval_runtime": 69.1351,
-      "eval_samples_per_second": 21.798,
-      "eval_steps_per_second": 5.453,
+      "eval_loss": 0.9725916385650635,
+      "eval_runtime": 62.9975,
+      "eval_samples_per_second": 23.922,
+      "eval_steps_per_second": 5.984,
       "step": 1600
     }
   ],
@@ -1243,7 +1243,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 4.2037767804184166e+17,
+  "total_flos": 3.937740637329162e+17,
   "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:69e121e7c25b7b21cddbd335368e2574f3f50d08e1b8e98dbd57d54cca8ee4c6
-size 6584
+oid sha256:11f83e9f934bc33e50720d217ae99dabaf0b3ef8a9013e81986c142bfd6f2b2c
+size 7160